2024-05-10T03:14:26.257 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T03:14:26.257 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/sessionmap} --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476 --verbose --name pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/orig.config.yaml 2024-05-10T03:14:26.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T03:14:26.395 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476 branch: wip-pdonnell-testing-20240509.171335-reef-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/sessionmap} email: pdonnell@redhat.com first_in_suite: false job_id: '7700476' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240509.171335-reef-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - client session with non-allowable root sha1: 50174015a7e79f951b988949abe236791f52d4da ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 50174015a7e79f951b988949abe236791f52d4da kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240509.171335-reef-debug sha1: 50174015a7e79f951b988949abe236791f52d4da owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4933 sha1: 50174015a7e79f951b988949abe236791f52d4da sleep_before_teardown: 0 subset: 420/512 suite: fs suite_branch: wip-pdonnell-testing-20240509.171335-reef-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50174015a7e79f951b988949abe236791f52d4da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1ggasP6PjOxGWODFby903l3yTc6RQhPb990pDLl36T4WN/LF3LpECGNapQ9EMT1HOx/bnblnJ2eYOgbYJKKMs= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPh9kCquJ9b7EwxZc1UICqq5ofYdFNn1MvupXfUlYLECOTZwj6aFtHoacQcBY1qgnVqdOqrK6p8don5fFwnbji8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_02:33:39 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T03:14:26.396 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa; will attempt to use it 2024-05-10T03:14:26.396 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa/tasks 2024-05-10T03:14:26.396 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T03:14:26.398 INFO:teuthology.task.internal:Checking packages... 2024-05-10T03:14:26.449 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '50174015a7e79f951b988949abe236791f52d4da' 2024-05-10T03:14:26.449 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T03:14:26.449 INFO:teuthology.packaging:ref: None 2024-05-10T03:14:26.449 INFO:teuthology.packaging:tag: None 2024-05-10T03:14:26.450 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T03:14:26.450 INFO:teuthology.packaging:sha1: 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:14:26.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T03:14:26.671 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1330.g50174015 2024-05-10T03:14:26.672 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T03:14:26.678 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T03:14:26.678 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T03:14:26.683 INFO:teuthology.task.internal:Saving configuration 2024-05-10T03:14:26.704 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T03:14:26.710 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T03:14:26.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 03:09:22.177088', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1ggasP6PjOxGWODFby903l3yTc6RQhPb990pDLl36T4WN/LF3LpECGNapQ9EMT1HOx/bnblnJ2eYOgbYJKKMs='} 2024-05-10T03:14:26.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 03:09:22.178969', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPh9kCquJ9b7EwxZc1UICqq5ofYdFNn1MvupXfUlYLECOTZwj6aFtHoacQcBY1qgnVqdOqrK6p8don5fFwnbji8='} 2024-05-10T03:14:26.761 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T03:14:26.767 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-10T03:14:26.767 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-10T03:14:26.767 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T03:14:26.840 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd17c5860d0>, signals=[15]) 2024-05-10T03:14:26.840 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T03:14:26.846 INFO:teuthology.task.internal:Opening connections... 2024-05-10T03:14:26.846 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:14:26.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:14:26.923 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:14:26.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:14:27.005 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T03:14:27.012 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-05-10T03:14:27.031 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-05-10T03:14:27.032 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T03:14:27.089 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-05-10T03:14:27.090 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T03:14:27.090 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T03:14:27.090 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T03:14:27.090 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-05-10T03:14:27.110 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-05-10T03:14:27.128 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-05-10T03:14:27.129 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-05-10T03:14:27.186 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-05-10T03:14:27.186 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-05-10T03:14:27.186 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-05-10T03:14:27.186 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-10T03:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-10T03:14:27.187 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-05-10T03:14:27.210 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T03:14:27.219 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T03:14:27.232 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T03:14:27.233 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2024-05-10T03:14:27.235 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-05-10T03:14:27.252 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T03:14:27.262 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T03:14:27.262 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2024-05-10T03:14:27.293 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-05-10T03:14:27.315 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T03:14:27.375 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T03:14:27.389 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-10T03:14:27.389 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-10T03:14:27.389 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-10T03:14:27.390 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T03:14:27.390 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T03:14:27.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:14:27.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-10T03:14:27.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:14:27.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-10T03:14:27.572 DEBUG:teuthology.task.kernel:sha1 for client.0 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:14:27.573 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-10T03:14:27.573 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-10T03:14:27.590 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-437.el9.x86_64 2024-05-10T03:14:27.591 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:14:27.591 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-10T03:14:27.591 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum upgrade -y linux-firmware 2024-05-10T03:14:27.653 DEBUG:teuthology.task.kernel:sha1 for client.1 is fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:14:27.654 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e"... 2024-05-10T03:14:27.654 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-05-10T03:14:27.671 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-437.el9.x86_64 2024-05-10T03:14:27.671 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:14:27.671 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-10T03:14:27.671 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum upgrade -y linux-firmware 2024-05-10T03:14:30.146 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.1 MB 00:00 2024-05-10T03:14:31.764 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.1 MB 00:02 2024-05-10T03:14:33.617 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-10T03:14:38.467 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-05-10T03:14:41.226 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-10T03:14:42.119 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-05-10T03:14:47.478 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-10T03:14:48.653 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-10T03:14:49.737 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-10T03:14:49.738 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2024-05-10T03:14:49.738 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-10T03:14:49.869 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T03:14:49.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:14:50.026 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-10T03:14:50.026 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-10T03:14:50.029 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi055.front.sepia.ceph.com to complete... 2024-05-10T03:14:53.578 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.2 MB 00:01 2024-05-10T03:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-10T03:14:56.272 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-10T03:15:01.321 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-05-10T03:15:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10 03:15:02 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-10T03:15:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:FINISHED --2024-05-10 03:15:02-- 2024-05-10T03:15:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:Total wall clock time: 12s 2024-05-10T03:15:02.166 INFO:teuthology.orchestra.run.smithi055.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-05-10T03:15:02.170 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.0... 2024-05-10T03:15:02.170 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-10T03:15:03.515 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-05-10T03:15:03.516 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-05-10T03:15:03.516 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-05-10T03:15:03.635 INFO:teuthology.task.kernel:Downloading kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.1... 2024-05-10T03:15:03.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e 2024-05-10T03:15:03.810 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ 2024-05-10T03:15:03.810 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-10T03:15:03.813 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi162.front.sepia.ceph.com to complete... 2024-05-10T03:15:04.362 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying... ######################################## 2024-05-10T03:15:04.488 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing... ######################################## 2024-05-10T03:15:04.603 INFO:teuthology.orchestra.run.smithi055.stdout:Updating / installing... 2024-05-10T03:15:12.885 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-10T03:15:16.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10 03:15:16 URL:https://2.chacra.ceph.com/r/kernel/testing/fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_gfb9a6a44107e-1.x86_64.rpm [357387737/357387737] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-10T03:15:16.483 INFO:teuthology.orchestra.run.smithi162.stderr:FINISHED --2024-05-10 03:15:16-- 2024-05-10T03:15:16.483 INFO:teuthology.orchestra.run.smithi162.stderr:Total wall clock time: 13s 2024-05-10T03:15:16.483 INFO:teuthology.orchestra.run.smithi162.stderr:Downloaded: 1 files, 341M in 12s (27.7 MB/s) 2024-05-10T03:15:16.485 INFO:teuthology.task.kernel:Installing kernel fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e on client.1... 2024-05-10T03:15:16.486 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-10T03:15:18.578 INFO:teuthology.orchestra.run.smithi162.stdout:Verifying... ######################################## 2024-05-10T03:15:18.794 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing... ######################################## 2024-05-10T03:15:18.928 INFO:teuthology.orchestra.run.smithi162.stdout:Updating / installing... 2024-05-10T03:15:26.848 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-6.9.0_rc3_gfb9a6a44107e-1 ######################################## 2024-05-10T03:15:53.594 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-10T03:15:53.594 DEBUG:teuthology.orchestra.run.smithi055:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-10T03:15:53.681 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.681 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.681 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.681 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-10T03:15:53.681 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-10T03:15:53.682 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-10T03:15:53.683 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-10T03:15:53.684 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-10T03:15:53.685 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-10T03:15:53.686 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-10T03:15:53.687 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-10T03:15:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-10T03:15:53.689 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-10T03:15:53.690 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-10T03:15:53.691 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-10T03:15:53.692 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-10T03:15:53.693 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-10T03:15:53.694 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-10T03:15:53.695 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-10T03:15:53.696 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-10T03:15:53.697 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-10T03:15:53.698 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-10T03:15:53.699 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-10T03:15:53.700 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-10T03:15:53.701 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-10T03:15:53.702 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-10T03:15:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-10T03:15:53.704 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-10T03:15:53.705 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-10T03:15:53.706 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-10T03:15:53.707 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-10T03:15:53.708 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-10T03:15:53.709 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-10T03:15:53.710 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-10T03:15:53.711 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-10T03:15:53.712 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-10T03:15:53.713 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-10T03:15:53.714 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-10T03:15:53.715 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-10T03:15:53.716 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-10T03:15:53.717 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-10T03:15:53.718 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-10T03:15:53.719 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-10T03:15:53.720 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-10T03:15:53.721 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-10T03:15:53.722 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-10T03:15:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-10T03:15:53.724 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-10T03:15:53.725 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-10T03:15:53.726 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-10T03:15:53.727 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-10T03:15:53.728 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-10T03:15:53.729 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-10T03:15:53.730 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-10T03:15:53.731 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-10T03:15:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-10T03:15:53.733 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-10T03:15:53.734 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-10T03:15:53.735 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-10T03:15:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-10T03:15:53.737 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-10T03:15:53.738 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-10T03:15:53.739 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-10T03:15:53.740 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-10T03:15:53.741 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-10T03:15:53.742 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-10T03:15:53.743 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-10T03:15:53.744 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-10T03:15:53.745 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-10T03:15:53.746 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-10T03:15:53.747 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-10T03:15:53.748 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-10T03:15:53.749 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-10T03:15:53.750 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-10T03:15:53.751 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-10T03:15:53.752 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-10T03:15:53.753 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-10T03:15:53.754 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-10T03:15:53.755 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-10T03:15:53.756 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-10T03:15:53.757 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-10T03:15:53.758 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-10T03:15:53.759 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-10T03:15:53.760 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-10T03:15:53.761 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-10T03:15:53.762 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-10T03:15:53.763 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-10T03:15:53.764 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-10T03:15:53.764 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-10T03:15:53.764 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-10T03:15:53.764 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-10T03:15:53.765 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-10T03:15:53.766 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-10T03:15:53.767 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-10T03:15:53.768 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-10T03:15:53.769 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-10T03:15:53.770 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-10T03:15:53.771 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-10T03:15:53.772 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-10T03:15:53.773 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-10T03:15:53.774 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-10T03:15:53.775 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-10T03:15:53.776 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-10T03:15:53.777 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-10T03:15:53.778 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-10T03:15:53.779 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-10T03:15:53.780 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-10T03:15:53.781 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-10T03:15:53.782 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-10T03:15:53.783 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-10T03:15:53.784 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-10T03:15:53.785 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-10T03:15:53.786 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-10T03:15:53.787 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-10T03:15:53.788 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-10T03:15:53.789 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-10T03:15:53.790 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-10T03:15:53.791 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-10T03:15:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-10T03:15:53.793 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-10T03:15:53.794 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-10T03:15:53.795 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-10T03:15:53.796 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-10T03:15:53.797 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-10T03:15:53.798 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-10T03:15:53.799 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-10T03:15:53.800 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-10T03:15:53.801 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-10T03:15:53.802 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-10T03:15:53.803 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-10T03:15:53.804 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-10T03:15:53.805 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-10T03:15:53.806 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-10T03:15:53.807 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-10T03:15:53.808 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-10T03:15:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-10T03:15:53.810 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-10T03:15:53.811 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-10T03:15:53.812 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-10T03:15:53.813 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T03:15:53.817 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.817 DEBUG:teuthology.orchestra.run.smithi055:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-10T03:15:53.839 INFO:teuthology.orchestra.run.smithi055.stdout:postinstall scriptlet (using /bin/sh): 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:for file in vmlinuz System.map config; do 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: fi 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:done 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:15:53.840 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.841 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2024-05-10T03:15:53.842 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.06 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 68.el9 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Group : Unspecified 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 8264254 2024-05-10T03:15:53.913 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : builder@centos.org 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : CentOS 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T03:15:53.914 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2024-05-10T03:15:53.915 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:15:53.915 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T03:15:53.915 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T03:15:53.915 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:53.916 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T03:15:54.506 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2024-05-10T03:15:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T03:15:55.959 INFO:teuthology.orchestra.run.smithi055.stderr:done 2024-05-10T03:15:55.961 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-05-10T03:15:55.989 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.VeZLVnVO3O 2024-05-10T03:15:55.989 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VeZLVnVO3O 2024-05-10T03:15:56.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.VeZLVnVO3O 2024-05-10T03:15:56.200 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.VeZLVnVO3O is 6KB 2024-05-10T03:15:56.249 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.VeZLVnVO3O 2024-05-10T03:15:56.263 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T03:15:56.325 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T03:15:56.325 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T03:15:56.325 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T03:15:56.325 INFO:teuthology.orchestra.run.smithi055.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-10T03:15:56.327 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:15:56.468 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2024-05-10T03:16:06.618 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-10T03:16:06.618 DEBUG:teuthology.orchestra.run.smithi162:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/System.map-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/config-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/boot/vmlinuz-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/System.map 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/config 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86 2024-05-10T03:16:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-10T03:16:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/arch/x86/kvm/kvm.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ansi_cprng.ko 2024-05-10T03:16:06.707 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/anubis.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/arc4.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_memcpy.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_pq.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_tx.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/async_xor.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/async_tx/raid6test.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authenc.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/authencesn.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blake2b_generic.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_common.ko 2024-05-10T03:16:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/blowfish_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/camellia_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast5_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast6_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cast_common.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/ccm.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cmac.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crc32_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_engine.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/crypto_user.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/cts.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/des_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/dh_generic.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/echainiv.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/essiv.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/fcrypt.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/khazad.ko 2024-05-10T03:16:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/lz4hc.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/md4.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/michael_mic.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcbc.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/pcrypt.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/rmd160.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/seed.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/serpent_generic.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tcrypt.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/tea.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_common.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/twofish_generic.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/vmac.ko 2024-05-10T03:16:06.710 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/wp512.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xcbc.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xor.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/crypto/xxhash_generic.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/acpi_pad.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/ec_sys.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/platform_profile.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbs.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/sbshc.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/acpi/video.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/acard-ahci.ko 2024-05-10T03:16:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ahci_platform.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/ata_generic.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/libahci_platform.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_acpi.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ali.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_amd.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_artop.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atiixp.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_atp867x.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd640.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cmd64x.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_cypress.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_efar.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt366.ko 2024-05-10T03:16:06.712 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt37x.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it8213.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_it821x.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_jmicron.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_marvell.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_mpiix.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_netcell.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ninja32.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87410.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_ns87415.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_oldpiix.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_opti.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_optidma.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pcmcia.ko 2024-05-10T03:16:06.713 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_piccolo.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_rdc.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sch.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_serverworks.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sil680.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sis.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_sl82c105.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_triflex.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pata_via.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/pdc_adma.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_inic162x.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_mv.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_nv.ko 2024-05-10T03:16:06.714 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_promise.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_qstor.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sil24.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sis.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_svw.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_sx4.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_uli.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_via.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ata/sata_vsc.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/auxdisplay/ks0108.ko 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap 2024-05-10T03:16:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/bcma/bcma.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/aoe/aoe.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/brd.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/drbd/drbd.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/floppy.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/loop.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-10T03:16:06.716 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/nbd.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/null_blk/null_blk.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/pktcdvd.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/rbd.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/virtio_blk.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/xen-blkfront.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/block/zram/zram.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hangcheck-timer.ko 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random 2024-05-10T03:16:06.717 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/via-rng.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/lp.ko 2024-05-10T03:16:06.718 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/mwave/mwave.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/ppdev.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tlclk.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/uv_mmtimer.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/char/virtio_console.ko 2024-05-10T03:16:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/ccp/ccp.ko 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common 2024-05-10T03:16:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-aes.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/padlock-sha.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dca/dca.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw 2024-05-10T03:16:06.721 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/amd64_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/e752x_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/edac_mce_amd.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3000_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i3200_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5100_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i5400_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7300_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i7core_edac.ko 2024-05-10T03:16:06.722 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/i82975x_edac.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/ie31200_edac.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/sb_edac.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/edac/x38_edac.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/edd.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-ich.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-appleir.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-aureal.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-axff.ko 2024-05-10T03:16:06.723 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-dr.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elecom.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-elo.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-emsff.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gaff.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gt683r.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-gyration.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-holtekff.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-hyperv.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-icade.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-keytouch.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-kye.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lcpower.ko 2024-05-10T03:16:06.724 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-led.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-lenovo.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-multitouch.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-ortek.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-penmount.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-petalynx.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-picolcd.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-pl.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-plantronics.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-primax.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-rmi.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-10T03:16:06.725 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-common.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-roccat.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-saitek.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-samsung.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sjoy.ko 2024-05-10T03:16:06.726 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sony.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-speedlink.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-steelseries.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-sunplus.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tivo.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-tmff.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-topseed.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-twinhan.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-uclogic.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-waltop.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-wiimote.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-xinmo.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zpff.ko 2024-05-10T03:16:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/hid-zydacron.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/uhid.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hid/wacom.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_balloon.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_utils.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hv/hv_vmbus.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/abituguru3.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7414.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ad7418.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adc128d818.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1025.ko 2024-05-10T03:16:06.728 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1026.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1029.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm1031.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adm9240.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ads7828.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7410.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7411.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7462.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7470.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7475.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/adt7x10.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/amc6821.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/applesmc.ko 2024-05-10T03:16:06.729 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asb100.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asc7621.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/atxp1.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/coretemp.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/dme1737.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds1621.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ds620.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc1403.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/emc6w201.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71805f.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f71882fg.ko 2024-05-10T03:16:06.730 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/f75375s.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fam15h_power.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/fschmd.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g760a.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/g762.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl518sm.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/gl520sm.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5500_temp.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/i5k_amb.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmaem.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ibmpex.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina209.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ina2xx.ko 2024-05-10T03:16:06.731 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/it87.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k10temp.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/k8temp.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lineage-pem.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm63.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm73.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm75.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm77.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm78.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm80.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm83.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm85.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm87.ko 2024-05-10T03:16:06.732 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm90.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm92.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm93.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95234.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95241.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/lm95245.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc2945.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4151.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4215.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4222.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4245.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4260.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/ltc4261.ko 2024-05-10T03:16:06.733 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max16065.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1619.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max1668.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max197.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6639.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6650.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/max6697.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/mcp3021.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6683.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775-core.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct6775.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/nct7802.ko 2024-05-10T03:16:06.734 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87360.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pc87427.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pcf8591.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-10T03:16:06.735 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/powr1220.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5627.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch5636.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht15.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sht21.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/shtc1.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/sis5595.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47b397.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m1.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/smsc47m192.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/thmc50.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp102.ko 2024-05-10T03:16:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp103.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp401.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/tmp421.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via-cputemp.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/via686a.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt1211.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/vt8231.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627ehf.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83627hf.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83781d.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83791d.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83792d.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83793.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83795.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l785ts.ko 2024-05-10T03:16:06.737 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/hwmon/w83l786ng.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-10T03:16:06.738 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-10T03:16:06.739 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-dev.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-mux.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-smbus.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/i2c/i2c-stub.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_core.ko 2024-05-10T03:16:06.740 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-10T03:16:06.741 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt 2024-05-10T03:16:06.742 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/gameport/gameport.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/joydev.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/appletouch.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/bcm5974.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/cyapatp.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-10T03:16:06.743 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/sermouse.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/altera_ps2.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/arc_ps2.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/serio/serio_raw.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/input/sparse-keymap.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-blinkm.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lm3530.ko 2024-05-10T03:16:06.744 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-lp3944.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/leds-ss4200.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/bcache/bcache.ko 2024-05-10T03:16:06.745 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-bio-prison.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache-smq.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-cache.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-crypt.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-delay.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-flakey.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-log-userspace.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-multipath.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-queue-length.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-raid.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-round-robin.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-service-time.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-switch.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-thin-pool.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/dm-verity.ko 2024-05-10T03:16:06.746 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid0.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid1.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid10.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/md/raid456.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptbase.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptctl.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptfc.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptlan.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptsas.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptscsih.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/message/fusion/mptspi.ko 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd 2024-05-10T03:16:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_ich.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/lpc_sch.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/mfd-core.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/sm501.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/viperboard.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/vx855.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/mfd/wl1273-core.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds9802als.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/apds990x.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/bh1770glc.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/cb710/cb710.ko 2024-05-10T03:16:06.748 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/at24.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/eeprom/max6875.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/enclosure.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/hpilo.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29003.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/isl29020.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-me.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei-txe.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/mei/mei.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-10T03:16:06.749 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_7xx1.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tifm_core.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/tsl2550.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_balloon.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/bonding/bonding.ko 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53 2024-05-10T03:16:06.750 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/dummy.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/eql.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-10T03:16:06.751 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-10T03:16:06.752 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-10T03:16:06.753 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-10T03:16:06.754 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec 2024-05-10T03:16:06.755 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-10T03:16:06.756 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/dnet.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ethoc.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/fealnx.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k 2024-05-10T03:16:06.757 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-10T03:16:06.758 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/jme.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-10T03:16:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia 2024-05-10T03:16:06.760 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-10T03:16:06.761 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis 2024-05-10T03:16:06.762 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun 2024-05-10T03:16:06.763 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom 2024-05-10T03:16:06.764 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/geneve.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ifb.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvlan.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/macvtap.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-10T03:16:06.765 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/mii.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/net_failover.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/netconsole.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/nlmon.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/amd.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-10T03:16:06.766 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/bcm87xx.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/broadcom.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/cicada.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/davicom.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/dp83640.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/et1011c.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/icplus.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/lxt.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/marvell.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/micrel.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/national.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/phylink.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/qsemi.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/realtek.ko 2024-05-10T03:16:06.767 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/smsc.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/ste10Xp.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/phy/vitesse.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_async.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppoe.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pppox.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/ppp/pptp.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slhc.ko 2024-05-10T03:16:06.768 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/slip/slip.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/sungem_phy.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tap.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_random.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/tun.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/veth.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/virtio_net.ko 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3 2024-05-10T03:16:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/vxlan/vxlan.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/net/xen-netfront.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ntb/ntb.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme-core.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/nvme/host/nvme.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport.ko 2024-05-10T03:16:06.770 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_cs.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_pc.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/parport/parport_serial.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pci/xen-pcifront.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/i82092.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/pd6729.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-10T03:16:06.771 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/acerhdf.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-10T03:16:06.772 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/hdaps.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/intel_ips.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/platform/x86/wmi.ko 2024-05-10T03:16:06.773 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/clients/pps_parport.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/pps/pps_core.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ptp/ptp_kvm.ko 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl 2024-05-10T03:16:06.774 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ras/amd/fmpm.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-em3027.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-10T03:16:06.775 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-max6900.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-10T03:16:06.776 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/rtc/rtc-x1205.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-9xxx.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-sas.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/3w-xxxx.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/BusLogic.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/a100u2w.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/advansys.ko 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx 2024-05-10T03:16:06.777 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/am53c974.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/atp870u.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bfa/bfa.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i 2024-05-10T03:16:06.778 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ch.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dc395x.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-10T03:16:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/dmx3191d.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/esp_scsi.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/fnic/fnic.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hpsa.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hptiop.ko 2024-05-10T03:16:06.780 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/hv_storvsc.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/initio.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ipr.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ips.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/isci/isci.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libfc/libfc.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/libsas/libsas.ko 2024-05-10T03:16:06.781 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/mvumi.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-10T03:16:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/pmcraid.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla1280.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/raid_class.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_debug.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-10T03:16:06.783 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/ses.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/st.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/stex.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/virtio_scsi.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/scsi/wd719x.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ssb/ssb.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-10T03:16:06.784 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_file.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_iblock.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_mod.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_pscsi.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/target_core_user.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-10T03:16:06.785 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-10T03:16:06.786 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_gsm.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/n_hdlc.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/nozomi.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/arc_uart.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core 2024-05-10T03:16:06.787 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_aec.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_cif.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_pci_generic.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/uio/uio_sercos3.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-acm.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usblp.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/class/usbtmc.ko 2024-05-10T03:16:06.788 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/mdc800.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/image/microtek.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/adutux.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/appledisplay.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi26.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/emi62.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ezusb.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/idmouse.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/iowarrior.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/ldusb.ko 2024-05-10T03:16:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/legousbtower.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usb3503.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usblcd.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/uss720.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/misc/yurex.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/phy/phy-generic.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/aircable.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ark3116.ko 2024-05-10T03:16:06.790 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ch341.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cp210x.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cyberjack.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/empeg.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/io_ti.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipaq.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ipw.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ir-usb.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-10T03:16:06.791 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mct_u232.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7720.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/mos7840.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/navman.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/omninet.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/opticon.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/option.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/oti6858.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/pl2303.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcaux.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/qcserial.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/quatech2.ko 2024-05-10T03:16:06.792 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/safe_serial.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/sierra.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ssu100.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/symbolserial.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_debug.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/visor.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/whiteheat.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/uas.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-10T03:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-karma.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/storage/usb-storage.ko 2024-05-10T03:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_net.ko 2024-05-10T03:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/vhost/vhost_scsi.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/apple_bl.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/lcd.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_balloon.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_mmio.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/virtio/virtio_ring.ko 2024-05-10T03:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2482.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/masters/ds2490.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-10T03:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/w1/wire.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/hpwdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/i6300esb.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ib700wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ibmasr.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-10T03:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/it87_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/machzwd.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/nv_tco.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/softdog.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/via_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-10T03:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/wdt_pci.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/watchdog/xen_wdt.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-evtchn.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntalloc.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-gntdev.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xen-privcmd.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs 2024-05-10T03:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/9p/9p.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/affs/affs.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/befs/befs.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/binfmt_misc.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/btrfs/btrfs.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cachefiles/cachefiles.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ceph/ceph.ko 2024-05-10T03:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/coda/coda.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/cramfs/cramfs.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/dlm/dlm.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/f2fs/f2fs.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/fat.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/msdos.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fat/vfat.ko 2024-05-10T03:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/cuse.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/fuse/fuse.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/gfs2/gfs2.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfs/hfs.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/hfsplus/hfsplus.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/isofs/isofs.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/jfs/jfs.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/lockd/lockd.ko 2024-05-10T03:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/minix/minix.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/netfs/netfs.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfs.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv3.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs/nfsv4.ko 2024-05-10T03:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/grace.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfs_common/nfs_acl.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nfsd/nfsd.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nilfs2/nilfs2.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-celtic.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-centeuro.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-croatian.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-cyrillic.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-gaelic.ko 2024-05-10T03:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-greek.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-iceland.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-inuit.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-roman.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-romanian.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/mac-turkish.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1250.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1251.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp1255.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp737.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp775.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp850.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp852.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp855.ko 2024-05-10T03:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp857.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp860.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp861.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp862.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp863.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp864.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp865.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp866.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp869.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp874.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp932.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp936.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp949.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_cp950.ko 2024-05-10T03:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_euc-jp.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-1.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-13.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-14.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-15.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-2.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-3.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-4.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-5.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-6.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-7.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_iso8859-9.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-r.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-ru.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_koi8-u.ko 2024-05-10T03:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/nls/nls_utf8.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/overlayfs/overlay.ko 2024-05-10T03:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/pstore/ramoops.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/reiserfs/reiserfs.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/romfs/romfs.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/client/cifs.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_arc4.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/smb/common/cifs_md4.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/squashfs/squashfs.ko 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv 2024-05-10T03:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/sysv/sysv.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/udf/udf.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/ufs/ufs.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/fs/xfs/xfs.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/rcu/rcutorture.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/torture.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-10T03:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/asn1_encoder.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc-itu-t.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crc8.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libarc4.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/crypto/libdes.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/libcrc32c.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lru_cache.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4_compress.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/lz4/lz4hc_compress.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/math/cordic.ko 2024-05-10T03:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/raid6/raid6_pq.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_bm.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_fsm.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/ts_kmp.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/lib/zstd/zstd_compress.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/mm/hwpoison-inject.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/6lowpan.ko 2024-05-10T03:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_dest.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_fragment.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_hop.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_mobility.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_routing.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/6lowpan/nhc_udp.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/garp.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/mrp.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/p8022.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/psnap.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/802/stp.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/8021q/8021q.ko 2024-05-10T03:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_fd.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_rdma.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/9p/9pnet_virtio.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/batman-adv/batman-adv.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/br_netfilter.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/bridge.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-10T03:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-10T03:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/ebtables.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ceph/libceph.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/failover.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/core/pktgen.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_diag.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv4.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dccp/dccp_ipv6.ko 2024-05-10T03:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dns_resolver/dns_resolver.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/dsa_core.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_brcm.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_dsa.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_none.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/dsa/tag_trailer.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4 2024-05-10T03:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ah4.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/esp4.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/fou.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/gre.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/inet_diag.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_gre.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_tunnel.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ip_vti.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipcomp.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/ipip.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-10T03:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-10T03:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_bic.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_dctcp.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_diag.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_highspeed.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_htcp.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_hybla.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_illinois.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_lp.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_scalable.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_vegas.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_veno.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_westwood.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tcp_yeah.ko 2024-05-10T03:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/tunnel4.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_diag.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/udp_tunnel.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ah6.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/esp6.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/fou6.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_tunnel.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ip6_vti.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/ipcomp6.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-10T03:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-10T03:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/sit.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/tunnel6.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/key/af_key.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_core.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_eth.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip.ko 2024-05-10T03:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ip6.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_netlink.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/l2tp/l2tp_ppp.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/llc/llc.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mac802154/mac802154.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/mpls/mpls_gso.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-10T03:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-10T03:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conncount.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack.ko 2024-05-10T03:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-10T03:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_log_syslog.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_irc.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_sip.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nf_tables.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_log.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_chain_nat.ko 2024-05-10T03:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_compat.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_ct.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fib.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_hash.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_limit.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_log.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_masq.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_nat.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_numgen.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_queue.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_quota.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_redir.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject.ko 2024-05-10T03:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_reject_inet.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_socket.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/nft_tunnel.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_AUDIT.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_CT.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_DSCP.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HL.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_HMARK.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LED.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_LOG.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NETMAP.ko 2024-05-10T03:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFLOG.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_RATEEST.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_SECMARK.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TEE.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TPROXY.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_TRACE.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_addrtype.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_bpf.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cgroup.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cluster.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_comment.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connbytes.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlabel.ko 2024-05-10T03:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connlimit.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_connmark.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_conntrack.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_cpu.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dccp.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_devgroup.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_dscp.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ecn.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_esp.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hashlimit.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_helper.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_hl.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipcomp.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_iprange.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_ipvs.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_l2tp.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_length.ko 2024-05-10T03:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_limit.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mac.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_mark.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_multiport.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nat.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_nfacct.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_osf.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_owner.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_physdev.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_pkttype.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_policy.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_quota.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_rateest.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_realm.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_recent.ko 2024-05-10T03:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_sctp.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_set.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_socket.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_state.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_statistic.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_string.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_tcpmss.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_time.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netfilter/xt_u32.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/netlink/netlink_diag.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/nsh/nsh.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/openvswitch.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-geneve.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-gre.ko 2024-05-10T03:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/openvswitch/vport-vxlan.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/packet/af_packet_diag.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_rdma.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/rds/rds_tcp.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_csum.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_gact.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_mirred.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_nat.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_pedit.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_police.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_simple.ko 2024-05-10T03:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_skbedit.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/act_vlan.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_basic.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_bpf.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_flow.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_fw.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_route.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/cls_u32.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_cmp.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_ipset.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_meta.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_nbyte.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_text.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/em_u32.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_choke.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_codel.ko 2024-05-10T03:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_drr.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_fq.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_gred.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hfsc.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_hhf.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_htb.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_ingress.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_mqprio_lib.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_multiq.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_netem.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_pie.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_plug.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_prio.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_qfq.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_red.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfb.ko 2024-05-10T03:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_sfq.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_tbf.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sched/sch_teql.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sctp/sctp_diag.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/sunrpc.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/tipc/tipc_diag.ko 2024-05-10T03:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/unix/unix_diag.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/kernel/security/keys/trusted-keys/trusted.ko 2024-05-10T03:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.alias.bin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.alias.bin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.bin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.builtin.modinfo 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep.bin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.devname 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.order 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.softdep 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.symbols.bin 2024-05-10T03:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout:/lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T03:16:06.845 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.845 DEBUG:teuthology.orchestra.run.smithi162:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:postinstall scriptlet (using /bin/sh): 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: /usr/bin/kernel-install add 6.9.0-rc3-gfb9a6a44107e /lib/modules/6.9.0-rc3-gfb9a6a44107e/vmlinuz 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:for file in vmlinuz System.map config; do 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e"; then 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: cp "/lib/modules/6.9.0-rc3-gfb9a6a44107e/${file}" "/boot/${file}-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: fi 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:done 2024-05-10T03:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -d "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" ] && \ 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" >/dev/null 2>&1; then 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: rm -rf "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: cp -r "/lib/modules/6.9.0-rc3-gfb9a6a44107e/dtb" "/boot/dtb-6.9.0-rc3-gfb9a6a44107e" 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-gfb9a6a44107e/modules.dep" ]; then 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: /usr/sbin/depmod 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-install remove 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout:fi 2024-05-10T03:16:06.872 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 68.el9 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8264254 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-10T03:16:06.945 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:16:06.946 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2024-05-10T03:16:06.947 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-10T03:16:06.947 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:06.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-10T03:16:07.529 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-05-10T03:16:08.853 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-10T03:16:08.883 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-05-10T03:16:08.886 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-05-10T03:16:08.906 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.Sj2UGB5jDj 2024-05-10T03:16:08.906 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Sj2UGB5jDj 2024-05-10T03:16:08.972 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.Sj2UGB5jDj 2024-05-10T03:16:09.110 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.Sj2UGB5jDj is 6KB 2024-05-10T03:16:09.134 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.Sj2UGB5jDj 2024-05-10T03:16:09.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2024-05-10T03:16:09.210 INFO:teuthology.orchestra.run.smithi162.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-10T03:16:09.210 INFO:teuthology.orchestra.run.smithi162.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-10T03:16:09.210 INFO:teuthology.orchestra.run.smithi162.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-10T03:16:09.210 INFO:teuthology.orchestra.run.smithi162.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e.conf 2024-05-10T03:16:09.212 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:16:09.410 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-05-10T03:16:26.488 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-10T03:16:26.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:16:26.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:16:39.422 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-10T03:16:39.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:16:39.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:16:44.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-10T03:16:53.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:16:53.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:16:57.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-05-10T03:17:06.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:17:06.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:17:09.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-05-10T03:17:12.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-10T03:17:21.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:17:21.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:17:24.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:17:24.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:17:27.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-05-10T03:17:42.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:17:42.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:17:42.769 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-10T03:17:43.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:17:43.394 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-10T03:17:43.394 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-05-10T03:17:43.436 INFO:teuthology.orchestra.run.smithi055.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:17:43.436 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:17:43.437 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T03:17:43.437 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T03:17:44.437 DEBUG:teuthology.parallel:result is None 2024-05-10T03:18:21.833 DEBUG:teuthology.orchestra.remote:timed out 2024-05-10T03:18:36.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:18:36.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:18:37.263 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-05-10T03:18:37.864 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:18:37.864 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-gfb9a6a44107e"... 2024-05-10T03:18:37.864 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-05-10T03:18:37.916 INFO:teuthology.orchestra.run.smithi162.stdout:6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:18:37.916 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-gfb9a6a44107e vs 6.9.0-rc3-gfb9a6a44107e 2024-05-10T03:18:37.917 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-10T03:18:37.917 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-10T03:18:38.918 DEBUG:teuthology.parallel:result is None 2024-05-10T03:18:38.922 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T03:18:38.935 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T03:18:38.936 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T03:18:38.944 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T03:18:39.001 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T03:18:39.009 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T03:18:39.027 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T03:18:39.027 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T03:18:39.033 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T03:18:39.108 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T03:18:39.119 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T03:18:39.119 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T03:18:39.127 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T03:18:39.258 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:18:39.277 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:18:39.300 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:18:39.352 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T03:18:39.358 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T03:18:39.365 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T03:18:39.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T03:18:39.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T03:18:39.460 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T03:18:39.471 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T03:18:39.471 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T03:18:39.477 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T03:18:39.530 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:18:39.617 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:18:39.713 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:18:39.803 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:18:39.897 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:18:39.897 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T03:18:39.992 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:18:40.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T03:18:40.179 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:18:40.259 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T03:18:40.339 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:18:40.339 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T03:18:40.419 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2024-05-10T03:18:40.428 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-05-10T03:18:40.543 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:18:40.572 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:18:41.050 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T03:18:41.059 INFO:teuthology.task.internal:Starting timer... 2024-05-10T03:18:41.059 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T03:18:41.096 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T03:18:41.180 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2024-05-10T03:18:41.281 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2024-05-10T03:18:41.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-05-10T03:18:41.398 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-05-10T03:18:41.404 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T03:18:41.404 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2024-05-10T03:18:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2024-05-10T03:18:41.470 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-05-10T03:18:41.514 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-05-10T03:18:41.514 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-05-10T03:18:41.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:18:41.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:18:41.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:18:41.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:18:41.672 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T03:18:41.672 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2024-05-10T03:18:41.746 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-05-10T03:18:41.816 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T03:18:41.827 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-10T03:18:41.833 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-10T03:18:41.970 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T03:18:42.009 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T03:18:42.011 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T03:25:41.793 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-05-10T03:25:41.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:25:41.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:25:41.903 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-05-10T03:25:42.059 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-05-10T03:25:42.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:25:42.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T03:25:42.168 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-05-10T03:25:42.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-05-10T03:25:42.325 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T03:25:42.336 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T03:25:42.336 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T03:25:42.337 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:25:42.344 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T03:25:42.344 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:25:42.434 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T03:25:42.440 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T03:25:42.476 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T03:25:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T03:25:42.547 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-05-10T03:25:42.568 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2024-05-10T03:25:42.577 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-05-10T03:25:42.602 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2024-05-10T03:25:42.614 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T03:25:42.646 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T03:25:42.655 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T03:25:42.690 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T03:25:42.782 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.793 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.842 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-10T03:25:42.851 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:25:42.852 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-10T03:25:42.852 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.852 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.852 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.852 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:25:42.854 INFO:teuthology.run_tasks:Running task install... 2024-05-10T03:25:42.863 DEBUG:teuthology.task.install:project ceph 2024-05-10T03:25:42.863 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}} 2024-05-10T03:25:42.863 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'} 2024-05-10T03:25:42.863 INFO:teuthology.task.install:Using flavor: default 2024-05-10T03:25:42.869 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T03:25:42.869 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T03:25:42.870 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T03:25:42.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:25:42.872 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T03:25:42.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:25:43.098 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/ 2024-05-10T03:25:43.098 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T03:25:43.103 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/ 2024-05-10T03:25:43.103 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T03:25:43.218 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T03:25:43.218 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:25:43.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T03:25:43.271 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T03:25:43.271 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:25:43.271 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T03:25:43.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T03:25:43.306 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T03:25:43.367 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T03:25:43.367 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T03:25:43.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T03:25:43.499 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T03:25:43.546 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T03:25:43.618 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T03:25:43.625 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-05-10T03:25:43.629 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-05-10T03:25:43.699 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2024-05-10T03:25:43.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2024-05-10T03:25:43.949 INFO:teuthology.orchestra.run.smithi162.stdout:54 files removed 2024-05-10T03:25:43.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T03:25:44.046 INFO:teuthology.orchestra.run.smithi055.stdout:54 files removed 2024-05-10T03:25:44.085 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T03:25:44.811 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 201 kB/s | 77 kB 00:00 2024-05-10T03:25:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 159 kB/s | 77 kB 00:00 2024-05-10T03:25:45.262 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-05-10T03:25:45.528 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-10T03:25:45.678 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-10T03:25:46.095 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-10T03:25:46.761 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.1 MB 00:01 2024-05-10T03:25:47.143 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.1 MB 00:00 2024-05-10T03:25:48.995 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-05-10T03:25:51.002 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-05-10T03:25:54.046 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-05-10T03:25:56.862 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-10T03:25:57.211 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-10T03:25:58.837 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-05-10T03:26:00.545 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-10T03:26:06.956 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-10T03:26:09.658 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T03:26:09.660 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T03:26:09.680 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T03:26:09.681 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T03:26:09.683 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T03:26:09.684 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T03:26:09.687 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T03:26:09.691 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T03:26:09.692 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T03:26:09.766 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el9 ceph 6.5 k 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el9 ceph 5.1 M 2024-05-10T03:26:09.790 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el9 ceph 824 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el9 ceph 144 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el9 ceph 1.4 M 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 139 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 3.5 M 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 7.4 M 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 49 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el9 ceph 7.5 M 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el9 ceph 38 M 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 264 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 224 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el9 ceph 220 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 18 k 2024-05-10T03:26:09.791 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el9 ceph 31 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 719 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el9 ceph 127 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el9 ceph 162 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el9 ceph 320 k 2024-05-10T03:26:09.792 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el9 ceph 298 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el9 ceph 99 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el9 ceph 87 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.0 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el9 ceph 170 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.2 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.0 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T03:26:09.793 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el9 ceph 17 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 24 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el9 ceph 2.0 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 246 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el9 ceph 4.9 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el9 ceph 18 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 16 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el9 ceph 25 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T03:26:09.794 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el9 ceph 162 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el9 ceph 468 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T03:26:09.795 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 4.2 M 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el9 ceph 45 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el9 ceph 129 k 2024-05-10T03:26:09.796 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T03:26:09.797 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T03:26:09.798 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T03:26:09.799 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout:Install 130 Packages 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 8 Packages 2024-05-10T03:26:09.800 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:26:09.801 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 199 M 2024-05-10T03:26:09.801 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-05-10T03:26:10.691 INFO:teuthology.orchestra.run.smithi055.stdout:(1/138): ceph-18.2.2-1330.g50174015.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2024-05-10T03:26:10.974 INFO:teuthology.orchestra.run.smithi055.stdout:(2/138): ceph-fuse-18.2.2-1330.g50174015.el9.x8 2.9 MB/s | 824 kB 00:00 2024-05-10T03:26:11.066 INFO:teuthology.orchestra.run.smithi055.stdout:(3/138): ceph-immutable-object-cache-18.2.2-133 1.6 MB/s | 144 kB 00:00 2024-05-10T03:26:11.192 INFO:teuthology.orchestra.run.smithi055.stdout:(4/138): ceph-base-18.2.2-1330.g50174015.el9.x8 6.6 MB/s | 5.1 MB 00:00 2024-05-10T03:26:11.534 INFO:teuthology.orchestra.run.smithi055.stdout:(5/138): ceph-mds-18.2.2-1330.g50174015.el9.x86 4.3 MB/s | 2.0 MB 00:00 2024-05-10T03:26:11.724 INFO:teuthology.orchestra.run.smithi055.stdout:(6/138): ceph-mgr-18.2.2-1330.g50174015.el9.x86 2.7 MB/s | 1.4 MB 00:00 2024-05-10T03:26:11.968 INFO:teuthology.orchestra.run.smithi055.stdout:(7/138): ceph-common-18.2.2-1330.g50174015.el9. 11 MB/s | 17 MB 00:01 2024-05-10T03:26:12.634 INFO:teuthology.orchestra.run.smithi055.stdout:(8/138): ceph-mon-18.2.2-1330.g50174015.el9.x86 4.4 MB/s | 4.9 MB 00:01 2024-05-10T03:26:12.785 INFO:teuthology.orchestra.run.smithi055.stdout:(9/138): ceph-radosgw-18.2.2-1330.g50174015.el9 9.3 MB/s | 7.5 MB 00:00 2024-05-10T03:26:12.810 INFO:teuthology.orchestra.run.smithi055.stdout:(10/138): ceph-selinux-18.2.2-1330.g50174015.el 144 kB/s | 25 kB 00:00 2024-05-10T03:26:13.010 INFO:teuthology.orchestra.run.smithi055.stdout:(11/138): cephfs-mirror-18.2.2-1330.g50174015.e 1.1 MB/s | 220 kB 00:00 2024-05-10T03:26:13.052 INFO:teuthology.orchestra.run.smithi055.stdout:(12/138): libcephfs-devel-18.2.2-1330.g50174015 765 kB/s | 31 kB 00:00 2024-05-10T03:26:13.361 INFO:teuthology.orchestra.run.smithi055.stdout:(13/138): ceph-osd-18.2.2-1330.g50174015.el9.x8 11 MB/s | 18 MB 00:01 2024-05-10T03:26:13.402 INFO:teuthology.orchestra.run.smithi055.stdout:(14/138): libcephfs2-18.2.2-1330.g50174015.el9. 2.0 MB/s | 719 kB 00:00 2024-05-10T03:26:13.436 INFO:teuthology.orchestra.run.smithi055.stdout:(15/138): libcephsqlite-18.2.2-1330.g50174015.e 2.1 MB/s | 162 kB 00:00 2024-05-10T03:26:13.470 INFO:teuthology.orchestra.run.smithi055.stdout:(16/138): librados-devel-18.2.2-1330.g50174015. 1.9 MB/s | 127 kB 00:00 2024-05-10T03:26:13.645 INFO:teuthology.orchestra.run.smithi055.stdout:(17/138): libradosstriper1-18.2.2-1330.g5017401 2.2 MB/s | 468 kB 00:00 2024-05-10T03:26:13.725 INFO:teuthology.orchestra.run.smithi055.stdout:(18/138): python3-ceph-argparse-18.2.2-1330.g50 572 kB/s | 45 kB 00:00 2024-05-10T03:26:13.788 INFO:teuthology.orchestra.run.smithi055.stdout:(19/138): python3-ceph-common-18.2.2-1330.g5017 2.1 MB/s | 129 kB 00:00 2024-05-10T03:26:13.846 INFO:teuthology.orchestra.run.smithi055.stdout:(20/138): python3-cephfs-18.2.2-1330.g50174015. 2.8 MB/s | 162 kB 00:00 2024-05-10T03:26:13.913 INFO:teuthology.orchestra.run.smithi055.stdout:(21/138): python3-rados-18.2.2-1330.g50174015.e 4.8 MB/s | 320 kB 00:00 2024-05-10T03:26:14.005 INFO:teuthology.orchestra.run.smithi055.stdout:(22/138): python3-rbd-18.2.2-1330.g50174015.el9 3.2 MB/s | 298 kB 00:00 2024-05-10T03:26:14.231 INFO:teuthology.orchestra.run.smithi055.stdout:(23/138): python3-rgw-18.2.2-1330.g50174015.el9 444 kB/s | 99 kB 00:00 2024-05-10T03:26:14.331 INFO:teuthology.orchestra.run.smithi055.stdout:(24/138): librgw2-18.2.2-1330.g50174015.el9.x86 4.9 MB/s | 4.2 MB 00:00 2024-05-10T03:26:14.356 INFO:teuthology.orchestra.run.smithi055.stdout:(25/138): rbd-fuse-18.2.2-1330.g50174015.el9.x8 703 kB/s | 87 kB 00:00 2024-05-10T03:26:14.415 INFO:teuthology.orchestra.run.smithi055.stdout:(26/138): rbd-nbd-18.2.2-1330.g50174015.el9.x86 2.9 MB/s | 170 kB 00:00 2024-05-10T03:26:14.465 INFO:teuthology.orchestra.run.smithi055.stdout:(27/138): ceph-grafana-dashboards-18.2.2-1330.g 496 kB/s | 24 kB 00:00 2024-05-10T03:26:14.524 INFO:teuthology.orchestra.run.smithi055.stdout:(28/138): ceph-mgr-cephadm-18.2.2-1330.g5017401 2.4 MB/s | 139 kB 00:00 2024-05-10T03:26:14.924 INFO:teuthology.orchestra.run.smithi055.stdout:(29/138): rbd-mirror-18.2.2-1330.g50174015.el9. 5.0 MB/s | 3.0 MB 00:00 2024-05-10T03:26:15.008 INFO:teuthology.orchestra.run.smithi055.stdout:(30/138): ceph-mgr-dashboard-18.2.2-1330.g50174 7.3 MB/s | 3.5 MB 00:00 2024-05-10T03:26:15.066 INFO:teuthology.orchestra.run.smithi055.stdout:(31/138): ceph-mgr-modules-core-18.2.2-1330.g50 4.2 MB/s | 246 kB 00:00 2024-05-10T03:26:15.125 INFO:teuthology.orchestra.run.smithi055.stdout:(32/138): ceph-mgr-rook-18.2.2-1330.g50174015.e 860 kB/s | 49 kB 00:00 2024-05-10T03:26:15.175 INFO:teuthology.orchestra.run.smithi055.stdout:(33/138): ceph-prometheus-alerts-18.2.2-1330.g5 336 kB/s | 16 kB 00:00 2024-05-10T03:26:15.250 INFO:teuthology.orchestra.run.smithi055.stdout:(34/138): ceph-volume-18.2.2-1330.g50174015.el9 3.5 MB/s | 264 kB 00:00 2024-05-10T03:26:15.309 INFO:teuthology.orchestra.run.smithi055.stdout:(35/138): cephadm-18.2.2-1330.g50174015.el9.noa 3.8 MB/s | 224 kB 00:00 2024-05-10T03:26:15.734 INFO:teuthology.orchestra.run.smithi055.stdout:(36/138): ceph-test-18.2.2-1330.g50174015.el9.x 13 MB/s | 38 MB 00:02 2024-05-10T03:26:15.760 INFO:teuthology.orchestra.run.smithi055.stdout:(37/138): cephfs-top-18.2.2-1330.g50174015.el9. 41 kB/s | 18 kB 00:00 2024-05-10T03:26:16.052 INFO:teuthology.orchestra.run.smithi055.stdout:(38/138): ceph-mgr-diskprediction-local-18.2.2- 6.6 MB/s | 7.4 MB 00:01 2024-05-10T03:26:16.260 INFO:teuthology.orchestra.run.smithi055.stdout:(39/138): fuse-2.9.9-15.el9.x86_64.rpm 153 kB/s | 80 kB 00:00 2024-05-10T03:26:16.560 INFO:teuthology.orchestra.run.smithi055.stdout:(40/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 847 kB/s | 253 kB 00:00 2024-05-10T03:26:16.619 INFO:teuthology.orchestra.run.smithi055.stdout:(41/138): libquadmath-11.4.1-3.el9.x86_64.rpm 345 kB/s | 195 kB 00:00 2024-05-10T03:26:16.745 INFO:teuthology.orchestra.run.smithi055.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 857 kB/s | 106 kB 00:00 2024-05-10T03:26:16.795 INFO:teuthology.orchestra.run.smithi055.stdout:(43/138): libgfortran-11.4.1-3.el9.x86_64.rpm 777 kB/s | 803 kB 00:01 2024-05-10T03:26:16.862 INFO:teuthology.orchestra.run.smithi055.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-10T03:26:16.912 INFO:teuthology.orchestra.run.smithi055.stdout:(45/138): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-05-10T03:26:16.938 INFO:teuthology.orchestra.run.smithi055.stdout:(46/138): python3-requests-2.25.1-8.el9.noarch. 880 kB/s | 125 kB 00:00 2024-05-10T03:26:16.971 INFO:teuthology.orchestra.run.smithi055.stdout:(47/138): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-10T03:26:17.163 INFO:teuthology.orchestra.run.smithi055.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 559 kB/s | 107 kB 00:00 2024-05-10T03:26:17.239 INFO:teuthology.orchestra.run.smithi055.stdout:(49/138): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-05-10T03:26:17.272 INFO:teuthology.orchestra.run.smithi055.stdout:(50/138): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-05-10T03:26:17.323 INFO:teuthology.orchestra.run.smithi055.stdout:(51/138): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-05-10T03:26:17.348 INFO:teuthology.orchestra.run.smithi055.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 440 kB/s | 33 kB 00:00 2024-05-10T03:26:17.381 INFO:teuthology.orchestra.run.smithi055.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-05-10T03:26:17.407 INFO:teuthology.orchestra.run.smithi055.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 314 kB/s | 18 kB 00:00 2024-05-10T03:26:17.440 INFO:teuthology.orchestra.run.smithi055.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-05-10T03:26:17.474 INFO:teuthology.orchestra.run.smithi055.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-05-10T03:26:17.499 INFO:teuthology.orchestra.run.smithi055.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 791 kB/s | 45 kB 00:00 2024-05-10T03:26:17.616 INFO:teuthology.orchestra.run.smithi055.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-05-10T03:26:17.650 INFO:teuthology.orchestra.run.smithi055.stdout:(59/138): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-10T03:26:17.675 INFO:teuthology.orchestra.run.smithi055.stdout:(60/138): libuuid-devel-2.37.4-18.el9.x86_64.rp 438 kB/s | 25 kB 00:00 2024-05-10T03:26:17.734 INFO:teuthology.orchestra.run.smithi055.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 570 kB/s | 47 kB 00:00 2024-05-10T03:26:17.767 INFO:teuthology.orchestra.run.smithi055.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-10T03:26:17.843 INFO:teuthology.orchestra.run.smithi055.stdout:(63/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-05-10T03:26:17.868 INFO:teuthology.orchestra.run.smithi055.stdout:(64/138): openblas-0.3.21-2.el9.x86_64.rpm 265 kB/s | 35 kB 00:00 2024-05-10T03:26:18.057 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-05-10T03:26:18.135 INFO:teuthology.orchestra.run.smithi055.stdout:(65/138): postgresql-13.14-1.el9.x86_64.rpm 6.0 MB/s | 1.6 MB 00:00 2024-05-10T03:26:18.285 INFO:teuthology.orchestra.run.smithi055.stdout:(66/138): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-10T03:26:18.335 INFO:teuthology.orchestra.run.smithi055.stdout:(67/138): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-05-10T03:26:18.427 INFO:teuthology.orchestra.run.smithi055.stdout:(68/138): openblas-openmp-0.3.21-2.el9.x86_64.r 7.2 MB/s | 4.7 MB 00:00 2024-05-10T03:26:18.453 INFO:teuthology.orchestra.run.smithi055.stdout:(69/138): postgresql-private-libs-13.14-1.el9.x 802 kB/s | 133 kB 00:00 2024-05-10T03:26:18.520 INFO:teuthology.orchestra.run.smithi055.stdout:(70/138): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-10T03:26:18.595 INFO:teuthology.orchestra.run.smithi055.stdout:(71/138): python3-jinja2-2.11.3-5.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-05-10T03:26:18.654 INFO:teuthology.orchestra.run.smithi055.stdout:(72/138): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-05-10T03:26:18.729 INFO:teuthology.orchestra.run.smithi055.stdout:(73/138): python3-jsonpointer-2.0-4.el9.noarch. 258 kB/s | 19 kB 00:00 2024-05-10T03:26:18.796 INFO:teuthology.orchestra.run.smithi055.stdout:(74/138): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-05-10T03:26:18.921 INFO:teuthology.orchestra.run.smithi055.stdout:(75/138): postgresql-server-13.14-1.el9.x86_64. 9.9 MB/s | 5.8 MB 00:00 2024-05-10T03:26:18.946 INFO:teuthology.orchestra.run.smithi055.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 232 kB/s | 35 kB 00:00 2024-05-10T03:26:19.064 INFO:teuthology.orchestra.run.smithi055.stdout:(77/138): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-05-10T03:26:19.097 INFO:teuthology.orchestra.run.smithi055.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-05-10T03:26:19.139 INFO:teuthology.orchestra.run.smithi055.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-10T03:26:19.172 INFO:teuthology.orchestra.run.smithi055.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-10T03:26:19.231 INFO:teuthology.orchestra.run.smithi055.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 463 kB/s | 42 kB 00:00 2024-05-10T03:26:19.265 INFO:teuthology.orchestra.run.smithi055.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-10T03:26:19.315 INFO:teuthology.orchestra.run.smithi055.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-10T03:26:19.340 INFO:teuthology.orchestra.run.smithi055.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 682 kB/s | 51 kB 00:00 2024-05-10T03:26:19.466 INFO:teuthology.orchestra.run.smithi055.stdout:(85/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 5.1 MB 00:00 2024-05-10T03:26:19.491 INFO:teuthology.orchestra.run.smithi055.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 307 kB/s | 54 kB 00:00 2024-05-10T03:26:19.533 INFO:teuthology.orchestra.run.smithi055.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 632 kB/s | 42 kB 00:00 2024-05-10T03:26:19.575 INFO:teuthology.orchestra.run.smithi055.stdout:(88/138): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-10T03:26:19.600 INFO:teuthology.orchestra.run.smithi055.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 873 kB/s | 58 kB 00:00 2024-05-10T03:26:19.650 INFO:teuthology.orchestra.run.smithi055.stdout:(90/138): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-10T03:26:19.707 INFO:teuthology.orchestra.run.smithi055.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 604 kB/s | 64 kB 00:00 2024-05-10T03:26:19.743 INFO:teuthology.orchestra.run.smithi055.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-05-10T03:26:19.993 INFO:teuthology.orchestra.run.smithi055.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 389 kB/s | 111 kB 00:00 2024-05-10T03:26:20.319 INFO:teuthology.orchestra.run.smithi055.stdout:(94/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-05-10T03:26:20.452 INFO:teuthology.orchestra.run.smithi055.stdout:(95/138): gperftools-libs-2.9.1-2.el9.x86_64.rp 436 kB/s | 309 kB 00:00 2024-05-10T03:26:20.477 INFO:teuthology.orchestra.run.smithi055.stdout:(96/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-05-10T03:26:20.619 INFO:teuthology.orchestra.run.smithi055.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 478 kB/s | 67 kB 00:00 2024-05-10T03:26:20.645 INFO:teuthology.orchestra.run.smithi055.stdout:(98/138): liboath-2.6.7-2.el9.x86_64.rpm 254 kB/s | 49 kB 00:00 2024-05-10T03:26:21.220 INFO:teuthology.orchestra.run.smithi055.stdout:(99/138): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-05-10T03:26:21.262 INFO:teuthology.orchestra.run.smithi055.stdout:(100/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.3 MB/s | 839 kB 00:00 2024-05-10T03:26:21.296 INFO:teuthology.orchestra.run.smithi055.stdout:(101/138): python3-asyncssh-2.13.2-5.el9.noarch 843 kB/s | 548 kB 00:00 2024-05-10T03:26:21.371 INFO:teuthology.orchestra.run.smithi055.stdout:(102/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 291 kB/s | 43 kB 00:00 2024-05-10T03:26:21.430 INFO:teuthology.orchestra.run.smithi055.stdout:(103/138): python3-cachetools-4.2.4-1.el9.noarc 194 kB/s | 32 kB 00:00 2024-05-10T03:26:21.455 INFO:teuthology.orchestra.run.smithi055.stdout:(104/138): python3-certifi-2021.10.8-2.el9.noar 95 kB/s | 15 kB 00:00 2024-05-10T03:26:21.605 INFO:teuthology.orchestra.run.smithi055.stdout:(105/138): python3-cheroot-8.6.0-4.el9.noarch.r 737 kB/s | 172 kB 00:00 2024-05-10T03:26:21.722 INFO:teuthology.orchestra.run.smithi055.stdout:(106/138): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-05-10T03:26:21.806 INFO:teuthology.orchestra.run.smithi055.stdout:(107/138): python3-google-auth-2.29.0-1.el9.noa 588 kB/s | 206 kB 00:00 2024-05-10T03:26:21.889 INFO:teuthology.orchestra.run.smithi055.stdout:(108/138): python3-jaraco-8.2.1-3.el9.noarch.rp 38 kB/s | 11 kB 00:00 2024-05-10T03:26:21.931 INFO:teuthology.orchestra.run.smithi055.stdout:(109/138): python3-jaraco-classes-3.2.1-5.el9.n 85 kB/s | 18 kB 00:00 2024-05-10T03:26:21.957 INFO:teuthology.orchestra.run.smithi055.stdout:(110/138): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-05-10T03:26:22.040 INFO:teuthology.orchestra.run.smithi055.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 130 kB/s | 19 kB 00:00 2024-05-10T03:26:22.065 INFO:teuthology.orchestra.run.smithi055.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-05-10T03:26:22.224 INFO:teuthology.orchestra.run.smithi055.stdout:(113/138): python3-more-itertools-8.12.0-2.el9. 501 kB/s | 79 kB 00:00 2024-05-10T03:26:22.249 INFO:teuthology.orchestra.run.smithi055.stdout:(114/138): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-05-10T03:26:22.341 INFO:teuthology.orchestra.run.smithi055.stdout:(115/138): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-05-10T03:26:22.367 INFO:teuthology.orchestra.run.smithi055.stdout:(116/138): python3-natsort-7.1.1-5.el9.noarch.r 409 kB/s | 58 kB 00:00 2024-05-10T03:26:22.484 INFO:teuthology.orchestra.run.smithi055.stdout:(117/138): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-05-10T03:26:22.542 INFO:teuthology.orchestra.run.smithi055.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 516 kB/s | 90 kB 00:00 2024-05-10T03:26:22.609 INFO:teuthology.orchestra.run.smithi055.stdout:(119/138): python3-pecan-1.4.2-3.el9.noarch.rpm 760 kB/s | 272 kB 00:00 2024-05-10T03:26:22.643 INFO:teuthology.orchestra.run.smithi055.stdout:(120/138): python3-repoze-lru-0.7-16.el9.noarch 195 kB/s | 31 kB 00:00 2024-05-10T03:26:22.793 INFO:teuthology.orchestra.run.smithi055.stdout:(121/138): python3-tempora-5.0.0-2.el9.noarch.r 240 kB/s | 36 kB 00:00 2024-05-10T03:26:22.818 INFO:teuthology.orchestra.run.smithi055.stdout:(122/138): python3-routes-2.5.1-5.el9.noarch.rp 685 kB/s | 188 kB 00:00 2024-05-10T03:26:22.869 INFO:teuthology.orchestra.run.smithi055.stdout:(123/138): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-05-10T03:26:23.052 INFO:teuthology.orchestra.run.smithi055.stdout:(124/138): python3-typing-extensions-4.4.0-2.el 198 kB/s | 51 kB 00:00 2024-05-10T03:26:23.111 INFO:teuthology.orchestra.run.smithi055.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 372 kB/s | 90 kB 00:00 2024-05-10T03:26:23.203 INFO:teuthology.orchestra.run.smithi055.stdout:(126/138): python3-webob-1.8.7-6.el9.noarch.rpm 599 kB/s | 230 kB 00:00 2024-05-10T03:26:23.245 INFO:teuthology.orchestra.run.smithi055.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-05-10T03:26:23.354 INFO:teuthology.orchestra.run.smithi055.stdout:(128/138): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-05-10T03:26:23.454 INFO:teuthology.orchestra.run.smithi055.stdout:(129/138): re2-20211101-3.el9.x86_64.rpm 767 kB/s | 192 kB 00:00 2024-05-10T03:26:23.695 INFO:teuthology.orchestra.run.smithi055.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-05-10T03:26:23.813 INFO:teuthology.orchestra.run.smithi055.stdout:(131/138): librados2-18.2.2-1330.g50174015.el9. 7.1 MB/s | 3.2 MB 00:00 2024-05-10T03:26:23.905 INFO:teuthology.orchestra.run.smithi055.stdout:(132/138): librbd1-18.2.2-1330.g50174015.el9.x8 6.7 MB/s | 3.0 MB 00:00 2024-05-10T03:26:24.201 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-10T03:26:24.280 INFO:teuthology.orchestra.run.smithi055.stdout:(133/138): NetworkManager-team-1.47.5-1.el9.x86 111 kB/s | 42 kB 00:00 2024-05-10T03:26:24.633 INFO:teuthology.orchestra.run.smithi055.stdout:(134/138): NetworkManager-tui-1.47.5-1.el9.x86_ 711 kB/s | 248 kB 00:00 2024-05-10T03:26:24.781 INFO:teuthology.orchestra.run.smithi055.stdout:(135/138): NetworkManager-1.47.5-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-05-10T03:26:24.897 INFO:teuthology.orchestra.run.smithi055.stdout:(136/138): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-10T03:26:25.064 INFO:teuthology.orchestra.run.smithi055.stdout:(137/138): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 2.4 MB 00:00 2024-05-10T03:26:25.215 INFO:teuthology.orchestra.run.smithi055.stdout:(138/138): openssl-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 1.3 MB 00:00 2024-05-10T03:26:25.221 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:26:25.222 INFO:teuthology.orchestra.run.smithi055.stdout:Total 13 MB/s | 199 MB 00:15 2024-05-10T03:26:26.888 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T03:26:26.889 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-10T03:26:26.909 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-10T03:26:26.911 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-10T03:26:26.912 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-10T03:26:26.914 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-10T03:26:26.916 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-10T03:26:26.920 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-10T03:26:26.921 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-10T03:26:26.992 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el9 ceph 6.5 k 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el9 ceph 5.1 M 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el9 ceph 824 k 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el9 ceph 144 k 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el9 ceph 1.4 M 2024-05-10T03:26:27.009 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 139 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 3.5 M 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 7.4 M 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 49 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el9 ceph 7.5 M 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el9 ceph 38 M 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 264 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 224 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el9 ceph 220 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 18 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-10T03:26:27.010 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el9 ceph 31 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 719 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el9 ceph 127 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el9 ceph 162 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el9 ceph 320 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el9 ceph 298 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el9 ceph 99 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el9 ceph 87 k 2024-05-10T03:26:27.011 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.0 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el9 ceph 170 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.2 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el9 ceph 3.0 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-10T03:26:27.012 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el9 ceph 17 M 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 24 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el9 ceph 2.0 M 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 246 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el9 ceph 4.9 M 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el9 ceph 18 M 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el9 ceph-noarch 16 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el9 ceph 25 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-10T03:26:27.013 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el9 ceph 162 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el9 ceph 468 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el9 ceph 4.2 M 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-10T03:26:27.014 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el9 ceph 45 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el9 ceph 129 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-10T03:26:27.015 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-10T03:26:27.016 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-10T03:26:27.017 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-10T03:26:27.018 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout:Install 130 Packages 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 8 Packages 2024-05-10T03:26:27.019 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:26:27.020 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 199 M 2024-05-10T03:26:27.020 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-05-10T03:26:27.955 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-05-10T03:26:27.969 INFO:teuthology.orchestra.run.smithi162.stdout:(1/138): ceph-18.2.2-1330.g50174015.el9.x86_64. 25 kB/s | 6.5 kB 00:00 2024-05-10T03:26:28.195 INFO:teuthology.orchestra.run.smithi162.stdout:(2/138): ceph-fuse-18.2.2-1330.g50174015.el9.x8 3.6 MB/s | 824 kB 00:00 2024-05-10T03:26:28.200 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-05-10T03:26:28.200 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-05-10T03:26:28.278 INFO:teuthology.orchestra.run.smithi162.stdout:(3/138): ceph-immutable-object-cache-18.2.2-133 1.7 MB/s | 144 kB 00:00 2024-05-10T03:26:28.562 INFO:teuthology.orchestra.run.smithi162.stdout:(4/138): ceph-base-18.2.2-1330.g50174015.el9.x8 6.0 MB/s | 5.1 MB 00:00 2024-05-10T03:26:28.629 INFO:teuthology.orchestra.run.smithi162.stdout:(5/138): ceph-mds-18.2.2-1330.g50174015.el9.x86 5.7 MB/s | 2.0 MB 00:00 2024-05-10T03:26:29.154 INFO:teuthology.orchestra.run.smithi162.stdout:(6/138): ceph-mgr-18.2.2-1330.g50174015.el9.x86 2.4 MB/s | 1.4 MB 00:00 2024-05-10T03:26:29.262 INFO:teuthology.orchestra.run.smithi162.stdout:(7/138): ceph-mon-18.2.2-1330.g50174015.el9.x86 7.7 MB/s | 4.9 MB 00:00 2024-05-10T03:26:29.513 INFO:teuthology.orchestra.run.smithi162.stdout:(8/138): ceph-common-18.2.2-1330.g50174015.el9. 9.3 MB/s | 17 MB 00:01 2024-05-10T03:26:29.571 INFO:teuthology.orchestra.run.smithi162.stdout:(9/138): ceph-selinux-18.2.2-1330.g50174015.el9 439 kB/s | 25 kB 00:00 2024-05-10T03:26:30.013 INFO:teuthology.orchestra.run.smithi162.stdout:(10/138): ceph-radosgw-18.2.2-1330.g50174015.el 10 MB/s | 7.5 MB 00:00 2024-05-10T03:26:30.072 INFO:teuthology.orchestra.run.smithi162.stdout:(11/138): cephfs-mirror-18.2.2-1330.g50174015.e 3.7 MB/s | 220 kB 00:00 2024-05-10T03:26:30.139 INFO:teuthology.orchestra.run.smithi162.stdout:(12/138): libcephfs-devel-18.2.2-1330.g50174015 472 kB/s | 31 kB 00:00 2024-05-10T03:26:30.289 INFO:teuthology.orchestra.run.smithi162.stdout:(13/138): libcephfs2-18.2.2-1330.g50174015.el9. 4.7 MB/s | 719 kB 00:00 2024-05-10T03:26:30.364 INFO:teuthology.orchestra.run.smithi162.stdout:(14/138): libcephsqlite-18.2.2-1330.g50174015.e 2.1 MB/s | 162 kB 00:00 2024-05-10T03:26:30.448 INFO:teuthology.orchestra.run.smithi162.stdout:(15/138): librados-devel-18.2.2-1330.g50174015. 1.5 MB/s | 127 kB 00:00 2024-05-10T03:26:30.540 INFO:teuthology.orchestra.run.smithi162.stdout:(16/138): libradosstriper1-18.2.2-1330.g5017401 5.0 MB/s | 468 kB 00:00 2024-05-10T03:26:30.845 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-05-10T03:26:30.846 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-05-10T03:26:31.073 INFO:teuthology.orchestra.run.smithi162.stdout:(17/138): librgw2-18.2.2-1330.g50174015.el9.x86 8.0 MB/s | 4.2 MB 00:00 2024-05-10T03:26:31.157 INFO:teuthology.orchestra.run.smithi162.stdout:(18/138): python3-ceph-argparse-18.2.2-1330.g50 546 kB/s | 45 kB 00:00 2024-05-10T03:26:31.232 INFO:teuthology.orchestra.run.smithi162.stdout:(19/138): python3-ceph-common-18.2.2-1330.g5017 1.7 MB/s | 129 kB 00:00 2024-05-10T03:26:31.316 INFO:teuthology.orchestra.run.smithi162.stdout:(20/138): python3-cephfs-18.2.2-1330.g50174015. 1.9 MB/s | 162 kB 00:00 2024-05-10T03:26:31.408 INFO:teuthology.orchestra.run.smithi162.stdout:(21/138): python3-rados-18.2.2-1330.g50174015.e 3.4 MB/s | 320 kB 00:00 2024-05-10T03:26:31.491 INFO:teuthology.orchestra.run.smithi162.stdout:(22/138): python3-rbd-18.2.2-1330.g50174015.el9 3.5 MB/s | 298 kB 00:00 2024-05-10T03:26:31.875 INFO:teuthology.orchestra.run.smithi162.stdout:(23/138): ceph-test-18.2.2-1330.g50174015.el9.x 16 MB/s | 38 MB 00:02 2024-05-10T03:26:31.909 INFO:teuthology.orchestra.run.smithi162.stdout:(24/138): python3-rgw-18.2.2-1330.g50174015.el9 239 kB/s | 99 kB 00:00 2024-05-10T03:26:31.934 INFO:teuthology.orchestra.run.smithi162.stdout:(25/138): rbd-fuse-18.2.2-1330.g50174015.el9.x8 1.5 MB/s | 87 kB 00:00 2024-05-10T03:26:32.001 INFO:teuthology.orchestra.run.smithi162.stdout:(26/138): rbd-nbd-18.2.2-1330.g50174015.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-10T03:26:32.068 INFO:teuthology.orchestra.run.smithi162.stdout:(27/138): ceph-grafana-dashboards-18.2.2-1330.g 370 kB/s | 24 kB 00:00 2024-05-10T03:26:32.160 INFO:teuthology.orchestra.run.smithi162.stdout:(28/138): rbd-mirror-18.2.2-1330.g50174015.el9. 12 MB/s | 3.0 MB 00:00 2024-05-10T03:26:32.193 INFO:teuthology.orchestra.run.smithi162.stdout:(29/138): ceph-mgr-cephadm-18.2.2-1330.g5017401 1.1 MB/s | 139 kB 00:00 2024-05-10T03:26:32.547 INFO:teuthology.orchestra.run.smithi162.stdout:(30/138): ceph-mgr-dashboard-18.2.2-1330.g50174 9.1 MB/s | 3.5 MB 00:00 2024-05-10T03:26:32.636 INFO:teuthology.orchestra.run.smithi162.stdout:(31/138): ceph-mgr-modules-core-18.2.2-1330.g50 2.8 MB/s | 246 kB 00:00 2024-05-10T03:26:32.761 INFO:teuthology.orchestra.run.smithi162.stdout:(32/138): ceph-mgr-diskprediction-local-18.2.2- 13 MB/s | 7.4 MB 00:00 2024-05-10T03:26:32.786 INFO:teuthology.orchestra.run.smithi162.stdout:(33/138): ceph-mgr-rook-18.2.2-1330.g50174015.e 330 kB/s | 49 kB 00:00 2024-05-10T03:26:32.811 INFO:teuthology.orchestra.run.smithi162.stdout:(34/138): ceph-prometheus-alerts-18.2.2-1330.g5 334 kB/s | 16 kB 00:00 2024-05-10T03:26:32.845 INFO:teuthology.orchestra.run.smithi162.stdout:(35/138): ceph-volume-18.2.2-1330.g50174015.el9 4.5 MB/s | 264 kB 00:00 2024-05-10T03:26:32.879 INFO:teuthology.orchestra.run.smithi162.stdout:(36/138): cephadm-18.2.2-1330.g50174015.el9.noa 3.3 MB/s | 224 kB 00:00 2024-05-10T03:26:32.904 INFO:teuthology.orchestra.run.smithi162.stdout:(37/138): cephfs-top-18.2.2-1330.g50174015.el9. 316 kB/s | 18 kB 00:00 2024-05-10T03:26:33.046 INFO:teuthology.orchestra.run.smithi162.stdout:(38/138): fuse-2.9.9-15.el9.x86_64.rpm 484 kB/s | 80 kB 00:00 2024-05-10T03:26:33.138 INFO:teuthology.orchestra.run.smithi162.stdout:(39/138): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-10T03:26:33.197 INFO:teuthology.orchestra.run.smithi162.stdout:(40/138): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-10T03:26:33.455 INFO:teuthology.orchestra.run.smithi162.stdout:(41/138): ceph-osd-18.2.2-1330.g50174015.el9.x8 4.1 MB/s | 18 MB 00:04 2024-05-10T03:26:33.489 INFO:teuthology.orchestra.run.smithi162.stdout:(42/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 724 kB/s | 253 kB 00:00 2024-05-10T03:26:33.567 INFO:teuthology.orchestra.run.smithi162.stdout:(43/138): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-05-10T03:26:33.597 INFO:teuthology.orchestra.run.smithi162.stdout:(44/138): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-10T03:26:33.623 INFO:teuthology.orchestra.run.smithi162.stdout:(45/138): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 125 kB 00:00 2024-05-10T03:26:33.648 INFO:teuthology.orchestra.run.smithi162.stdout:(46/138): python3-ply-3.11-14.el9.noarch.rpm 556 kB/s | 106 kB 00:00 2024-05-10T03:26:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:(47/138): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-10T03:26:34.107 INFO:teuthology.orchestra.run.smithi162.stdout:(48/138): boost-program-options-1.75.0-8.el9.x8 252 kB/s | 107 kB 00:00 2024-05-10T03:26:34.216 INFO:teuthology.orchestra.run.smithi162.stdout:(49/138): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-05-10T03:26:34.416 INFO:teuthology.orchestra.run.smithi162.stdout:(50/138): automake-1.16.2-8.el9.noarch.rpm 870 kB/s | 689 kB 00:00 2024-05-10T03:26:34.549 INFO:teuthology.orchestra.run.smithi162.stdout:(51/138): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-10T03:26:34.574 INFO:teuthology.orchestra.run.smithi162.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 207 kB/s | 33 kB 00:00 2024-05-10T03:26:34.608 INFO:teuthology.orchestra.run.smithi162.stdout:(53/138): flex-2.6.4-9.el9.x86_64.rpm 801 kB/s | 313 kB 00:00 2024-05-10T03:26:34.683 INFO:teuthology.orchestra.run.smithi162.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 170 kB/s | 18 kB 00:00 2024-05-10T03:26:34.700 INFO:teuthology.orchestra.run.smithi162.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-05-10T03:26:34.792 INFO:teuthology.orchestra.run.smithi162.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-05-10T03:26:34.818 INFO:teuthology.orchestra.run.smithi162.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-05-10T03:26:35.001 INFO:teuthology.orchestra.run.smithi162.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-10T03:26:35.076 INFO:teuthology.orchestra.run.smithi162.stdout:(59/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-05-10T03:26:35.110 INFO:teuthology.orchestra.run.smithi162.stdout:(60/138): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-05-10T03:26:35.144 INFO:teuthology.orchestra.run.smithi162.stdout:(61/138): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-05-10T03:26:35.185 INFO:teuthology.orchestra.run.smithi162.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-10T03:26:35.219 INFO:teuthology.orchestra.run.smithi162.stdout:(63/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-10T03:26:35.253 INFO:teuthology.orchestra.run.smithi162.stdout:(64/138): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-10T03:26:35.578 INFO:teuthology.orchestra.run.smithi162.stdout:(65/138): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-10T03:26:35.728 INFO:teuthology.orchestra.run.smithi162.stdout:(66/138): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-05-10T03:26:35.837 INFO:teuthology.orchestra.run.smithi162.stdout:(67/138): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-10T03:26:37.012 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-05-10T03:26:37.105 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/146 2024-05-10T03:26:37.181 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-05-10T03:26:37.293 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-05-10T03:26:37.352 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-05-10T03:26:37.391 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-05-10T03:26:37.497 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-05-10T03:26:37.925 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-05-10T03:26:37.937 INFO:teuthology.orchestra.run.smithi162.stdout:(68/138): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-05-10T03:26:38.038 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:26:38.225 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:26:38.256 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:26:38.399 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:26:38.439 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 10/146 2024-05-10T03:26:38.763 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/146 2024-05-10T03:26:38.828 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/146 2024-05-10T03:26:38.852 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:26:39.067 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:26:39.094 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 14/146 2024-05-10T03:26:39.203 INFO:teuthology.orchestra.run.smithi162.stdout:(69/138): openblas-openmp-0.3.21-2.el9.x86_64.r 1.2 MB/s | 4.7 MB 00:04 2024-05-10T03:26:39.612 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 14/146 2024-05-10T03:26:39.612 INFO:teuthology.orchestra.run.smithi162.stdout:(70/138): python3-devel-3.9.18-3.el9.x86_64.rpm 599 kB/s | 244 kB 00:00 2024-05-10T03:26:39.774 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 15/146 2024-05-10T03:26:39.845 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 15/146 2024-05-10T03:26:39.954 INFO:teuthology.orchestra.run.smithi162.stdout:(71/138): python3-jinja2-2.11.3-5.el9.noarch.rp 727 kB/s | 248 kB 00:00 2024-05-10T03:26:40.026 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:26:40.079 INFO:teuthology.orchestra.run.smithi162.stdout:(72/138): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-05-10T03:26:40.152 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:26:40.204 INFO:teuthology.orchestra.run.smithi162.stdout:(73/138): python3-jsonpointer-2.0-4.el9.noarch. 154 kB/s | 19 kB 00:00 2024-05-10T03:26:40.350 INFO:teuthology.orchestra.run.smithi162.stdout:(74/138): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-10T03:26:40.480 INFO:teuthology.orchestra.run.smithi162.stdout:(75/138): python3-markupsafe-1.1.1-12.el9.x86_6 269 kB/s | 35 kB 00:00 2024-05-10T03:26:40.609 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:26:40.676 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-05-10T03:26:40.733 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-05-10T03:26:40.869 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-05-10T03:26:40.924 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-05-10T03:26:40.964 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-05-10T03:26:41.117 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-05-10T03:26:41.201 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/146 2024-05-10T03:26:41.242 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-05-10T03:26:41.303 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/146 2024-05-10T03:26:41.420 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/146 2024-05-10T03:26:41.461 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el9. 27/146 2024-05-10T03:26:41.507 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 28/146 2024-05-10T03:26:41.607 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_6 29/146 2024-05-10T03:26:41.713 INFO:teuthology.orchestra.run.smithi162.stdout:(76/138): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-05-10T03:26:41.815 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-05-10T03:26:41.890 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-05-10T03:26:42.064 INFO:teuthology.orchestra.run.smithi162.stdout:(77/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 717 kB/s | 250 kB 00:00 2024-05-10T03:26:42.414 INFO:teuthology.orchestra.run.smithi162.stdout:(78/138): python3-oauthlib-3.1.1-5.el9.noarch.r 636 kB/s | 222 kB 00:00 2024-05-10T03:26:42.599 INFO:teuthology.orchestra.run.smithi162.stdout:(79/138): python3-packaging-20.9-5.el9.noarch.r 423 kB/s | 77 kB 00:00 2024-05-10T03:26:42.602 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-05-10T03:26:42.706 INFO:teuthology.orchestra.run.smithi162.stdout:(80/138): python3-prettytable-0.7.2-27.el9.noar 392 kB/s | 42 kB 00:00 2024-05-10T03:26:42.755 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-05-10T03:26:42.800 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 34/146 2024-05-10T03:26:42.838 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el9.x86_6 35/146 2024-05-10T03:26:42.896 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-05-10T03:26:42.964 INFO:teuthology.orchestra.run.smithi162.stdout:(81/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 615 kB/s | 159 kB 00:00 2024-05-10T03:26:43.099 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 37/146 2024-05-10T03:26:43.196 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 38/146 2024-05-10T03:26:43.351 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : re2-1:20211101-3.el9.x86_64 39/146 2024-05-10T03:26:43.391 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-05-10T03:26:43.407 INFO:teuthology.orchestra.run.smithi162.stdout:(82/138): python3-pyasn1-modules-0.4.8-6.el9.no 631 kB/s | 279 kB 00:00 2024-05-10T03:26:43.466 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-05-10T03:26:43.524 INFO:teuthology.orchestra.run.smithi162.stdout:(83/138): python3-pytz-2021.1-5.el9.noarch.rpm 441 kB/s | 51 kB 00:00 2024-05-10T03:26:43.565 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-05-10T03:26:43.617 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/146 2024-05-10T03:26:43.632 INFO:teuthology.orchestra.run.smithi162.stdout:(84/138): python3-requests-oauthlib-1.3.0-12.el 498 kB/s | 54 kB 00:00 2024-05-10T03:26:43.691 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 44/146 2024-05-10T03:26:43.749 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-05-10T03:26:43.806 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-05-10T03:26:43.866 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-05-10T03:26:44.078 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-05-10T03:26:44.454 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-05-10T03:26:44.524 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 50/146 2024-05-10T03:26:44.683 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-05-10T03:26:44.754 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 52/146 2024-05-10T03:26:44.849 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-05-10T03:26:44.892 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/146 2024-05-10T03:26:44.934 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-05-10T03:26:44.983 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-05-10T03:26:45.034 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-05-10T03:26:45.106 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-05-10T03:26:45.174 INFO:teuthology.orchestra.run.smithi162.stdout:(85/138): python3-babel-2.9.1-2.el9.noarch.rpm 844 kB/s | 6.0 MB 00:07 2024-05-10T03:26:45.174 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-05-10T03:26:45.230 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-05-10T03:26:45.302 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-05-10T03:26:45.341 INFO:teuthology.orchestra.run.smithi162.stdout:(86/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.0 MB/s | 5.1 MB 00:04 2024-05-10T03:26:45.360 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-05-10T03:26:45.366 INFO:teuthology.orchestra.run.smithi162.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 218 kB/s | 42 kB 00:00 2024-05-10T03:26:45.436 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-05-10T03:26:45.488 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-05-10T03:26:45.508 INFO:teuthology.orchestra.run.smithi162.stdout:(88/138): uuid-1.6.2-55.el9.x86_64.rpm 410 kB/s | 58 kB 00:00 2024-05-10T03:26:45.536 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-05-10T03:26:45.628 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-05-10T03:26:45.641 INFO:teuthology.orchestra.run.smithi162.stdout:(89/138): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 706 kB/s | 94 kB 00:00 2024-05-10T03:26:45.758 INFO:teuthology.orchestra.run.smithi162.stdout:(90/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 549 kB/s | 64 kB 00:00 2024-05-10T03:26:45.794 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-05-10T03:26:45.860 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-05-10T03:26:45.892 INFO:teuthology.orchestra.run.smithi162.stdout:(91/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 338 kB/s | 45 kB 00:00 2024-05-10T03:26:45.926 INFO:teuthology.orchestra.run.smithi162.stdout:(92/138): socat-1.7.4.1-5.el9.x86_64.rpm 522 kB/s | 305 kB 00:00 2024-05-10T03:26:46.534 INFO:teuthology.orchestra.run.smithi162.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 173 kB/s | 111 kB 00:00 2024-05-10T03:26:46.568 INFO:teuthology.orchestra.run.smithi162.stdout:(94/138): gperftools-libs-2.9.1-2.el9.x86_64.rp 482 kB/s | 309 kB 00:00 2024-05-10T03:26:46.726 INFO:teuthology.orchestra.run.smithi162.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-05-10T03:26:46.872 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-05-10T03:26:46.893 INFO:teuthology.orchestra.run.smithi162.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 293 kB/s | 49 kB 00:00 2024-05-10T03:26:46.924 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-05-10T03:26:46.957 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-05-10T03:26:47.002 INFO:teuthology.orchestra.run.smithi162.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-05-10T03:26:47.076 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-05-10T03:26:47.094 INFO:teuthology.orchestra.run.smithi162.stdout:(98/138): libarrow-9.0.0-11.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-05-10T03:26:47.115 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 73/146 2024-05-10T03:26:47.294 INFO:teuthology.orchestra.run.smithi162.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-05-10T03:26:47.336 INFO:teuthology.orchestra.run.smithi162.stdout:(100/138): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-05-10T03:26:47.361 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.21-2.el9.x86_64 74/146 2024-05-10T03:26:47.386 INFO:teuthology.orchestra.run.smithi162.stdout:(101/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-05-10T03:26:47.387 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 75/146 2024-05-10T03:26:47.445 INFO:teuthology.orchestra.run.smithi162.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 298 kB/s | 32 kB 00:00 2024-05-10T03:26:47.504 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-05-10T03:26:47.520 INFO:teuthology.orchestra.run.smithi162.stdout:(103/138): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-05-10T03:26:47.571 INFO:teuthology.orchestra.run.smithi162.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-10T03:26:47.621 INFO:teuthology.orchestra.run.smithi162.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-10T03:26:47.696 INFO:teuthology.orchestra.run.smithi162.stdout:(106/138): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-10T03:26:47.721 INFO:teuthology.orchestra.run.smithi162.stdout:(107/138): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-10T03:26:47.772 INFO:teuthology.orchestra.run.smithi162.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-05-10T03:26:47.797 INFO:teuthology.orchestra.run.smithi162.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-05-10T03:26:47.881 INFO:teuthology.orchestra.run.smithi162.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-10T03:26:48.048 INFO:teuthology.orchestra.run.smithi162.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 79 kB/s | 20 kB 00:00 2024-05-10T03:26:48.131 INFO:teuthology.orchestra.run.smithi162.stdout:(112/138): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-05-10T03:26:48.165 INFO:teuthology.orchestra.run.smithi162.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-05-10T03:26:48.232 INFO:teuthology.orchestra.run.smithi162.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 793 kB/s | 79 kB 00:00 2024-05-10T03:26:48.257 INFO:teuthology.orchestra.run.smithi162.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 631 kB/s | 58 kB 00:00 2024-05-10T03:26:48.359 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-05-10T03:26:48.374 INFO:teuthology.orchestra.run.smithi162.stdout:(116/138): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-05-10T03:26:48.399 INFO:teuthology.orchestra.run.smithi162.stdout:(117/138): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-05-10T03:26:48.466 INFO:teuthology.orchestra.run.smithi162.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 988 kB/s | 90 kB 00:00 2024-05-10T03:26:48.500 INFO:teuthology.orchestra.run.smithi162.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-05-10T03:26:48.504 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-05-10T03:26:48.600 INFO:teuthology.orchestra.run.smithi162.stdout:(120/138): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-10T03:26:48.625 INFO:teuthology.orchestra.run.smithi162.stdout:(121/138): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-05-10T03:26:48.992 INFO:teuthology.orchestra.run.smithi162.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 91 kB/s | 36 kB 00:00 2024-05-10T03:26:49.055 INFO:teuthology.orchestra.run.smithi162.stdout:(123/138): python3-typing-extensions-4.4.0-2.el 119 kB/s | 51 kB 00:00 2024-05-10T03:26:49.159 INFO:teuthology.orchestra.run.smithi162.stdout:(124/138): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-05-10T03:26:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 619 kB/s | 90 kB 00:00 2024-05-10T03:26:49.310 INFO:teuthology.orchestra.run.smithi162.stdout:(126/138): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-05-10T03:26:49.335 INFO:teuthology.orchestra.run.smithi162.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2024-05-10T03:26:49.386 INFO:teuthology.orchestra.run.smithi162.stdout:(128/138): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-10T03:26:49.553 INFO:teuthology.orchestra.run.smithi162.stdout:(129/138): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-10T03:26:49.720 INFO:teuthology.orchestra.run.smithi162.stdout:(130/138): librados2-18.2.2-1330.g50174015.el9. 9.7 MB/s | 3.2 MB 00:00 2024-05-10T03:26:50.045 INFO:teuthology.orchestra.run.smithi162.stdout:(131/138): librbd1-18.2.2-1330.g50174015.el9.x8 6.1 MB/s | 3.0 MB 00:00 2024-05-10T03:26:50.120 INFO:teuthology.orchestra.run.smithi162.stdout:(132/138): NetworkManager-1.47.5-1.el9.x86_64.r 5.8 MB/s | 2.3 MB 00:00 2024-05-10T03:26:50.170 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-05-10T03:26:50.178 INFO:teuthology.orchestra.run.smithi162.stdout:(133/138): NetworkManager-team-1.47.5-1.el9.x86 728 kB/s | 42 kB 00:00 2024-05-10T03:26:50.237 INFO:teuthology.orchestra.run.smithi162.stdout:(134/138): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-10T03:26:50.269 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-05-10T03:26:50.304 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-05-10T03:26:50.305 INFO:teuthology.orchestra.run.smithi162.stdout:(135/138): NetworkManager-libnm-1.47.5-1.el9.x8 7.1 MB/s | 1.8 MB 00:00 2024-05-10T03:26:50.340 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 82/146 2024-05-10T03:26:50.371 INFO:teuthology.orchestra.run.smithi162.stdout:(136/138): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-10T03:26:50.530 INFO:teuthology.orchestra.run.smithi162.stdout:(137/138): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-10T03:26:50.736 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-05-10T03:26:50.781 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el9.x86_64 84/146 2024-05-10T03:26:50.906 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el9.x86_64 84/146 2024-05-10T03:26:50.938 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 85/146 2024-05-10T03:26:50.996 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-05-10T03:26:51.506 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-05-10T03:26:51.542 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el9.x86_64 88/146 2024-05-10T03:26:51.644 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el9.x86_64 88/146 2024-05-10T03:26:51.708 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 89/146 2024-05-10T03:26:54.454 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:26:54.499 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:26:55.198 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:26:55.244 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 91/146 2024-05-10T03:26:55.458 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 91/146 2024-05-10T03:26:55.458 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T03:26:55.458 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T03:26:55.458 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:26:55.512 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 92/146 2024-05-10T03:27:06.093 INFO:teuthology.orchestra.run.smithi162.stdout:(138/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 741 kB/s | 16 MB 00:22 2024-05-10T03:27:06.105 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-05-10T03:27:06.105 INFO:teuthology.orchestra.run.smithi162.stdout:Total 5.1 MB/s | 199 MB 00:39 2024-05-10T03:27:08.672 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 92/146 2024-05-10T03:27:08.672 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2024-05-10T03:27:08.672 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2024-05-10T03:27:08.672 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2024-05-10T03:27:08.672 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2024-05-10T03:27:08.673 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2024-05-10T03:27:08.673 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /root 2024-05-10T03:27:08.673 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2024-05-10T03:27:08.673 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:08.770 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-05-10T03:27:08.905 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 93/146 2024-05-10T03:27:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 93/146 2024-05-10T03:27:08.958 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:08.958 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T03:27:08.958 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:09.001 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-05-10T03:27:09.001 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-05-10T03:27:10.791 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 94/146 2024-05-10T03:27:10.872 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 94/146 2024-05-10T03:27:10.872 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:10.872 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T03:27:10.873 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T03:27:10.873 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T03:27:10.873 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:11.124 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 95/146 2024-05-10T03:27:11.205 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 95/146 2024-05-10T03:27:11.205 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:11.206 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T03:27:11.206 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T03:27:11.206 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T03:27:11.206 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:11.327 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-05-10T03:27:11.328 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-05-10T03:27:11.758 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 96/146 2024-05-10T03:27:11.830 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 96/146 2024-05-10T03:27:11.830 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:11.830 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T03:27:11.830 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T03:27:11.830 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T03:27:11.831 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:11.985 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-05-10T03:27:12.052 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-05-10T03:27:12.131 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-05-10T03:27:12.377 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-05-10T03:27:12.466 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-05-10T03:27:12.563 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-05-10T03:27:12.669 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-05-10T03:27:12.849 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-05-10T03:27:12.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-05-10T03:27:12.963 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-05-10T03:27:12.996 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.9-15.el9.x86_64 107/146 2024-05-10T03:27:13.630 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el9.noarch 108/146 2024-05-10T03:27:13.666 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el9.noarch 108/146 2024-05-10T03:27:13.718 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 109/146 2024-05-10T03:27:13.846 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 110/146 2024-05-10T03:27:13.877 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 111/146 2024-05-10T03:27:15.256 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 111/146 2024-05-10T03:27:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 112/146 2024-05-10T03:27:16.478 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 112/146 2024-05-10T03:27:16.521 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 113/146 2024-05-10T03:27:16.675 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 113/146 2024-05-10T03:27:16.868 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 114/146 2024-05-10T03:27:16.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 115/146 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 115/146 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T03:27:16.972 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:17.032 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 116/146 2024-05-10T03:27:17.056 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-05-10T03:27:17.071 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 116/146 2024-05-10T03:27:17.158 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/146 2024-05-10T03:27:17.178 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el9.x86_64 117/146 2024-05-10T03:27:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-05-10T03:27:17.244 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 118/146 2024-05-10T03:27:17.344 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-05-10T03:27:17.404 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-05-10T03:27:17.443 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-05-10T03:27:17.521 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-05-10T03:27:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-05-10T03:27:18.032 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:27:18.107 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-45.el9.x86_64 119/146 2024-05-10T03:27:18.144 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 120/146 2024-05-10T03:27:18.208 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 120/146 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T03:27:18.225 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:27:18.278 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 121/146 2024-05-10T03:27:18.351 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 121/146 2024-05-10T03:27:18.352 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:18.352 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T03:27:18.352 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:27:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 10/146 2024-05-10T03:27:18.454 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 122/146 2024-05-10T03:27:18.531 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 122/146 2024-05-10T03:27:18.531 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:18.531 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T03:27:18.532 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T03:27:18.532 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T03:27:18.532 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/146 2024-05-10T03:27:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/146 2024-05-10T03:27:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:27:18.950 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:27:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 14/146 2024-05-10T03:27:19.188 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 123/146 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 123/146 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T03:27:19.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 14/146 2024-05-10T03:27:19.467 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 15/146 2024-05-10T03:27:19.555 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 15/146 2024-05-10T03:27:19.770 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:27:19.895 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:27:20.328 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 16/146 2024-05-10T03:27:20.383 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-05-10T03:27:20.435 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-05-10T03:27:20.532 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-05-10T03:27:20.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-05-10T03:27:20.622 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-05-10T03:27:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-05-10T03:27:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/146 2024-05-10T03:27:20.841 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-05-10T03:27:20.895 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/146 2024-05-10T03:27:21.003 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/146 2024-05-10T03:27:21.045 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el9. 27/146 2024-05-10T03:27:21.093 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 28/146 2024-05-10T03:27:21.200 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_6 29/146 2024-05-10T03:27:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-05-10T03:27:21.456 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-05-10T03:27:22.153 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-05-10T03:27:22.270 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-05-10T03:27:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 34/146 2024-05-10T03:27:22.417 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el9.x86_6 35/146 2024-05-10T03:27:22.482 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-05-10T03:27:22.683 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 37/146 2024-05-10T03:27:22.770 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 38/146 2024-05-10T03:27:22.917 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-3.el9.x86_64 39/146 2024-05-10T03:27:22.957 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-05-10T03:27:23.022 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-05-10T03:27:23.110 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-05-10T03:27:23.159 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/146 2024-05-10T03:27:23.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 44/146 2024-05-10T03:27:23.266 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-05-10T03:27:23.320 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-05-10T03:27:23.381 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-05-10T03:27:23.579 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-05-10T03:27:23.603 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 124/146 2024-05-10T03:27:23.624 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-05-10T03:27:23.657 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 125/146 2024-05-10T03:27:23.676 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 50/146 2024-05-10T03:27:23.711 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 126/146 2024-05-10T03:27:23.752 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 127/146 2024-05-10T03:27:23.805 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-05-10T03:27:23.848 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 52/146 2024-05-10T03:27:23.900 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-05-10T03:27:23.938 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/146 2024-05-10T03:27:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-05-10T03:27:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-05-10T03:27:24.054 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 128/146 2024-05-10T03:27:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-05-10T03:27:24.107 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 129/146 2024-05-10T03:27:24.122 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-05-10T03:27:24.135 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_ 130/146 2024-05-10T03:27:24.170 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 131/146 2024-05-10T03:27:24.192 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-05-10T03:27:24.245 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-05-10T03:27:24.246 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 132/146 2024-05-10T03:27:24.300 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-05-10T03:27:24.348 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-05-10T03:27:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-05-10T03:27:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-05-10T03:27:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-05-10T03:27:24.604 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-05-10T03:27:24.760 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-05-10T03:27:24.830 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-05-10T03:27:25.218 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:27:25.269 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:27:25.329 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:27:25.780 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-05-10T03:27:25.829 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-05-10T03:27:25.861 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-05-10T03:27:25.985 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-05-10T03:27:26.029 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 73/146 2024-05-10T03:27:26.276 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 74/146 2024-05-10T03:27:26.300 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 75/146 2024-05-10T03:27:26.407 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-05-10T03:27:26.774 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 134/146 2024-05-10T03:27:26.901 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 135/146 2024-05-10T03:27:26.981 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-05-10T03:27:27.135 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-05-10T03:27:27.171 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-05-10T03:27:27.209 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 139/146 2024-05-10T03:27:27.209 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 140/146 2024-05-10T03:27:27.242 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-05-10T03:27:27.345 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 140/146 2024-05-10T03:27:27.397 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-05-10T03:27:27.417 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 141/146 2024-05-10T03:27:27.417 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-05-10T03:27:27.554 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-05-10T03:27:27.593 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 143/146 2024-05-10T03:27:27.628 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:27:27.629 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:27:27.965 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:27:27.965 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 145/146 2024-05-10T03:27:28.110 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 145/146 2024-05-10T03:27:28.245 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:27:28.516 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:27:28.517 INFO:teuthology.orchestra.run.smithi055.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-10T03:27:28.517 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:29.029 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-05-10T03:27:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-05-10T03:27:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-05-10T03:27:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 82/146 2024-05-10T03:27:29.584 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-05-10T03:27:29.627 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el9.x86_64 84/146 2024-05-10T03:27:29.747 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el9.x86_64 84/146 2024-05-10T03:27:29.774 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 85/146 2024-05-10T03:27:29.830 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-05-10T03:27:30.302 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-05-10T03:27:30.329 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el9.x86_64 88/146 2024-05-10T03:27:30.404 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el9.x86_64 88/146 2024-05-10T03:27:30.431 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 89/146 2024-05-10T03:27:32.705 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:27:32.752 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:27:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 90/146 2024-05-10T03:27:33.430 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 91/146 2024-05-10T03:27:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 91/146 2024-05-10T03:27:33.589 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-10T03:27:33.589 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-10T03:27:33.589 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:33.631 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 92/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el9.x86_64 1/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 2/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 3/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 4/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 6/146 2024-05-10T03:27:34.160 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 7/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 10/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 11/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 12/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_ 14/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 15/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 16/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el9.x86_6 17/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 18/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el9.x86_64 19/146 2024-05-10T03:27:34.161 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el9. 21/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_6 22/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 23/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 24/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 25/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 26/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 27/146 2024-05-10T03:27:34.162 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 28/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 30/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 31/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 34/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/146 2024-05-10T03:27:34.173 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 36/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el9.noarch 37/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 38/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 39/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 40/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 41/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-05-10T03:27:34.174 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 47/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 51/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-05-10T03:27:34.175 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 60/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 61/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 64/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 65/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 66/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-13.14-1.el9.x86_64 67/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 68/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 69/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 70/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 72/146 2024-05-10T03:27:34.176 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 73/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-05-10T03:27:34.177 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 89/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 91/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-05-10T03:27:34.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 107/146 2024-05-10T03:27:34.179 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-05-10T03:27:34.180 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 124/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 125/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : re2-1:20211101-3.el9.x86_64 129/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el9.x86_64 131/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el9.x86_64 133/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 135/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 136/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 137/146 2024-05-10T03:27:34.181 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 138/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 139/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 140/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 141/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 142/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 143/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 144/146 2024-05-10T03:27:34.182 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 145/146 2024-05-10T03:27:36.372 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:27:36.372 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:36.372 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T03:27:36.373 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.374 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T03:27:36.375 INFO:teuthology.orchestra.run.smithi055.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T03:27:36.376 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T03:27:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T03:27:36.378 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T03:27:36.379 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T03:27:36.380 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T03:27:36.381 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T03:27:36.382 INFO:teuthology.orchestra.run.smithi055.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T03:27:36.383 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T03:27:36.383 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T03:27:36.383 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T03:27:36.383 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:27:36.383 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-05-10T03:27:37.181 DEBUG:teuthology.parallel:result is None 2024-05-10T03:27:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 92/146 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-05-10T03:27:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:46.666 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 93/146 2024-05-10T03:27:46.713 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 93/146 2024-05-10T03:27:46.713 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:46.713 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-10T03:27:46.714 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 94/146 2024-05-10T03:27:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 94/146 2024-05-10T03:27:48.525 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:48.525 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-10T03:27:48.526 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T03:27:48.526 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-10T03:27:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:48.753 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 95/146 2024-05-10T03:27:48.823 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 95/146 2024-05-10T03:27:48.823 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:48.823 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-10T03:27:48.823 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T03:27:48.823 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-10T03:27:48.824 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 96/146 2024-05-10T03:27:49.402 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 96/146 2024-05-10T03:27:49.402 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:49.402 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-10T03:27:49.403 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T03:27:49.403 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-10T03:27:49.403 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:49.538 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-05-10T03:27:49.601 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-05-10T03:27:49.674 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-05-10T03:27:49.909 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-05-10T03:27:49.967 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-05-10T03:27:50.075 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-05-10T03:27:50.194 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-05-10T03:27:50.373 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-05-10T03:27:50.424 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-05-10T03:27:50.464 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-05-10T03:27:50.498 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 107/146 2024-05-10T03:27:51.132 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el9.noarch 108/146 2024-05-10T03:27:51.170 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el9.noarch 108/146 2024-05-10T03:27:51.219 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 109/146 2024-05-10T03:27:51.322 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 110/146 2024-05-10T03:27:51.355 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 111/146 2024-05-10T03:27:52.615 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 111/146 2024-05-10T03:27:52.656 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 112/146 2024-05-10T03:27:53.752 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 112/146 2024-05-10T03:27:53.781 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 113/146 2024-05-10T03:27:53.927 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 113/146 2024-05-10T03:27:54.108 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 114/146 2024-05-10T03:27:54.133 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 115/146 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 115/146 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-10T03:27:54.209 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:54.267 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 116/146 2024-05-10T03:27:54.303 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 116/146 2024-05-10T03:27:54.408 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el9.x86_64 117/146 2024-05-10T03:27:54.484 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 118/146 2024-05-10T03:27:55.277 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-45.el9.x86_64 119/146 2024-05-10T03:27:55.320 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 120/146 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 120/146 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-10T03:27:55.414 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:55.462 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 121/146 2024-05-10T03:27:55.545 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 121/146 2024-05-10T03:27:55.545 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:55.545 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-10T03:27:55.546 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 122/146 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 122/146 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-10T03:27:55.725 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:27:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 123/146 2024-05-10T03:27:56.216 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 123/146 2024-05-10T03:27:56.217 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-10T03:27:56.217 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-10T03:27:56.217 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T03:27:56.217 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-10T03:27:56.217 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:00.246 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 124/146 2024-05-10T03:28:00.305 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 125/146 2024-05-10T03:28:00.367 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 126/146 2024-05-10T03:28:00.402 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 127/146 2024-05-10T03:28:00.704 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 128/146 2024-05-10T03:28:00.756 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 129/146 2024-05-10T03:28:00.784 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_ 130/146 2024-05-10T03:28:00.826 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 131/146 2024-05-10T03:28:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 132/146 2024-05-10T03:28:01.782 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:28:01.833 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:28:01.902 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 133/146 2024-05-10T03:28:03.274 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 134/146 2024-05-10T03:28:03.373 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 135/146 2024-05-10T03:28:03.478 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-05-10T03:28:03.625 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-05-10T03:28:03.666 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-05-10T03:28:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 139/146 2024-05-10T03:28:03.703 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 140/146 2024-05-10T03:28:03.831 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 140/146 2024-05-10T03:28:03.879 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 141/146 2024-05-10T03:28:03.879 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-05-10T03:28:04.015 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-05-10T03:28:04.046 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 143/146 2024-05-10T03:28:04.080 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:28:04.080 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:28:04.392 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 144/146 2024-05-10T03:28:04.392 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 145/146 2024-05-10T03:28:04.554 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 145/146 2024-05-10T03:28:04.686 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:28:04.948 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:28:04.949 INFO:teuthology.orchestra.run.smithi162.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-10T03:28:04.949 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:10.557 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el9.x86_64 1/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 2/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 3/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 4/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 6/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 7/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 8/146 2024-05-10T03:28:10.558 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 9/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 10/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 11/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 12/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 13/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_ 14/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 15/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 16/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el9.x86_6 17/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el9.x86 18/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el9.x86_64 19/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el9. 21/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_6 22/146 2024-05-10T03:28:10.559 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 23/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 24/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 25/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 26/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 27/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 28/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/146 2024-05-10T03:28:10.560 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noa 30/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.n 31/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 34/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 36/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el9.noarch 37/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 38/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 39/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 40/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 41/146 2024-05-10T03:28:10.573 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 47/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 51/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-05-10T03:28:10.574 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 60/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 61/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 64/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 65/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 66/146 2024-05-10T03:28:10.575 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.14-1.el9.x86_64 67/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 68/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 69/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 70/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 72/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 73/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-05-10T03:28:10.576 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-05-10T03:28:10.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-05-10T03:28:10.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-05-10T03:28:10.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-05-10T03:28:10.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-05-10T03:28:10.577 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-05-10T03:28:10.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-05-10T03:28:10.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-05-10T03:28:10.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-05-10T03:28:10.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-05-10T03:28:10.578 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 89/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 91/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-05-10T03:28:10.579 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-05-10T03:28:10.580 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-05-10T03:28:10.580 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/146 2024-05-10T03:28:10.580 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/146 2024-05-10T03:28:10.581 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 107/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-05-10T03:28:10.582 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 124/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 125/146 2024-05-10T03:28:10.583 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-3.el9.x86_64 129/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el9.x86_64 131/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el9.x86_64 133/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 135/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 136/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 137/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 138/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 139/146 2024-05-10T03:28:10.584 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 140/146 2024-05-10T03:28:10.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 141/146 2024-05-10T03:28:10.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 142/146 2024-05-10T03:28:10.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 143/146 2024-05-10T03:28:10.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 144/146 2024-05-10T03:28:10.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 145/146 2024-05-10T03:28:12.631 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 146/146 2024-05-10T03:28:12.631 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:12.631 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-05-10T03:28:12.631 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-10T03:28:12.631 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.632 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.633 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:18.2.2-1330.g50174015.el9.noarch 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-10T03:28:12.634 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-10T03:28:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-10T03:28:12.636 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.14-1.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-10T03:28:12.637 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-10T03:28:12.638 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-05-10T03:28:12.639 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-10T03:28:12.640 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el9.x86_64 2024-05-10T03:28:12.641 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-3.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-10T03:28:12.642 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-10T03:28:12.643 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:12.643 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-05-10T03:28:13.217 DEBUG:teuthology.parallel:result is None 2024-05-10T03:28:13.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:28:13.364 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T03:28:13.443 INFO:teuthology.orchestra.run.smithi055.stdout:18.2.2-1330.g50174015.el9 2024-05-10T03:28:13.443 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el9 2024-05-10T03:28:13.443 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T03:28:13.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T03:28:13.665 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T03:28:13.717 INFO:teuthology.orchestra.run.smithi162.stdout:18.2.2-1330.g50174015.el9 2024-05-10T03:28:13.717 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el9 2024-05-10T03:28:13.717 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T03:28:13.719 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T03:28:13.719 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:13.719 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T03:28:13.820 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:13.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T03:28:13.895 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T03:28:13.895 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:13.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T03:28:13.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T03:28:14.133 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:14.133 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T03:28:14.198 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T03:28:14.301 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T03:28:14.301 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:14.301 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T03:28:14.396 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T03:28:14.522 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:14.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T03:28:14.588 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T03:28:14.691 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T03:28:14.691 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:14.691 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T03:28:14.784 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T03:28:14.873 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:14.873 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T03:28:14.934 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T03:28:15.040 INFO:teuthology.run_tasks:Running task ceph... 2024-05-10T03:28:15.128 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-10T03:28:15.128 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /var/log/ceph 2024-05-10T03:28:15.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2024-05-10T03:28:15.226 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-10T03:28:15.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-10T03:28:15.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-10T03:28:15.314 INFO:tasks.ceph:Creating extra log directories... 2024-05-10T03:28:15.314 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-10T03:28:15.322 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-10T03:28:15.412 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-10T03:28:15.412 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-10T03:28:15.412 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-10T03:28:15.412 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:15.412 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-10T03:28:15.471 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:15.716 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:15.811 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:15.811 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-10T03:28:15.856 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:16.042 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:16.117 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-10T03:28:16.117 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-10T03:28:16.118 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476', 'branch': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/sessionmap}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7700476', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240509.171335-reef-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4933, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'sleep_before_teardown': 0, 'subset': '420/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '50174015a7e79f951b988949abe236791f52d4da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi055.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1ggasP6PjOxGWODFby903l3yTc6RQhPb990pDLl36T4WN/LF3LpECGNapQ9EMT1HOx/bnblnJ2eYOgbYJKKMs=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPh9kCquJ9b7EwxZc1UICqq5ofYdFNn1MvupXfUlYLECOTZwj6aFtHoacQcBY1qgnVqdOqrK6p8don5fFwnbji8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'client session with non-allowable root'], 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-10_02:33:39', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-10T03:28:16.118 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-10T03:28:16.126 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-10T03:28:16.180 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/run/ceph 2024-05-10T03:28:16.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2024-05-10T03:28:16.271 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:16.272 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T03:28:16.327 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T03:28:16.327 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2024-05-10T03:28:16.379 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T03:28:16.379 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-10 03:27:28.955962968 +0000 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-10 03:25:13.376764385 +0000 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-10 03:25:13.376764385 +0000 2024-05-10T03:28:16.380 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-10 03:25:13.376764385 +0000 2024-05-10T03:28:16.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T03:28:16.458 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-10T03:28:16.459 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-10T03:28:16.459 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000477581 s, 1.1 MB/s 2024-05-10T03:28:16.462 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T03:28:16.517 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2024-05-10T03:28:16.571 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T03:28:16.571 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-10 03:27:28.956962947 +0000 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-10 03:25:13.992752012 +0000 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-10 03:25:13.992752012 +0000 2024-05-10T03:28:16.572 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-10 03:25:13.992752012 +0000 2024-05-10T03:28:16.572 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T03:28:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-10T03:28:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-10T03:28:16.648 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000413102 s, 1.2 MB/s 2024-05-10T03:28:16.651 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T03:28:16.708 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 675 Links: 1 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-10 03:27:28.957962927 +0000 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-10 03:25:14.617739459 +0000 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-10 03:25:14.617739459 +0000 2024-05-10T03:28:16.763 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-10 03:25:14.617739459 +0000 2024-05-10T03:28:16.764 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T03:28:16.845 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-10T03:28:16.845 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-10T03:28:16.845 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000413381 s, 1.2 MB/s 2024-05-10T03:28:16.850 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T03:28:16.911 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2024-05-10T03:28:16.963 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 6h/6d Inode: 695 Links: 1 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-05-10 03:27:28.958962907 +0000 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-05-10 03:25:15.235727046 +0000 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-05-10 03:25:15.235727046 +0000 2024-05-10T03:28:16.964 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-05-10 03:25:15.235727046 +0000 2024-05-10T03:28:16.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T03:28:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-05-10T03:28:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-05-10T03:28:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000404058 s, 1.3 MB/s 2024-05-10T03:28:17.044 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T03:28:17.104 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:17.104 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:17.104 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T03:28:17.149 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T03:28:17.149 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-10 03:28:05.401788509 +0000 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-10 03:23:54.617492288 +0000 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-10 03:23:54.617492288 +0000 2024-05-10T03:28:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-10 03:23:54.617492288 +0000 2024-05-10T03:28:17.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T03:28:17.334 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-10T03:28:17.335 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-10T03:28:17.335 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000462151 s, 1.1 MB/s 2024-05-10T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T03:28:17.379 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-05-10T03:28:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T03:28:17.461 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:17.461 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-10 03:28:05.402788488 +0000 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-10 03:23:55.188480403 +0000 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-10 03:23:55.188480403 +0000 2024-05-10T03:28:17.462 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-10 03:23:55.188480403 +0000 2024-05-10T03:28:17.462 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T03:28:17.574 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-10T03:28:17.574 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-10T03:28:17.574 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000377373 s, 1.4 MB/s 2024-05-10T03:28:17.578 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T03:28:17.626 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-05-10T03:28:17.721 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-10 03:28:05.403788468 +0000 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-10 03:23:55.801467644 +0000 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-10 03:23:55.801467644 +0000 2024-05-10T03:28:17.722 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-10 03:23:55.801467644 +0000 2024-05-10T03:28:17.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T03:28:17.799 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-10T03:28:17.799 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-10T03:28:17.799 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000369467 s, 1.4 MB/s 2024-05-10T03:28:17.803 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T03:28:17.860 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-05-10T03:28:17.907 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 692 Links: 1 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-05-10 03:28:05.404788448 +0000 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-05-10 03:23:56.430454552 +0000 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-05-10 03:23:56.430454552 +0000 2024-05-10T03:28:17.908 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-05-10 03:23:56.430454552 +0000 2024-05-10T03:28:17.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T03:28:17.978 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-05-10T03:28:17.979 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-05-10T03:28:17.979 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000303022 s, 1.7 MB/s 2024-05-10T03:28:17.982 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T03:28:18.034 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:18.035 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-10T03:28:18.035 INFO:tasks.ceph:Generating config... 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] debug client = 20 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] debug ms = 1 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] pid file = 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-10T03:28:18.036 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-10T03:28:18.036 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-10T03:28:18.036 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-10T03:28:18.036 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-10T03:28:18.037 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-10T03:28:18.037 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-10T03:28:18.038 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-10T03:28:18.038 INFO:tasks.ceph:Setting up mon.a... 2024-05-10T03:28:18.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-10T03:28:18.258 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.keyring 2024-05-10T03:28:18.275 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-10T03:28:18.483 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T03:28:18.568 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.55'), ('mon.b', '172.21.15.162'), ('mon.c', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]')] 2024-05-10T03:28:18.568 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-10T03:28:18.569 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:18.569 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-10T03:28:18.625 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.55 --add b 172.21.15.162 --addv c '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:18.856 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:18.856 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: generated fsid 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = pacific 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 0 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:fsid 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:last_changed 2024-05-10T03:28:18.853289+0000 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:created 2024-05-10T03:28:18.853289+0000 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:min_mon_release 16 (pacific) 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:election_strategy: 1 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.b 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:2: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.c 2024-05-10T03:28:18.857 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-10T03:28:18.867 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-10T03:28:18.921 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 340d5548-dd78-4f40-a3a8-44ee99f8e0d8... 2024-05-10T03:28:18.922 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:28:18.929 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.073 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:28:19.074 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.075 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on skipped message = False 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-10T03:28:19.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.077 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:28:19.078 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:28:19.079 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:28:19.080 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.081 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-10T03:28:19.082 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-10T03:28:19.083 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:28:19.084 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.085 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:28:19.086 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.087 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on skipped message = False 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-05-10T03:28:19.088 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.089 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-05-10T03:28:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-05-10T03:28:19.091 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:28:19.092 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.093 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-05-10T03:28:19.094 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2024-05-10T03:28:19.095 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-05-10T03:28:19.096 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2024-05-10T03:28:19.096 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:28:19.096 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:28:19.096 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:28:19.113 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-10T03:28:19.113 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-10T03:28:19.328 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-10T03:28:19.328 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:19.328 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-10T03:28:19.379 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:19.379 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-10T03:28:19.430 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:28:19.430 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:19.431 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-10T03:28:19.431 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T03:28:19.548 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:28:19.548 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:19.647 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:28:19.648 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:19.648 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-10T03:28:19.648 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-10T03:28:19.754 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:28:19.754 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:19.845 INFO:tasks.ceph:Setting up mon nodes... 2024-05-10T03:28:19.845 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-10T03:28:19.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-10T03:28:20.081 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-10T03:28:20.098 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-10T03:28:20.317 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-10T03:28:20.333 INFO:tasks.ceph:Setting up mds nodes... 2024-05-10T03:28:20.334 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-10T03:28:20.569 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-10T03:28:20.585 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-10T03:28:20.674 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-10T03:28:20.902 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-10T03:28:20.920 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-10T03:28:21.010 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-10T03:28:21.208 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-10T03:28:21.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-10T03:28:21.304 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-10T03:28:21.541 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-10T03:28:21.556 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-10T03:28:21.630 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-10T03:28:21.630 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T03:28:21.829 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-10T03:28:21.884 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-10T03:28:22.081 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-10T03:28:22.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-10T03:28:22.326 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-10T03:28:22.389 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-10T03:28:22.575 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-10T03:28:22.627 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-10T03:28:22.627 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-10T03:28:22.627 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-10T03:28:22.723 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:22.723 INFO:tasks.ceph:role: osd.0 2024-05-10T03:28:22.723 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:28:22.723 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:22.840 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:24.124 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-10T03:28:24.132 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-10T03:28:24.132 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-10T03:28:24.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-10T03:28:24.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-10T03:28:24.399 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:24.399 INFO:tasks.ceph:role: osd.1 2024-05-10T03:28:24.399 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:28:24.400 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:24.513 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:24.514 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:24.514 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:24.514 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:24.514 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:25.792 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-10T03:28:25.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-10T03:28:25.800 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-10T03:28:25.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-10T03:28:25.981 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-10T03:28:26.064 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:26.064 INFO:tasks.ceph:role: osd.2 2024-05-10T03:28:26.064 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:28:26.064 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-10T03:28:26.175 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:26.175 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:26.175 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:26.175 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:26.176 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:27.456 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-10T03:28:27.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-10T03:28:27.463 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-10T03:28:27.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-10T03:28:27.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-10T03:28:27.729 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:27.730 INFO:tasks.ceph:role: osd.3 2024-05-10T03:28:27.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:28:27.730 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:27.844 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:27.845 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:29.145 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2024-05-10T03:28:29.151 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2024-05-10T03:28:29.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-10T03:28:29.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-10T03:28:29.340 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:29.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.571+0000 7fb886716640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-10T03:28:29.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.572+0000 7fb886716640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-10T03:28:29.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.572+0000 7fb886716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T03:28:29.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.572+0000 7fb886716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T03:28:29.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.572+0000 7fb886716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-10T03:28:29.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:29.575+0000 7fb886716640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-10T03:28:32.836 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-10T03:28:32.919 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:33.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.124+0000 7fb95ba1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-10T03:28:33.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.125+0000 7fb95ba1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-10T03:28:33.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.125+0000 7fb95ba1c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T03:28:33.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.125+0000 7fb95ba1c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T03:28:33.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.125+0000 7fb95ba1c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-10T03:28:33.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:33.127+0000 7fb95ba1c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-10T03:28:36.138 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-10T03:28:36.228 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:36.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.434+0000 7fe2eb41e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-10T03:28:36.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.435+0000 7fe2eb41e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-10T03:28:36.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.435+0000 7fe2eb41e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T03:28:36.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.435+0000 7fe2eb41e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T03:28:36.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.435+0000 7fe2eb41e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-10T03:28:36.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:36.438+0000 7fe2eb41e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-10T03:28:39.713 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-10T03:28:39.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:40.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.011+0000 7f8caefc2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-10T03:28:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.012+0000 7f8caefc2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-10T03:28:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.012+0000 7f8caefc2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T03:28:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.012+0000 7f8caefc2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T03:28:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.012+0000 7f8caefc2640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-10T03:28:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:28:40.015+0000 7f8caefc2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-10T03:28:43.266 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-10T03:28:43.354 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-10T03:28:43.446 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:43.447 INFO:tasks.ceph:role: osd.4 2024-05-10T03:28:43.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:28:43.447 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-10T03:28:43.536 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:43.536 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:43.536 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:43.537 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:43.537 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:43.537 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:43.537 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:43.944 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-05-10T03:28:43.951 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-05-10T03:28:43.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-10T03:28:44.099 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-10T03:28:44.189 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-10T03:28:44.272 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:44.272 INFO:tasks.ceph:role: osd.5 2024-05-10T03:28:44.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:28:44.273 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-10T03:28:44.354 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:44.354 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:44.355 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:44.710 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-05-10T03:28:44.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-05-10T03:28:44.717 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-10T03:28:44.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-10T03:28:44.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-10T03:28:44.999 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:44.999 INFO:tasks.ceph:role: osd.6 2024-05-10T03:28:44.999 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:28:44.999 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:45.086 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:45.087 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:45.087 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:45.494 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-05-10T03:28:45.502 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-05-10T03:28:45.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-10T03:28:45.632 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-10T03:28:45.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-10T03:28:45.795 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-10T03:28:45.795 INFO:tasks.ceph:role: osd.7 2024-05-10T03:28:45.795 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:28:45.795 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-10T03:28:45.876 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-10T03:28:45.876 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-10T03:28:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-10T03:28:46.120 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:46.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:28:46.290 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-05-10T03:28:46.296 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-05-10T03:28:46.296 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-10T03:28:46.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-10T03:28:46.584 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:46.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.793+0000 7f8c66501640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-10T03:28:46.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.794+0000 7f8c66501640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-10T03:28:46.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.794+0000 7f8c66501640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T03:28:46.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.794+0000 7f8c66501640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T03:28:46.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.794+0000 7f8c66501640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-10T03:28:46.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:46.798+0000 7f8c66501640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-10T03:28:50.067 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-10T03:28:50.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:50.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.343+0000 7fccfb29d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-10T03:28:50.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.343+0000 7fccfb29d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-10T03:28:50.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.343+0000 7fccfb29d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T03:28:50.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.343+0000 7fccfb29d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T03:28:50.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.343+0000 7fccfb29d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-10T03:28:50.350 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:50.346+0000 7fccfb29d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-10T03:28:53.588 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-10T03:28:53.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:53.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.871+0000 7fbbb58af640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-10T03:28:53.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.872+0000 7fbbb58af640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-10T03:28:53.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.872+0000 7fbbb58af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T03:28:53.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.872+0000 7fbbb58af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T03:28:53.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.872+0000 7fbbb58af640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-10T03:28:53.878 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:53.874+0000 7fbbb58af640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-10T03:28:57.090 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-10T03:28:57.171 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:28:57.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.368+0000 7f8a50a32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-10T03:28:57.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.369+0000 7f8a50a32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-10T03:28:57.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.369+0000 7f8a50a32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T03:28:57.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.369+0000 7f8a50a32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T03:28:57.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.369+0000 7f8a50a32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-10T03:28:57.375 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:28:57.371+0000 7f8a50a32640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-10T03:29:00.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-10T03:29:00.448 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-10T03:29:00.448 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.448 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-10T03:29:00.544 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.544 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-10T03:29:00.634 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.634 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-10T03:29:00.722 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-10T03:29:00.813 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.814 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-10T03:29:00.901 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.901 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-10T03:29:00.990 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:00.990 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-10T03:29:01.082 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-10T03:29:01.171 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.171 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-10T03:29:01.254 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.255 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-10T03:29:01.341 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.341 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-10T03:29:01.428 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.428 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-10T03:29:01.509 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-10T03:29:01.600 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.600 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-10T03:29:01.687 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:01.688 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-10T03:29:01.747 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:01.747 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-10T03:29:01.801 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.801 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-10T03:29:01.857 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:01.857 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-10T03:29:01.909 INFO:tasks.ceph:Adding keys to all mons... 2024-05-10T03:29:01.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-10T03:29:01.917 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-10T03:29:01.995 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2024-05-10T03:29:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw== 2024-05-10T03:29:01.995 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg== 2024-05-10T03:29:01.996 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[client.2] 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg== 2024-05-10T03:29:01.997 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw== 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi162.stdout:[client.3] 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg== 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.y] 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw== 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw== 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-10T03:29:01.998 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.0] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.1] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.2] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.3] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.x] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA== 2024-05-10T03:29:01.999 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.4] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.5] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.6] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.7] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[client.2] 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg== 2024-05-10T03:29:02.000 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2024-05-10T03:29:02.001 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw== 2024-05-10T03:29:02.001 INFO:teuthology.orchestra.run.smithi055.stdout:[client.3] 2024-05-10T03:29:02.001 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg== 2024-05-10T03:29:02.002 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T03:29:02.050 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T03:29:02.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:02.274 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:02.486 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:02.492 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:02.708 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:02.714 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:02.930 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:02.936 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:03.144 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:03.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:03.362 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:03.367 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:03.581 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T03:29:03.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-10T03:29:03.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:03.796 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:04.011 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:04.015 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-10T03:29:04.229 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.448 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.454 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.663 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.668 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.880 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:04.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-10T03:29:05.097 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.102 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.314 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.320 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.539 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.753 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-10T03:29:05.973 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-10T03:29:05.973 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-10T03:29:06.060 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T03:29:06.306 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-10T03:29:06.389 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-10T03:29:06.473 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T03:29:06.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-10T03:29:06.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-10T03:29:06.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-10T03:29:07.065 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-10T03:29:07.139 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:29:07.147 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-10T03:29:07.200 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-10T03:29:07.200 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-10T03:29:07.201 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-10T03:29:07.205 INFO:tasks.ceph.mon.a:Started 2024-05-10T03:29:07.206 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-10T03:29:07.206 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-10T03:29:07.213 INFO:tasks.ceph.mon.b:Started 2024-05-10T03:29:07.213 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-10T03:29:07.214 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-10T03:29:07.221 INFO:tasks.ceph.mon.c:Started 2024-05-10T03:29:07.221 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-10T03:29:07.222 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-10T03:29:07.222 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-10T03:29:07.229 INFO:tasks.ceph.mgr.y:Started 2024-05-10T03:29:07.229 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-10T03:29:07.229 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-10T03:29:07.236 INFO:tasks.ceph.mgr.x:Started 2024-05-10T03:29:07.236 INFO:tasks.ceph:Setting crush tunables to default 2024-05-10T03:29:07.236 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd crush tunables default 2024-05-10T03:29:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.477+0000 7ff498400640 1 Processor -- start 2024-05-10T03:29:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.477+0000 7ff498400640 1 -- start start 2024-05-10T03:29:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff498400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff498400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b3590 con 0x7ff4900b4450 2024-05-10T03:29:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff498400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b29c0 con 0x7ff49006b120 2024-05-10T03:29:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff498400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900b2c60 con 0x7ff490166480 2024-05-10T03:29:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff495a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff4900b4450 0x7ff4900b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52646/0 (socket says 172.21.15.55:52646) 2024-05-10T03:29:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 learned_addr learned my addr 172.21.15.55:0/1281673241 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.479+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.480+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-10T03:29:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.480+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.480+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.480+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:07.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.681+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:07.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.682+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-10T03:29:07.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.682+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:07.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.682+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:07.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:07.682+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:08.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.082+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:08.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.082+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-10T03:29:08.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.082+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:08.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.082+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:08.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.083+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:08.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.884+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:08.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.885+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-10T03:29:08.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.885+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:08.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.885+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:08.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:08.885+0000 7ff496400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 msgr2=0x7ff4901688a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff490166480 0x7ff4901688a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v1:172.21.15.162:6789/0 conn(0x7ff49006b120 legacy=0x7ff49006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v1:172.21.15.55:6789/0 conn(0x7ff4900b4450 legacy=0x7ff4900b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4900b2c60 con 0x7ff4700036f0 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff470008da0 con 0x7ff4700061f0 2024-05-10T03:29:10.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.479+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:10.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:10.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.480+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:10.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.681+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:10.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:10.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:10.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:10.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:10.682+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:11.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.083+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.083+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.084+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:11.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.885+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:11.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:11.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:11.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.886+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.887+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.887+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:11.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:11.887+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:29:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 msgr2=0x7ff4700086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T03:29:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff4700061f0 0x7ff4700086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 msgr2=0x7ff470005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700036f0 0x7ff470005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 reap_dead start 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900b2f40 con 0x7ff4900b4450 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.479+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff470005ea0 con 0x7ff470004040 2024-05-10T03:29:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4900b2c60 con 0x7ff4700066b0 2024-05-10T03:29:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.480+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.481+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.481+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.481+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.481+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.682+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.683+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.683+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.683+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:13.683+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.084+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.084+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.085+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.085+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:14.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.085+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.886+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:14.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.887+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:14.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.887+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:14.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.887+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:14.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:14.887+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T03:29:16.280 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:29:16.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:29:16.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 msgr2=0x7ff470003b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T03:29:16.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.55:3300/0 conn(0x7ff4700066b0 0x7ff470003b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:16.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v1:172.21.15.162:6789/0 conn(0x7ff470004040 legacy=0x7ff470004540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:16.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 >> v1:172.21.15.55:6789/0 conn(0x7ff4900b4450 legacy=0x7ff47000b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4900b2c60 con 0x7ff47000eac0 2024-05-10T03:29:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.480+0000 7ff48ea00640 1 -- 172.21.15.55:0/1281673241 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff470008da0 con 0x7ff47000f620 2024-05-10T03:29:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.481+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.481+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.481+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.481+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.482+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T03:29:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.683+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.683+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:16.684+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.085+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.085+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495000640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.086+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T03:29:17.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.888+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> v2:172.21.15.162:3300/0 conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.888+0000 7ff495000640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.888+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 msgr2=0x7ff47000fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:17.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.888+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000f620 0x7ff47000fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.889+0000 7ff495a00640 1 -- 172.21.15.55:0/1281673241 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470010910 con 0x7ff47000eac0 2024-05-10T03:29:17.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.890+0000 7ff495a00640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff47000eac0 0x7ff47000ef40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff484007fd0 tx=0x7ff48402bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31abc667426b4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:17.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.891+0000 7ff48fe00640 1 -- 172.21.15.55:0/1281673241 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48402c580 con 0x7ff47000eac0 2024-05-10T03:29:17.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.891+0000 7ff48fe00640 1 -- 172.21.15.55:0/1281673241 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484030070 con 0x7ff47000eac0 2024-05-10T03:29:17.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.891+0000 7ff48fe00640 1 -- 172.21.15.55:0/1281673241 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48402c8e0 con 0x7ff47000eac0 2024-05-10T03:29:17.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 -- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff47000eac0 msgr2=0x7ff47000ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff47000eac0 0x7ff47000ef40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff484007fd0 tx=0x7ff48402bc80 comp rx=0 tx=0).stop 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff496400640 1 -- 172.21.15.55:0/1281673241 reap_dead start 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 -- 172.21.15.55:0/1281673241 shutdown_connections 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 --2- 172.21.15.55:0/1281673241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff47000eac0 0x7ff47000ef40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 -- 172.21.15.55:0/1281673241 >> 172.21.15.55:0/1281673241 conn(0x7ff490070900 msgr2=0x7ff490070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 -- 172.21.15.55:0/1281673241 shutdown_connections 2024-05-10T03:29:17.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.893+0000 7ff498400640 1 -- 172.21.15.55:0/1281673241 wait complete. 2024-05-10T03:29:17.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.897+0000 7ff498400640 1 Processor -- start 2024-05-10T03:29:17.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.898+0000 7ff498400640 1 -- start start 2024-05-10T03:29:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000eac0 0x7ff490142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff49013f8a0 0x7ff49013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490156c80 con 0x7ff490140670 2024-05-10T03:29:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901433b0 con 0x7ff49013f8a0 2024-05-10T03:29:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff498400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49015a870 con 0x7ff47000eac0 2024-05-10T03:29:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.900+0000 7ff495000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48518/0 (socket says 172.21.15.55:48518) 2024-05-10T03:29:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 -- 172.21.15.55:0/841144194 learned_addr learned my addr 172.21.15.55:0/841144194 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495a00640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000eac0 0x7ff490142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff496400640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff49013f8a0 0x7ff49013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 -- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000eac0 msgr2=0x7ff490142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000eac0 0x7ff490142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 -- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff49013f8a0 msgr2=0x7ff49013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff49013f8a0 0x7ff49013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.901+0000 7ff495000640 1 -- 172.21.15.55:0/841144194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900b2f40 con 0x7ff490140670 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.902+0000 7ff495000640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff47c003fc0 tx=0x7ff47c0178c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.902+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c007020 con 0x7ff490140670 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.902+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c01d040 con 0x7ff490140670 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.902+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c007420 con 0x7ff490140670 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.902+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4901401f0 con 0x7ff490140670 2024-05-10T03:29:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.903+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff490143600 con 0x7ff490140670 2024-05-10T03:29:17.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.903+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff47c006ad0 con 0x7ff490140670 2024-05-10T03:29:17.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.903+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff47c001c80 con 0x7ff490140670 2024-05-10T03:29:17.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.904+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490067220 con 0x7ff490140670 2024-05-10T03:29:17.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.908+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff47c025020 con 0x7ff490140670 2024-05-10T03:29:17.943 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:17.939+0000 7fbf0de1f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:29:17.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.984+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff49006d510 con 0x7ff490140670 2024-05-10T03:29:17.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.992+0000 7ff48d600640 1 -- 172.21.15.55:0/841144194 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff47c0017e0 con 0x7ff490140670 2024-05-10T03:29:17.997 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 msgr2=0x7ff490156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff47c003fc0 tx=0x7ff47c0178c0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 shutdown_connections 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff49013f8a0 0x7ff49013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff490140670 0x7ff490156740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 --2- 172.21.15.55:0/841144194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff47000eac0 0x7ff490142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:17.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 >> 172.21.15.55:0/841144194 conn(0x7ff490070900 msgr2=0x7ff49013dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:18.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 shutdown_connections 2024-05-10T03:29:18.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:17.994+0000 7ff498400640 1 -- 172.21.15.55:0/841144194 wait complete. 2024-05-10T03:29:18.017 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:18.012+0000 7ff5b7f23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T03:29:18.026 INFO:tasks.ceph:check_enable_crimson: False 2024-05-10T03:29:18.026 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-10T03:29:18.027 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:18.027 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-10T03:29:18.103 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:18.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-10T03:29:18.132 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:18.128+0000 7fbf0de1f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:29:18.193 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:18.188+0000 7fbf0de1f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:29:18.220 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:18.215+0000 7ff5b7f23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:29:18.229 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:18.229 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-10T03:29:18.288 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:18.283+0000 7ff5b7f23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T03:29:18.305 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:18.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-10T03:29:18.423 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:18.423 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-10T03:29:18.486 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:18.481+0000 7fbf0de1f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:29:18.497 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:18.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-10T03:29:18.543 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:18.539+0000 7fbf0de1f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:29:18.599 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:18.595+0000 7ff5b7f23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T03:29:18.605 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:18.605 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-10T03:29:18.655 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:18.650+0000 7ff5b7f23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T03:29:18.719 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:29:18.719 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-10T03:29:18.786 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 097e253b-6207-4031-9378-098510bfc2b2 0 2024-05-10T03:29:19.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.074+0000 7f7238000640 1 Processor -- start 2024-05-10T03:29:19.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.074+0000 7f7238000640 1 -- start start 2024-05-10T03:29:19.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.076+0000 7f7238000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7230071db0 0x7f7230158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.076+0000 7f7238000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f72300ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.076+0000 7f7238000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72300708b0 con 0x7f72300b3860 2024-05-10T03:29:19.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.076+0000 7f7238000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300b2490 con 0x7f7230071db0 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.076+0000 7f7238000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230070050 con 0x7f723006a4b0 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7236000640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f72300b3860 0x7f72300b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57772/0 (socket says 172.21.15.162:57772) 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7236000640 1 -- 172.21.15.162:0/4106848533 learned_addr learned my addr 172.21.15.162:0/4106848533 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7234c00640 1 --2- 172.21.15.162:0/4106848533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f72300ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7235600640 1 --2- 172.21.15.162:0/4106848533 >> v2:172.21.15.55:3300/0 conn(0x7f7230071db0 0x7f7230158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f722fe00640 1 -- 172.21.15.162:0/4106848533 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274662168 0 0) 0x7f72300708b0 con 0x7f72300b3860 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f722fe00640 1 -- 172.21.15.162:0/4106848533 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7218003520 con 0x7f72300b3860 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7234c00640 1 -- 172.21.15.162:0/4106848533 >> v2:172.21.15.55:3300/0 conn(0x7f7230071db0 msgr2=0x7f7230158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7234c00640 1 --2- 172.21.15.162:0/4106848533 >> v2:172.21.15.55:3300/0 conn(0x7f7230071db0 0x7f7230158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7234c00640 1 -- 172.21.15.162:0/4106848533 >> v1:172.21.15.162:6789/0 conn(0x7f72300b3860 legacy=0x7f72300b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.082 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.077+0000 7f7234c00640 1 -- 172.21.15.162:0/4106848533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723015b940 con 0x7f723006a4b0 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.078+0000 7f7234c00640 1 --2- 172.21.15.162:0/4106848533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f72300ccf90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097a0 tx=0x7f721c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5a66540a81f15c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f722fe00640 1 -- 172.21.15.162:0/4106848533 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c02ae40 con 0x7f723006a4b0 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f722fe00640 1 -- 172.21.15.162:0/4106848533 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c032440 con 0x7f723006a4b0 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f722fe00640 1 -- 172.21.15.162:0/4106848533 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0327d0 con 0x7f723006a4b0 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 -- 172.21.15.162:0/4106848533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 msgr2=0x7f72300ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 --2- 172.21.15.162:0/4106848533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f72300ccf90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097a0 tx=0x7f721c02a0f0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 -- 172.21.15.162:0/4106848533 shutdown_connections 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 --2- 172.21.15.162:0/4106848533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f72300ccf90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 --2- 172.21.15.162:0/4106848533 >> v2:172.21.15.55:3300/0 conn(0x7f7230071db0 0x7f7230158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 -- 172.21.15.162:0/4106848533 >> 172.21.15.162:0/4106848533 conn(0x7f72300cbb90 msgr2=0x7f72300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.084 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 -- 172.21.15.162:0/4106848533 shutdown_connections 2024-05-10T03:29:19.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.079+0000 7f7238000640 1 -- 172.21.15.162:0/4106848533 wait complete. 2024-05-10T03:29:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.082+0000 7f7238000640 1 Processor -- start 2024-05-10T03:29:19.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.082+0000 7f7238000640 1 -- start start 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f7230156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72300b3860 0x7f72301309d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230131570 con 0x7f72300b3860 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300acb20 con 0x7f7230071db0 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.083+0000 7f7238000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230157500 con 0x7f723006a4b0 2024-05-10T03:29:19.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7235600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f7230156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58836/0 (socket says 172.21.15.162:58836) 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 -- 172.21.15.162:0/3542456206 learned_addr learned my addr 172.21.15.162:0/3542456206 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7236000640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72300b3860 0x7f72301309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 -- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 msgr2=0x7f7230156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f7230156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 -- 172.21.15.162:0/3542456206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72300b3860 msgr2=0x7f72301309d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72300b3860 0x7f72301309d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 -- 172.21.15.162:0/3542456206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009400 con 0x7f7230071db0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7235600640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f7230156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.084+0000 7f7234c00640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f721c035040 tx=0x7f721c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c03d8f0 con 0x7f7230071db0 2024-05-10T03:29:19.090 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c03da70 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c03de20 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72300acd70 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72300ad330 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f721c039040 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.085+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f721c03b4d0 con 0x7f7230071db0 2024-05-10T03:29:19.091 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.086+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7230066f60 con 0x7f7230071db0 2024-05-10T03:29:19.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.089+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f721c041070 con 0x7f7230071db0 2024-05-10T03:29:19.172 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.167+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "097e253b-6207-4031-9378-098510bfc2b2", "id": 0} v 0) v1 -- 0x7f7230067220 con 0x7f7230071db0 2024-05-10T03:29:19.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.176+0000 7f722d600640 1 -- 172.21.15.162:0/3542456206 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "097e253b-6207-4031-9378-098510bfc2b2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f721c0540b0 con 0x7f7230071db0 2024-05-10T03:29:19.180 INFO:teuthology.orchestra.run.smithi162.stdout:0 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.177+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 msgr2=0x7f72300ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.177+0000 7f7238000640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f721c035040 tx=0x7f721c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.177+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 shutdown_connections 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72300b3860 0x7f72301309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7230071db0 0x7f72300ade10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 --2- 172.21.15.162:0/3542456206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f723006a4b0 0x7f7230156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 >> 172.21.15.162:0/3542456206 conn(0x7f72300cbb90 msgr2=0x7f723015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 shutdown_connections 2024-05-10T03:29:19.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.178+0000 7f7238000640 1 -- 172.21.15.162:0/3542456206 wait complete. 2024-05-10T03:29:19.205 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new f73e2adc-a6b9-45d7-abc3-cb47949005ae 1 2024-05-10T03:29:19.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.443+0000 7fa4c9a00640 1 Processor -- start 2024-05-10T03:29:19.448 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.444+0000 7fa4c9a00640 1 -- start start 2024-05-10T03:29:19.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c9a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4c40b0c40 0x7fa4c40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.451 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c9a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4c40cc0a0 0x7fa4c40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c9a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40c7060 con 0x7fa4c40b1590 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c9a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40cd3b0 con 0x7fa4c40cc0a0 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c9a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40c7300 con 0x7fa4c40b0c40 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c2a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4c40b0c40 0x7fa4c40b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c2a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4c40b0c40 0x7fa4c40b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58852/0 (socket says 172.21.15.162:58852) 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c2000640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa4c40b1590 0x7fa4c40cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57778/0 (socket says 172.21.15.162:57778) 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.446+0000 7fa4c2a00640 1 -- 172.21.15.162:0/211725021 learned_addr learned my addr 172.21.15.162:0/211725021 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c3400640 1 --2- 172.21.15.162:0/211725021 >> v2:172.21.15.55:3300/0 conn(0x7fa4c40cc0a0 0x7fa4c40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c2a00640 1 -- 172.21.15.162:0/211725021 >> v2:172.21.15.55:3300/0 conn(0x7fa4c40cc0a0 msgr2=0x7fa4c40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c2a00640 1 --2- 172.21.15.162:0/211725021 >> v2:172.21.15.55:3300/0 conn(0x7fa4c40cc0a0 0x7fa4c40cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c2a00640 1 -- 172.21.15.162:0/211725021 >> v1:172.21.15.162:6789/0 conn(0x7fa4c40b1590 legacy=0x7fa4c40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c2a00640 1 -- 172.21.15.162:0/211725021 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40c75e0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.452 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.447+0000 7fa4c1600640 1 -- 172.21.15.162:0/211725021 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609776856 0 0) 0x7fa4c40c7060 con 0x7fa4c40b1590 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.448+0000 7fa4c2a00640 1 --2- 172.21.15.162:0/211725021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c40b1050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40cece0 tx=0x7fa4b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a412eb371c526d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.448+0000 7fa4c1600640 1 -- 172.21.15.162:0/211725021 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032440 con 0x7fa4c40b0c40 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.448+0000 7fa4c1600640 1 -- 172.21.15.162:0/211725021 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b40325a0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.448+0000 7fa4c1600640 1 -- 172.21.15.162:0/211725021 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032930 con 0x7fa4c40b0c40 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 -- 172.21.15.162:0/211725021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 msgr2=0x7fa4c40b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/211725021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c40b1050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40cece0 tx=0x7fa4b402a270 comp rx=0 tx=0).stop 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 -- 172.21.15.162:0/211725021 shutdown_connections 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/211725021 >> v2:172.21.15.55:3300/0 conn(0x7fa4c40cc0a0 0x7fa4c40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.454 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/211725021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c40b1050 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.455 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 -- 172.21.15.162:0/211725021 >> 172.21.15.162:0/211725021 conn(0x7fa4c40cdbe0 msgr2=0x7fa4c40cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.455 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 -- 172.21.15.162:0/211725021 shutdown_connections 2024-05-10T03:29:19.455 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.449+0000 7fa4c9a00640 1 -- 172.21.15.162:0/211725021 wait complete. 2024-05-10T03:29:19.456 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.452+0000 7fa4c9a00640 1 Processor -- start 2024-05-10T03:29:19.456 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.452+0000 7fa4c9a00640 1 -- start start 2024-05-10T03:29:19.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c40b1590 0x7fa4c414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4c40cc0a0 0x7fa4c415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c415d180 con 0x7fa4c40b1590 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c414c6a0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c9a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40b5840 con 0x7fa4c40cc0a0 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.453+0000 7fa4c2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58864/0 (socket says 172.21.15.162:58864) 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 -- 172.21.15.162:0/4099561923 learned_addr learned my addr 172.21.15.162:0/4099561923 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c3400640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4c40cc0a0 0x7fa4c415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2000640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c40b1590 0x7fa4c414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 -- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4c40cc0a0 msgr2=0x7fa4c415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4c40cc0a0 0x7fa4c415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.460 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 -- 172.21.15.162:0/4099561923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c40b1590 msgr2=0x7fa4c414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c40b1590 0x7fa4c414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 -- 172.21.15.162:0/4099561923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b40094d0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.454+0000 7fa4c2a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032440 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4030b80 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b403d420 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c414e4b0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c415cac0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa4b4039040 con 0x7fa4c40b0c40 2024-05-10T03:29:19.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.455+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa4b4046890 con 0x7fa4c40b0c40 2024-05-10T03:29:19.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.456+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c415d2c0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.458+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa4b4065dc0 con 0x7fa4c40b0c40 2024-05-10T03:29:19.503 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.499+0000 7fbf0de1f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:29:19.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.537+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f73e2adc-a6b9-45d7-abc3-cb47949005ae", "id": 1} v 0) v1 -- 0x7fa4c4067930 con 0x7fa4c40b0c40 2024-05-10T03:29:19.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.546+0000 7fa4baa00640 1 -- 172.21.15.162:0/4099561923 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f73e2adc-a6b9-45d7-abc3-cb47949005ae", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa4b4054030 con 0x7fa4c40b0c40 2024-05-10T03:29:19.550 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 msgr2=0x7fa4c415dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b40046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 shutdown_connections 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4c40cc0a0 0x7fa4c415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c40b1590 0x7fa4c414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 --2- 172.21.15.162:0/4099561923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4c40b0c40 0x7fa4c415dfd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 >> 172.21.15.162:0/4099561923 conn(0x7fa4c40cdbe0 msgr2=0x7fa4c415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 shutdown_connections 2024-05-10T03:29:19.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.548+0000 7fa4c9a00640 1 -- 172.21.15.162:0/4099561923 wait complete. 2024-05-10T03:29:19.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new de765a8c-93e6-4aba-a97e-06bfb1c026d0 2 2024-05-10T03:29:19.615 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.610+0000 7fbf0de1f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:29:19.670 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.665+0000 7fbf0de1f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:29:19.703 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:19.698+0000 7ff5b7f23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T03:29:19.820 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:19.815+0000 7ff5b7f23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T03:29:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.825+0000 7f3038a00640 1 Processor -- start 2024-05-10T03:29:19.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.826+0000 7f3038a00640 1 -- start start 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3038a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f303415d700 0x7f303415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3038a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 0x7f30340ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3038a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340b6d00 con 0x7f30340d1ba0 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3038a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303415e050 con 0x7f30340cdf70 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3038a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340b8ff0 con 0x7f303415d700 2024-05-10T03:29:19.833 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f302be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f303415d700 0x7f303415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f302be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f303415d700 0x7f303415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58868/0 (socket says 172.21.15.162:58868) 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f302be00640 1 -- 172.21.15.162:0/2207836938 learned_addr learned my addr 172.21.15.162:0/2207836938 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3032800640 1 --1- 172.21.15.162:0/2207836938 >> v1:172.21.15.162:6789/0 conn(0x7f30340d1ba0 0x7f30340b65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57782/0 (socket says 172.21.15.162:57782) 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f3033200640 1 --2- 172.21.15.162:0/2207836938 >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 0x7f30340ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.828+0000 7f302be00640 1 -- 172.21.15.162:0/2207836938 >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 msgr2=0x7f30340ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.829+0000 7f302be00640 1 --2- 172.21.15.162:0/2207836938 >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 0x7f30340ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.829+0000 7f302be00640 1 -- 172.21.15.162:0/2207836938 >> v1:172.21.15.162:6789/0 conn(0x7f30340d1ba0 legacy=0x7f30340b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.829+0000 7f302be00640 1 -- 172.21.15.162:0/2207836938 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340b7b50 con 0x7f303415d700 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.829+0000 7f3031e00640 1 -- 172.21.15.162:0/2207836938 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597356117 0 0) 0x7f3024001110 con 0x7f30340d1ba0 2024-05-10T03:29:19.834 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.829+0000 7f3033200640 1 --2- 172.21.15.162:0/2207836938 >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 0x7f30340ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:19.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.830+0000 7f302be00640 1 --2- 172.21.15.162:0/2207836938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f303415db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30340cfa10 tx=0x7f301c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7440665c8f00abba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.830+0000 7f3031e00640 1 -- 172.21.15.162:0/2207836938 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c032440 con 0x7f303415d700 2024-05-10T03:29:19.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.830+0000 7f3031e00640 1 -- 172.21.15.162:0/2207836938 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c0325a0 con 0x7f303415d700 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.830+0000 7f3031e00640 1 -- 172.21.15.162:0/2207836938 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c032930 con 0x7f303415d700 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 -- 172.21.15.162:0/2207836938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 msgr2=0x7f303415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 --2- 172.21.15.162:0/2207836938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f303415db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30340cfa10 tx=0x7f301c02a270 comp rx=0 tx=0).stop 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 -- 172.21.15.162:0/2207836938 shutdown_connections 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 --2- 172.21.15.162:0/2207836938 >> v2:172.21.15.55:3300/0 conn(0x7f30340cdf70 0x7f30340ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 --2- 172.21.15.162:0/2207836938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f303415db10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 -- 172.21.15.162:0/2207836938 >> 172.21.15.162:0/2207836938 conn(0x7f30340d0620 msgr2=0x7f30340d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 -- 172.21.15.162:0/2207836938 shutdown_connections 2024-05-10T03:29:19.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.831+0000 7f3038a00640 1 -- 172.21.15.162:0/2207836938 wait complete. 2024-05-10T03:29:19.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.834+0000 7f3038a00640 1 Processor -- start 2024-05-10T03:29:19.838 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.834+0000 7f3038a00640 1 -- start start 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.835+0000 7f3038a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.835+0000 7f3038a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 0x7f30340cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.835+0000 7f3038a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f30340cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3038a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c9cf0 con 0x7f30340d1ba0 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3038a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341e61e0 con 0x7f303415d700 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3038a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30341e69f0 con 0x7f30340cdf70 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3033200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f30340cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:38262/0 (socket says 172.21.15.162:38262) 2024-05-10T03:29:19.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 -- 172.21.15.162:0/1293533228 learned_addr learned my addr 172.21.15.162:0/1293533228 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f302be00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 0x7f30340cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 -- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 msgr2=0x7f30340cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f30340cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 -- 172.21.15.162:0/1293533228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 msgr2=0x7f30340cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 0x7f30340cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3032800640 1 -- 172.21.15.162:0/1293533228 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0094d0 con 0x7f30340cdf70 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f3033200640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f30340cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.836+0000 7f302be00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 0x7f30340cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f3032800640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f302400c430 tx=0x7f302400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:19.843 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240074e0 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024007660 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30340c81f0 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024004a20 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341e6710 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f302401d3f0 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.837+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f3024013830 con 0x7f30340cdf70 2024-05-10T03:29:19.844 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.838+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303404e290 con 0x7f30340cdf70 2024-05-10T03:29:19.845 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.841+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f30340b6d00 con 0x7f30340cdf70 2024-05-10T03:29:19.876 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.872+0000 7fbf0de1f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:29:19.877 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:19.872+0000 7ff5b7f23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T03:29:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.919+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "de765a8c-93e6-4aba-a97e-06bfb1c026d0", "id": 2} v 0) v1 -- 0x7f30340d14a0 con 0x7f30340cdf70 2024-05-10T03:29:19.934 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.930+0000 7f302aa00640 1 -- 172.21.15.162:0/1293533228 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de765a8c-93e6-4aba-a97e-06bfb1c026d0", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3024013580 con 0x7f30340cdf70 2024-05-10T03:29:19.935 INFO:teuthology.orchestra.run.smithi162.stdout:2 2024-05-10T03:29:19.935 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 msgr2=0x7f30341e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f302400c430 tx=0x7f302400c8f0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 shutdown_connections 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f303415d700 0x7f30340cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30340d1ba0 0x7f30340cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 --2- 172.21.15.162:0/1293533228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30340cdf70 0x7f30341e5ac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.931+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 >> 172.21.15.162:0/1293533228 conn(0x7f30340d0620 msgr2=0x7f30341e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:19.936 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.932+0000 7fbf0de1f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:29:19.936 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.932+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 shutdown_connections 2024-05-10T03:29:19.937 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:19.932+0000 7f3038a00640 1 -- 172.21.15.162:0/1293533228 wait complete. 2024-05-10T03:29:19.959 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 60999af4-c269-4ee9-8303-8441b4aed779 3 2024-05-10T03:29:20.007 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:19.996+0000 7fbf0de1f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:29:20.090 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:20.085+0000 7ff5b7f23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T03:29:20.137 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:20.133+0000 7fbf0de1f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:29:20.159 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:20.155+0000 7ff5b7f23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T03:29:20.225 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:20.220+0000 7ff5b7f23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T03:29:20.253 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.249+0000 7f9b15e00640 1 Processor -- start 2024-05-10T03:29:20.253 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.249+0000 7f9b15e00640 1 -- start start 2024-05-10T03:29:20.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b100b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b1015d700 0x7f9b1015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100b7b50 con 0x7f9b100cdf70 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1015e050 con 0x7f9b1015d700 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100b8ff0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15400640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9b100cdf70 0x7f9b100ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57790/0 (socket says 172.21.15.162:57790) 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b15400640 1 -- 172.21.15.162:0/127504532 learned_addr learned my addr 172.21.15.162:0/127504532 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b14a00640 1 --2- 172.21.15.162:0/127504532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b100b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b0fe00640 1 --2- 172.21.15.162:0/127504532 >> v2:172.21.15.55:3300/0 conn(0x7f9b1015d700 0x7f9b1015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.251+0000 7f9b0f400640 1 -- 172.21.15.162:0/127504532 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973844025 0 0) 0x7f9b100b7b50 con 0x7f9b100cdf70 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.252+0000 7f9b14a00640 1 -- 172.21.15.162:0/127504532 >> v2:172.21.15.55:3300/0 conn(0x7f9b1015d700 msgr2=0x7f9b1015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.252+0000 7f9b14a00640 1 --2- 172.21.15.162:0/127504532 >> v2:172.21.15.55:3300/0 conn(0x7f9b1015d700 0x7f9b1015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.252+0000 7f9b14a00640 1 -- 172.21.15.162:0/127504532 >> v1:172.21.15.162:6789/0 conn(0x7f9b100cdf70 legacy=0x7f9b100ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.252+0000 7f9b14a00640 1 -- 172.21.15.162:0/127504532 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100b6f40 con 0x7f9b100d1ba0 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.253+0000 7f9b14a00640 1 --2- 172.21.15.162:0/127504532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b100b65f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b040097a0 tx=0x7f9b0402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdfb04bede09ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:20.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.253+0000 7f9b0f400640 1 -- 172.21.15.162:0/127504532 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04032440 con 0x7f9b100d1ba0 2024-05-10T03:29:20.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.253+0000 7f9b0f400640 1 -- 172.21.15.162:0/127504532 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b040325a0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.253+0000 7f9b0f400640 1 -- 172.21.15.162:0/127504532 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04032930 con 0x7f9b100d1ba0 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 -- 172.21.15.162:0/127504532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 msgr2=0x7f9b100b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 --2- 172.21.15.162:0/127504532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b100b65f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b040097a0 tx=0x7f9b0402a180 comp rx=0 tx=0).stop 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 -- 172.21.15.162:0/127504532 shutdown_connections 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 --2- 172.21.15.162:0/127504532 >> v2:172.21.15.55:3300/0 conn(0x7f9b1015d700 0x7f9b1015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 --2- 172.21.15.162:0/127504532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b100b65f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 -- 172.21.15.162:0/127504532 >> 172.21.15.162:0/127504532 conn(0x7f9b100d0620 msgr2=0x7f9b100d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 -- 172.21.15.162:0/127504532 shutdown_connections 2024-05-10T03:29:20.259 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.254+0000 7f9b15e00640 1 -- 172.21.15.162:0/127504532 wait complete. 2024-05-10T03:29:20.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.256+0000 7f9b15e00640 1 Processor -- start 2024-05-10T03:29:20.261 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.257+0000 7f9b15e00640 1 -- start start 2024-05-10T03:29:20.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 0x7f9b1014c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.264 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b1015d700 0x7f9b1015fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100c9ea0 con 0x7f9b1015d700 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b100cb600 con 0x7f9b100cdf70 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b15e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101f7d40 con 0x7f9b100d1ba0 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b14a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 0x7f9b1014c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b0fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:38276/0 (socket says 172.21.15.162:38276) 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.258+0000 7f9b0fe00640 1 -- 172.21.15.162:0/392429073 learned_addr learned my addr 172.21.15.162:0/392429073 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b15400640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b1015d700 0x7f9b1015fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 -- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 msgr2=0x7f9b1014c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 0x7f9b1014c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 -- 172.21.15.162:0/392429073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b1015d700 msgr2=0x7f9b1015fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b1015d700 0x7f9b1015fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 -- 172.21.15.162:0/392429073 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04009400 con 0x7f9b100d1ba0 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b14a00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 0x7f9b1014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:20.265 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0fe00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00d4f0 tx=0x7f9afc00d9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc004240 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc0043a0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc003850 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.259+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b100c92b0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.260+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b100c97e0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.260+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9afc003cb0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.260+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9afc012ba0 con 0x7f9b100d1ba0 2024-05-10T03:29:20.266 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.261+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b100ca110 con 0x7f9b100d1ba0 2024-05-10T03:29:20.267 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.263+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9afc012e60 con 0x7f9b100d1ba0 2024-05-10T03:29:20.346 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.342+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "60999af4-c269-4ee9-8303-8441b4aed779", "id": 3} v 0) v1 -- 0x7f9b1004e290 con 0x7f9b100d1ba0 2024-05-10T03:29:20.353 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.349+0000 7f9b0cc00640 1 -- 172.21.15.162:0/392429073 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60999af4-c269-4ee9-8303-8441b4aed779", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9afc012420 con 0x7f9b100d1ba0 2024-05-10T03:29:20.353 INFO:teuthology.orchestra.run.smithi162.stdout:3 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 msgr2=0x7f9b101f7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00d4f0 tx=0x7f9afc00d9b0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 shutdown_connections 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b1015d700 0x7f9b1015fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b100d1ba0 0x7f9b101f7220 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 --2- 172.21.15.162:0/392429073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b100cdf70 0x7f9b1014c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.350+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 >> 172.21.15.162:0/392429073 conn(0x7f9b100d0620 msgr2=0x7f9b100b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:20.355 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.351+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 shutdown_connections 2024-05-10T03:29:20.356 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.351+0000 7f9b15e00640 1 -- 172.21.15.162:0/392429073 wait complete. 2024-05-10T03:29:20.358 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:20.353+0000 7ff5b7f23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T03:29:20.379 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new af007dd1-0644-42f1-9149-ae514bb1d110 4 2024-05-10T03:29:20.629 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:20.625+0000 7fbf0de1f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:29:20.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.633+0000 7f86ef200640 1 Processor -- start 2024-05-10T03:29:20.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.633+0000 7f86ef200640 1 -- start start 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.635+0000 7f86ef200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86f00ce940 0x7f86f015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.635+0000 7f86ef200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f86f00ce490 0x7f86f006a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.635+0000 7f86ef200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f00b1dd0 con 0x7f86f00d2430 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.635+0000 7f86ef200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f00b20b0 con 0x7f86f00ce490 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.635+0000 7f86ef200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f00b2390 con 0x7f86f00ce940 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86e7e00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f86f00d2430 0x7f86f00cdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57792/0 (socket says 172.21.15.162:57792) 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86e7e00640 1 -- 172.21.15.162:0/3713610292 learned_addr learned my addr 172.21.15.162:0/3713610292 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-05-10T03:29:20.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ede00640 1 --2- 172.21.15.162:0/3713610292 >> v2:172.21.15.162:3300/0 conn(0x7f86f00ce940 0x7f86f015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ee800640 1 --2- 172.21.15.162:0/3713610292 >> v2:172.21.15.55:3300/0 conn(0x7f86f00ce490 0x7f86f006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ed400640 1 -- 172.21.15.162:0/3713610292 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504243533 0 0) 0x7f86f00b1dd0 con 0x7f86f00d2430 2024-05-10T03:29:20.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ed400640 1 -- 172.21.15.162:0/3713610292 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d4003520 con 0x7f86f00d2430 2024-05-10T03:29:20.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ede00640 1 -- 172.21.15.162:0/3713610292 >> v2:172.21.15.55:3300/0 conn(0x7f86f00ce490 msgr2=0x7f86f006a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:20.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ede00640 1 --2- 172.21.15.162:0/3713610292 >> v2:172.21.15.55:3300/0 conn(0x7f86f00ce490 0x7f86f006a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ede00640 1 -- 172.21.15.162:0/3713610292 >> v1:172.21.15.162:6789/0 conn(0x7f86f00d2430 legacy=0x7f86f00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.636+0000 7f86ede00640 1 -- 172.21.15.162:0/3713610292 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f006a110 con 0x7f86f00ce940 2024-05-10T03:29:20.642 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.637+0000 7f86ede00640 1 --2- 172.21.15.162:0/3713610292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00ce940 0x7f86f015d680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f86f00d0da0 tx=0x7f86e002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f24e8b0e3e2604 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.638+0000 7f86ed400640 1 -- 172.21.15.162:0/3713610292 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e002ae60 con 0x7f86f00ce940 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.638+0000 7f86ed400640 1 -- 172.21.15.162:0/3713610292 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e0032440 con 0x7f86f00ce940 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.638+0000 7f86ed400640 1 -- 172.21.15.162:0/3713610292 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00327d0 con 0x7f86f00ce940 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 -- 172.21.15.162:0/3713610292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00ce940 msgr2=0x7f86f015d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 --2- 172.21.15.162:0/3713610292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00ce940 0x7f86f015d680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f86f00d0da0 tx=0x7f86e002a110 comp rx=0 tx=0).stop 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 -- 172.21.15.162:0/3713610292 shutdown_connections 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 --2- 172.21.15.162:0/3713610292 >> v2:172.21.15.55:3300/0 conn(0x7f86f00ce490 0x7f86f006a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 --2- 172.21.15.162:0/3713610292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00ce940 0x7f86f015d680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 -- 172.21.15.162:0/3713610292 >> 172.21.15.162:0/3713610292 conn(0x7f86f00b5770 msgr2=0x7f86f00d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:20.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 -- 172.21.15.162:0/3713610292 shutdown_connections 2024-05-10T03:29:20.644 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.639+0000 7f86ef200640 1 -- 172.21.15.162:0/3713610292 wait complete. 2024-05-10T03:29:20.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.641+0000 7f86ef200640 1 Processor -- start 2024-05-10T03:29:20.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.641+0000 7f86ef200640 1 -- start start 2024-05-10T03:29:20.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 0x7f86f01f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0156540 0x7f86f0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0157a00 con 0x7f86f0156540 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00c66f0 con 0x7f86f00d2430 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ef200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0156e90 con 0x7f86f00ce940 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ede00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 0x7f86f01f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58916/0 (socket says 172.21.15.162:58916) 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 -- 172.21.15.162:0/1416951580 learned_addr learned my addr 172.21.15.162:0/1416951580 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ee800640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0156540 0x7f86f0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 -- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 msgr2=0x7f86f01f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 0x7f86f01f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 -- 172.21.15.162:0/1416951580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0156540 msgr2=0x7f86f0156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0156540 0x7f86f0156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86e7e00640 1 -- 172.21.15.162:0/1416951580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009400 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.643+0000 7f86ede00640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 0x7f86f01f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86e7e00640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c440 tx=0x7f86d800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8005ce0 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8005e40 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f015e4c0 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8007270 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f00c68f0 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.644+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f86d801c440 con 0x7f86f00d2430 2024-05-10T03:29:20.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.645+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f86d801cc00 con 0x7f86f00d2430 2024-05-10T03:29:20.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.645+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f004e290 con 0x7f86f00d2430 2024-05-10T03:29:20.652 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.648+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f86d8021060 con 0x7f86f00d2430 2024-05-10T03:29:20.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.727+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "af007dd1-0644-42f1-9149-ae514bb1d110", "id": 4} v 0) v1 -- 0x7f86f00d14b0 con 0x7f86f00d2430 2024-05-10T03:29:20.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.735+0000 7f86e6000640 1 -- 172.21.15.162:0/1416951580 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af007dd1-0644-42f1-9149-ae514bb1d110", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f86d8017070 con 0x7f86f00d2430 2024-05-10T03:29:20.740 INFO:teuthology.orchestra.run.smithi162.stdout:4 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.736+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 msgr2=0x7f86f0156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.736+0000 7f86ef200640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c440 tx=0x7f86d800c900 comp rx=0 tx=0).stop 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 shutdown_connections 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0156540 0x7f86f0156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f00d2430 0x7f86f0156000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 --2- 172.21.15.162:0/1416951580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f00ce940 0x7f86f01f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:20.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 >> 172.21.15.162:0/1416951580 conn(0x7f86f00b5770 msgr2=0x7f86f00d3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:20.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 shutdown_connections 2024-05-10T03:29:20.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:20.737+0000 7f86ef200640 1 -- 172.21.15.162:0/1416951580 wait complete. 2024-05-10T03:29:20.765 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 9bcde535-4e7a-4394-8fd3-bf35c240183e 5 2024-05-10T03:29:20.838 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:20.833+0000 7ff5b7f23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T03:29:20.945 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:20.941+0000 7fbf0de1f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:29:21.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.002+0000 7f20f5e00640 1 Processor -- start 2024-05-10T03:29:21.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.003+0000 7f20f5e00640 1 -- start start 2024-05-10T03:29:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.004+0000 7f20f5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20f015d680 0x7f20f00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.004+0000 7f20f5e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f20f00ce2c0 0x7f20f006a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.004+0000 7f20f5e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f015da90 con 0x7f20f00ce940 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.004+0000 7f20f5e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00ce6d0 con 0x7f20f015d680 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.004+0000 7f20f5e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00b1dd0 con 0x7f20f00ce2c0 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f4a00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f20f00ce940 0x7f20f00d2430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57800/0 (socket says 172.21.15.162:57800) 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f4a00640 1 -- 172.21.15.162:0/2722842555 learned_addr learned my addr 172.21.15.162:0/2722842555 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f5400640 1 --2- 172.21.15.162:0/2722842555 >> v2:172.21.15.162:3300/0 conn(0x7f20f00ce2c0 0x7f20f006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20efe00640 1 --2- 172.21.15.162:0/2722842555 >> v2:172.21.15.55:3300/0 conn(0x7f20f015d680 0x7f20f00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20ef400640 1 -- 172.21.15.162:0/2722842555 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857328594 0 0) 0x7f20f015da90 con 0x7f20f00ce940 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20ef400640 1 -- 172.21.15.162:0/2722842555 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d8003520 con 0x7f20f00ce940 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f5400640 1 -- 172.21.15.162:0/2722842555 >> v2:172.21.15.55:3300/0 conn(0x7f20f015d680 msgr2=0x7f20f00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f5400640 1 --2- 172.21.15.162:0/2722842555 >> v2:172.21.15.55:3300/0 conn(0x7f20f015d680 0x7f20f00cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f5400640 1 -- 172.21.15.162:0/2722842555 >> v1:172.21.15.162:6789/0 conn(0x7f20f00ce940 legacy=0x7f20f00d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.005+0000 7f20f5400640 1 -- 172.21.15.162:0/2722842555 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00b20b0 con 0x7f20f00ce2c0 2024-05-10T03:29:21.011 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.006+0000 7fbf0de1f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:29:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.007+0000 7f20f5400640 1 --2- 172.21.15.162:0/2722842555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f00ce2c0 0x7f20f006a910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ca0a0 tx=0x7f20e802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d8d57e19f100c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.007+0000 7f20ef400640 1 -- 172.21.15.162:0/2722842555 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007740 con 0x7f20f00ce2c0 2024-05-10T03:29:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.007+0000 7f20ef400640 1 -- 172.21.15.162:0/2722842555 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e802d050 con 0x7f20f00ce2c0 2024-05-10T03:29:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.007+0000 7f20ef400640 1 -- 172.21.15.162:0/2722842555 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007ac0 con 0x7f20f00ce2c0 2024-05-10T03:29:21.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 -- 172.21.15.162:0/2722842555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f00ce2c0 msgr2=0x7f20f006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 --2- 172.21.15.162:0/2722842555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f00ce2c0 0x7f20f006a910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ca0a0 tx=0x7f20e802a5f0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 -- 172.21.15.162:0/2722842555 shutdown_connections 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 --2- 172.21.15.162:0/2722842555 >> v2:172.21.15.55:3300/0 conn(0x7f20f015d680 0x7f20f00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 --2- 172.21.15.162:0/2722842555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f00ce2c0 0x7f20f006a910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 -- 172.21.15.162:0/2722842555 >> 172.21.15.162:0/2722842555 conn(0x7f20f00b5770 msgr2=0x7f20f00d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 -- 172.21.15.162:0/2722842555 shutdown_connections 2024-05-10T03:29:21.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.008+0000 7f20f5e00640 1 -- 172.21.15.162:0/2722842555 wait complete. 2024-05-10T03:29:21.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.010+0000 7f20f5e00640 1 Processor -- start 2024-05-10T03:29:21.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.010+0000 7f20f5e00640 1 -- start start 2024-05-10T03:29:21.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20f00ce2c0 0x7f20f01f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f01556b0 con 0x7f20f00ce2c0 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f01f3f10 con 0x7f20f015d680 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0154a80 con 0x7f20f00ce940 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20efe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20efe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:38288/0 (socket says 172.21.15.162:38288) 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58936/0 (socket says 172.21.15.162:58936) 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20efe00640 1 -- 172.21.15.162:0/3861642775 learned_addr learned my addr 172.21.15.162:0/3861642775 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.012+0000 7f20f4a00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20f00ce2c0 0x7f20f01f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 -- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 msgr2=0x7f20f0154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 -- 172.21.15.162:0/3861642775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20f00ce2c0 msgr2=0x7f20f01f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20f00ce2c0 0x7f20f01f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 -- 172.21.15.162:0/3861642775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8009a70 con 0x7f20f015d680 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20efe00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:21.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5400640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f20f00d4080 tx=0x7f20e8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e803b820 con 0x7f20f015d680 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e803b980 con 0x7f20f015d680 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e803bcf0 con 0x7f20f015d680 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.013+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f0160de0 con 0x7f20f015d680 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.014+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f0155ba0 con 0x7f20f015d680 2024-05-10T03:29:21.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.014+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f20e8003c90 con 0x7f20f015d680 2024-05-10T03:29:21.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.014+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f20e80047a0 con 0x7f20f015d680 2024-05-10T03:29:21.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.015+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f004e290 con 0x7f20f015d680 2024-05-10T03:29:21.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.017+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f20e8049060 con 0x7f20f015d680 2024-05-10T03:29:21.100 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.096+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9bcde535-4e7a-4394-8fd3-bf35c240183e", "id": 5} v 0) v1 -- 0x7f20f00d14b0 con 0x7f20f015d680 2024-05-10T03:29:21.108 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.104+0000 7f20ecc00640 1 -- 172.21.15.162:0/3861642775 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9bcde535-4e7a-4394-8fd3-bf35c240183e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f20e803f070 con 0x7f20f015d680 2024-05-10T03:29:21.108 INFO:teuthology.orchestra.run.smithi162.stdout:5 2024-05-10T03:29:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 msgr2=0x7f20f01608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f20f00d4080 tx=0x7f20e8004a40 comp rx=0 tx=0).stop 2024-05-10T03:29:21.109 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 shutdown_connections 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f015d680 0x7f20f01608a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f00ce940 0x7f20f0154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 --2- 172.21.15.162:0/3861642775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20f00ce2c0 0x7f20f01f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 >> 172.21.15.162:0/3861642775 conn(0x7f20f00b5770 msgr2=0x7f20f006a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 shutdown_connections 2024-05-10T03:29:21.110 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.105+0000 7f20f5e00640 1 -- 172.21.15.162:0/3861642775 wait complete. 2024-05-10T03:29:21.134 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 877cf826-8802-4a9f-815c-d81c9aafff27 6 2024-05-10T03:29:21.145 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.140+0000 7ff5b7f23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T03:29:21.153 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.148+0000 7fbf0de1f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:29:21.208 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.203+0000 7ff5b7f23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T03:29:21.345 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.340+0000 7ff5b7f23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T03:29:21.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.376+0000 7ff9d0c00640 1 Processor -- start 2024-05-10T03:29:21.381 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.377+0000 7ff9d0c00640 1 -- start start 2024-05-10T03:29:21.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.378+0000 7ff9d0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 0x7ff9cc142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.378+0000 7ff9d0c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc06ad20 con 0x7ff9cc15fe50 2024-05-10T03:29:21.383 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.378+0000 7ff9d0c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc06a110 con 0x7ff9cc0b2550 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9d0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc06a3f0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 0x7ff9cc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9ca800640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff9cc0b2550 0x7ff9cc06b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57810/0 (socket says 172.21.15.162:57810) 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9ca800640 1 -- 172.21.15.162:0/1665458137 learned_addr learned my addr 172.21.15.162:0/1665458137 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9c3e00640 1 -- 172.21.15.162:0/1665458137 >> v1:172.21.15.162:6789/0 conn(0x7ff9cc0b2550 legacy=0x7ff9cc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9c3e00640 1 -- 172.21.15.162:0/1665458137 >> v1:172.21.15.55:6789/0 conn(0x7ff9cc15fe50 legacy=0x7ff9cc141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9c9400640 1 -- 172.21.15.162:0/1665458137 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643118748 0 0) 0x7ff9cc06a110 con 0x7ff9cc0b2550 2024-05-10T03:29:21.384 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.379+0000 7ff9c3e00640 1 -- 172.21.15.162:0/1665458137 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc06a6d0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.380+0000 7ff9c3e00640 1 --2- 172.21.15.162:0/1665458137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 0x7ff9cc142440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40097d0 tx=0x7ff9b402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5da6eedf7f4f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.381+0000 7ff9c9400640 1 -- 172.21.15.162:0/1665458137 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4032440 con 0x7ff9cc0b2140 2024-05-10T03:29:21.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.381+0000 7ff9c9400640 1 -- 172.21.15.162:0/1665458137 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b40325a0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.385 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.381+0000 7ff9c9400640 1 -- 172.21.15.162:0/1665458137 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4032910 con 0x7ff9cc0b2140 2024-05-10T03:29:21.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.381+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1665458137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 msgr2=0x7ff9cc142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.386 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.381+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1665458137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 0x7ff9cc142440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40097d0 tx=0x7ff9b402a290 comp rx=0 tx=0).stop 2024-05-10T03:29:21.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.382+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1665458137 shutdown_connections 2024-05-10T03:29:21.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.382+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1665458137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc0b2140 0x7ff9cc142440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.382+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1665458137 >> 172.21.15.162:0/1665458137 conn(0x7ff9cc0cbb90 msgr2=0x7ff9cc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.382+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1665458137 shutdown_connections 2024-05-10T03:29:21.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.382+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1665458137 wait complete. 2024-05-10T03:29:21.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.385+0000 7ff9d0c00640 1 Processor -- start 2024-05-10T03:29:21.390 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.386+0000 7ff9d0c00640 1 -- start start 2024-05-10T03:29:21.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0b2550 0x7ff9cc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.394 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc15fe50 0x7ff9cc13af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc071f60 con 0x7ff9cc0b2550 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15d2f0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9d0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc070c70 con 0x7ff9cc15fe50 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.388+0000 7ff9c9e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58946/0 (socket says 172.21.15.162:58946) 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 -- 172.21.15.162:0/1356673393 learned_addr learned my addr 172.21.15.162:0/1356673393 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9ca800640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc15fe50 0x7ff9cc13af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 -- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc15fe50 msgr2=0x7ff9cc13af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c3e00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0b2550 0x7ff9cc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc15fe50 0x7ff9cc13af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 -- 172.21.15.162:0/1356673393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0b2550 msgr2=0x7ff9cc15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.395 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0b2550 0x7ff9cc15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 -- 172.21.15.162:0/1356673393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4009400 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.389+0000 7ff9c9e00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c400 tx=0x7ff9bc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0052d0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9bc005430 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc070a20 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc003830 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc0711a0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff9bc003c90 con 0x7ff9cc0b2140 2024-05-10T03:29:21.396 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.390+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff9bc012420 con 0x7ff9cc0b2140 2024-05-10T03:29:21.397 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.391+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc0ceee0 con 0x7ff9cc0b2140 2024-05-10T03:29:21.397 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.393+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff9cc070a20 con 0x7ff9cc0b2140 2024-05-10T03:29:21.476 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.472+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "877cf826-8802-4a9f-815c-d81c9aafff27", "id": 6} v 0) v1 -- 0x7ff9cc066f60 con 0x7ff9cc0b2140 2024-05-10T03:29:21.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.480+0000 7ff9c2000640 1 -- 172.21.15.162:0/1356673393 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "877cf826-8802-4a9f-815c-d81c9aafff27", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9bc017070 con 0x7ff9cc0b2140 2024-05-10T03:29:21.484 INFO:teuthology.orchestra.run.smithi162.stdout:6 2024-05-10T03:29:21.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 msgr2=0x7ff9cc0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c400 tx=0x7ff9bc00c8c0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 shutdown_connections 2024-05-10T03:29:21.486 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9cc15fe50 0x7ff9cc13af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9cc0b2550 0x7ff9cc15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 --2- 172.21.15.162:0/1356673393 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9cc0b2140 0x7ff9cc0704e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 >> 172.21.15.162:0/1356673393 conn(0x7ff9cc0cbb90 msgr2=0x7ff9cc15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 shutdown_connections 2024-05-10T03:29:21.487 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.482+0000 7ff9d0c00640 1 -- 172.21.15.162:0/1356673393 wait complete. 2024-05-10T03:29:21.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 0051db81-3299-4de5-a2af-a56a1e4c9ba7 7 2024-05-10T03:29:21.518 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.514+0000 7fbf0de1f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:29:21.578 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.574+0000 7fbf0de1f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:29:21.687 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.682+0000 7ff5b7f23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T03:29:21.745 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.740+0000 7ff5b7f23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T03:29:21.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.766+0000 7f2a3ea00640 1 Processor -- start 2024-05-10T03:29:21.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.766+0000 7f2a3ea00640 1 -- start start 2024-05-10T03:29:21.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a380d1ba0 0x7f2a380b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3ea00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a380ce680 0x7f2a3815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3ea00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380b7b50 con 0x7f2a3815d700 2024-05-10T03:29:21.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3ea00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3815e050 con 0x7f2a380d1ba0 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3ea00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380b8ff0 con 0x7f2a380ce680 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a380ce680 0x7f2a3815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.768+0000 7f2a3e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a380ce680 0x7f2a3815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58952/0 (socket says 172.21.15.162:58952) 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3e000640 1 -- 172.21.15.162:0/4280448825 learned_addr learned my addr 172.21.15.162:0/4280448825 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3d600640 1 --2- 172.21.15.162:0/4280448825 >> v2:172.21.15.55:3300/0 conn(0x7f2a380d1ba0 0x7f2a380b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3e000640 1 -- 172.21.15.162:0/4280448825 >> v2:172.21.15.55:3300/0 conn(0x7f2a380d1ba0 msgr2=0x7f2a380b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3e000640 1 --2- 172.21.15.162:0/4280448825 >> v2:172.21.15.55:3300/0 conn(0x7f2a380d1ba0 0x7f2a380b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3e000640 1 -- 172.21.15.162:0/4280448825 >> v1:172.21.15.55:6789/0 conn(0x7f2a3815d700 legacy=0x7f2a380cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:21.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.769+0000 7f2a3e000640 1 -- 172.21.15.162:0/4280448825 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380b6f40 con 0x7f2a380ce680 2024-05-10T03:29:21.774 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.769+0000 7fbf0de1f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.770+0000 7f2a3e000640 1 --2- 172.21.15.162:0/4280448825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a380ce680 0x7f2a3815db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2a380d3e00 tx=0x7f2a3002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835382cbb2144393 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.770+0000 7f2a37e00640 1 -- 172.21.15.162:0/4280448825 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30031600 con 0x7f2a380ce680 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.770+0000 7f2a37e00640 1 -- 172.21.15.162:0/4280448825 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3002d050 con 0x7f2a380ce680 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.770+0000 7f2a37e00640 1 -- 172.21.15.162:0/4280448825 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30031980 con 0x7f2a380ce680 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 -- 172.21.15.162:0/4280448825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a380ce680 msgr2=0x7f2a3815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/4280448825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a380ce680 0x7f2a3815db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2a380d3e00 tx=0x7f2a3002a6f0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 -- 172.21.15.162:0/4280448825 shutdown_connections 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/4280448825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a380ce680 0x7f2a3815db10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/4280448825 >> v2:172.21.15.55:3300/0 conn(0x7f2a380d1ba0 0x7f2a380b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 -- 172.21.15.162:0/4280448825 >> 172.21.15.162:0/4280448825 conn(0x7f2a380d0620 msgr2=0x7f2a380d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 -- 172.21.15.162:0/4280448825 shutdown_connections 2024-05-10T03:29:21.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.771+0000 7f2a3ea00640 1 -- 172.21.15.162:0/4280448825 wait complete. 2024-05-10T03:29:21.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.774+0000 7f2a3ea00640 1 Processor -- start 2024-05-10T03:29:21.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.774+0000 7f2a3ea00640 1 -- start start 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a380d1ba0 0x7f2a3814d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a38157eb0 0x7f2a381e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381e77e0 con 0x7f2a380d1ba0 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38158ce0 con 0x7f2a3815d700 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3ea00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38159f00 con 0x7f2a38157eb0 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.775+0000 7f2a3cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:58966/0 (socket says 172.21.15.162:58966) 2024-05-10T03:29:21.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 -- 172.21.15.162:0/995984123 learned_addr learned my addr 172.21.15.162:0/995984123 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3e000640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a38157eb0 0x7f2a381e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3d600640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a380d1ba0 0x7f2a3814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 -- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a38157eb0 msgr2=0x7f2a381e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a38157eb0 0x7f2a381e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 -- 172.21.15.162:0/995984123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a380d1ba0 msgr2=0x7f2a3814d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a380d1ba0 0x7f2a3814d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 -- 172.21.15.162:0/995984123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009b30 con 0x7f2a3815d700 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.776+0000 7f2a3cc00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b3f0 tx=0x7f2a2400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24011830 con 0x7f2a3815d700 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24011990 con 0x7f2a3815d700 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a381e6910 con 0x7f2a3815d700 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24011d40 con 0x7f2a3815d700 2024-05-10T03:29:21.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3814d970 con 0x7f2a3815d700 2024-05-10T03:29:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2a24004320 con 0x7f2a3815d700 2024-05-10T03:29:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.777+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f2a2401a400 con 0x7f2a3815d700 2024-05-10T03:29:21.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.778+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a381e7920 con 0x7f2a3815d700 2024-05-10T03:29:21.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.780+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f2a24038dc0 con 0x7f2a3815d700 2024-05-10T03:29:21.863 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.859+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0051db81-3299-4de5-a2af-a56a1e4c9ba7", "id": 7} v 0) v1 -- 0x7f2a3804e290 con 0x7f2a3815d700 2024-05-10T03:29:21.871 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.867+0000 7f2a35600640 1 -- 172.21.15.162:0/995984123 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0051db81-3299-4de5-a2af-a56a1e4c9ba7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a24027070 con 0x7f2a3815d700 2024-05-10T03:29:21.872 INFO:teuthology.orchestra.run.smithi162.stdout:7 2024-05-10T03:29:21.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 msgr2=0x7f2a381e4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:21.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b3f0 tx=0x7f2a2400b8b0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 shutdown_connections 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a38157eb0 0x7f2a381e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a3815d700 0x7f2a381e4740 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 --2- 172.21.15.162:0/995984123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a380d1ba0 0x7f2a3814d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 >> 172.21.15.162:0/995984123 conn(0x7f2a380d0620 msgr2=0x7f2a3815c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 shutdown_connections 2024-05-10T03:29:21.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-05-10T03:29:21.869+0000 7f2a3ea00640 1 -- 172.21.15.162:0/995984123 wait complete. 2024-05-10T03:29:21.896 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-10T03:29:21.896 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-10T03:29:21.904 INFO:tasks.ceph.osd.0:Started 2024-05-10T03:29:21.904 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-10T03:29:21.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-10T03:29:21.912 INFO:tasks.ceph.osd.1:Started 2024-05-10T03:29:21.912 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-10T03:29:21.912 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-10T03:29:21.916 INFO:tasks.ceph.osd.2:Started 2024-05-10T03:29:21.917 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-10T03:29:21.917 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-10T03:29:21.922 INFO:tasks.ceph.osd.3:Started 2024-05-10T03:29:21.922 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-10T03:29:21.922 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-10T03:29:21.927 INFO:tasks.ceph.osd.4:Started 2024-05-10T03:29:21.928 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-10T03:29:21.928 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-10T03:29:21.933 INFO:tasks.ceph.osd.5:Started 2024-05-10T03:29:21.933 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-10T03:29:21.933 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-10T03:29:21.938 INFO:tasks.ceph.osd.6:Started 2024-05-10T03:29:21.938 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-10T03:29:21.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-10T03:29:21.943 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:21.939+0000 7ff5b7f23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T03:29:21.944 INFO:tasks.ceph.osd.7:Started 2024-05-10T03:29:21.944 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:21.957 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:21.953+0000 7fbf0de1f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:29:22.052 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:22.048+0000 7fbf0de1f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:29:22.155 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:22.151+0000 7fbf0de1f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:29:22.200 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:22.195+0000 7ff5b7f23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T03:29:22.216 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:22.211+0000 7fbf0de1f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:29:22.272 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:22.267+0000 7ff5b7f23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T03:29:22.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.275+0000 7f2202000640 1 Processor -- start 2024-05-10T03:29:22.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.275+0000 7f2202000640 1 -- start start 2024-05-10T03:29:22.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f2202000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21fc15fe50 0x7f21fc141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:22.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f2202000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:22.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f2202000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc06ad20 con 0x7f21fc0b2550 2024-05-10T03:29:22.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f2202000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21fc141f50 con 0x7f21fc15fe50 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f2202000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc06a110 con 0x7f21fc0b2140 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f21faa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f21fb400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21fc15fe50 0x7f21fc141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f21fbe00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f21fc0b2550 0x7f21fc06b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:42652/0 (socket says 172.21.15.55:42652) 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f21faa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:56092/0 (socket says 172.21.15.55:56092) 2024-05-10T03:29:22.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.277+0000 7f21fbe00640 1 -- 172.21.15.55:0/3713328965 learned_addr learned my addr 172.21.15.55:0/3713328965 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.278+0000 7f21faa00640 1 -- 172.21.15.55:0/3713328965 >> v2:172.21.15.162:3300/0 conn(0x7f21fc15fe50 msgr2=0x7f21fc141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.278+0000 7f21faa00640 1 --2- 172.21.15.55:0/3713328965 >> v2:172.21.15.162:3300/0 conn(0x7f21fc15fe50 0x7f21fc141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.278+0000 7f21faa00640 1 -- 172.21.15.55:0/3713328965 >> v1:172.21.15.162:6789/0 conn(0x7f21fc0b2550 legacy=0x7f21fc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.278+0000 7f21faa00640 1 -- 172.21.15.55:0/3713328965 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc06a3f0 con 0x7f21fc0b2140 2024-05-10T03:29:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.278+0000 7f21fa000640 1 -- 172.21.15.55:0/3713328965 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900452111 0 0) 0x7f21fc06ad20 con 0x7f21fc0b2550 2024-05-10T03:29:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.279+0000 7f21faa00640 1 --2- 172.21.15.55:0/3713328965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097a0 tx=0x7f21e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4aa4aa2dd6dc2a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.279+0000 7f21fa000640 1 -- 172.21.15.55:0/3713328965 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8032440 con 0x7f21fc0b2140 2024-05-10T03:29:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.279+0000 7f21fa000640 1 -- 172.21.15.55:0/3713328965 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e80325a0 con 0x7f21fc0b2140 2024-05-10T03:29:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.279+0000 7f21fa000640 1 -- 172.21.15.55:0/3713328965 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8032930 con 0x7f21fc0b2140 2024-05-10T03:29:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 -- 172.21.15.55:0/3713328965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 msgr2=0x7f21fc142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 --2- 172.21.15.55:0/3713328965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097a0 tx=0x7f21e802a210 comp rx=0 tx=0).stop 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 -- 172.21.15.55:0/3713328965 shutdown_connections 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 --2- 172.21.15.55:0/3713328965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2140 0x7f21fc142440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 --2- 172.21.15.55:0/3713328965 >> v2:172.21.15.162:3300/0 conn(0x7f21fc15fe50 0x7f21fc141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 -- 172.21.15.55:0/3713328965 >> 172.21.15.55:0/3713328965 conn(0x7f21fc0cbb90 msgr2=0x7f21fc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 -- 172.21.15.55:0/3713328965 shutdown_connections 2024-05-10T03:29:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.280+0000 7f2202000640 1 -- 172.21.15.55:0/3713328965 wait complete. 2024-05-10T03:29:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.283+0000 7f2202000640 1 Processor -- start 2024-05-10T03:29:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.283+0000 7f2202000640 1 -- start start 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21fc0b2140 0x7f21fc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 0x7f21fc1e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0ae470 con 0x7f21fc15fe50 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc15b4a0 con 0x7f21fc0b2140 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.284+0000 7f2202000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc142830 con 0x7f21fc0b2550 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:22.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:56096/0 (socket says 172.21.15.55:56096) 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 -- 172.21.15.55:0/4278974941 learned_addr learned my addr 172.21.15.55:0/4278974941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21fbe00640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 0x7f21fc1e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21fb400640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21fc0b2140 0x7f21fc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 -- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21fc0b2140 msgr2=0x7f21fc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21fc0b2140 0x7f21fc15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 -- 172.21.15.55:0/4278974941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 msgr2=0x7f21fc1e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 0x7f21fc1e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21faa00640 1 -- 172.21.15.55:0/4278974941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009400 con 0x7f21fc0b2550 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.285+0000 7f21fbe00640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 0x7f21fc1e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.286+0000 7f21faa00640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21e8035040 tx=0x7f21e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.286+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8006e10 con 0x7f21fc0b2550 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.286+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8032e90 con 0x7f21fc0b2550 2024-05-10T03:29:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.286+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc0ae220 con 0x7f21fc0b2550 2024-05-10T03:29:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.286+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8030be0 con 0x7f21fc0b2550 2024-05-10T03:29:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.287+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc070b10 con 0x7f21fc0b2550 2024-05-10T03:29:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.287+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f21e8039040 con 0x7f21fc0b2550 2024-05-10T03:29:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.287+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f21e8030e30 con 0x7f21fc0b2550 2024-05-10T03:29:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.288+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc066f60 con 0x7f21fc0b2550 2024-05-10T03:29:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.293+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f21e8041030 con 0x7f21fc0b2550 2024-05-10T03:29:22.359 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:22.355+0000 7ff5b7f23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T03:29:22.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.359+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21fc067220 con 0x7f21fc0b2550 2024-05-10T03:29:22.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.360+0000 7f21f3400640 1 -- 172.21.15.55:0/4278974941 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f21e8037070 con 0x7f21fc0b2550 2024-05-10T03:29:22.365 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:22.365 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:21.862076+0000","last_up_change":"0.000000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:22.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.361+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 msgr2=0x7f21fc0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.361+0000 7f2202000640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21e8035040 tx=0x7f21e80046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 shutdown_connections 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21fc15fe50 0x7f21fc1e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21fc0b2550 0x7f21fc0ad9b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 --2- 172.21.15.55:0/4278974941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21fc0b2140 0x7f21fc15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 >> 172.21.15.55:0/4278974941 conn(0x7f21fc0cbb90 msgr2=0x7f21fc0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 shutdown_connections 2024-05-10T03:29:22.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:22.362+0000 7f2202000640 1 -- 172.21.15.55:0/4278974941 wait complete. 2024-05-10T03:29:22.388 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-10T03:29:22.388 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-10T03:29:22.416 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:22.412+0000 7ff5b7f23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T03:29:22.670 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-05-10T03:29:22.666+0000 7f58aa443640 -1 Falling back to public interface 2024-05-10T03:29:22.674 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-05-10T03:29:22.670+0000 7f9755c11640 -1 Falling back to public interface 2024-05-10T03:29:22.686 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-10T03:29:22.682+0000 7f0116819640 -1 Falling back to public interface 2024-05-10T03:29:22.699 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-10T03:29:22.695+0000 7f1be2024640 -1 Falling back to public interface 2024-05-10T03:29:22.734 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-05-10T03:29:22.730+0000 7f015ba1f640 -1 Falling back to public interface 2024-05-10T03:29:22.758 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-05-10T03:29:22.753+0000 7fb26560f640 -1 Falling back to public interface 2024-05-10T03:29:22.764 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-10T03:29:22.759+0000 7f7e55486640 -1 Falling back to public interface 2024-05-10T03:29:22.771 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:22.767+0000 7fbf0de1f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:29:22.773 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-10T03:29:22.768+0000 7f9f5c557640 -1 Falling back to public interface 2024-05-10T03:29:22.906 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:22.902+0000 7fbf0de1f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:29:22.986 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:22.981+0000 7ff5b7f23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T03:29:23.122 INFO:tasks.ceph.mgr.y.smithi055.stderr:2024-05-10T03:29:23.117+0000 7ff5b7f23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T03:29:23.194 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:23.435 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.430+0000 7ff827400640 1 Processor -- start 2024-05-10T03:29:23.435 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.430+0000 7ff827400640 1 -- start start 2024-05-10T03:29:23.437 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.432+0000 7ff827400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff820165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.437 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.432+0000 7ff827400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200715e0 con 0x7ff8200b2700 2024-05-10T03:29:23.437 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.432+0000 7ff827400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820166310 con 0x7ff82006b6e0 2024-05-10T03:29:23.437 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.432+0000 7ff827400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200711c0 con 0x7ff8201659c0 2024-05-10T03:29:23.437 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.433+0000 7ff825400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff8200b2700 0x7ff8200b2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35212/0 (socket says 172.21.15.55:35212) 2024-05-10T03:29:23.438 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.433+0000 7ff825400640 1 -- 172.21.15.55:0/1632942295 learned_addr learned my addr 172.21.15.55:0/1632942295 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:23.439 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.433+0000 7ff81fe00640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff820165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.439 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.433+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107479347 0 0) 0x7ff818001490 con 0x7ff8200b2700 2024-05-10T03:29:23.439 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.433+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff808003520 con 0x7ff8200b2700 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1571172833 0 0) 0x7ff808003520 con 0x7ff8200b2700 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 msgr2=0x7ff820165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff820165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 >> v1:172.21.15.162:6789/0 conn(0x7ff82006b6e0 legacy=0x7ff8200ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8201667f0 con 0x7ff8200b2700 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7ff818001b80 con 0x7ff8200b2700 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 >> v1:172.21.15.55:6789/0 conn(0x7ff8200b2700 legacy=0x7ff8200b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff8080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808003f20 0x7ff8080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff808005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200711c0 con 0x7ff808004d00 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800bbc0 con 0x7ff808003f20 2024-05-10T03:29:23.440 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800bd80 con 0x7ff8201659c0 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff825400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff808005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.434+0000 7ff824a00640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff8080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.435+0000 7ff825400640 1 -- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 msgr2=0x7ff8080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.435+0000 7ff825400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8201659c0 0x7ff8080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.435+0000 7ff825400640 1 -- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808003f20 msgr2=0x7ff8080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.435+0000 7ff825400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff808003f20 0x7ff8080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.441 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.435+0000 7ff825400640 1 -- 172.21.15.55:0/1632942295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808003520 con 0x7ff808004d00 2024-05-10T03:29:23.442 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff825400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff808005110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8200b21f0 tx=0x7ff8080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e90ef6880680eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:23.442 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81802b9a0 con 0x7ff808004d00 2024-05-10T03:29:23.442 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81802bb00 con 0x7ff808004d00 2024-05-10T03:29:23.442 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff81f400640 1 -- 172.21.15.55:0/1632942295 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818034470 con 0x7ff808004d00 2024-05-10T03:29:23.442 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff827400640 1 -- 172.21.15.55:0/1632942295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 msgr2=0x7ff808005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff827400640 1 --2- 172.21.15.55:0/1632942295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff808005110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8200b21f0 tx=0x7ff8080057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.436+0000 7ff825400640 1 -- 172.21.15.55:0/1632942295 reap_dead start 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.437+0000 7ff827400640 1 -- 172.21.15.55:0/1632942295 shutdown_connections 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.437+0000 7ff827400640 1 -- 172.21.15.55:0/1632942295 >> 172.21.15.55:0/1632942295 conn(0x7ff8200cbb90 msgr2=0x7ff8200cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.437+0000 7ff827400640 1 -- 172.21.15.55:0/1632942295 shutdown_connections 2024-05-10T03:29:23.443 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.437+0000 7ff827400640 1 -- 172.21.15.55:0/1632942295 wait complete. 2024-05-10T03:29:23.444 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.439+0000 7ff827400640 1 Processor -- start 2024-05-10T03:29:23.444 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.439+0000 7ff827400640 1 -- start start 2024-05-10T03:29:23.447 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.447 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 0x7ff8200a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.447 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff82015a0a0 0x7ff8200a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:23.447 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200ae3b0 con 0x7ff808004d00 2024-05-10T03:29:23.447 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82014bb10 con 0x7ff820149aa0 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff827400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82015da40 con 0x7ff82015a0a0 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff824a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff824a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43380/0 (socket says 172.21.15.55:43380) 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.441+0000 7ff824a00640 1 -- 172.21.15.55:0/3066953158 learned_addr learned my addr 172.21.15.55:0/3066953158 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff81fe00640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 0x7ff8200a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff825400640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff82015a0a0 0x7ff8200a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 -- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff82015a0a0 msgr2=0x7ff8200a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff82015a0a0 0x7ff8200a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 -- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 msgr2=0x7ff8200a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.448 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 0x7ff8200a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 -- 172.21.15.55:0/3066953158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82014a340 con 0x7ff808004d00 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff81fe00640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 0x7ff8200a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff824a00640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e580 tx=0x7ff81400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140052e0 con 0x7ff808004d00 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff814005440 con 0x7ff808004d00 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814003810 con 0x7ff808004d00 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.442+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82015dc90 con 0x7ff808004d00 2024-05-10T03:29:23.449 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.443+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82015e250 con 0x7ff808004d00 2024-05-10T03:29:23.450 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.444+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff820066f60 con 0x7ff808004d00 2024-05-10T03:29:23.450 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.444+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 2) v1 ==== 44528+0+0 (secure 0 0 0) 0x7ff814003970 con 0x7ff808004d00 2024-05-10T03:29:23.450 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.444+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff814049320 con 0x7ff808004d00 2024-05-10T03:29:23.451 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.446+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff81401bd30 con 0x7ff808004d00 2024-05-10T03:29:23.520 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.515+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff820067220 con 0x7ff808004d00 2024-05-10T03:29:23.522 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.517+0000 7ff81cc00640 1 -- 172.21.15.55:0/3066953158 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff81401bb50 con 0x7ff808004d00 2024-05-10T03:29:23.522 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-10T03:29:23.522 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":10,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:21.862076+0000","last_up_change":"0.000000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 msgr2=0x7ff82015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e580 tx=0x7ff81400ea40 comp rx=0 tx=0).stop 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 shutdown_connections 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff82015a0a0 0x7ff8200a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff820149aa0 0x7ff8200a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 --2- 172.21.15.55:0/3066953158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff808004d00 0x7ff82015d460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 >> 172.21.15.55:0/3066953158 conn(0x7ff8200cbb90 msgr2=0x7ff8200b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:23.523 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 shutdown_connections 2024-05-10T03:29:23.524 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:23.518+0000 7ff827400640 1 -- 172.21.15.55:0/3066953158 wait complete. 2024-05-10T03:29:23.542 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-10T03:29:23.955 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-05-10T03:29:23.951+0000 7fbeaee00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-10T03:29:26.292 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-05-10T03:29:26.288+0000 7f58aa443640 -1 osd.4 0 log_to_monitors true 2024-05-10T03:29:26.318 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-05-10T03:29:26.314+0000 7f9755c11640 -1 osd.5 0 log_to_monitors true 2024-05-10T03:29:26.350 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-05-10T03:29:26.346+0000 7f015ba1f640 -1 osd.6 0 log_to_monitors true 2024-05-10T03:29:26.395 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-05-10T03:29:26.391+0000 7fb26560f640 -1 osd.7 0 log_to_monitors true 2024-05-10T03:29:26.564 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-10T03:29:26.559+0000 7f0116819640 -1 osd.2 0 log_to_monitors true 2024-05-10T03:29:26.568 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-10T03:29:26.564+0000 7f1be2024640 -1 osd.3 0 log_to_monitors true 2024-05-10T03:29:26.597 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-10T03:29:26.593+0000 7f7e55486640 -1 osd.0 0 log_to_monitors true 2024-05-10T03:29:26.682 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-10T03:29:26.678+0000 7f9f5c557640 -1 osd.1 0 log_to_monitors true 2024-05-10T03:29:30.194 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-10T03:29:30.189+0000 7f0111200640 -1 osd.2 0 waiting for initial osdmap 2024-05-10T03:29:30.202 INFO:tasks.ceph.osd.2.smithi055.stderr:2024-05-10T03:29:30.197+0000 7f0107200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.248 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-10T03:29:30.244+0000 7f7e50e00640 -1 osd.0 0 waiting for initial osdmap 2024-05-10T03:29:30.256 INFO:tasks.ceph.osd.0.smithi055.stderr:2024-05-10T03:29:30.252+0000 7f7e46400640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.305 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-10T03:29:30.300+0000 7f9f57e00640 -1 osd.1 0 waiting for initial osdmap 2024-05-10T03:29:30.312 INFO:tasks.ceph.osd.1.smithi055.stderr:2024-05-10T03:29:30.308+0000 7f9f4d400640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.344 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-10T03:29:30.340+0000 7f1bdc800640 -1 osd.3 0 waiting for initial osdmap 2024-05-10T03:29:30.349 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:30.356 INFO:tasks.ceph.osd.3.smithi055.stderr:2024-05-10T03:29:30.351+0000 7f1bd2800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.444 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-05-10T03:29:30.441+0000 7f0156800640 -1 osd.6 0 waiting for initial osdmap 2024-05-10T03:29:30.453 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-05-10T03:29:30.450+0000 7f014c800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.502 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-05-10T03:29:30.499+0000 7fb260000640 -1 osd.7 0 waiting for initial osdmap 2024-05-10T03:29:30.514 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-05-10T03:29:30.508+0000 7fb256000640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.532 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-05-10T03:29:30.529+0000 7f58a5200640 -1 osd.4 0 waiting for initial osdmap 2024-05-10T03:29:30.537 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-05-10T03:29:30.534+0000 7f589b200640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.561 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-05-10T03:29:30.558+0000 7f9750400640 -1 osd.5 0 waiting for initial osdmap 2024-05-10T03:29:30.565 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.560+0000 7fe73dc00640 1 Processor -- start 2024-05-10T03:29:30.565 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.561+0000 7fe73dc00640 1 -- start start 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.562+0000 7fe73dc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe73815fe50 0x7fe738141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe73dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe73dc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73806ad20 con 0x7fe7380b2140 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe73dc00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe738141f50 con 0x7fe73815fe50 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe73dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73806a110 con 0x7fe7380b2550 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe73815fe50 0x7fe738141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe737400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe73815fe50 0x7fe738141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43412/0 (socket says 172.21.15.55:43412) 2024-05-10T03:29:30.568 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 -- 172.21.15.55:0/3892651817 learned_addr learned my addr 172.21.15.55:0/3892651817 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:30.569 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-05-10T03:29:30.565+0000 7f9746400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T03:29:30.569 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 -- 172.21.15.55:0/3892651817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 msgr2=0x7fe73806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:30.569 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 --2- 172.21.15.55:0/3892651817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.569 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.563+0000 7fe736a00640 1 -- 172.21.15.55:0/3892651817 >> v1:172.21.15.162:6789/0 conn(0x7fe7380b2140 legacy=0x7fe738142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.570 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.564+0000 7fe735600640 1 -- 172.21.15.55:0/3892651817 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570301116 0 0) 0x7fe73806ad20 con 0x7fe7380b2140 2024-05-10T03:29:30.570 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.564+0000 7fe736a00640 1 -- 172.21.15.55:0/3892651817 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73806a3f0 con 0x7fe73815fe50 2024-05-10T03:29:30.570 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.565+0000 7fe736a00640 1 --2- 172.21.15.55:0/3892651817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe738141a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe728009e50 tx=0x7fe72802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c03d94b5177f95 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.565+0000 7fe735600640 1 -- 172.21.15.55:0/3892651817 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728031600 con 0x7fe73815fe50 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.565+0000 7fe735600640 1 -- 172.21.15.55:0/3892651817 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72802d050 con 0x7fe73815fe50 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.565+0000 7fe735600640 1 -- 172.21.15.55:0/3892651817 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728031980 con 0x7fe73815fe50 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 -- 172.21.15.55:0/3892651817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 msgr2=0x7fe738141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 --2- 172.21.15.55:0/3892651817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe738141a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe728009e50 tx=0x7fe72802a7a0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 -- 172.21.15.55:0/3892651817 shutdown_connections 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 --2- 172.21.15.55:0/3892651817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 --2- 172.21.15.55:0/3892651817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe738141a10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 -- 172.21.15.55:0/3892651817 >> 172.21.15.55:0/3892651817 conn(0x7fe7380cbb90 msgr2=0x7fe7380cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 -- 172.21.15.55:0/3892651817 shutdown_connections 2024-05-10T03:29:30.571 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.566+0000 7fe73dc00640 1 -- 172.21.15.55:0/3892651817 wait complete. 2024-05-10T03:29:30.573 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 Processor -- start 2024-05-10T03:29:30.573 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 -- start start 2024-05-10T03:29:30.576 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7380b2140 0x7fe7381f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.576 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7381441f0 con 0x7fe73815fe50 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73815c020 con 0x7fe7380b2140 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.570+0000 7fe73dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73815c270 con 0x7fe7380b2550 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43420/0 (socket says 172.21.15.55:43420) 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 -- 172.21.15.55:0/1048858923 learned_addr learned my addr 172.21.15.55:0/1048858923 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe736000640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe736a00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7380b2140 0x7fe7381f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 -- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 msgr2=0x7fe73815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73815af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 -- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7380b2140 msgr2=0x7fe7381f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7380b2140 0x7fe7381f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.577 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 -- 172.21.15.55:0/1048858923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009b30 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.571+0000 7fe737400640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe73000e7f0 tx=0x7fe73000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.572+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730009510 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.572+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73000ee40 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.572+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73815c550 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.572+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73000cb80 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.572+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73815cb10 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.573+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe738066f60 con 0x7fe73815fe50 2024-05-10T03:29:30.578 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.574+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fe73000cce0 con 0x7fe73815fe50 2024-05-10T03:29:30.579 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.575+0000 7fe72ea00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 0x7fe71406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:30.579 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.575+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe7300886b0 con 0x7fe73815fe50 2024-05-10T03:29:30.580 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.575+0000 7fe736a00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 0x7fe71406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:30.581 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.576+0000 7fe736a00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 0x7fe71406d0b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe728033000 tx=0x7fe728009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:30.584 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.581+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe7300545d0 con 0x7fe73815fe50 2024-05-10T03:29:30.776 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.771+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe738067220 con 0x7fe73815fe50 2024-05-10T03:29:30.777 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.773+0000 7fe72ea00640 1 -- 172.21.15.55:0/1048858923 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe730021020 con 0x7fe73815fe50 2024-05-10T03:29:30.778 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-10T03:29:30.778 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":12,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:27.968181+0000","last_up_change":"0.000000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 msgr2=0x7fe71406d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 0x7fe71406d0b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe728033000 tx=0x7fe728009000 comp rx=0 tx=0).stop 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 msgr2=0x7fe73815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe73000e7f0 tx=0x7fe73000ebf0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 shutdown_connections 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe73815fe50 0x7fe73815dd90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.780 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7380b2550 0x7fe73815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.781 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe71406aba0 0x7fe71406d0b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.781 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 --2- 172.21.15.55:0/1048858923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7380b2140 0x7fe7381f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:30.781 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.775+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 >> 172.21.15.55:0/1048858923 conn(0x7fe7380cbb90 msgr2=0x7fe7380b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:30.781 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.776+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 shutdown_connections 2024-05-10T03:29:30.781 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:30.776+0000 7fe73dc00640 1 -- 172.21.15.55:0/1048858923 wait complete. 2024-05-10T03:29:30.799 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-10T03:29:37.606 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:37.877 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.873+0000 7f936de00640 1 Processor -- start 2024-05-10T03:29:37.877 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.873+0000 7f936de00640 1 -- start start 2024-05-10T03:29:37.879 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.875+0000 7f936de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f936815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.879 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936de00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93680d25a0 con 0x7f93681618b0 2024-05-10T03:29:37.879 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936de00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93680b28d0 con 0x7f93680b2170 2024-05-10T03:29:37.879 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93680d2b20 con 0x7f93680b6680 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936d400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f93681618b0 0x7f936806aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50908/0 (socket says 172.21.15.55:50908) 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936d400640 1 -- 172.21.15.55:0/3067064798 learned_addr learned my addr 172.21.15.55:0/3067064798 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f936ca00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f936815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.876+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308676715 0 0) 0x7f93680d25a0 con 0x7f93681618b0 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c003520 con 0x7f93681618b0 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955221790 0 0) 0x7f934c003520 con 0x7f93681618b0 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 msgr2=0x7f936815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f936815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 >> v1:172.21.15.162:6789/0 conn(0x7f93680b2170 legacy=0x7f93681611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93680d2840 con 0x7f93681618b0 2024-05-10T03:29:37.882 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f93600019f0 con 0x7f93681618b0 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 >> v1:172.21.15.55:6789/0 conn(0x7f93681618b0 legacy=0x7f936806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f934c003a10 0x7f934c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c0047c0 0x7f934c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f934c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93680d2b20 con 0x7f934c003a10 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00bb10 con 0x7f934c0047c0 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00bcd0 con 0x7f93680b6680 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.877+0000 7f936ca00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f934c003a10 0x7f934c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936d400640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f934c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f9367e00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c0047c0 0x7f934c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936ca00640 1 -- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 msgr2=0x7f934c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936ca00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93680b6680 0x7f934c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.883 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936ca00640 1 -- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c0047c0 msgr2=0x7f934c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:37.884 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936ca00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c0047c0 0x7f934c004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.884 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f936ca00640 1 -- 172.21.15.55:0/3067064798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c003520 con 0x7f934c003a10 2024-05-10T03:29:37.884 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.878+0000 7f9367e00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c0047c0 0x7f934c004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.879+0000 7f936ca00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f934c003a10 0x7f934c003e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9350009ac0 tx=0x7f934c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157188bdcbcb1892 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.879+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350031400 con 0x7f934c003a10 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.879+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935002d050 con 0x7f934c003a10 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.879+0000 7f9367400640 1 -- 172.21.15.55:0/3067064798 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350031780 con 0x7f934c003a10 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 -- 172.21.15.55:0/3067064798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f934c003a10 msgr2=0x7f934c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 --2- 172.21.15.55:0/3067064798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f934c003a10 0x7f934c003e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9350009ac0 tx=0x7f934c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936d400640 1 -- 172.21.15.55:0/3067064798 reap_dead start 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 -- 172.21.15.55:0/3067064798 shutdown_connections 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 -- 172.21.15.55:0/3067064798 >> 172.21.15.55:0/3067064798 conn(0x7f93680b5730 msgr2=0x7f93680d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:37.885 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 -- 172.21.15.55:0/3067064798 shutdown_connections 2024-05-10T03:29:37.886 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.880+0000 7f936de00640 1 -- 172.21.15.55:0/3067064798 wait complete. 2024-05-10T03:29:37.886 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.883+0000 7f936de00640 1 Processor -- start 2024-05-10T03:29:37.886 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.883+0000 7f936de00640 1 -- start start 2024-05-10T03:29:37.890 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c003a10 0x7f936814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.890 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f936815dc60 0x7f93681600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.890 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.890 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936815db20 con 0x7f9368160600 2024-05-10T03:29:37.890 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936814ba90 con 0x7f934c003a10 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93681f4c00 con 0x7f936815dc60 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38842/0 (socket says 172.21.15.55:38842) 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.885+0000 7f936d400640 1 -- 172.21.15.55:0/3905350715 learned_addr learned my addr 172.21.15.55:0/3905350715 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f9367e00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f936815dc60 0x7f93681600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936ca00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c003a10 0x7f936814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 -- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f936815dc60 msgr2=0x7f93681600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f936815dc60 0x7f93681600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 -- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c003a10 msgr2=0x7f936814b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c003a10 0x7f936814b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 -- 172.21.15.55:0/3905350715 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936815e4d0 con 0x7f9368160600 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f936d400640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f936806c220 tx=0x7f9360007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:37.891 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.886+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000eb60 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.887+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000ece0 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.887+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360013660 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.887+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936815f330 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.887+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936815e720 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.888+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368067740 con 0x7f9368160600 2024-05-10T03:29:37.892 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.889+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f93600137c0 con 0x7f9368160600 2024-05-10T03:29:37.893 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.890+0000 7f9364c00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 0x7f934406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:37.893 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.890+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9360089430 con 0x7f9368160600 2024-05-10T03:29:37.894 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.890+0000 7f936ca00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 0x7f934406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:37.895 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.892+0000 7f936ca00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 0x7f934406d110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9350033000 tx=0x7f9350008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:37.899 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:37.896+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f93600548d0 con 0x7f9368160600 2024-05-10T03:29:38.109 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.106+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936815f4c0 con 0x7f9368160600 2024-05-10T03:29:38.111 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.107+0000 7f9364c00640 1 -- 172.21.15.55:0/3905350715 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f9360057ef0 con 0x7f9368160600 2024-05-10T03:29:38.112 INFO:teuthology.misc.health.smithi055.stdout: 2024-05-10T03:29:38.112 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":17,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:35.275824+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:38.114 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.110+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 msgr2=0x7f934406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.114 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.110+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 0x7f934406d110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9350033000 tx=0x7f9350008030 comp rx=0 tx=0).stop 2024-05-10T03:29:38.114 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.110+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 msgr2=0x7f93681f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.114 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.110+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f936806c220 tx=0x7f9360007bc0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 shutdown_connections 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9368160600 0x7f93681f3a00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f936815dc60 0x7f93681600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f934406ac00 0x7f934406d110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 --2- 172.21.15.55:0/3905350715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f934c003a10 0x7f936814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 >> 172.21.15.55:0/3905350715 conn(0x7f93680b5730 msgr2=0x7f936815b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 shutdown_connections 2024-05-10T03:29:38.115 INFO:teuthology.misc.health.smithi055.stderr:2024-05-10T03:29:38.111+0000 7f936de00640 1 -- 172.21.15.55:0/3905350715 wait complete. 2024-05-10T03:29:38.135 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-10T03:29:38.135 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-10T03:29:38.135 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:29:38.136 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:29:38.143 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:29:38.143 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:29:38.144 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:29:38.150 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:29:38.151 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:29:38.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:29:38.159 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:29:38.159 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:29:38.159 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:29:38.167 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:29:38.167 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-10T03:29:38.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:38.371 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:29:38.372 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:29:38.389 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:29:38.389 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:29:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.416+0000 7fd6be000640 1 Processor -- start 2024-05-10T03:29:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.416+0000 7fd6be000640 1 -- start start 2024-05-10T03:29:38.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6be000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd6b80b6bf0 0x7fd6b815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6be000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6b80ceb50 0x7fd6b80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6be000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b806bcb0 con 0x7fd6b80cdef0 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6be000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b806c270 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6be000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b806bf90 con 0x7fd6b80ceb50 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6b7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd6b80b6bf0 0x7fd6b815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.418+0000 7fd6b7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd6b80b6bf0 0x7fd6b815dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38876/0 (socket says 172.21.15.55:38876) 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7400640 1 -- 172.21.15.55:0/1499160663 learned_addr learned my addr 172.21.15.55:0/1499160663 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7e00640 1 --2- 172.21.15.55:0/1499160663 >> v2:172.21.15.162:3300/0 conn(0x7fd6b80ceb50 0x7fd6b80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7400640 1 -- 172.21.15.55:0/1499160663 >> v2:172.21.15.162:3300/0 conn(0x7fd6b80ceb50 msgr2=0x7fd6b80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7400640 1 --2- 172.21.15.55:0/1499160663 >> v2:172.21.15.162:3300/0 conn(0x7fd6b80ceb50 0x7fd6b80d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7400640 1 -- 172.21.15.55:0/1499160663 >> v1:172.21.15.162:6789/0 conn(0x7fd6b80cdef0 legacy=0x7fd6b80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.419+0000 7fd6b7400640 1 -- 172.21.15.55:0/1499160663 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80d1360 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.420+0000 7fd6b7400640 1 --2- 172.21.15.55:0/1499160663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b815dd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd6b808c780 tx=0x7fd6a802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea2ae1d7a20c449 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.420+0000 7fd6b6000640 1 -- 172.21.15.55:0/1499160663 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032640 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.420+0000 7fd6b6000640 1 -- 172.21.15.55:0/1499160663 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80327a0 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.420+0000 7fd6b6000640 1 -- 172.21.15.55:0/1499160663 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032b30 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 -- 172.21.15.55:0/1499160663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 msgr2=0x7fd6b815dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 --2- 172.21.15.55:0/1499160663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b815dd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd6b808c780 tx=0x7fd6a802a300 comp rx=0 tx=0).stop 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 -- 172.21.15.55:0/1499160663 shutdown_connections 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 --2- 172.21.15.55:0/1499160663 >> v2:172.21.15.162:3300/0 conn(0x7fd6b80ceb50 0x7fd6b80d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 --2- 172.21.15.55:0/1499160663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b815dd50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 -- 172.21.15.55:0/1499160663 >> 172.21.15.55:0/1499160663 conn(0x7fd6b80d32c0 msgr2=0x7fd6b80d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 -- 172.21.15.55:0/1499160663 shutdown_connections 2024-05-10T03:29:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.421+0000 7fd6be000640 1 -- 172.21.15.55:0/1499160663 wait complete. 2024-05-10T03:29:38.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.424+0000 7fd6be000640 1 Processor -- start 2024-05-10T03:29:38.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.424+0000 7fd6be000640 1 -- start start 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6b80cdef0 0x7fd6b814a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 0x7fd6b815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b81544c0 con 0x7fd6b80b6bf0 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b81f4290 con 0x7fd6b815c750 2024-05-10T03:29:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6be000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8157030 con 0x7fd6b80cdef0 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.425+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 0x7fd6b815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38886/0 (socket says 172.21.15.55:38886) 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7400640 1 -- 172.21.15.55:0/2597940080 learned_addr learned my addr 172.21.15.55:0/2597940080 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b6a00640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6b80cdef0 0x7fd6b814a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 -- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6b80cdef0 msgr2=0x7fd6b814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6b80cdef0 0x7fd6b814a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 -- 172.21.15.55:0/2597940080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 msgr2=0x7fd6b8153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 -- 172.21.15.55:0/2597940080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a80094d0 con 0x7fd6b815c750 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7400640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.426+0000 7fd6b7e00640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 0x7fd6b815cb60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000eb70 tx=0x7fd6b000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.427+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000cd20 con 0x7fd6b815c750 2024-05-10T03:29:38.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.427+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000f040 con 0x7fd6b815c750 2024-05-10T03:29:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.427+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00146c0 con 0x7fd6b815c750 2024-05-10T03:29:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.427+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b815d0a0 con 0x7fd6b815c750 2024-05-10T03:29:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.427+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80b4580 con 0x7fd6b815c750 2024-05-10T03:29:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.429+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b8067220 con 0x7fd6b815c750 2024-05-10T03:29:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.430+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd6b0014820 con 0x7fd6b815c750 2024-05-10T03:29:38.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.431+0000 7fd6af400640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 0x7fd69406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.431+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd6b0089ed0 con 0x7fd6b815c750 2024-05-10T03:29:38.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.431+0000 7fd6b7400640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 0x7fd69406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.432+0000 7fd6b7400640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 0x7fd69406d110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8035040 tx=0x7fd6a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.436+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd6b0055490 con 0x7fd6b815c750 2024-05-10T03:29:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.632+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6b806ac90 con 0x7fd6b815c750 2024-05-10T03:29:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.634+0000 7fd6af400640 1 -- 172.21.15.55:0/2597940080 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fd6b0018090 con 0x7fd6b815c750 2024-05-10T03:29:38.638 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:38.639 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:35.275824+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 msgr2=0x7fd69406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 0x7fd69406d110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8035040 tx=0x7fd6a802e000 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 msgr2=0x7fd6b815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 0x7fd6b815cb60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000eb70 tx=0x7fd6b000c490 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 shutdown_connections 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd69406ac00 0x7fd69406d110 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b815c750 0x7fd6b815cb60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6b80cdef0 0x7fd6b814a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 --2- 172.21.15.55:0/2597940080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6b80b6bf0 0x7fd6b8153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 >> 172.21.15.55:0/2597940080 conn(0x7fd6b80d32c0 msgr2=0x7fd6b80d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 shutdown_connections 2024-05-10T03:29:38.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.637+0000 7fd6be000640 1 -- 172.21.15.55:0/2597940080 wait complete. 2024-05-10T03:29:38.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:29:38.664 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:29:38.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.942+0000 7fd7d2a00640 1 Processor -- start 2024-05-10T03:29:38.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.943+0000 7fd7d2a00640 1 -- start start 2024-05-10T03:29:38.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7cc06e070 0x7fd7cc0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0b42c0 con 0x7fd7cc15d7b0 2024-05-10T03:29:38.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b45a0 con 0x7fd7cc06e070 2024-05-10T03:29:38.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b4880 con 0x7fd7cc06c940 2024-05-10T03:29:38.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:56480/0 (socket says 172.21.15.55:56480) 2024-05-10T03:29:38.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d0c00640 1 -- 172.21.15.55:0/4293616836 learned_addr learned my addr 172.21.15.55:0/4293616836 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:38.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.945+0000 7fd7d2000640 1 --2- 172.21.15.55:0/4293616836 >> v2:172.21.15.162:3300/0 conn(0x7fd7cc06e070 0x7fd7cc0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.946+0000 7fd7d2000640 1 -- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 msgr2=0x7fd7cc06db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.946+0000 7fd7d2000640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.946+0000 7fd7d2000640 1 -- 172.21.15.55:0/4293616836 >> v1:172.21.15.162:6789/0 conn(0x7fd7cc15d7b0 legacy=0x7fd7cc06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.946+0000 7fd7d2000640 1 -- 172.21.15.55:0/4293616836 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0b4b60 con 0x7fd7cc06e070 2024-05-10T03:29:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.946+0000 7fd7d0c00640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:29:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.948+0000 7fd7d2000640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc0acbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009d90 tx=0x7fd7c402a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b21ed4a344f0e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.948+0000 7fd7cbe00640 1 -- 172.21.15.55:0/4293616836 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4007740 con 0x7fd7cc06e070 2024-05-10T03:29:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.948+0000 7fd7cbe00640 1 -- 172.21.15.55:0/4293616836 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c402d050 con 0x7fd7cc06e070 2024-05-10T03:29:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.948+0000 7fd7cbe00640 1 -- 172.21.15.55:0/4293616836 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4007ac0 con 0x7fd7cc06e070 2024-05-10T03:29:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 -- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 msgr2=0x7fd7cc0acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc0acbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009d90 tx=0x7fd7c402a530 comp rx=0 tx=0).stop 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 -- 172.21.15.55:0/4293616836 shutdown_connections 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc06c940 0x7fd7cc06db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/4293616836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc0acbe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 -- 172.21.15.55:0/4293616836 >> 172.21.15.55:0/4293616836 conn(0x7fd7cc070eb0 msgr2=0x7fd7cc0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 -- 172.21.15.55:0/4293616836 shutdown_connections 2024-05-10T03:29:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.949+0000 7fd7d2a00640 1 -- 172.21.15.55:0/4293616836 wait complete. 2024-05-10T03:29:38.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.951+0000 7fd7d2a00640 1 Processor -- start 2024-05-10T03:29:38.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.951+0000 7fd7d2a00640 1 -- start start 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc15d7b0 0x7fd7cc151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc155690 con 0x7fd7cc142790 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1526e0 con 0x7fd7cc06e070 2024-05-10T03:29:38.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b0e70 con 0x7fd7cc15d7b0 2024-05-10T03:29:38.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38896/0 (socket says 172.21.15.55:38896) 2024-05-10T03:29:38.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.953+0000 7fd7d2000640 1 -- 172.21.15.55:0/1461438377 learned_addr learned my addr 172.21.15.55:0/1461438377 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:38.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 -- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc15d7b0 msgr2=0x7fd7cc151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d0c00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc15d7b0 0x7fd7cc151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d1600640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc15d7b0 0x7fd7cc151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 -- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 msgr2=0x7fd7cc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 -- 172.21.15.55:0/1461438377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80094d0 con 0x7fd7cc142790 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2000640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009eb0 tx=0x7fd7c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c403b820 con 0x7fd7cc142790 2024-05-10T03:29:38.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c403b980 con 0x7fd7cc142790 2024-05-10T03:29:38.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c403bd10 con 0x7fd7cc142790 2024-05-10T03:29:38.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.954+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4009a70 con 0x7fd7cc142790 2024-05-10T03:29:38.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.955+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc1424d0 con 0x7fd7cc142790 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.956+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc151bb0 con 0x7fd7cc142790 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.957+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd7c402ad50 con 0x7fd7cc142790 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.957+0000 7fd7c9600640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 0x7fd7ac06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.958+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7c40aa6c0 con 0x7fd7cc142790 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.958+0000 7fd7d1600640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 0x7fd7ac06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:38.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.959+0000 7fd7d1600640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 0x7fd7ac06d140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000f60 tx=0x7fd7c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:38.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:38.963+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd7c4075b50 con 0x7fd7cc142790 2024-05-10T03:29:39.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.163+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7cc0b0fe0 con 0x7fd7cc142790 2024-05-10T03:29:39.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.163+0000 7fd7c9600640 1 -- 172.21.15.55:0/1461438377 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7c404c350 con 0x7fd7cc142790 2024-05-10T03:29:39.167 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:29:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 msgr2=0x7fd7ac06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 0x7fd7ac06d140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000f60 tx=0x7fd7c0009040 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 msgr2=0x7fd7cc142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009eb0 tx=0x7fd7c4004a40 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 shutdown_connections 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc142790 0x7fd7cc142ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7cc15d7b0 0x7fd7cc151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd7ac06ac30 0x7fd7ac06d140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 --2- 172.21.15.55:0/1461438377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7cc06e070 0x7fd7cc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.166+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 >> 172.21.15.55:0/1461438377 conn(0x7fd7cc070eb0 msgr2=0x7fd7cc157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.167+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 shutdown_connections 2024-05-10T03:29:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.167+0000 7fd7d2a00640 1 -- 172.21.15.55:0/1461438377 wait complete. 2024-05-10T03:29:39.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:29:39.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.467+0000 7fa46f200640 1 Processor -- start 2024-05-10T03:29:39.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.467+0000 7fa46f200640 1 -- start start 2024-05-10T03:29:39.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.469+0000 7fa46f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa47015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.469+0000 7fa46f200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4700b2170 0x7fa4701611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.469+0000 7fa46f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47006bac0 con 0x7fa4701616e0 2024-05-10T03:29:39.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.469+0000 7fa46f200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4700d2b20 con 0x7fa4700b2170 2024-05-10T03:29:39.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.469+0000 7fa46f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015db20 con 0x7fa4700b6680 2024-05-10T03:29:39.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa46e800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4701616e0 0x7fa4700cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50934/0 (socket says 172.21.15.55:50934) 2024-05-10T03:29:39.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa46e800640 1 -- 172.21.15.55:0/1204109575 learned_addr learned my addr 172.21.15.55:0/1204109575 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:39.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa465400640 1 --2- 172.21.15.55:0/1204109575 >> v2:172.21.15.162:3300/0 conn(0x7fa4700b2170 0x7fa4701611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa46de00640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa47015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518176268 0 0) 0x7fa47006bac0 con 0x7fa4701616e0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.470+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa454003520 con 0x7fa4701616e0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104737176 0 0) 0x7fa454003520 con 0x7fa4701616e0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 msgr2=0x7fa47015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa47015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 >> v2:172.21.15.162:3300/0 conn(0x7fa4700b2170 msgr2=0x7fa4701611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 --2- 172.21.15.55:0/1204109575 >> v2:172.21.15.162:3300/0 conn(0x7fa4700b2170 0x7fa4701611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700d25a0 con 0x7fa4701616e0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa468001670 con 0x7fa4701616e0 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 >> v1:172.21.15.55:6789/0 conn(0x7fa4701616e0 legacy=0x7fa4700cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4540039e0 0x7fa454003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4540047c0 0x7fa454004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa454005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700d2b20 con 0x7fa4540039e0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015db20 con 0x7fa4540047c0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45400b980 con 0x7fa4700b6680 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4540039e0 0x7fa454003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa465400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4540047c0 0x7fa454004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46e800640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa454005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 -- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 msgr2=0x7fa454005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa454005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 -- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4540047c0 msgr2=0x7fa454004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4540047c0 0x7fa454004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.471+0000 7fa46de00640 1 -- 172.21.15.55:0/1204109575 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700d25a0 con 0x7fa4540039e0 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.472+0000 7fa465400640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4540047c0 0x7fa454004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:39.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.472+0000 7fa46e800640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4700b6680 0x7fa454005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.472+0000 7fa46de00640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4540039e0 0x7fa454003e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa460009fc0 tx=0x7fa4540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6c4894c073b67a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:39.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600305f0 con 0x7fa4540039e0 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46000f040 con 0x7fa4540039e0 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46d400640 1 -- 172.21.15.55:0/1204109575 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460030990 con 0x7fa4540039e0 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46f200640 1 -- 172.21.15.55:0/1204109575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4540039e0 msgr2=0x7fa454003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46f200640 1 --2- 172.21.15.55:0/1204109575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4540039e0 0x7fa454003e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa460009fc0 tx=0x7fa4540057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.473+0000 7fa46e800640 1 -- 172.21.15.55:0/1204109575 reap_dead start 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.474+0000 7fa46f200640 1 -- 172.21.15.55:0/1204109575 shutdown_connections 2024-05-10T03:29:39.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.474+0000 7fa46f200640 1 -- 172.21.15.55:0/1204109575 >> 172.21.15.55:0/1204109575 conn(0x7fa4700b5730 msgr2=0x7fa4700d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:39.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.474+0000 7fa46f200640 1 -- 172.21.15.55:0/1204109575 shutdown_connections 2024-05-10T03:29:39.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.474+0000 7fa46f200640 1 -- 172.21.15.55:0/1204109575 wait complete. 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.476+0000 7fa46f200640 1 Processor -- start 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.476+0000 7fa46f200640 1 -- start start 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 0x7fa4700cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 0x7fa4700a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700c9510 con 0x7fa4700aa1f0 2024-05-10T03:29:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700cc200 con 0x7fa47015a410 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4700aa600 con 0x7fa4540039e0 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38922/0 (socket says 172.21.15.55:38922) 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 -- 172.21.15.55:0/1712671936 learned_addr learned my addr 172.21.15.55:0/1712671936 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa465400640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 0x7fa4700a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46de00640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 0x7fa4700cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 -- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 msgr2=0x7fa4700cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 0x7fa4700cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 -- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 msgr2=0x7fa4700a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 0x7fa4700a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46e800640 1 -- 172.21.15.55:0/1712671936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700cc450 con 0x7fa4700aa1f0 2024-05-10T03:29:39.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa465400640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 0x7fa4700a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.478+0000 7fa46de00640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 0x7fa4700cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa46e800640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa470161100 tx=0x7fa46800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468007230 con 0x7fa4700aa1f0 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa468007390 con 0x7fa4700aa1f0 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4700cc730 con 0x7fa4700aa1f0 2024-05-10T03:29:39.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46800dec0 con 0x7fa4700aa1f0 2024-05-10T03:29:39.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.479+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4700cccf0 con 0x7fa4700aa1f0 2024-05-10T03:29:39.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.480+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47004e290 con 0x7fa4700aa1f0 2024-05-10T03:29:39.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.481+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa4680047e0 con 0x7fa4700aa1f0 2024-05-10T03:29:39.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.482+0000 7fa466a00640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 0x7fa44c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:39.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.482+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa468088d00 con 0x7fa4700aa1f0 2024-05-10T03:29:39.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.483+0000 7fa46de00640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 0x7fa44c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:39.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.484+0000 7fa46de00640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 0x7fa44c06d140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa460035000 tx=0x7fa46002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:39.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.487+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa4680541c0 con 0x7fa4700aa1f0 2024-05-10T03:29:39.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.730+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4700b1e70 con 0x7fa4700aa1f0 2024-05-10T03:29:39.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.731+0000 7fa466a00640 1 -- 172.21.15.55:0/1712671936 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fa4680577e0 con 0x7fa4700aa1f0 2024-05-10T03:29:39.735 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:39.735 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T03:29:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2024-05-10T03:29:39.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 msgr2=0x7fa44c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 0x7fa44c06d140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa460035000 tx=0x7fa46002f000 comp rx=0 tx=0).stop 2024-05-10T03:29:39.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 msgr2=0x7fa4701f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:39.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa470161100 tx=0x7fa46800db80 comp rx=0 tx=0).stop 2024-05-10T03:29:39.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 shutdown_connections 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4700aa1f0 0x7fa4701f6e40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa44c06ac30 0x7fa44c06d140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa47015a410 0x7fa4700a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 --2- 172.21.15.55:0/1712671936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4540039e0 0x7fa4700cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 >> 172.21.15.55:0/1712671936 conn(0x7fa4700b5730 msgr2=0x7fa470160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.734+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 shutdown_connections 2024-05-10T03:29:39.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:39.735+0000 7fa46f200640 1 -- 172.21.15.55:0/1712671936 wait complete. 2024-05-10T03:29:39.765 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:40.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.037+0000 7ff9fb400640 1 Processor -- start 2024-05-10T03:29:40.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.037+0000 7ff9fb400640 1 -- start start 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.040+0000 7ff9fb400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff9f415fe50 0x7ff9f4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.040+0000 7ff9fb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9f40b2140 0x7ff9f4142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.040+0000 7ff9fb400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f406ad20 con 0x7ff9f40b2550 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.040+0000 7ff9fb400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f4141f50 con 0x7ff9f40b2140 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.040+0000 7ff9fb400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f406a110 con 0x7ff9f415fe50 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9f40b2140 0x7ff9f4142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9f40b2140 0x7ff9f4142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38934/0 (socket says 172.21.15.55:38934) 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 -- 172.21.15.55:0/3238253580 learned_addr learned my addr 172.21.15.55:0/3238253580 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f8a00640 1 --2- 172.21.15.55:0/3238253580 >> v2:172.21.15.162:3300/0 conn(0x7ff9f415fe50 0x7ff9f4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 -- 172.21.15.55:0/3238253580 >> v2:172.21.15.162:3300/0 conn(0x7ff9f415fe50 msgr2=0x7ff9f4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 --2- 172.21.15.55:0/3238253580 >> v2:172.21.15.162:3300/0 conn(0x7ff9f415fe50 0x7ff9f4141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 -- 172.21.15.55:0/3238253580 >> v1:172.21.15.162:6789/0 conn(0x7ff9f40b2550 legacy=0x7ff9f406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.041+0000 7ff9f3e00640 1 -- 172.21.15.55:0/3238253580 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f406a3f0 con 0x7ff9f40b2140 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.042+0000 7ff9f3e00640 1 --2- 172.21.15.55:0/3238253580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2140 0x7ff9f4142440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff9f406e520 tx=0x7ff9e002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba54d5d746296e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9f3400640 1 -- 172.21.15.55:0/3238253580 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0032440 con 0x7ff9f40b2140 2024-05-10T03:29:40.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9f3400640 1 -- 172.21.15.55:0/3238253580 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e00325a0 con 0x7ff9f40b2140 2024-05-10T03:29:40.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9f3400640 1 -- 172.21.15.55:0/3238253580 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0032930 con 0x7ff9f40b2140 2024-05-10T03:29:40.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 -- 172.21.15.55:0/3238253580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2140 msgr2=0x7ff9f4142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 --2- 172.21.15.55:0/3238253580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2140 0x7ff9f4142440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff9f406e520 tx=0x7ff9e002a2c0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 -- 172.21.15.55:0/3238253580 shutdown_connections 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 --2- 172.21.15.55:0/3238253580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2140 0x7ff9f4142440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 --2- 172.21.15.55:0/3238253580 >> v2:172.21.15.162:3300/0 conn(0x7ff9f415fe50 0x7ff9f4141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.043+0000 7ff9fb400640 1 -- 172.21.15.55:0/3238253580 >> 172.21.15.55:0/3238253580 conn(0x7ff9f40cbb90 msgr2=0x7ff9f40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.044+0000 7ff9fb400640 1 -- 172.21.15.55:0/3238253580 shutdown_connections 2024-05-10T03:29:40.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.044+0000 7ff9fb400640 1 -- 172.21.15.55:0/3238253580 wait complete. 2024-05-10T03:29:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.046+0000 7ff9fb400640 1 Processor -- start 2024-05-10T03:29:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.046+0000 7ff9fb400640 1 -- start start 2024-05-10T03:29:40.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9f40b2140 0x7ff9f40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9f415fe50 0x7ff9f4137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4143a70 con 0x7ff9f40b2550 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4137900 con 0x7ff9f40b2140 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9fb400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4157020 con 0x7ff9f415fe50 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38950/0 (socket says 172.21.15.55:38950) 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.048+0000 7ff9f3e00640 1 -- 172.21.15.55:0/855350995 learned_addr learned my addr 172.21.15.55:0/855350995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f8a00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9f40b2140 0x7ff9f40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f9400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9f415fe50 0x7ff9f4137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 -- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9f415fe50 msgr2=0x7ff9f4137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9f415fe50 0x7ff9f4137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 -- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9f40b2140 msgr2=0x7ff9f40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9f40b2140 0x7ff9f40af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 -- 172.21.15.55:0/855350995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00094d0 con 0x7ff9f40b2550 2024-05-10T03:29:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.049+0000 7ff9f3e00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0000c00 tx=0x7ff9e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.050+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0032440 con 0x7ff9f40b2550 2024-05-10T03:29:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.050+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0030ba0 con 0x7ff9f40b2550 2024-05-10T03:29:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.050+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e003d440 con 0x7ff9f40b2550 2024-05-10T03:29:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.050+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f4138e80 con 0x7ff9f40b2550 2024-05-10T03:29:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.050+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f4155850 con 0x7ff9f40b2550 2024-05-10T03:29:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.051+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f4066f60 con 0x7ff9f40b2550 2024-05-10T03:29:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.052+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9e0039040 con 0x7ff9f40b2550 2024-05-10T03:29:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.053+0000 7ff9f0c00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 0x7ff9d406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.053+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9e00a9ba0 con 0x7ff9f40b2550 2024-05-10T03:29:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.054+0000 7ff9f8a00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 0x7ff9d406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.055+0000 7ff9f8a00640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 0x7ff9d406d110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80079a0 tx=0x7ff9e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.058+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff9e0075040 con 0x7ff9f40b2550 2024-05-10T03:29:40.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.257+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f4067220 con 0x7ff9f40b2550 2024-05-10T03:29:40.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.259+0000 7ff9f0c00640 1 -- 172.21.15.55:0/855350995 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7ff9e0037090 con 0x7ff9f40b2550 2024-05-10T03:29:40.263 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:40.263 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:35.275824+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 msgr2=0x7ff9d406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 0x7ff9d406d110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80079a0 tx=0x7ff9e8008040 comp rx=0 tx=0).stop 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 msgr2=0x7ff9f40afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0000c00 tx=0x7ff9e00046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 shutdown_connections 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9f415fe50 0x7ff9f4137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9f40b2550 0x7ff9f40afdf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9d406ac00 0x7ff9d406d110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 --2- 172.21.15.55:0/855350995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9f40b2140 0x7ff9f40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 >> 172.21.15.55:0/855350995 conn(0x7ff9f40cbb90 msgr2=0x7ff9f415efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 shutdown_connections 2024-05-10T03:29:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.262+0000 7ff9fb400640 1 -- 172.21.15.55:0/855350995 wait complete. 2024-05-10T03:29:40.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:29:40.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:29:40.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.571+0000 7fc497000640 1 Processor -- start 2024-05-10T03:29:40.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.571+0000 7fc497000640 1 -- start start 2024-05-10T03:29:40.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.573+0000 7fc497000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 0x7fc49015d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.573+0000 7fc497000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4900ce260 0x7fc49006a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.573+0000 7fc497000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4900ceb00 con 0x7fc4900b1bc0 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.573+0000 7fc497000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49015dcd0 con 0x7fc4900ce260 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.573+0000 7fc497000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900d2150 con 0x7fc4900b2110 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc496600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4900ce260 0x7fc49006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc495200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc4900b1bc0 0x7fc4900cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50944/0 (socket says 172.21.15.55:50944) 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc495200640 1 -- 172.21.15.55:0/2594655683 learned_addr learned my addr 172.21.15.55:0/2594655683 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:40.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc495c00640 1 --2- 172.21.15.55:0/2594655683 >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 0x7fc49015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc496600640 1 -- 172.21.15.55:0/2594655683 >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 msgr2=0x7fc49015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc496600640 1 --2- 172.21.15.55:0/2594655683 >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 0x7fc49015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc496600640 1 -- 172.21.15.55:0/2594655683 >> v1:172.21.15.55:6789/0 conn(0x7fc4900b1bc0 legacy=0x7fc4900cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc496600640 1 -- 172.21.15.55:0/2594655683 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900b2910 con 0x7fc4900ce260 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.574+0000 7fc495c00640 1 --2- 172.21.15.55:0/2594655683 >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 0x7fc49015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:29:40.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.575+0000 7fc496600640 1 --2- 172.21.15.55:0/2594655683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc49006a110 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc49006c910 tx=0x7fc48802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe04019679fc4fb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.576+0000 7fc48fe00640 1 -- 172.21.15.55:0/2594655683 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48802ae60 con 0x7fc4900ce260 2024-05-10T03:29:40.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.576+0000 7fc48fe00640 1 -- 172.21.15.55:0/2594655683 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488032440 con 0x7fc4900ce260 2024-05-10T03:29:40.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.576+0000 7fc48fe00640 1 -- 172.21.15.55:0/2594655683 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880327d0 con 0x7fc4900ce260 2024-05-10T03:29:40.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.576+0000 7fc497000640 1 -- 172.21.15.55:0/2594655683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 msgr2=0x7fc49006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.576+0000 7fc497000640 1 --2- 172.21.15.55:0/2594655683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc49006a110 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc49006c910 tx=0x7fc48802a110 comp rx=0 tx=0).stop 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 -- 172.21.15.55:0/2594655683 shutdown_connections 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 --2- 172.21.15.55:0/2594655683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc49006a110 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 --2- 172.21.15.55:0/2594655683 >> v2:172.21.15.162:3300/0 conn(0x7fc4900b2110 0x7fc49015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 -- 172.21.15.55:0/2594655683 >> 172.21.15.55:0/2594655683 conn(0x7fc4900d30d0 msgr2=0x7fc4900d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 -- 172.21.15.55:0/2594655683 shutdown_connections 2024-05-10T03:29:40.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.577+0000 7fc497000640 1 -- 172.21.15.55:0/2594655683 wait complete. 2024-05-10T03:29:40.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.579+0000 7fc497000640 1 Processor -- start 2024-05-10T03:29:40.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.579+0000 7fc497000640 1 -- start start 2024-05-10T03:29:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4900b1bc0 0x7fc4900b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 0x7fc4900b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901f5d00 con 0x7fc4900ce260 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490159e20 con 0x7fc4900b1bc0 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc497000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901f44b0 con 0x7fc4900b2110 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc496600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc496600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38958/0 (socket says 172.21.15.55:38958) 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc496600640 1 -- 172.21.15.55:0/3875686294 learned_addr learned my addr 172.21.15.55:0/3875686294 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc495200640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 0x7fc4900b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc495c00640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4900b1bc0 0x7fc4900b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc496600640 1 -- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 msgr2=0x7fc4900b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.581+0000 7fc496600640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 0x7fc4900b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc496600640 1 -- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4900b1bc0 msgr2=0x7fc4900b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc496600640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4900b1bc0 0x7fc4900b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc496600640 1 -- 172.21.15.55:0/3875686294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009400 con 0x7fc4900ce260 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc495200640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 0x7fc4900b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc496600640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc488009890 tx=0x7fc4880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488030850 con 0x7fc4900ce260 2024-05-10T03:29:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4880309d0 con 0x7fc4900ce260 2024-05-10T03:29:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488030d80 con 0x7fc4900ce260 2024-05-10T03:29:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49015a070 con 0x7fc4900ce260 2024-05-10T03:29:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.582+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4901f4140 con 0x7fc4900ce260 2024-05-10T03:29:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.583+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4901f4620 con 0x7fc4900ce260 2024-05-10T03:29:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.585+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc488039040 con 0x7fc4900ce260 2024-05-10T03:29:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.586+0000 7fc48d600640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 0x7fc47006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.586+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4880a9ae0 con 0x7fc4900ce260 2024-05-10T03:29:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.586+0000 7fc495c00640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 0x7fc47006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.588+0000 7fc495c00640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 0x7fc47006d140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc484009ed0 tx=0x7fc48400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.590+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc488075090 con 0x7fc4900ce260 2024-05-10T03:29:40.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.790+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc49004e290 con 0x7fc4900ce260 2024-05-10T03:29:40.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.791+0000 7fc48d600640 1 -- 172.21.15.55:0/3875686294 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4880786b0 con 0x7fc4900ce260 2024-05-10T03:29:40.795 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 msgr2=0x7fc47006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 0x7fc47006d140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc484009ed0 tx=0x7fc48400b040 comp rx=0 tx=0).stop 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 msgr2=0x7fc4901f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc488009890 tx=0x7fc4880046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 shutdown_connections 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4900ce260 0x7fc4901f4cd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4900b2110 0x7fc4900b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc47006ac30 0x7fc47006d140 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 --2- 172.21.15.55:0/3875686294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4900b1bc0 0x7fc4900b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:40.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 >> 172.21.15.55:0/3875686294 conn(0x7fc4900d30d0 msgr2=0x7fc4900b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:40.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 shutdown_connections 2024-05-10T03:29:40.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:40.794+0000 7fc497000640 1 -- 172.21.15.55:0/3875686294 wait complete. 2024-05-10T03:29:40.822 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:29:41.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.097+0000 7fecd7200640 1 Processor -- start 2024-05-10T03:29:41.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.097+0000 7fecd7200640 1 -- start start 2024-05-10T03:29:41.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.099+0000 7fecd7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fecd01612f0 0x7fecd015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.099+0000 7fecd7200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fecd006b520 0x7fecd00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.099+0000 7fecd7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd00ce610 0x7fecd00b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.099+0000 7fecd7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd015dc80 con 0x7fecd01612f0 2024-05-10T03:29:41.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd7200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd00cfa00 con 0x7fecd006b520 2024-05-10T03:29:41.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd00cea20 con 0x7fecd00ce610 2024-05-10T03:29:41.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fecd01612f0 0x7fecd015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fecd01612f0 0x7fecd015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38966/0 (socket says 172.21.15.55:38966) 2024-05-10T03:29:41.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd5e00640 1 -- 172.21.15.55:0/4033933793 learned_addr learned my addr 172.21.15.55:0/4033933793 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd5400640 1 --2- 172.21.15.55:0/4033933793 >> v2:172.21.15.162:3300/0 conn(0x7fecd006b520 0x7fecd00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.100+0000 7fecd6800640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd00ce610 0x7fecd00b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.101+0000 7fecd5e00640 1 -- 172.21.15.55:0/4033933793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd00ce610 msgr2=0x7fecd00b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.101+0000 7fecd5e00640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd00ce610 0x7fecd00b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.101+0000 7fecd5e00640 1 -- 172.21.15.55:0/4033933793 >> v2:172.21.15.162:3300/0 conn(0x7fecd006b520 msgr2=0x7fecd00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.101+0000 7fecd5e00640 1 --2- 172.21.15.55:0/4033933793 >> v2:172.21.15.162:3300/0 conn(0x7fecd006b520 0x7fecd00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.101+0000 7fecd5e00640 1 -- 172.21.15.55:0/4033933793 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd006a480 con 0x7fecd01612f0 2024-05-10T03:29:41.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.102+0000 7fecd5e00640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd01612f0 0x7fecd015d740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecd00d3f90 tx=0x7fecc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156241e77652179 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.102+0000 7fecd4a00640 1 -- 172.21.15.55:0/4033933793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4032440 con 0x7fecd01612f0 2024-05-10T03:29:41.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.102+0000 7fecd4a00640 1 -- 172.21.15.55:0/4033933793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40325a0 con 0x7fecd01612f0 2024-05-10T03:29:41.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.102+0000 7fecd4a00640 1 -- 172.21.15.55:0/4033933793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4032930 con 0x7fecd01612f0 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 -- 172.21.15.55:0/4033933793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd01612f0 msgr2=0x7fecd015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd01612f0 0x7fecd015d740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecd00d3f90 tx=0x7fecc402a210 comp rx=0 tx=0).stop 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 -- 172.21.15.55:0/4033933793 shutdown_connections 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd00ce610 0x7fecd00b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 --2- 172.21.15.55:0/4033933793 >> v2:172.21.15.162:3300/0 conn(0x7fecd006b520 0x7fecd00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 --2- 172.21.15.55:0/4033933793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd01612f0 0x7fecd015d740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 -- 172.21.15.55:0/4033933793 >> 172.21.15.55:0/4033933793 conn(0x7fecd00d0830 msgr2=0x7fecd00d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:41.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 -- 172.21.15.55:0/4033933793 shutdown_connections 2024-05-10T03:29:41.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.103+0000 7fecd7200640 1 -- 172.21.15.55:0/4033933793 wait complete. 2024-05-10T03:29:41.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.106+0000 7fecd7200640 1 Processor -- start 2024-05-10T03:29:41.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.106+0000 7fecd7200640 1 -- start start 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd00ce610 0x7fecd01606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd01612f0 0x7fecd01f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd015f280 con 0x7fecd006b520 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd01f4220 con 0x7fecd00ce610 2024-05-10T03:29:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.107+0000 7fecd7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd015f3f0 con 0x7fecd01612f0 2024-05-10T03:29:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38982/0 (socket says 172.21.15.55:38982) 2024-05-10T03:29:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 -- 172.21.15.55:0/2804471077 learned_addr learned my addr 172.21.15.55:0/2804471077 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 -- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd01612f0 msgr2=0x7fecd01f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd01612f0 0x7fecd01f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 -- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd00ce610 msgr2=0x7fecd01606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd00ce610 0x7fecd01606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 -- 172.21.15.55:0/2804471077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc40094d0 con 0x7fecd006b520 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.108+0000 7fecd5e00640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.109+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4032440 con 0x7fecd006b520 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.109+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4030b80 con 0x7fecd006b520 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.109+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd0160250 con 0x7fecd006b520 2024-05-10T03:29:41.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.109+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc403d440 con 0x7fecd006b520 2024-05-10T03:29:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.109+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd0145950 con 0x7fecd006b520 2024-05-10T03:29:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.110+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd01603e0 con 0x7fecd006b520 2024-05-10T03:29:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.111+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fecc4039040 con 0x7fecd006b520 2024-05-10T03:29:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.112+0000 7fecca000640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 0x7fecb406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.112+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fecc40a9d10 con 0x7fecd006b520 2024-05-10T03:29:41.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.113+0000 7fecd5400640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 0x7fecb406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.114+0000 7fecd5400640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 0x7fecb406d110 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd00d2d60 tx=0x7fecc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.117+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fecc4075110 con 0x7fecd006b520 2024-05-10T03:29:41.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.325+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fecd004e290 con 0x7fecd006b520 2024-05-10T03:29:41.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.327+0000 7fecca000640 1 -- 172.21.15.55:0/2804471077 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fecc4041090 con 0x7fecd006b520 2024-05-10T03:29:41.330 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:29:41.331 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:29:41.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 msgr2=0x7fecb406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 0x7fecb406d110 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd00d2d60 tx=0x7fecc0008040 comp rx=0 tx=0).stop 2024-05-10T03:29:41.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 msgr2=0x7fecd0145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 shutdown_connections 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecd01612f0 0x7fecd01f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecb406ac00 0x7fecb406d110 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd00ce610 0x7fecd01606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.329+0000 7fecd7200640 1 --2- 172.21.15.55:0/2804471077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecd006b520 0x7fecd0145410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.330+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 >> 172.21.15.55:0/2804471077 conn(0x7fecd00d0830 msgr2=0x7fecd015bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.330+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 shutdown_connections 2024-05-10T03:29:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.330+0000 7fecd7200640 1 -- 172.21.15.55:0/2804471077 wait complete. 2024-05-10T03:29:41.358 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:29:41.358 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:29:41.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.640+0000 7feaa5200640 1 Processor -- start 2024-05-10T03:29:41.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.640+0000 7feaa5200640 1 -- start start 2024-05-10T03:29:41.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.642+0000 7feaa5200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feaa0071e70 0x7feaa015f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.642+0000 7feaa5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7feaa5200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa0071b80 con 0x7feaa015c350 2024-05-10T03:29:41.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7feaa5200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa0072280 con 0x7feaa0071e70 2024-05-10T03:29:41.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7feaa5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa015bc20 con 0x7feaa015c760 2024-05-10T03:29:41.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:56572/0 (socket says 172.21.15.55:56572) 2024-05-10T03:29:41.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7fea9fe00640 1 -- 172.21.15.55:0/1358822151 learned_addr learned my addr 172.21.15.55:0/1358822151 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:41.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7fea9ea00640 1 --1- 172.21.15.55:0/1358822151 >> v1:172.21.15.162:6789/0 conn(0x7feaa015c350 0x7feaa006b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:35628/0 (socket says 172.21.15.55:35628) 2024-05-10T03:29:41.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.643+0000 7fea9f400640 1 --2- 172.21.15.55:0/1358822151 >> v2:172.21.15.162:3300/0 conn(0x7feaa0071e70 0x7feaa015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.644+0000 7fea9fe00640 1 -- 172.21.15.55:0/1358822151 >> v2:172.21.15.162:3300/0 conn(0x7feaa0071e70 msgr2=0x7feaa015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.644+0000 7fea9fe00640 1 --2- 172.21.15.55:0/1358822151 >> v2:172.21.15.162:3300/0 conn(0x7feaa0071e70 0x7feaa015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.644+0000 7fea9fe00640 1 -- 172.21.15.55:0/1358822151 >> v1:172.21.15.162:6789/0 conn(0x7feaa015c350 legacy=0x7feaa006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.644+0000 7fea9fe00640 1 -- 172.21.15.55:0/1358822151 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa015f910 con 0x7feaa015c760 2024-05-10T03:29:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.646+0000 7fea9fe00640 1 --2- 172.21.15.55:0/1358822151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea98009e50 tx=0x7fea9802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba6b9126ee9bab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7fea9e000640 1 -- 172.21.15.55:0/1358822151 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9802aea0 con 0x7feaa015c760 2024-05-10T03:29:41.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7fea9e000640 1 -- 172.21.15.55:0/1358822151 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9802d050 con 0x7feaa015c760 2024-05-10T03:29:41.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7fea9e000640 1 -- 172.21.15.55:0/1358822151 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98031820 con 0x7feaa015c760 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 -- 172.21.15.55:0/1358822151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 msgr2=0x7feaa00cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 --2- 172.21.15.55:0/1358822151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fea98009e50 tx=0x7fea9802a680 comp rx=0 tx=0).stop 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 -- 172.21.15.55:0/1358822151 shutdown_connections 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 --2- 172.21.15.55:0/1358822151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c760 0x7feaa00cb900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 --2- 172.21.15.55:0/1358822151 >> v2:172.21.15.162:3300/0 conn(0x7feaa0071e70 0x7feaa015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.647+0000 7feaa5200640 1 -- 172.21.15.55:0/1358822151 >> 172.21.15.55:0/1358822151 conn(0x7feaa006c070 msgr2=0x7feaa006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.648+0000 7feaa5200640 1 -- 172.21.15.55:0/1358822151 shutdown_connections 2024-05-10T03:29:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.648+0000 7feaa5200640 1 -- 172.21.15.55:0/1358822151 wait complete. 2024-05-10T03:29:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.650+0000 7feaa5200640 1 Processor -- start 2024-05-10T03:29:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.650+0000 7feaa5200640 1 -- start start 2024-05-10T03:29:41.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 0x7feaa0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c350 0x7feaa00ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00c50d0 con 0x7feaa015c760 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00c73e0 con 0x7feaa0071e70 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7feaa5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00cabe0 con 0x7feaa015c350 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38998/0 (socket says 172.21.15.55:38998) 2024-05-10T03:29:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7fea9fe00640 1 -- 172.21.15.55:0/2029988644 learned_addr learned my addr 172.21.15.55:0/2029988644 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7fea9ea00640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c350 0x7feaa00ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.652+0000 7fea9f400640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 0x7feaa0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 -- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c350 msgr2=0x7feaa00ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c350 0x7feaa00ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 -- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 msgr2=0x7feaa0154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 0x7feaa0154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 -- 172.21.15.55:0/2029988644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea98009b30 con 0x7feaa015c760 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9f400640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 0x7feaa0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:29:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea9fe00640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea98034040 tx=0x7fea98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.653+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9803d860 con 0x7feaa015c760 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.654+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9803d9c0 con 0x7feaa015c760 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.654+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9803dd70 con 0x7feaa015c760 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.654+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa00c52e0 con 0x7feaa015c760 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.654+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa00c58a0 con 0x7feaa015c760 2024-05-10T03:29:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.655+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa00cdc50 con 0x7feaa015c760 2024-05-10T03:29:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.656+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fea9803b400 con 0x7feaa015c760 2024-05-10T03:29:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.657+0000 7fea97400640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 0x7fea8006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.657+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea980a9e00 con 0x7feaa015c760 2024-05-10T03:29:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.658+0000 7fea9f400640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 0x7fea8006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.659+0000 7fea9f400640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 0x7fea8006d110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea90009af0 tx=0x7fea90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.662+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fea98075200 con 0x7feaa015c760 2024-05-10T03:29:41.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:41.862+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feaa0067220 con 0x7feaa015c760 2024-05-10T03:29:42.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.714+0000 7fea97400640 1 -- 172.21.15.55:0/2029988644 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea980410b0 con 0x7feaa015c760 2024-05-10T03:29:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:29:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 msgr2=0x7fea8006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 0x7fea8006d110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea90009af0 tx=0x7fea90008040 comp rx=0 tx=0).stop 2024-05-10T03:29:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 msgr2=0x7feaa013f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea98034040 tx=0x7fea98004a40 comp rx=0 tx=0).stop 2024-05-10T03:29:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 shutdown_connections 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaa015c760 0x7feaa013f380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaa015c350 0x7feaa00ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea8006ac00 0x7fea8006d110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 --2- 172.21.15.55:0/2029988644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaa0071e70 0x7feaa0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 >> 172.21.15.55:0/2029988644 conn(0x7feaa006c070 msgr2=0x7feaa00cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.720+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 shutdown_connections 2024-05-10T03:29:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:42.721+0000 7feaa5200640 1 -- 172.21.15.55:0/2029988644 wait complete. 2024-05-10T03:29:42.759 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T03:29:43.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.028+0000 7f28dd000640 1 Processor -- start 2024-05-10T03:29:43.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.028+0000 7f28dd000640 1 -- start start 2024-05-10T03:29:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.030+0000 7f28dd000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28d815b3b0 0x7f28d815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.030+0000 7f28dd000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80700a0 con 0x7f28d8154370 2024-05-10T03:29:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28dd000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d8070380 con 0x7f28d815aee0 2024-05-10T03:29:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28dd000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d8070660 con 0x7f28d815b3b0 2024-05-10T03:29:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28d7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28d815b3b0 0x7f28d815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28d7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28d815b3b0 0x7f28d815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59700/0 (socket says 172.21.15.55:59700) 2024-05-10T03:29:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28d7e00640 1 -- 172.21.15.55:0/1984972851 learned_addr learned my addr 172.21.15.55:0/1984972851 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.031+0000 7f28d7e00640 1 -- 172.21.15.55:0/1984972851 >> v1:172.21.15.162:6789/0 conn(0x7f28d815aee0 legacy=0x7f28d8070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.032+0000 7f28d7e00640 1 -- 172.21.15.55:0/1984972851 >> v1:172.21.15.55:6789/0 conn(0x7f28d8154370 legacy=0x7f28d80b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.032+0000 7f28d7e00640 1 -- 172.21.15.55:0/1984972851 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8070940 con 0x7f28d815b3b0 2024-05-10T03:29:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.032+0000 7f28d6000640 1 -- 172.21.15.55:0/1984972851 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219614696 0 0) 0x7f28d80700a0 con 0x7f28d8154370 2024-05-10T03:29:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.033+0000 7f28d7e00640 1 --2- 172.21.15.55:0/1984972851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815b3b0 0x7f28d815b7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009e50 tx=0x7f28d002a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e94c9a41a32ef14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.033+0000 7f28d6000640 1 -- 172.21.15.55:0/1984972851 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0031600 con 0x7f28d815b3b0 2024-05-10T03:29:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.033+0000 7f28d6000640 1 -- 172.21.15.55:0/1984972851 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d002d050 con 0x7f28d815b3b0 2024-05-10T03:29:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.033+0000 7f28d6000640 1 -- 172.21.15.55:0/1984972851 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0031980 con 0x7f28d815b3b0 2024-05-10T03:29:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 -- 172.21.15.55:0/1984972851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815b3b0 msgr2=0x7f28d815b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 --2- 172.21.15.55:0/1984972851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815b3b0 0x7f28d815b7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009e50 tx=0x7f28d002a720 comp rx=0 tx=0).stop 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 -- 172.21.15.55:0/1984972851 shutdown_connections 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 --2- 172.21.15.55:0/1984972851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815b3b0 0x7f28d815b7c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 -- 172.21.15.55:0/1984972851 >> 172.21.15.55:0/1984972851 conn(0x7f28d80b1810 msgr2=0x7f28d806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 -- 172.21.15.55:0/1984972851 shutdown_connections 2024-05-10T03:29:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.034+0000 7f28dd000640 1 -- 172.21.15.55:0/1984972851 wait complete. 2024-05-10T03:29:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.036+0000 7f28dd000640 1 Processor -- start 2024-05-10T03:29:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.037+0000 7f28dd000640 1 -- start start 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d8154370 0x7f28d80ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28d815b3b0 0x7f28d8155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80add10 con 0x7f28d815aee0 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8144af0 con 0x7f28d8154370 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28dd000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80ade90 con 0x7f28d815b3b0 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28d6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28d6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59702/0 (socket says 172.21.15.55:59702) 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.038+0000 7f28d6a00640 1 -- 172.21.15.55:0/106889235 learned_addr learned my addr 172.21.15.55:0/106889235 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d7400640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d8154370 0x7f28d80ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d7e00640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28d815b3b0 0x7f28d8155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 -- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28d815b3b0 msgr2=0x7f28d8155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28d815b3b0 0x7f28d8155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 -- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d8154370 msgr2=0x7f28d80ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d8154370 0x7f28d80ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 -- 172.21.15.55:0/106889235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0009b30 con 0x7f28d815aee0 2024-05-10T03:29:43.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.039+0000 7f28d6a00640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3c0 tx=0x7f28c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.040+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c40035f0 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.040+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4003750 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.040+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4003b00 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.040+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d8158330 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.040+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d8157a00 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.041+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8159760 con 0x7f28d815aee0 2024-05-10T03:29:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.042+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f28c4003c60 con 0x7f28d815aee0 2024-05-10T03:29:43.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.044+0000 7f28cf400640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 0x7f28b806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:43.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.044+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f28c4088bb0 con 0x7f28d815aee0 2024-05-10T03:29:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.044+0000 7f28d7400640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 0x7f28b806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:43.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.046+0000 7f28d7400640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 0x7f28b806d140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f60 tx=0x7f28c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:43.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.049+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f28c4053ec0 con 0x7f28d815aee0 2024-05-10T03:29:43.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.253+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f28d804e370 con 0x7f28d815aee0 2024-05-10T03:29:43.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.736+0000 7f28cf400640 1 -- 172.21.15.55:0/106889235 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f28c4017070 con 0x7f28d815aee0 2024-05-10T03:29:43.742 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:29:43.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.743+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 msgr2=0x7f28b806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.743+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 0x7f28b806d140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f60 tx=0x7f28c8009040 comp rx=0 tx=0).stop 2024-05-10T03:29:43.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.743+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 msgr2=0x7f28d81557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:43.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.743+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3c0 tx=0x7f28c400c880 comp rx=0 tx=0).stop 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 shutdown_connections 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28d815b3b0 0x7f28d8155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28d815aee0 0x7f28d81557d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f28b806ac30 0x7f28b806d140 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f60 tx=0x7f28c8009040 comp rx=0 tx=0).stop 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 --2- 172.21.15.55:0/106889235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d8154370 0x7f28d80ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:43.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.745+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 >> 172.21.15.55:0/106889235 conn(0x7f28d80b1810 msgr2=0x7f28d8153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.753+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 shutdown_connections 2024-05-10T03:29:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:43.754+0000 7f28dd000640 1 -- 172.21.15.55:0/106889235 wait complete. 2024-05-10T03:29:43.792 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:29:44.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.037+0000 7f2996800640 1 Processor -- start 2024-05-10T03:29:44.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.037+0000 7f2996800640 1 -- start start 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.039+0000 7f2996800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29900b23f0 0x7f2990161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.039+0000 7f2996800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29900d29f0 con 0x7f29900ce740 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.039+0000 7f2996800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29901623d0 con 0x7f29900b8d70 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.039+0000 7f2996800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29900d1de0 con 0x7f29900b23f0 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.040+0000 7f298fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f29900ce740 0x7f299015cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53826/0 (socket says 172.21.15.55:53826) 2024-05-10T03:29:44.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.040+0000 7f298fe00640 1 -- 172.21.15.55:0/1013708907 learned_addr learned my addr 172.21.15.55:0/1013708907 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.040+0000 7f298f400640 1 --2- 172.21.15.55:0/1013708907 >> v2:172.21.15.162:3300/0 conn(0x7f29900b23f0 0x7f2990161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.040+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433026914 0 0) 0x7f29900d29f0 con 0x7f29900ce740 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.040+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003520 con 0x7f29900ce740 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2922291951 0 0) 0x7f2974003520 con 0x7f29900ce740 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 >> v2:172.21.15.162:3300/0 conn(0x7f29900b23f0 msgr2=0x7f2990161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:44.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 --2- 172.21.15.55:0/1013708907 >> v2:172.21.15.162:3300/0 conn(0x7f29900b23f0 0x7f2990161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 >> v1:172.21.15.162:6789/0 conn(0x7f29900b8d70 legacy=0x7f29900ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900d20c0 con 0x7f29900ce740 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2988001670 con 0x7f29900ce740 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 >> v1:172.21.15.55:6789/0 conn(0x7f29900ce740 legacy=0x7f299015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2974003970 0x7f2974003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 0x7f2974004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2974005110 0x7f297400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900d1de0 con 0x7f29740047c0 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297400c080 con 0x7f2974005110 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297400c240 con 0x7f2974003970 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.041+0000 7f298ea00640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 0x7f2974004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298fe00640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2974005110 0x7f297400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298ea00640 1 -- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2974003970 msgr2=0x7f2974003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:44.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298ea00640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2974003970 0x7f2974003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298ea00640 1 -- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2974005110 msgr2=0x7f297400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:44.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298ea00640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2974005110 0x7f297400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298ea00640 1 -- 172.21.15.55:0/1013708907 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974003520 con 0x7f29740047c0 2024-05-10T03:29:44.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.042+0000 7f298fe00640 1 -- 172.21.15.55:0/1013708907 reap_dead start 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.043+0000 7f298ea00640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 0x7f2974004bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2978002010 tx=0x7f29740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38df2fa0a5e3e52d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.043+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978031600 con 0x7f29740047c0 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.043+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297802d050 con 0x7f29740047c0 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.043+0000 7f298e000640 1 -- 172.21.15.55:0/1013708907 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780319c0 con 0x7f29740047c0 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 -- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 msgr2=0x7f2974004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 0x7f2974004bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2978002010 tx=0x7f29740057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 -- 172.21.15.55:0/1013708907 shutdown_connections 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 --2- 172.21.15.55:0/1013708907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29740047c0 0x7f2974004bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 -- 172.21.15.55:0/1013708907 >> 172.21.15.55:0/1013708907 conn(0x7f29900d0850 msgr2=0x7f29900d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 -- 172.21.15.55:0/1013708907 shutdown_connections 2024-05-10T03:29:44.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.044+0000 7f2996800640 1 -- 172.21.15.55:0/1013708907 wait complete. 2024-05-10T03:29:44.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.046+0000 7f2996800640 1 Processor -- start 2024-05-10T03:29:44.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.046+0000 7f2996800640 1 -- start start 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29740047c0 0x7f29900ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 0x7f29900aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900ce480 con 0x7f2990150330 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900cabe0 con 0x7f29900aa6f0 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f2996800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29900cb3f0 con 0x7f29740047c0 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f298fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f298fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59722/0 (socket says 172.21.15.55:59722) 2024-05-10T03:29:44.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f298fe00640 1 -- 172.21.15.55:0/3762558952 learned_addr learned my addr 172.21.15.55:0/3762558952 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.048+0000 7f298f400640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29740047c0 0x7f29900ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 -- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29740047c0 msgr2=0x7f29900ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29740047c0 0x7f29900ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 -- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 msgr2=0x7f29900aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298ea00640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 0x7f29900aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 0x7f29900aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 -- 172.21.15.55:0/3762558952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900b9300 con 0x7f2990150330 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298ea00640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 0x7f29900aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f298fe00640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29900cfb80 tx=0x7f298800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988010510 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2988010690 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29900cccb0 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988010a40 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.049+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29900cae30 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.050+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29900cce40 con 0x7f2990150330 2024-05-10T03:29:44.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.052+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2988010ba0 con 0x7f2990150330 2024-05-10T03:29:44.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.053+0000 7f2987400640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.053+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2988089160 con 0x7f2990150330 2024-05-10T03:29:44.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.053+0000 7f298f400640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.054+0000 7f298f400640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29800047b0 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.056+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2988054380 con 0x7f2990150330 2024-05-10T03:29:44.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.293+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2990067220 con 0x7f2990150330 2024-05-10T03:29:44.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.307+0000 7f2987400640 1 -- 172.21.15.55:0/3762558952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f29880220c0 con 0x7f2990150330 2024-05-10T03:29:44.311 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T03:29:44.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 msgr2=0x7f296c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29800047b0 tx=0x7f2980008040 comp rx=0 tx=0).stop 2024-05-10T03:29:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 msgr2=0x7f2990150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29900cfb80 tx=0x7f298800e960 comp rx=0 tx=0).stop 2024-05-10T03:29:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 shutdown_connections 2024-05-10T03:29:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2990150330 0x7f2990150740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29900aa6f0 0x7f29900aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.311+0000 7f2996800640 1 --2- 172.21.15.55:0/3762558952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29740047c0 0x7f29900ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.312+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 >> 172.21.15.55:0/3762558952 conn(0x7f29900d0850 msgr2=0x7f29900b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.312+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 shutdown_connections 2024-05-10T03:29:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.312+0000 7f2996800640 1 -- 172.21.15.55:0/3762558952 wait complete. 2024-05-10T03:29:44.360 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T03:29:44.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T03:29:44.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.600+0000 7fb6d7200640 1 Processor -- start 2024-05-10T03:29:44.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.600+0000 7fb6d7200640 1 -- start start 2024-05-10T03:29:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb6d00b8970 0x7fb6d00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d7200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 0x7fb6d00ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d00ce690 con 0x7fb6d00d12b0 2024-05-10T03:29:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d00d2b00 con 0x7fb6d00b8970 2024-05-10T03:29:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d7200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d006a2b0 con 0x7fb6d00cdd40 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d5e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb6d00d12b0 0x7fb6d00b65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53828/0 (socket says 172.21.15.55:53828) 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.602+0000 7fb6d5e00640 1 -- 172.21.15.55:0/2598302117 learned_addr learned my addr 172.21.15.55:0/2598302117 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d5400640 1 --2- 172.21.15.55:0/2598302117 >> v2:172.21.15.55:3300/0 conn(0x7fb6d00b8970 0x7fb6d00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d6800640 1 --2- 172.21.15.55:0/2598302117 >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 0x7fb6d00ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d4a00640 1 -- 172.21.15.55:0/2598302117 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749837864 0 0) 0x7fb6d00ce690 con 0x7fb6d00d12b0 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d4a00640 1 -- 172.21.15.55:0/2598302117 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b8003520 con 0x7fb6d00d12b0 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d5400640 1 -- 172.21.15.55:0/2598302117 >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 msgr2=0x7fb6d00ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d5400640 1 --2- 172.21.15.55:0/2598302117 >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 0x7fb6d00ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d5400640 1 -- 172.21.15.55:0/2598302117 >> v1:172.21.15.55:6789/0 conn(0x7fb6d00d12b0 legacy=0x7fb6d00b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d5400640 1 -- 172.21.15.55:0/2598302117 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d006a590 con 0x7fb6d00b8970 2024-05-10T03:29:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.603+0000 7fb6d6800640 1 --2- 172.21.15.55:0/2598302117 >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 0x7fb6d00ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.604+0000 7fb6d5400640 1 --2- 172.21.15.55:0/2598302117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00b8970 0x7fb6d00d2400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00b63a0 tx=0x7fb6bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319fa37488ca9360 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.604+0000 7fb6d4a00640 1 -- 172.21.15.55:0/2598302117 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc02ae60 con 0x7fb6d00b8970 2024-05-10T03:29:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.604+0000 7fb6d4a00640 1 -- 172.21.15.55:0/2598302117 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc032440 con 0x7fb6d00b8970 2024-05-10T03:29:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.604+0000 7fb6d4a00640 1 -- 172.21.15.55:0/2598302117 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0327b0 con 0x7fb6d00b8970 2024-05-10T03:29:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 -- 172.21.15.55:0/2598302117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00b8970 msgr2=0x7fb6d00d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 --2- 172.21.15.55:0/2598302117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00b8970 0x7fb6d00d2400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00b63a0 tx=0x7fb6bc02a110 comp rx=0 tx=0).stop 2024-05-10T03:29:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 -- 172.21.15.55:0/2598302117 shutdown_connections 2024-05-10T03:29:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 --2- 172.21.15.55:0/2598302117 >> v2:172.21.15.162:3300/0 conn(0x7fb6d00cdd40 0x7fb6d00ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 --2- 172.21.15.55:0/2598302117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00b8970 0x7fb6d00d2400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 -- 172.21.15.55:0/2598302117 >> 172.21.15.55:0/2598302117 conn(0x7fb6d00d3100 msgr2=0x7fb6d00d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 -- 172.21.15.55:0/2598302117 shutdown_connections 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.605+0000 7fb6d7200640 1 -- 172.21.15.55:0/2598302117 wait complete. 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.608+0000 7fb6d7200640 1 Processor -- start 2024-05-10T03:29:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.608+0000 7fb6d7200640 1 -- start start 2024-05-10T03:29:44.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d00b8970 0x7fb6d0157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d00cdd40 0x7fb6d01575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00cd5e0 con 0x7fb6d00d12b0 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d0146d90 con 0x7fb6d00cdd40 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d015a060 con 0x7fb6d00b8970 2024-05-10T03:29:44.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59744/0 (socket says 172.21.15.55:59744) 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.609+0000 7fb6d6800640 1 -- 172.21.15.55:0/4218008747 learned_addr learned my addr 172.21.15.55:0/4218008747 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d5e00640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d00b8970 0x7fb6d0157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d5400640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d00cdd40 0x7fb6d01575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 -- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d00b8970 msgr2=0x7fb6d0157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d00b8970 0x7fb6d0157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 -- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d00cdd40 msgr2=0x7fb6d01575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d00cdd40 0x7fb6d01575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 -- 172.21.15.55:0/4218008747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc009400 con 0x7fb6d00d12b0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6d6800640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00cd50 tx=0x7fb6cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00f680 con 0x7fb6d00d12b0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00f800 con 0x7fb6d00d12b0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.610+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00fbb0 con 0x7fb6d00d12b0 2024-05-10T03:29:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.611+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d00a9b10 con 0x7fb6d00d12b0 2024-05-10T03:29:44.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.611+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d01455c0 con 0x7fb6d00d12b0 2024-05-10T03:29:44.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.612+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0145750 con 0x7fb6d00d12b0 2024-05-10T03:29:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.613+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb6cc00b040 con 0x7fb6d00d12b0 2024-05-10T03:29:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.613+0000 7fb6ca000640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 0x7fb6b006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.614+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6cc089ea0 con 0x7fb6d00d12b0 2024-05-10T03:29:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.614+0000 7fb6d5e00640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 0x7fb6b006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.616+0000 7fb6d5e00640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 0x7fb6b006d140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005880 tx=0x7fb6c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:44.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.618+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb6cc054f50 con 0x7fb6d00d12b0 2024-05-10T03:29:44.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:44.818+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb6d0067220 con 0x7fb6d00d12b0 2024-05-10T03:29:45.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.318+0000 7fb6ca000640 1 -- 172.21.15.55:0/4218008747 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb6cc01d070 con 0x7fb6d00d12b0 2024-05-10T03:29:45.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 msgr2=0x7fb6b006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:45.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 0x7fb6b006d140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4005880 tx=0x7fb6c4009040 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 msgr2=0x7fb6d0157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00cd50 tx=0x7fb6cc00a650 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 shutdown_connections 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6d00d12b0 0x7fb6d0157b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb6b006ac30 0x7fb6b006d140 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d00cdd40 0x7fb6d01575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 --2- 172.21.15.55:0/4218008747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d00b8970 0x7fb6d0157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 >> 172.21.15.55:0/4218008747 conn(0x7fb6d00d3100 msgr2=0x7fb6d01604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 shutdown_connections 2024-05-10T03:29:45.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.322+0000 7fb6d7200640 1 -- 172.21.15.55:0/4218008747 wait complete. 2024-05-10T03:29:45.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:29:45.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.638+0000 7f94f5400640 1 Processor -- start 2024-05-10T03:29:45.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.638+0000 7f94f5400640 1 -- start start 2024-05-10T03:29:45.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f015f2e0 0x7f94f00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94f0155c70 0x7f94f00cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94f006b120 0x7f94f006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f015f6f0 con 0x7f94f0155c70 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f00b3e60 con 0x7f94f006b120 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94f5400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f015bee0 con 0x7f94f015f2e0 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94ede00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94f0155c70 0x7f94f00cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94ede00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94f0155c70 0x7f94f00cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59760/0 (socket says 172.21.15.55:59760) 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.641+0000 7f94ede00640 1 -- 172.21.15.55:0/2025491154 learned_addr learned my addr 172.21.15.55:0/2025491154 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ee800640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f015f2e0 0x7f94f00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ef200640 1 --2- 172.21.15.55:0/2025491154 >> v2:172.21.15.162:3300/0 conn(0x7f94f006b120 0x7f94f006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ede00640 1 -- 172.21.15.55:0/2025491154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f015f2e0 msgr2=0x7f94f00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ede00640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f015f2e0 0x7f94f00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ede00640 1 -- 172.21.15.55:0/2025491154 >> v2:172.21.15.162:3300/0 conn(0x7f94f006b120 msgr2=0x7f94f006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ede00640 1 --2- 172.21.15.55:0/2025491154 >> v2:172.21.15.162:3300/0 conn(0x7f94f006b120 0x7f94f006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.642+0000 7f94ede00640 1 -- 172.21.15.55:0/2025491154 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f006ba70 con 0x7f94f0155c70 2024-05-10T03:29:45.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.643+0000 7f94ede00640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f00cb930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94f006fe00 tx=0x7f94dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3651322cc0264306 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:45.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.643+0000 7f94ed400640 1 -- 172.21.15.55:0/2025491154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc032440 con 0x7f94f0155c70 2024-05-10T03:29:45.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.644+0000 7f94ed400640 1 -- 172.21.15.55:0/2025491154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc0325a0 con 0x7f94f0155c70 2024-05-10T03:29:45.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.644+0000 7f94ed400640 1 -- 172.21.15.55:0/2025491154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc032930 con 0x7f94f0155c70 2024-05-10T03:29:45.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.644+0000 7f94f5400640 1 -- 172.21.15.55:0/2025491154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 msgr2=0x7f94f00cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:45.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.644+0000 7f94f5400640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f00cb930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94f006fe00 tx=0x7f94dc02a210 comp rx=0 tx=0).stop 2024-05-10T03:29:45.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 -- 172.21.15.55:0/2025491154 shutdown_connections 2024-05-10T03:29:45.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 --2- 172.21.15.55:0/2025491154 >> v2:172.21.15.162:3300/0 conn(0x7f94f006b120 0x7f94f006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f00cb930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 --2- 172.21.15.55:0/2025491154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f015f2e0 0x7f94f00cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 -- 172.21.15.55:0/2025491154 >> 172.21.15.55:0/2025491154 conn(0x7f94f006c120 msgr2=0x7f94f006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:45.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 -- 172.21.15.55:0/2025491154 shutdown_connections 2024-05-10T03:29:45.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.645+0000 7f94f5400640 1 -- 172.21.15.55:0/2025491154 wait complete. 2024-05-10T03:29:45.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.647+0000 7f94f5400640 1 Processor -- start 2024-05-10T03:29:45.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.647+0000 7f94f5400640 1 -- start start 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 0x7f94f00a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94f015f2e0 0x7f94f01ef970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01ef520 con 0x7f94f0155c70 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01f0cb0 con 0x7f94f015f2e0 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94f5400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00a9550 con 0x7f94f006b120 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59762/0 (socket says 172.21.15.55:59762) 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 -- 172.21.15.55:0/1914862608 learned_addr learned my addr 172.21.15.55:0/1914862608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ee800640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 0x7f94f00a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ef200640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94f015f2e0 0x7f94f01ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 -- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 msgr2=0x7f94f00a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 0x7f94f00a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 -- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94f015f2e0 msgr2=0x7f94f01ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.649+0000 7f94ede00640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94f015f2e0 0x7f94f01ef970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94ede00640 1 -- 172.21.15.55:0/1914862608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0094d0 con 0x7f94f0155c70 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94ee800640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 0x7f94f00a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94ede00640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc032440 con 0x7f94f0155c70 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc030b80 con 0x7f94f0155c70 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc03d440 con 0x7f94f0155c70 2024-05-10T03:29:45.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f00a9390 con 0x7f94f0155c70 2024-05-10T03:29:45.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.650+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f0154000 con 0x7f94f0155c70 2024-05-10T03:29:45.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.651+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f0066f60 con 0x7f94f0155c70 2024-05-10T03:29:45.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.652+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f94dc039040 con 0x7f94f0155c70 2024-05-10T03:29:45.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.653+0000 7f94e6a00640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 0x7f94d006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:45.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.654+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f94dc0a9e20 con 0x7f94f0155c70 2024-05-10T03:29:45.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.654+0000 7f94ee800640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 0x7f94d006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:45.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.655+0000 7f94ee800640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 0x7f94d006d110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:45.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.658+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f94dc074f10 con 0x7f94f0155c70 2024-05-10T03:29:45.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:45.851+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f94f00a9390 con 0x7f94f0155c70 2024-05-10T03:29:46.412 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:29:46.421 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:29:47.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.767+0000 7f94e6a00640 1 -- 172.21.15.55:0/1914862608 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f94dc078530 con 0x7f94f0155c70 2024-05-10T03:29:47.772 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data_ec' created 2024-05-10T03:29:47.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.773+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 msgr2=0x7f94d006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:47.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.773+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 0x7f94d006d110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0008040 comp rx=0 tx=0).stop 2024-05-10T03:29:47.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.773+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 msgr2=0x7f94f0153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:47.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.773+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:47.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 shutdown_connections 2024-05-10T03:29:47.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94d006ac00 0x7f94d006d110 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0008040 comp rx=0 tx=0).stop 2024-05-10T03:29:47.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94f015f2e0 0x7f94f01ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:47.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94f0155c70 0x7f94f0153ac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:47.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 --2- 172.21.15.55:0/1914862608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94f006b120 0x7f94f00a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:47.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.774+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 >> 172.21.15.55:0/1914862608 conn(0x7f94f006c120 msgr2=0x7f94f0070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:47.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.775+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 shutdown_connections 2024-05-10T03:29:47.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:47.775+0000 7f94f5400640 1 -- 172.21.15.55:0/1914862608 wait complete. 2024-05-10T03:29:47.817 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T03:29:48.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.082+0000 7fc1da200640 1 Processor -- start 2024-05-10T03:29:48.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.083+0000 7fc1da200640 1 -- start start 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.085+0000 7fc1da200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1d40cf350 0x7fc1d40cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.085+0000 7fc1da200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1d40b3410 0x7fc1d40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.085+0000 7fc1da200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d4162400 con 0x7fc1d4140700 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.085+0000 7fc1da200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40b3d60 con 0x7fc1d40cf350 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.085+0000 7fc1da200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d4161880 con 0x7fc1d40b3410 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1d40cf350 0x7fc1d40cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1d40cf350 0x7fc1d40cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59772/0 (socket says 172.21.15.55:59772) 2024-05-10T03:29:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 -- 172.21.15.55:0/3890046116 learned_addr learned my addr 172.21.15.55:0/3890046116 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:48.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1d3e00640 1 --2- 172.21.15.55:0/3890046116 >> v2:172.21.15.162:3300/0 conn(0x7fc1d40b3410 0x7fc1d40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 -- 172.21.15.55:0/3890046116 >> v2:172.21.15.162:3300/0 conn(0x7fc1d40b3410 msgr2=0x7fc1d40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:48.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 --2- 172.21.15.55:0/3890046116 >> v2:172.21.15.162:3300/0 conn(0x7fc1d40b3410 0x7fc1d40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 -- 172.21.15.55:0/3890046116 >> v1:172.21.15.162:6789/0 conn(0x7fc1d4140700 legacy=0x7fc1d4143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:48.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.086+0000 7fc1caa00640 1 -- 172.21.15.55:0/3890046116 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d4161b60 con 0x7fc1d40cf350 2024-05-10T03:29:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.087+0000 7fc1caa00640 1 --2- 172.21.15.55:0/3890046116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d40cf760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40b5fa0 tx=0x7fc1bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf0cac2817560f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.088+0000 7fc1d2a00640 1 -- 172.21.15.55:0/3890046116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc032440 con 0x7fc1d40cf350 2024-05-10T03:29:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.088+0000 7fc1d2a00640 1 -- 172.21.15.55:0/3890046116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0325a0 con 0x7fc1d40cf350 2024-05-10T03:29:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.088+0000 7fc1d2a00640 1 -- 172.21.15.55:0/3890046116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc032930 con 0x7fc1d40cf350 2024-05-10T03:29:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.088+0000 7fc1da200640 1 -- 172.21.15.55:0/3890046116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 msgr2=0x7fc1d40cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.088+0000 7fc1da200640 1 --2- 172.21.15.55:0/3890046116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d40cf760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40b5fa0 tx=0x7fc1bc02a1a0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 -- 172.21.15.55:0/3890046116 shutdown_connections 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 --2- 172.21.15.55:0/3890046116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d40cf760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 --2- 172.21.15.55:0/3890046116 >> v2:172.21.15.162:3300/0 conn(0x7fc1d40b3410 0x7fc1d40b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 -- 172.21.15.55:0/3890046116 >> 172.21.15.55:0/3890046116 conn(0x7fc1d40b5390 msgr2=0x7fc1d40b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 -- 172.21.15.55:0/3890046116 shutdown_connections 2024-05-10T03:29:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.089+0000 7fc1da200640 1 -- 172.21.15.55:0/3890046116 wait complete. 2024-05-10T03:29:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.091+0000 7fc1da200640 1 Processor -- start 2024-05-10T03:29:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.091+0000 7fc1da200640 1 -- start start 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1d40b3410 0x7fc1d413e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 0x7fc1d4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d41f80c0 con 0x7fc1d40cf350 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d415a9e0 con 0x7fc1d40b3410 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1da200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d41f7de0 con 0x7fc1d4140700 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1caa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1caa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59784/0 (socket says 172.21.15.55:59784) 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1caa00640 1 -- 172.21.15.55:0/2209668461 learned_addr learned my addr 172.21.15.55:0/2209668461 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.093+0000 7fc1d3400640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1d40b3410 0x7fc1d413e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1d3e00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 0x7fc1d4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 -- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 msgr2=0x7fc1d4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 0x7fc1d4159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 -- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1d40b3410 msgr2=0x7fc1d413e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1d40b3410 0x7fc1d413e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 -- 172.21.15.55:0/2209668461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0094d0 con 0x7fc1d40cf350 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1d3e00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 0x7fc1d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1caa00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:48.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc032440 con 0x7fc1d40cf350 2024-05-10T03:29:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc030b80 con 0x7fc1d40cf350 2024-05-10T03:29:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d413da90 con 0x7fc1d40cf350 2024-05-10T03:29:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.094+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc03d420 con 0x7fc1d40cf350 2024-05-10T03:29:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.095+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d413e050 con 0x7fc1d40cf350 2024-05-10T03:29:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.096+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d413d8f0 con 0x7fc1d40cf350 2024-05-10T03:29:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.097+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1bc039040 con 0x7fc1d40cf350 2024-05-10T03:29:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.098+0000 7fc1cbe00640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 0x7fc1b006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.099+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fc1bc0aa310 con 0x7fc1d40cf350 2024-05-10T03:29:48.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.099+0000 7fc1d3400640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 0x7fc1b006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:48.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.100+0000 7fc1d3400640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 0x7fc1b006d110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006fd0 tx=0x7fc1c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:48.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.104+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc1d413d8f0 con 0x7fc1d40cf350 2024-05-10T03:29:48.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.303+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc1d413d8f0 con 0x7fc1d40cf350 2024-05-10T03:29:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.781+0000 7fc1cbe00640 1 -- 172.21.15.55:0/2209668461 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc1d413d8f0 con 0x7fc1d40cf350 2024-05-10T03:29:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.787+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 msgr2=0x7fc1b006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.787+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 0x7fc1b006d110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006fd0 tx=0x7fc1c4008040 comp rx=0 tx=0).stop 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.787+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 msgr2=0x7fc1d413c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.787+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 shutdown_connections 2024-05-10T03:29:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1d4140700 0x7fc1d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1d40cf350 0x7fc1d413c550 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1b006ac00 0x7fc1b006d110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 --2- 172.21.15.55:0/2209668461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1d40b3410 0x7fc1d413e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.788+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 >> 172.21.15.55:0/2209668461 conn(0x7fc1d40b5390 msgr2=0x7fc1d406a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:48.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.790+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 shutdown_connections 2024-05-10T03:29:48.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:48.793+0000 7fc1da200640 1 -- 172.21.15.55:0/2209668461 wait complete. 2024-05-10T03:29:48.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T03:29:49.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.086+0000 7f0837e00640 1 Processor -- start 2024-05-10T03:29:49.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.086+0000 7f0837e00640 1 -- start start 2024-05-10T03:29:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.088+0000 7f0837e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f083815c660 0x7f0838141a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0837e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cf960 0x7f083815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0837e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083806d430 con 0x7f08380cfd70 2024-05-10T03:29:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0837e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380b4620 con 0x7f083815c660 2024-05-10T03:29:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0837e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380d0340 con 0x7f08380cf960 2024-05-10T03:29:49.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0836a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f083815c660 0x7f0838141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0836a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f083815c660 0x7f0838141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59794/0 (socket says 172.21.15.55:59794) 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0836a00640 1 -- 172.21.15.55:0/1446727558 learned_addr learned my addr 172.21.15.55:0/1446727558 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.089+0000 7f0836000640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cf960 0x7f083815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.090+0000 7f0836a00640 1 -- 172.21.15.55:0/1446727558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cf960 msgr2=0x7f083815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.090+0000 7f0836a00640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cf960 0x7f083815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.090+0000 7f0836a00640 1 -- 172.21.15.55:0/1446727558 >> v1:172.21.15.55:6789/0 conn(0x7f08380cfd70 legacy=0x7f083815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.090+0000 7f0836a00640 1 -- 172.21.15.55:0/1446727558 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380d0620 con 0x7f083815c660 2024-05-10T03:29:49.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.091+0000 7f0836a00640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815c660 0x7f0838141a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0830009e50 tx=0x7f083002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aa4bd58808d626 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.091+0000 7f0835600640 1 -- 172.21.15.55:0/1446727558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830031600 con 0x7f083815c660 2024-05-10T03:29:49.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.091+0000 7f0835600640 1 -- 172.21.15.55:0/1446727558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083002d050 con 0x7f083815c660 2024-05-10T03:29:49.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.091+0000 7f0835600640 1 -- 172.21.15.55:0/1446727558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830031980 con 0x7f083815c660 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 -- 172.21.15.55:0/1446727558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815c660 msgr2=0x7f0838141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815c660 0x7f0838141a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0830009e50 tx=0x7f083002a710 comp rx=0 tx=0).stop 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 -- 172.21.15.55:0/1446727558 shutdown_connections 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cf960 0x7f083815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 --2- 172.21.15.55:0/1446727558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815c660 0x7f0838141a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 -- 172.21.15.55:0/1446727558 >> 172.21.15.55:0/1446727558 conn(0x7f08380b2a90 msgr2=0x7f08380b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 -- 172.21.15.55:0/1446727558 shutdown_connections 2024-05-10T03:29:49.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.092+0000 7f0837e00640 1 -- 172.21.15.55:0/1446727558 wait complete. 2024-05-10T03:29:49.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.094+0000 7f0837e00640 1 Processor -- start 2024-05-10T03:29:49.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.095+0000 7f0837e00640 1 -- start start 2024-05-10T03:29:49.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cfd70 0x7f08380cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f083815c660 0x7f083815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08381516f0 con 0x7f08380cf960 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380cbf50 con 0x7f083815c660 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0837e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380cc1a0 con 0x7f08380cfd70 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0836a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0836a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59800/0 (socket says 172.21.15.55:59800) 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.096+0000 7f0836a00640 1 -- 172.21.15.55:0/2747415594 learned_addr learned my addr 172.21.15.55:0/2747415594 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836000640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cfd70 0x7f08380cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0837400640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f083815c660 0x7f083815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 -- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cfd70 msgr2=0x7f08380cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cfd70 0x7f08380cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 -- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f083815c660 msgr2=0x7f083815dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f083815c660 0x7f083815dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 -- 172.21.15.55:0/2747415594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830009b30 con 0x7f08380cf960 2024-05-10T03:29:49.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f0836a00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f083002ad00 tx=0x7f0830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.097+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083003b820 con 0x7f08380cf960 2024-05-10T03:29:49.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.098+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083003b980 con 0x7f08380cf960 2024-05-10T03:29:49.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.098+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083003bcf0 con 0x7f08380cf960 2024-05-10T03:29:49.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.098+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08380cb5b0 con 0x7f08380cf960 2024-05-10T03:29:49.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.098+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08380ca590 con 0x7f08380cf960 2024-05-10T03:29:49.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.099+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083804e290 con 0x7f08380cf960 2024-05-10T03:29:49.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.100+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0830038050 con 0x7f08380cf960 2024-05-10T03:29:49.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.101+0000 7f082ea00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 0x7f081406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.101+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f08300a9f00 con 0x7f08380cf960 2024-05-10T03:29:49.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.102+0000 7f0836000640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 0x7f081406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.103+0000 7f0836000640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 0x7f081406d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0820006fd0 tx=0x7f0820008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.106+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0830074ce0 con 0x7f08380cf960 2024-05-10T03:29:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.355+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f083815b1e0 con 0x7f08380cf960 2024-05-10T03:29:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.367+0000 7f082ea00640 1 -- 172.21.15.55:0/2747415594 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0830036020 con 0x7f08380cf960 2024-05-10T03:29:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:added data pool 4 to fsmap 2024-05-10T03:29:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 msgr2=0x7f081406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 0x7f081406d140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0820006fd0 tx=0x7f0820008040 comp rx=0 tx=0).stop 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 msgr2=0x7f08380cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f083002ad00 tx=0x7f0830004a40 comp rx=0 tx=0).stop 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 shutdown_connections 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f081406ac30 0x7f081406d140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f083815c660 0x7f083815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08380cfd70 0x7f08380cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 --2- 172.21.15.55:0/2747415594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08380cf960 0x7f08380cb830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 >> 172.21.15.55:0/2747415594 conn(0x7f08380b2a90 msgr2=0x7f083813fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.370+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 shutdown_connections 2024-05-10T03:29:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.371+0000 7f0837e00640 1 -- 172.21.15.55:0/2747415594 wait complete. 2024-05-10T03:29:49.400 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-05-10T03:29:49.396+0000 7f3d33800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:29:49.402 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:29:49.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.672+0000 7f1eccc00640 1 Processor -- start 2024-05-10T03:29:49.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.672+0000 7f1eccc00640 1 -- start start 2024-05-10T03:29:49.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1eccc00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ec80cdef0 0x7f1ec815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1eccc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec806bcb0 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1eccc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec806bf90 con 0x7f1ec80ceb50 2024-05-10T03:29:49.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1eccc00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec806c270 con 0x7f1ec80cdef0 2024-05-10T03:29:49.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1ec5e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1ec80b6bf0 0x7f1ec80ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53846/0 (socket says 172.21.15.55:53846) 2024-05-10T03:29:49.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1ec5e00640 1 -- 172.21.15.55:0/3306518400 learned_addr learned my addr 172.21.15.55:0/3306518400 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040138705 0 0) 0x7f1ec806bcb0 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac003520 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.675+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3306518400 >> v2:172.21.15.162:3300/0 conn(0x7f1ec80cdef0 0x7f1ec815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030536742 0 0) 0x7f1eac003520 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 >> v2:172.21.15.162:3300/0 conn(0x7f1ec80cdef0 msgr2=0x7f1ec815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 --2- 172.21.15.55:0/3306518400 >> v2:172.21.15.162:3300/0 conn(0x7f1ec80cdef0 0x7f1ec815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 >> v1:172.21.15.162:6789/0 conn(0x7f1ec80ceb50 legacy=0x7f1ec80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80d1360 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f1eb8001670 con 0x7f1ec80b6bf0 2024-05-10T03:29:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 >> v1:172.21.15.55:6789/0 conn(0x7f1ec80b6bf0 legacy=0x7f1ec80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1eac003970 0x7f1eac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eac0047c0 0x7f1eac004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1eac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec806c270 con 0x7f1eac005230 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac00c1b0 con 0x7f1eac0047c0 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac00c370 con 0x7f1eac003970 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1eac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec5400640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eac0047c0 0x7f1eac004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.676+0000 7f1ec5e00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1eac003970 0x7f1eac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 -- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1eac003970 msgr2=0x7f1eac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1eac003970 0x7f1eac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 -- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eac0047c0 msgr2=0x7f1eac004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1eac0047c0 0x7f1eac004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 -- 172.21.15.55:0/3306518400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac003520 con 0x7f1eac005230 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec5e00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1eac003970 0x7f1eac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.677+0000 7f1ec6800640 1 -- 172.21.15.55:0/3306518400 reap_dead start 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1eac00b980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007330 tx=0x7f1eac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e15b813cf974fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0003b90 con 0x7f1eac005230 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec0003cf0 con 0x7f1eac005230 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1ec4a00640 1 -- 172.21.15.55:0/3306518400 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0004260 con 0x7f1eac005230 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1eccc00640 1 -- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 msgr2=0x7f1eac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.678+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1eac00b980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0007330 tx=0x7f1eac0057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.679+0000 7f1eccc00640 1 -- 172.21.15.55:0/3306518400 shutdown_connections 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.679+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3306518400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1eac00b980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.679+0000 7f1eccc00640 1 -- 172.21.15.55:0/3306518400 >> 172.21.15.55:0/3306518400 conn(0x7f1ec80d32c0 msgr2=0x7f1ec80d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.679+0000 7f1eccc00640 1 -- 172.21.15.55:0/3306518400 shutdown_connections 2024-05-10T03:29:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.679+0000 7f1eccc00640 1 -- 172.21.15.55:0/3306518400 wait complete. 2024-05-10T03:29:49.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.681+0000 7f1eccc00640 1 Processor -- start 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.681+0000 7f1eccc00640 1 -- start start 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ec814a6d0 0x7f1ec80b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ec814aef0 0x7f1ec814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec814bc10 con 0x7f1eac005230 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80b3fa0 con 0x7f1ec814aef0 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1eccc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80b4e00 con 0x7f1ec814a6d0 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59818/0 (socket says 172.21.15.55:59818) 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 -- 172.21.15.55:0/3573728935 learned_addr learned my addr 172.21.15.55:0/3573728935 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5400640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ec814a6d0 0x7f1ec80b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 -- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ec814a6d0 msgr2=0x7f1ec80b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ec814a6d0 0x7f1ec80b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 -- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ec814aef0 msgr2=0x7f1ec814b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec6800640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ec814aef0 0x7f1ec814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ec814aef0 0x7f1ec814b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.683+0000 7f1ec5e00640 1 -- 172.21.15.55:0/3573728935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80b41f0 con 0x7f1eac005230 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1ec5e00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800d7e0 tx=0x7f1eb800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80074e0 con 0x7f1eac005230 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8007640 con 0x7f1eac005230 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004a00 con 0x7f1eac005230 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec80b44d0 con 0x7f1eac005230 2024-05-10T03:29:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.684+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80b4a90 con 0x7f1eac005230 2024-05-10T03:29:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.685+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec80680c0 con 0x7f1eac005230 2024-05-10T03:29:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.686+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1eb8004b60 con 0x7f1eac005230 2024-05-10T03:29:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.687+0000 7f1ebe000640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 0x7f1ea806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.687+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1eb8089280 con 0x7f1eac005230 2024-05-10T03:29:49.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.688+0000 7f1ec5400640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 0x7f1ea806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.689+0000 7f1ec5400640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 0x7f1ea806d110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4007fd0 tx=0x7f1eb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:49.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.692+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1eb8054180 con 0x7f1eac005230 2024-05-10T03:29:49.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.933+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ec8067220 con 0x7f1eac005230 2024-05-10T03:29:49.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.935+0000 7f1ebe000640 1 -- 172.21.15.55:0/3573728935 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f1eb80577a0 con 0x7f1eac005230 2024-05-10T03:29:49.939 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:49.939 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:29:49.357204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T03:29:49.939 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-05-10T03:29:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 msgr2=0x7f1ea806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 0x7f1ea806d110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4007fd0 tx=0x7f1eb4009040 comp rx=0 tx=0).stop 2024-05-10T03:29:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 msgr2=0x7f1ec80b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800d7e0 tx=0x7f1eb800dca0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 shutdown_connections 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1ea806ac00 0x7f1ea806d110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ec814aef0 0x7f1ec814b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ec814a6d0 0x7f1ec80b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 --2- 172.21.15.55:0/3573728935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eac005230 0x7f1ec80b5290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.937+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 >> 172.21.15.55:0/3573728935 conn(0x7f1ec80d32c0 msgr2=0x7f1ec815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.938+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 shutdown_connections 2024-05-10T03:29:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:49.938+0000 7f1eccc00640 1 -- 172.21.15.55:0/3573728935 wait complete. 2024-05-10T03:29:49.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:50.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.233+0000 7f23c7200640 1 Processor -- start 2024-05-10T03:29:50.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.233+0000 7f23c7200640 1 -- start start 2024-05-10T03:29:50.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23c7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f23c815d5e0 0x7f23c8168510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23c7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c806bac0 con 0x7f23c80b2170 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23c7200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c8161fc0 con 0x7f23c80b6680 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23c7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c80d2b20 con 0x7f23c815d5e0 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23bd400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f23c80b2170 0x7f23c81618b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53862/0 (socket says 172.21.15.55:53862) 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23bd400640 1 -- 172.21.15.55:0/2148901510 learned_addr learned my addr 172.21.15.55:0/2148901510 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.235+0000 7f23c6800640 1 --2- 172.21.15.55:0/2148901510 >> v2:172.21.15.55:3300/0 conn(0x7f23c815d5e0 0x7f23c8168510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.236+0000 7f23c5400640 1 -- 172.21.15.55:0/2148901510 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028965940 0 0) 0x7f23c806bac0 con 0x7f23c80b2170 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.236+0000 7f23c5400640 1 -- 172.21.15.55:0/2148901510 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23ac003520 con 0x7f23c80b2170 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.236+0000 7f23c6800640 1 -- 172.21.15.55:0/2148901510 >> v1:172.21.15.162:6789/0 conn(0x7f23c80b6680 legacy=0x7f23c81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.236+0000 7f23c6800640 1 -- 172.21.15.55:0/2148901510 >> v1:172.21.15.55:6789/0 conn(0x7f23c80b2170 legacy=0x7f23c81618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.236+0000 7f23c6800640 1 -- 172.21.15.55:0/2148901510 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80d25a0 con 0x7f23c815d5e0 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.237+0000 7f23c6800640 1 --2- 172.21.15.55:0/2148901510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c8168510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d90 tx=0x7f23c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad411a6c655ec72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.237+0000 7f23c5400640 1 -- 172.21.15.55:0/2148901510 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0007740 con 0x7f23c815d5e0 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.237+0000 7f23c5400640 1 -- 172.21.15.55:0/2148901510 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c002d050 con 0x7f23c815d5e0 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.237+0000 7f23c5400640 1 -- 172.21.15.55:0/2148901510 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0007aa0 con 0x7f23c815d5e0 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 -- 172.21.15.55:0/2148901510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 msgr2=0x7f23c8168510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 --2- 172.21.15.55:0/2148901510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c8168510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d90 tx=0x7f23c002a6f0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 -- 172.21.15.55:0/2148901510 shutdown_connections 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 --2- 172.21.15.55:0/2148901510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c8168510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 -- 172.21.15.55:0/2148901510 >> 172.21.15.55:0/2148901510 conn(0x7f23c80b5730 msgr2=0x7f23c80d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 -- 172.21.15.55:0/2148901510 shutdown_connections 2024-05-10T03:29:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.238+0000 7f23c7200640 1 -- 172.21.15.55:0/2148901510 wait complete. 2024-05-10T03:29:50.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.240+0000 7f23c7200640 1 Processor -- start 2024-05-10T03:29:50.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.240+0000 7f23c7200640 1 -- start start 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23c80b2170 0x7f23c81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23c80b6680 0x7f23c806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8160590 con 0x7f23c815d5e0 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8152b00 con 0x7f23c80b2170 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8152630 con 0x7f23c80b6680 2024-05-10T03:29:50.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59836/0 (socket says 172.21.15.55:59836) 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.242+0000 7f23c6800640 1 -- 172.21.15.55:0/2798576944 learned_addr learned my addr 172.21.15.55:0/2798576944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 -- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23c80b6680 msgr2=0x7f23c806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23c80b6680 0x7f23c806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 -- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23c80b2170 msgr2=0x7f23c81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23c80b2170 0x7f23c81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 -- 172.21.15.55:0/2798576944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0009a70 con 0x7f23c815d5e0 2024-05-10T03:29:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c6800640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d90 tx=0x7f23c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0003b90 con 0x7f23c815d5e0 2024-05-10T03:29:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0003cf0 con 0x7f23c815d5e0 2024-05-10T03:29:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c003b900 con 0x7f23c815d5e0 2024-05-10T03:29:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c815e3a0 con 0x7f23c815d5e0 2024-05-10T03:29:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.243+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c815e650 con 0x7f23c815d5e0 2024-05-10T03:29:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.245+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c8067720 con 0x7f23c815d5e0 2024-05-10T03:29:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.245+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f23c003ba60 con 0x7f23c815d5e0 2024-05-10T03:29:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.246+0000 7f23bea00640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 0x7f23a406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.246+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23c00aa560 con 0x7f23c815d5e0 2024-05-10T03:29:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.247+0000 7f23c5e00640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 0x7f23a406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.248+0000 7f23c5e00640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 0x7f23a406d110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000f60 tx=0x7f23b80079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.251+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f23c00753b0 con 0x7f23c815d5e0 2024-05-10T03:29:50.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.445+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23c804e290 con 0x7f23c815d5e0 2024-05-10T03:29:50.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.447+0000 7f23bea00640 1 -- 172.21.15.55:0/2798576944 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f23c00789d0 con 0x7f23c815d5e0 2024-05-10T03:29:50.451 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:50.452 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 msgr2=0x7f23a406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 0x7f23a406d110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23b8000f60 tx=0x7f23b80079b0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 msgr2=0x7f23c806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009d90 tx=0x7f23c0004a40 comp rx=0 tx=0).stop 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 shutdown_connections 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23c815d5e0 0x7f23c806b600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23c80b6680 0x7f23c806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f23a406ac00 0x7f23a406d110 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 --2- 172.21.15.55:0/2798576944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23c80b2170 0x7f23c81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 >> 172.21.15.55:0/2798576944 conn(0x7f23c80b5730 msgr2=0x7f23c80b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 shutdown_connections 2024-05-10T03:29:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.450+0000 7f23c7200640 1 -- 172.21.15.55:0/2798576944 wait complete. 2024-05-10T03:29:50.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:50.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.751+0000 7f32ad400640 1 Processor -- start 2024-05-10T03:29:50.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.751+0000 7f32ad400640 1 -- start start 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32ad400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32a80b2170 0x7f32a815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32ad400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f32a806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32ad400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a80d25a0 con 0x7f32a80b6680 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32ad400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a80d2b20 con 0x7f32a80b2170 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32ad400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a815db20 con 0x7f32a81618b0 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32a7e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f32a80b6680 0x7f32a81611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53878/0 (socket says 172.21.15.55:53878) 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.753+0000 7f32a7e00640 1 -- 172.21.15.55:0/2254564471 learned_addr learned my addr 172.21.15.55:0/2254564471 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a7400640 1 --2- 172.21.15.55:0/2254564471 >> v2:172.21.15.162:3300/0 conn(0x7f32a80b2170 0x7f32a815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32aca00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f32a806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148437084 0 0) 0x7f32a80d25a0 con 0x7f32a80b6680 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c003520 con 0x7f32a80b6680 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970872771 0 0) 0x7f328c003520 con 0x7f32a80b6680 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 msgr2=0x7f32a806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f32a806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 >> v2:172.21.15.162:3300/0 conn(0x7f32a80b2170 msgr2=0x7f32a815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 --2- 172.21.15.55:0/2254564471 >> v2:172.21.15.162:3300/0 conn(0x7f32a80b2170 0x7f32a815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80b28d0 con 0x7f32a80b6680 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f32980019f0 con 0x7f32a80b6680 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 >> v1:172.21.15.55:6789/0 conn(0x7f32a80b6680 legacy=0x7f32a81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c0039e0 0x7f328c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f328c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f328c004d00 0x7f328c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80d2b20 con 0x7f328c004d00 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a815db20 con 0x7f328c0039e0 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.754+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c00ba30 con 0x7f32a81618b0 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f328c004d00 0x7f328c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32a7400640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f328c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32a7e00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c0039e0 0x7f328c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 -- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 msgr2=0x7f328c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:50.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a81618b0 0x7f328c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 -- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c0039e0 msgr2=0x7f328c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:50.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c0039e0 0x7f328c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.755+0000 7f32aca00640 1 -- 172.21.15.55:0/2254564471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c003520 con 0x7f328c004d00 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.756+0000 7f32aca00640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f328c004d00 0x7f328c005110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009a90 tx=0x7f328c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853ad7731f447472 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.756+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0031400 con 0x7f328c004d00 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a002d050 con 0x7f328c004d00 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32a6a00640 1 -- 172.21.15.55:0/2254564471 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0031780 con 0x7f328c004d00 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32ad400640 1 -- 172.21.15.55:0/2254564471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f328c004d00 msgr2=0x7f328c005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32ad400640 1 --2- 172.21.15.55:0/2254564471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f328c004d00 0x7f328c005110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009a90 tx=0x7f328c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32aca00640 1 -- 172.21.15.55:0/2254564471 reap_dead start 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32ad400640 1 -- 172.21.15.55:0/2254564471 shutdown_connections 2024-05-10T03:29:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.757+0000 7f32ad400640 1 -- 172.21.15.55:0/2254564471 >> 172.21.15.55:0/2254564471 conn(0x7f32a80b5730 msgr2=0x7f32a80d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.758+0000 7f32ad400640 1 -- 172.21.15.55:0/2254564471 shutdown_connections 2024-05-10T03:29:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.758+0000 7f32ad400640 1 -- 172.21.15.55:0/2254564471 wait complete. 2024-05-10T03:29:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.760+0000 7f32ad400640 1 Processor -- start 2024-05-10T03:29:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.760+0000 7f32ad400640 1 -- start start 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c004d00 0x7f32a814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a80c9730 0x7f32a80c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80c9310 con 0x7f32a80cbfb0 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80ccdb0 con 0x7f328c004d00 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32ad400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a815ffe0 con 0x7f32a80c9730 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59862/0 (socket says 172.21.15.55:59862) 2024-05-10T03:29:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 -- 172.21.15.55:0/2689967400 learned_addr learned my addr 172.21.15.55:0/2689967400 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32a7e00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c004d00 0x7f32a814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32a7400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a80c9730 0x7f32a80c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 -- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a80c9730 msgr2=0x7f32a80c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a80c9730 0x7f32a80c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 -- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c004d00 msgr2=0x7f32a814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c004d00 0x7f32a814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.762+0000 7f32aca00640 1 -- 172.21.15.55:0/2689967400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80cc900 con 0x7f32a80cbfb0 2024-05-10T03:29:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f32aca00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32a0034040 tx=0x7f32a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a003b610 con 0x7f32a80cbfb0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a003b770 con 0x7f32a80cbfb0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a003bae0 con 0x7f32a80cbfb0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a814ba00 con 0x7f32a80cbfb0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.763+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a8160510 con 0x7f32a80cbfb0 2024-05-10T03:29:50.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.764+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a8067740 con 0x7f32a80cbfb0 2024-05-10T03:29:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.765+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f32a0038050 con 0x7f32a80cbfb0 2024-05-10T03:29:50.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.767+0000 7f329fe00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 0x7f328406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:50.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.767+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32a0000e40 con 0x7f32a80cbfb0 2024-05-10T03:29:50.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.767+0000 7f32a7e00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 0x7f328406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:50.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.768+0000 7f32a7e00640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 0x7f328406d110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3298002230 tx=0x7f3298009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.771+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f32a0074790 con 0x7f32a80cbfb0 2024-05-10T03:29:50.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.973+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a804e290 con 0x7f32a80cbfb0 2024-05-10T03:29:50.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.976+0000 7f329fe00640 1 -- 172.21.15.55:0/2689967400 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f32a0036090 con 0x7f32a80cbfb0 2024-05-10T03:29:50.981 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:50.981 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 msgr2=0x7f328406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 0x7f328406d110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3298002230 tx=0x7f3298009040 comp rx=0 tx=0).stop 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 msgr2=0x7f32a80cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32a0034040 tx=0x7f32a0004730 comp rx=0 tx=0).stop 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 shutdown_connections 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a80cbfb0 0x7f32a80cc3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32a80c9730 0x7f32a80c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f328406ac00 0x7f328406d110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 --2- 172.21.15.55:0/2689967400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328c004d00 0x7f32a814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:50.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.979+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 >> 172.21.15.55:0/2689967400 conn(0x7f32a80b5730 msgr2=0x7f32a815b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:50.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.980+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 shutdown_connections 2024-05-10T03:29:50.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:50.980+0000 7f32ad400640 1 -- 172.21.15.55:0/2689967400 wait complete. 2024-05-10T03:29:51.023 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:29:51.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.299+0000 7f3d91800640 1 Processor -- start 2024-05-10T03:29:51.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.300+0000 7f3d91800640 1 -- start start 2024-05-10T03:29:51.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d91800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d8c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d91800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d8c06d7e0 0x7f3d8c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d91800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c06d350 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d91800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8c06c740 con 0x7f3d8c06d7e0 2024-05-10T03:29:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d91800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c06ca20 con 0x7f3d8c144710 2024-05-10T03:29:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d90e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3d8c0b2fc0 0x7f3d8c15eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53892/0 (socket says 172.21.15.55:53892) 2024-05-10T03:29:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d90e00640 1 -- 172.21.15.55:0/1927229427 learned_addr learned my addr 172.21.15.55:0/1927229427 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.302+0000 7f3d8be00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d8c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d83e00640 1 --2- 172.21.15.55:0/1927229427 >> v2:172.21.15.162:3300/0 conn(0x7f3d8c06d7e0 0x7f3d8c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616668894 0 0) 0x7f3d8c06d350 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d70003520 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3567873171 0 0) 0x7f3d70003520 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 msgr2=0x7f3d8c144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d8c144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 >> v2:172.21.15.162:3300/0 conn(0x7f3d8c06d7e0 msgr2=0x7f3d8c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 --2- 172.21.15.55:0/1927229427 >> v2:172.21.15.162:3300/0 conn(0x7f3d8c06d7e0 0x7f3d8c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c06cd00 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f3d840019f0 con 0x7f3d8c0b2fc0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 >> v1:172.21.15.55:6789/0 conn(0x7f3d8c0b2fc0 legacy=0x7f3d8c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700039e0 0x7f3d70003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d700047c0 0x7f3d70004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d70005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c06c740 con 0x7f3d700047c0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c06ca20 con 0x7f3d700039e0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.303+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7000ba80 con 0x7f3d8c144710 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d700047c0 0x7f3d70004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d8be00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700039e0 0x7f3d70003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d90e00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d70005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 -- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 msgr2=0x7f3d70005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c144710 0x7f3d70005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 -- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700039e0 msgr2=0x7f3d70003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700039e0 0x7f3d70003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.304+0000 7f3d83e00640 1 -- 172.21.15.55:0/1927229427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70003520 con 0x7f3d700047c0 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.305+0000 7f3d83e00640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d700047c0 0x7f3d70004bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0ae870 tx=0x7f3d700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2a68dd2bec6b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.305+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031400 con 0x7f3d700047c0 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.305+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7402d050 con 0x7f3d700047c0 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.305+0000 7f3d8b400640 1 -- 172.21.15.55:0/1927229427 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74031780 con 0x7f3d700047c0 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 -- 172.21.15.55:0/1927229427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d700047c0 msgr2=0x7f3d70004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 --2- 172.21.15.55:0/1927229427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d700047c0 0x7f3d70004bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0ae870 tx=0x7f3d700057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d90e00640 1 -- 172.21.15.55:0/1927229427 reap_dead start 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 -- 172.21.15.55:0/1927229427 shutdown_connections 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 -- 172.21.15.55:0/1927229427 >> 172.21.15.55:0/1927229427 conn(0x7f3d8c06a110 msgr2=0x7f3d8c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 -- 172.21.15.55:0/1927229427 shutdown_connections 2024-05-10T03:29:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.306+0000 7f3d91800640 1 -- 172.21.15.55:0/1927229427 wait complete. 2024-05-10T03:29:51.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.309+0000 7f3d91800640 1 Processor -- start 2024-05-10T03:29:51.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.309+0000 7f3d91800640 1 -- start start 2024-05-10T03:29:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.310+0000 7f3d91800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 0x7f3d8c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d91800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d91800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c0b0280 0x7f3d8c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d91800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c152bd0 con 0x7f3d8c071a50 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d91800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c070ff0 con 0x7f3d700047c0 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d91800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1f7be0 con 0x7f3d8c0b0280 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59882/0 (socket says 172.21.15.55:59882) 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 -- 172.21.15.55:0/3384500912 learned_addr learned my addr 172.21.15.55:0/3384500912 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d90e00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c0b0280 0x7f3d8c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d8be00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 0x7f3d8c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 -- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c0b0280 msgr2=0x7f3d8c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c0b0280 0x7f3d8c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 -- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 msgr2=0x7f3d8c15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 0x7f3d8c15a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d83e00640 1 -- 172.21.15.55:0/3384500912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0b0690 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.311+0000 7f3d8be00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 0x7f3d8c15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d83e00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d74034040 tx=0x7f3d74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74003bb0 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74003d10 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c15e1d0 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74044610 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.312+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c15e790 con 0x7f3d8c071a50 2024-05-10T03:29:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.313+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c066ce0 con 0x7f3d8c071a50 2024-05-10T03:29:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.314+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3d74038070 con 0x7f3d8c071a50 2024-05-10T03:29:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.315+0000 7f3d88c00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 0x7f3d6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.315+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3d740a9870 con 0x7f3d8c071a50 2024-05-10T03:29:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.316+0000 7f3d8be00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 0x7f3d6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.317+0000 7f3d8be00640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 0x7f3d6806d140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009160 tx=0x7f3d7c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.320+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3d74074800 con 0x7f3d8c071a50 2024-05-10T03:29:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.564+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d8c15e980 con 0x7f3d8c071a50 2024-05-10T03:29:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.566+0000 7f3d88c00640 1 -- 172.21.15.55:0/3384500912 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f3d74009690 con 0x7f3d8c071a50 2024-05-10T03:29:51.570 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:51.570 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:29:49.357204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T03:29:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-05-10T03:29:51.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 msgr2=0x7f3d6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 0x7f3d6806d140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009160 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 msgr2=0x7f3d8c0afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d74034040 tx=0x7f3d74004730 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 shutdown_connections 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d8c0b0280 0x7f3d8c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d8c071a50 0x7f3d8c0afd40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d6806ac30 0x7f3d6806d140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 --2- 172.21.15.55:0/3384500912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d700047c0 0x7f3d8c15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 >> 172.21.15.55:0/3384500912 conn(0x7f3d8c06a110 msgr2=0x7f3d8c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 shutdown_connections 2024-05-10T03:29:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.569+0000 7f3d91800640 1 -- 172.21.15.55:0/3384500912 wait complete. 2024-05-10T03:29:51.596 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.882+0000 7ff7bd600640 1 Processor -- start 2024-05-10T03:29:51.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.882+0000 7ff7bd600640 1 -- start start 2024-05-10T03:29:51.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.884+0000 7ff7bd600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b80b4170 0x7ff7b80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.884+0000 7ff7bd600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b80cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.884+0000 7ff7bd600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80b4ac0 con 0x7ff7b815a5d0 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.884+0000 7ff7bd600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b815ad30 con 0x7ff7b80b4170 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.884+0000 7ff7bd600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b815b010 con 0x7ff7b80ccc80 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b80b4170 0x7ff7b80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b80b4170 0x7ff7b80b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59890/0 (socket says 172.21.15.55:59890) 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 -- 172.21.15.55:0/3003785806 learned_addr learned my addr 172.21.15.55:0/3003785806 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7bcc00640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b80cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 -- 172.21.15.55:0/3003785806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 msgr2=0x7ff7b80cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b80cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 -- 172.21.15.55:0/3003785806 >> v1:172.21.15.162:6789/0 conn(0x7ff7b815a5d0 legacy=0x7ff7b80cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.885+0000 7ff7b7400640 1 -- 172.21.15.55:0/3003785806 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b815b2f0 con 0x7ff7b80b4170 2024-05-10T03:29:51.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.886+0000 7ff7b7400640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b80b4170 0x7ff7b80b4580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7b806edf0 tx=0x7ff7a402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d8ef1ec113036e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.887+0000 7ff7b6a00640 1 -- 172.21.15.55:0/3003785806 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4032440 con 0x7ff7b80b4170 2024-05-10T03:29:51.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.887+0000 7ff7b6a00640 1 -- 172.21.15.55:0/3003785806 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a40325a0 con 0x7ff7b80b4170 2024-05-10T03:29:51.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.887+0000 7ff7b6a00640 1 -- 172.21.15.55:0/3003785806 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4032930 con 0x7ff7b80b4170 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.887+0000 7ff7bd600640 1 -- 172.21.15.55:0/3003785806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b80b4170 msgr2=0x7ff7b80b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.887+0000 7ff7bd600640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b80b4170 0x7ff7b80b4580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7b806edf0 tx=0x7ff7a402a200 comp rx=0 tx=0).stop 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 -- 172.21.15.55:0/3003785806 shutdown_connections 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b80cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 --2- 172.21.15.55:0/3003785806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b80b4170 0x7ff7b80b4580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 -- 172.21.15.55:0/3003785806 >> 172.21.15.55:0/3003785806 conn(0x7ff7b806a110 msgr2=0x7ff7b806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 -- 172.21.15.55:0/3003785806 shutdown_connections 2024-05-10T03:29:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.888+0000 7ff7bd600640 1 -- 172.21.15.55:0/3003785806 wait complete. 2024-05-10T03:29:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.890+0000 7ff7bd600640 1 Processor -- start 2024-05-10T03:29:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.890+0000 7ff7bd600640 1 -- start start 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 0x7ff7b8140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b8166d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8165ee0 con 0x7ff7b815a5d0 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81419a0 con 0x7ff7b80b4170 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bd600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8166610 con 0x7ff7b80ccc80 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bcc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bcc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59894/0 (socket says 172.21.15.55:59894) 2024-05-10T03:29:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7bcc00640 1 -- 172.21.15.55:0/507422317 learned_addr learned my addr 172.21.15.55:0/507422317 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7b7400640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b8166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.892+0000 7ff7b7e00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 0x7ff7b8140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 -- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 msgr2=0x7ff7b8166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b8166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 -- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 msgr2=0x7ff7b8140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 0x7ff7b8140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 -- 172.21.15.55:0/507422317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40094d0 con 0x7ff7b815a5d0 2024-05-10T03:29:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7b7e00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 0x7ff7b8140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bcc00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cc00 tx=0x7ff7b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ae80 con 0x7ff7b815a5d0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000ce10 con 0x7ff7b815a5d0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0003b90 con 0x7ff7b815a5d0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.893+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8141430 con 0x7ff7b815a5d0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.894+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b8155830 con 0x7ff7b815a5d0 2024-05-10T03:29:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.895+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b8066ce0 con 0x7ff7b815a5d0 2024-05-10T03:29:51.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.896+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff7b000f3f0 con 0x7ff7b815a5d0 2024-05-10T03:29:51.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.897+0000 7ff7afe00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 0x7ff79806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:51.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.897+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff7b0089920 con 0x7ff7b815a5d0 2024-05-10T03:29:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.897+0000 7ff7b7e00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 0x7ff79806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:51.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.899+0000 7ff7b7e00640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 0x7ff79806d140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8006fd0 tx=0x7ff7a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:51.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:51.902+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff7b00548a0 con 0x7ff7b815a5d0 2024-05-10T03:29:52.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.103+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b804e290 con 0x7ff7b815a5d0 2024-05-10T03:29:52.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.104+0000 7ff7afe00640 1 -- 172.21.15.55:0/507422317 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7ff7b001d020 con 0x7ff7b815a5d0 2024-05-10T03:29:52.108 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:52.109 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:52.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.107+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 msgr2=0x7ff79806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.107+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 0x7ff79806d140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8006fd0 tx=0x7ff7a8008040 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.107+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 msgr2=0x7ff7b81652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.107+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cc00 tx=0x7ff7b000a5c0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 shutdown_connections 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b815a5d0 0x7ff7b81652d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7b80ccc80 0x7ff7b8166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff79806ac30 0x7ff79806d140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 --2- 172.21.15.55:0/507422317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7b80b4170 0x7ff7b8140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 >> 172.21.15.55:0/507422317 conn(0x7ff7b806a110 msgr2=0x7ff7b815e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 shutdown_connections 2024-05-10T03:29:52.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.108+0000 7ff7bd600640 1 -- 172.21.15.55:0/507422317 wait complete. 2024-05-10T03:29:52.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:29:44.296174+0000', 'modified': '2024-05-10T03:29:49.357204+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2047857054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 2047857054}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2047857054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-10T03:29:52.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:29:52.543 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-10T03:29:52.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.841+0000 7fa7cec00640 1 Processor -- start 2024-05-10T03:29:52.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.841+0000 7fa7cec00640 1 -- start start 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 0x7fa7c8143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7cec00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7c806b120 0x7fa7c806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7cec00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c80cdbe0 con 0x7fa7c80b2040 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7cec00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c815e8b0 con 0x7fa7c806b120 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7cec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80ce140 con 0x7fa7c81435d0 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7ce200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7c806b120 0x7fa7c806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7ce200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7c806b120 0x7fa7c806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37234/0 (socket says 172.21.15.55:37234) 2024-05-10T03:29:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.844+0000 7fa7ce200640 1 -- 172.21.15.55:0/2629872360 learned_addr learned my addr 172.21.15.55:0/2629872360 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7cce00640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 0x7fa7c8143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7ce200640 1 -- 172.21.15.55:0/2629872360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 msgr2=0x7fa7c8143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7ce200640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 0x7fa7c8143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7ce200640 1 -- 172.21.15.55:0/2629872360 >> v1:172.21.15.162:6789/0 conn(0x7fa7c80b2040 legacy=0x7fa7c80ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7ce200640 1 -- 172.21.15.55:0/2629872360 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c806ba70 con 0x7fa7c806b120 2024-05-10T03:29:52.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.845+0000 7fa7cce00640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 0x7fa7c8143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:52.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.846+0000 7fa7ce200640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c806b120 0x7fa7c806b530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009e50 tx=0x7fa7c002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98b79c2e1a2b678 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.846+0000 7fa7c7e00640 1 -- 172.21.15.55:0/2629872360 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0031600 con 0x7fa7c806b120 2024-05-10T03:29:52.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.846+0000 7fa7c7e00640 1 -- 172.21.15.55:0/2629872360 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c002d050 con 0x7fa7c806b120 2024-05-10T03:29:52.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.846+0000 7fa7c7e00640 1 -- 172.21.15.55:0/2629872360 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0031980 con 0x7fa7c806b120 2024-05-10T03:29:52.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 -- 172.21.15.55:0/2629872360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c806b120 msgr2=0x7fa7c806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c806b120 0x7fa7c806b530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009e50 tx=0x7fa7c002a630 comp rx=0 tx=0).stop 2024-05-10T03:29:52.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 -- 172.21.15.55:0/2629872360 shutdown_connections 2024-05-10T03:29:52.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c806b120 0x7fa7c806b530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 --2- 172.21.15.55:0/2629872360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c81435d0 0x7fa7c8143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 -- 172.21.15.55:0/2629872360 >> 172.21.15.55:0/2629872360 conn(0x7fa7c806c730 msgr2=0x7fa7c80b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 -- 172.21.15.55:0/2629872360 shutdown_connections 2024-05-10T03:29:52.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.847+0000 7fa7cec00640 1 -- 172.21.15.55:0/2629872360 wait complete. 2024-05-10T03:29:52.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.850+0000 7fa7cec00640 1 Processor -- start 2024-05-10T03:29:52.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.850+0000 7fa7cec00640 1 -- start start 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c806b120 0x7fa7c813eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c81435d0 0x7fa7c80afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c813f2e0 con 0x7fa7c80b2040 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80ae420 con 0x7fa7c81435d0 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.851+0000 7fa7cec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c81583c0 con 0x7fa7c806b120 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37238/0 (socket says 172.21.15.55:37238) 2024-05-10T03:29:52.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 -- 172.21.15.55:0/3527878520 learned_addr learned my addr 172.21.15.55:0/3527878520 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7ce200640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c81435d0 0x7fa7c80afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cd800640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c806b120 0x7fa7c813eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 -- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c806b120 msgr2=0x7fa7c813eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c806b120 0x7fa7c813eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 -- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c81435d0 msgr2=0x7fa7c80afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c81435d0 0x7fa7c80afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.852+0000 7fa7cce00640 1 -- 172.21.15.55:0/3527878520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0009b30 con 0x7fa7c80b2040 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7cce00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c820 tx=0x7fa7b400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7c5600640 1 -- 172.21.15.55:0/3527878520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004220 con 0x7fa7c80b2040 2024-05-10T03:29:52.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7c5600640 1 -- 172.21.15.55:0/3527878520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4004380 con 0x7fa7c80b2040 2024-05-10T03:29:52.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7c5600640 1 -- 172.21.15.55:0/3527878520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4005bb0 con 0x7fa7c80b2040 2024-05-10T03:29:52.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c80acc50 con 0x7fa7c80b2040 2024-05-10T03:29:52.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.853+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c81588f0 con 0x7fa7c80b2040 2024-05-10T03:29:52.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.855+0000 7fa7c5600640 1 -- 172.21.15.55:0/3527878520 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa7b4005d10 con 0x7fa7c80b2040 2024-05-10T03:29:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.856+0000 7fa7c5600640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 0x7fa7a806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.856+0000 7fa7c5600640 1 -- 172.21.15.55:0/3527878520 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7b4089a00 con 0x7fa7c80b2040 2024-05-10T03:29:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.857+0000 7fa7cd800640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 0x7fa7a806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.857+0000 7fa7d716a740 1 Processor -- start 2024-05-10T03:29:52.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.857+0000 7fa7d716a740 1 -- start start 2024-05-10T03:29:52.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.858+0000 7fa7cd800640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 0x7fa7a806d110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00a750 tx=0x7fa7bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce44270 0x55863d20dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d228f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d14ca70 con 0x55863ce1b3b0 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d14a200 con 0x55863ce23590 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55863d145af0 con 0x55863ce44270 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37254/0 (socket says 172.21.15.55:37254) 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.859+0000 7fa7cce00640 1 -- 172.21.15.55:0/1520365174 learned_addr learned my addr 172.21.15.55:0/1520365174 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cd800640 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce44270 0x55863d20dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7ce200640 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d228f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 -- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce44270 msgr2=0x55863d20dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce44270 0x55863d20dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 -- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 msgr2=0x55863d228f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d228f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 -- 172.21.15.55:0/1520365174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863d110b90 con 0x55863ce1b3b0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7cce00640 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4054a10 tx=0x7fa7b4058cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bab5ecff129c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7ba000640 1 -- 172.21.15.55:0/1520365174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b405c340 con 0x55863ce1b3b0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7ba000640 1 -- 172.21.15.55:0/1520365174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4060900 con 0x55863ce1b3b0 2024-05-10T03:29:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.860+0000 7fa7ba000640 1 -- 172.21.15.55:0/1520365174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ce10 con 0x55863ce1b3b0 2024-05-10T03:29:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 -- 172.21.15.55:0/1520365174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 msgr2=0x55863d228a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4054a10 tx=0x7fa7b4058cd0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 -- 172.21.15.55:0/1520365174 shutdown_connections 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d228f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d228a00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 --2- 172.21.15.55:0/1520365174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce44270 0x55863d20dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.861+0000 7fa7d716a740 1 -- 172.21.15.55:0/1520365174 >> 172.21.15.55:0/1520365174 conn(0x55863ce383c0 msgr2=0x55863d1e3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.862+0000 7fa7d716a740 1 -- 172.21.15.55:0/1520365174 shutdown_connections 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.862+0000 7fa7d716a740 1 -- 172.21.15.55:0/1520365174 wait complete. 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.863+0000 7fa7d716a740 1 -- start start 2024-05-10T03:29:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.863+0000 7fa7d716a740 1 Processor -- start 2024-05-10T03:29:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce23590 0x55863d208fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d2094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d0e7220 con 0x55863ce1b3b0 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d0e6ec0 con 0x55863ce44270 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55863ceacf20 con 0x55863ce23590 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7d716a740 10 client.0 _ll_register_callbacks cb 0x55863d13f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37268/0 (socket says 172.21.15.55:37268) 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 -- 172.21.15.55:0/4005358729 learned_addr learned my addr 172.21.15.55:0/4005358729 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 -- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce23590 msgr2=0x55863d208fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce23590 0x55863d208fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 -- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 msgr2=0x55863d2094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d2094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.865+0000 7fa7cd800640 1 -- 172.21.15.55:0/4005358729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c81588f0 con 0x55863ce1b3b0 2024-05-10T03:29:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7cd800640 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00ce40 tx=0x7fa7bc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14ce77e5ab40d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7aea00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc01f670 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7bc013e10 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55863d16fc90 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7bc01f7d0 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7d716a740 10 client.4383 fetch_fsmap learned FSMap version 7 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55863d144170 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.866+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55863d16fc90 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fa7bc01f930 con 0x55863ce1b3b0 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 10 client.4383 fetch_fsmap finished waiting for FSMap version 7 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 10 client.4383 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 20 client.4383 populate_metadata read hostname 'smithi055' 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863ce02e40 0x55863d224650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 4 client.4383 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:29:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 --> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55863ce393c0 con 0x55863ce02e40 2024-05-10T03:29:52.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.867+0000 7fa7cce00640 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863ce02e40 0x55863d224650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.868+0000 7fa7cce00640 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863ce02e40 0x55863d224650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8127bbaaaa86cecf server_cookie=25992436bbcd051e in_seq=0 out_seq=0 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.868+0000 7fa7aea00640 10 client.4383 ms_handle_connect on v2:172.21.15.162:6835/2047857054 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.870+0000 7fa7aea00640 1 -- 172.21.15.55:0/4005358729 <== mds.0 v2:172.21.15.162:6835/2047857054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55863ce393c0 con 0x55863ce02e40 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.870+0000 7fa7aea00640 10 client.4383 handle_command_reply: tid=0 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 client.4383 shutdown 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 msgr2=0x55863d211690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00ce40 tx=0x7fa7bc004680 comp rx=0 tx=0).stop 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 shutdown_connections 2024-05-10T03:29:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863ce02e40 0x55863d224650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d2094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce23590 0x55863d208fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4005358729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce1b3b0 0x55863d211690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7aea00640 0 client.4383 ms_handle_reset on v2:172.21.15.162:6835/2047857054 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 >> 172.21.15.55:0/4005358729 conn(0x55863ce383c0 msgr2=0x55863d1e3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 shutdown_connections 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 1 -- 172.21.15.55:0/4005358729 wait complete. 2024-05-10T03:29:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.871+0000 7fa7d716a740 20 client.4383 trim_cache size 0 max 16384 2024-05-10T03:29:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.889+0000 7fa7d716a740 1 Processor -- start 2024-05-10T03:29:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.889+0000 7fa7d716a740 1 -- start start 2024-05-10T03:29:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d241b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d208e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d1ddc70 con 0x55863ce44270 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d1d9680 con 0x55863ce23590 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.890+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55863d1dde10 con 0x55863ce1b3b0 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37282/0 (socket says 172.21.15.55:37282) 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 -- 172.21.15.55:0/4054215486 learned_addr learned my addr 172.21.15.55:0/4054215486 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7cce00640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d208e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7cd800640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d241b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 -- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 msgr2=0x55863d241b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d241b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 -- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 msgr2=0x55863d208e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d208e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7ce200640 1 -- 172.21.15.55:0/4054215486 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863d179130 con 0x55863ce44270 2024-05-10T03:29:52.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7cce00640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d208e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:52.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.891+0000 7fa7cd800640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d241b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:52.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.892+0000 7fa7ce200640 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009f70 tx=0x7fa7c002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e74306c825f4ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.892+0000 7fa7baa00640 1 -- 172.21.15.55:0/4054215486 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c003c650 con 0x55863ce44270 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.892+0000 7fa7baa00640 1 -- 172.21.15.55:0/4054215486 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c003c7b0 con 0x55863ce44270 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.892+0000 7fa7baa00640 1 -- 172.21.15.55:0/4054215486 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c003cb20 con 0x55863ce44270 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 -- 172.21.15.55:0/4054215486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 msgr2=0x55863d2093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009f70 tx=0x7fa7c002c000 comp rx=0 tx=0).stop 2024-05-10T03:29:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 -- 172.21.15.55:0/4054215486 shutdown_connections 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce44270 0x55863d2093a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce23590 0x55863d208e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 --2- 172.21.15.55:0/4054215486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d241b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 -- 172.21.15.55:0/4054215486 >> 172.21.15.55:0/4054215486 conn(0x55863ce383c0 msgr2=0x55863d20d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 -- 172.21.15.55:0/4054215486 shutdown_connections 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.893+0000 7fa7d716a740 1 -- 172.21.15.55:0/4054215486 wait complete. 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.894+0000 7fa7d716a740 1 -- start start 2024-05-10T03:29:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.895+0000 7fa7d716a740 1 Processor -- start 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d208e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d1e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d1e3fe0 con 0x55863ce23590 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863d2299d0 con 0x55863ce44270 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55863d1e21f0 con 0x55863ce1b3b0 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7d716a740 10 client.0 _ll_register_callbacks cb 0x7fa7bc00c520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.896+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37288/0 (socket says 172.21.15.55:37288) 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 -- 172.21.15.55:0/2335854733 learned_addr learned my addr 172.21.15.55:0/2335854733 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:52.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7ce200640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d1e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cd800640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d208e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 -- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 msgr2=0x55863d208e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d208e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 -- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 msgr2=0x55863d1e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d1e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 -- 172.21.15.55:0/2335854733 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c806ba70 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7cce00640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4007410 tx=0x7fa7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7386795f3531033 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7aea00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b405c340 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7b4086070 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.897+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7b4060900 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7b40568b0 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 10 client.4389 fetch_fsmap learned FSMap version 7 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55863d0e6430 con 0x55863ce23590 2024-05-10T03:29:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4060900 con 0x55863ce23590 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fa7b4079050 con 0x55863ce23590 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 10 client.4389 fetch_fsmap finished waiting for FSMap version 7 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 10 client.4389 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 20 client.4389 populate_metadata read hostname 'smithi055' 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863d28b3d0 0x55863d1e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 4 client.4389 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:29:52.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.898+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 --> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55863ce4a6c0 con 0x55863d28b3d0 2024-05-10T03:29:52.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.899+0000 7fa7cd800640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863d28b3d0 0x55863d1e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.900+0000 7fa7cd800640 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863d28b3d0 0x55863d1e3560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e31cae0d549f81 server_cookie=933a4be7e3b40ba0 in_seq=0 out_seq=0 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.900+0000 7fa7aea00640 10 client.4389 ms_handle_connect on v2:172.21.15.162:6835/2047857054 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.900+0000 7fa7aea00640 1 -- 172.21.15.55:0/2335854733 <== mds.0 v2:172.21.15.162:6835/2047857054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55863ce4a6c0 con 0x55863d28b3d0 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.900+0000 7fa7aea00640 10 client.4389 handle_command_reply: tid=0 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.900+0000 7fa7d716a740 1 client.4389 shutdown 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 msgr2=0x55863d2093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4007410 tx=0x7fa7b40046c0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 shutdown_connections 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x55863d28b3d0 0x55863d1e3560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55863ce44270 0x55863d1e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55863ce23590 0x55863d2093a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 --2- 172.21.15.55:0/2335854733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55863ce1b3b0 0x55863d208e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7aea00640 0 client.4389 ms_handle_reset on v2:172.21.15.162:6835/2047857054 2024-05-10T03:29:52.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 >> 172.21.15.55:0/2335854733 conn(0x55863ce383c0 msgr2=0x55863d1df790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 shutdown_connections 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 1 -- 172.21.15.55:0/2335854733 wait complete. 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.901+0000 7fa7d716a740 20 client.4389 trim_cache size 0 max 16384 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4294, 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:29:52.907 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 8.597053184, 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 7, 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 25, 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 24, 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 14.510234577 2024-05-10T03:29:52.908 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 msgr2=0x7fa7a806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 0x7fa7a806d110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00a750 tx=0x7fa7bc008040 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 msgr2=0x7fa7c81578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400c820 tx=0x7fa7b400cce0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 shutdown_connections 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa7a806ac00 0x7fa7a806d110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c81435d0 0x7fa7c80afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7c80b2040 0x7fa7c81578a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 --2- 172.21.15.55:0/3527878520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7c806b120 0x7fa7c813eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 >> 172.21.15.55:0/3527878520 conn(0x7fa7c806c730 msgr2=0x7fa7c8140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 shutdown_connections 2024-05-10T03:29:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:52.902+0000 7fa7cec00640 1 -- 172.21.15.55:0/3527878520 wait complete. 2024-05-10T03:29:52.931 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T03:29:52.931 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:52.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:29:53.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.196+0000 7f88ebe00640 1 Processor -- start 2024-05-10T03:29:53.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.196+0000 7f88ebe00640 1 -- start start 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88ebe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88ec06d430 0x7f88ec141b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88ebe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0b4b90 con 0x7f88ec0b5970 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88ebe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec0d0e80 con 0x7f88ec06d430 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88ebe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec06d840 con 0x7f88ec0b5020 2024-05-10T03:29:53.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88eaa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88ec06d430 0x7f88ec141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88eaa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88ec06d430 0x7f88ec141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37296/0 (socket says 172.21.15.55:37296) 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.199+0000 7f88eaa00640 1 -- 172.21.15.55:0/3578724453 learned_addr learned my addr 172.21.15.55:0/3578724453 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88eaa00640 1 -- 172.21.15.55:0/3578724453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 msgr2=0x7f88ec0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88ea000640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88eaa00640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88eaa00640 1 -- 172.21.15.55:0/3578724453 >> v1:172.21.15.162:6789/0 conn(0x7f88ec0b5970 legacy=0x7f88ec15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88eaa00640 1 -- 172.21.15.55:0/3578724453 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0b5d80 con 0x7f88ec06d430 2024-05-10T03:29:53.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.200+0000 7f88ea000640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:29:53.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.201+0000 7f88eaa00640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec06d430 0x7f88ec141b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e434c71cf95b24 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:53.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.201+0000 7f88e9600640 1 -- 172.21.15.55:0/3578724453 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc032440 con 0x7f88ec06d430 2024-05-10T03:29:53.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.201+0000 7f88e9600640 1 -- 172.21.15.55:0/3578724453 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc0325a0 con 0x7f88ec06d430 2024-05-10T03:29:53.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.201+0000 7f88e9600640 1 -- 172.21.15.55:0/3578724453 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc032930 con 0x7f88ec06d430 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 -- 172.21.15.55:0/3578724453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec06d430 msgr2=0x7f88ec141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec06d430 0x7f88ec141b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc02a210 comp rx=0 tx=0).stop 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 -- 172.21.15.55:0/3578724453 shutdown_connections 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 --2- 172.21.15.55:0/3578724453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec06d430 0x7f88ec141b00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 -- 172.21.15.55:0/3578724453 >> 172.21.15.55:0/3578724453 conn(0x7f88ec0b6680 msgr2=0x7f88ec0b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 -- 172.21.15.55:0/3578724453 shutdown_connections 2024-05-10T03:29:53.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.202+0000 7f88ebe00640 1 -- 172.21.15.55:0/3578724453 wait complete. 2024-05-10T03:29:53.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.205+0000 7f88ebe00640 1 Processor -- start 2024-05-10T03:29:53.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.205+0000 7f88ebe00640 1 -- start start 2024-05-10T03:29:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ec159b10 0x7f88ec06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1414f0 con 0x7f88ec0b5970 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0cba10 con 0x7f88ec159b10 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.206+0000 7f88ebe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0c6880 con 0x7f88ec0b5020 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37310/0 (socket says 172.21.15.55:37310) 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 -- 172.21.15.55:0/47736875 learned_addr learned my addr 172.21.15.55:0/47736875 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 -- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 msgr2=0x7f88ec0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 -- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ec159b10 msgr2=0x7f88ec06c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ec159b10 0x7f88ec06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 -- 172.21.15.55:0/47736875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc0094d0 con 0x7f88ec0b5970 2024-05-10T03:29:53.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.207+0000 7f88ea000640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88d400c820 tx=0x7f88d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:53.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.208+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d4007a30 con 0x7f88ec0b5970 2024-05-10T03:29:53.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.208+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4007b90 con 0x7f88ec0b5970 2024-05-10T03:29:53.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.208+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d400ce10 con 0x7f88ec0b5970 2024-05-10T03:29:53.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.208+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec0c7900 con 0x7f88ec0b5970 2024-05-10T03:29:53.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.208+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec0c75d0 con 0x7f88ec0b5970 2024-05-10T03:29:53.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.209+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec0ce7c0 con 0x7f88ec0b5970 2024-05-10T03:29:53.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.210+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f88d40113f0 con 0x7f88ec0b5970 2024-05-10T03:29:53.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.211+0000 7f88e2a00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 0x7f88c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.211+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f88d40897f0 con 0x7f88ec0b5970 2024-05-10T03:29:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.212+0000 7f88eaa00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 0x7f88c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.213+0000 7f88eaa00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 0x7f88c806d140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88dc034000 tx=0x7f88dc02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:53.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.216+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f88ec0ce7c0 con 0x7f88ec0b5970 2024-05-10T03:29:53.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.453+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f88ec0ce7c0 con 0x7f88ec0b5970 2024-05-10T03:29:53.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.454+0000 7f88e2a00640 1 -- 172.21.15.55:0/47736875 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f88ec0ce7c0 con 0x7f88ec0b5970 2024-05-10T03:29:53.457 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T03:29:53.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.456+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 msgr2=0x7f88c806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:53.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.456+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 0x7f88c806d140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88dc034000 tx=0x7f88dc02c000 comp rx=0 tx=0).stop 2024-05-10T03:29:53.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.456+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 msgr2=0x7f88ec06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:53.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.456+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88d400c820 tx=0x7f88d400cce0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 shutdown_connections 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88c806ac30 0x7f88c806d140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88ec159b10 0x7f88ec06c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88ec0b5970 0x7f88ec06bd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 --2- 172.21.15.55:0/47736875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88ec0b5020 0x7f88ec0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 >> 172.21.15.55:0/47736875 conn(0x7f88ec0b6680 msgr2=0x7f88ec0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 shutdown_connections 2024-05-10T03:29:53.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:53.457+0000 7f88ebe00640 1 -- 172.21.15.55:0/47736875 wait complete. 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T03:29:53.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:29:53.488 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T03:29:53.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:53.488 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:29:53.542 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:29:53.542 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:29:53.542 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:29:53.542 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42459sec preferred_lft 42459sec 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:29:53.543 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:29:53.544 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:29:53.544 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:29:53.544 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:29:53.545 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T03:29:53.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T03:29:53.545 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:29:53.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:53.970 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:29:54.048 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:29:54.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:54.051 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:29:54.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:29:54.116 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:29:54.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:54.819 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:29:54.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:54.880 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-10T03:29:54.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:54.939 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:29:54.939 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:29:54.939 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T03:29:54.939 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T03:29:54.939 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:29:55.070 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T03:29:55.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T03:29:55.071 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:29:55.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:55.321 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:29:55.321 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:29:55.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:29:55.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T03:29:55.321 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:29:55.466 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T03:29:55.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:29:55.466 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:55.511 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T03:29:55.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:29:55.512 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:55.607 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-10T03:29:55.700 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T03:29:55.700 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T03:29:55.781 INFO:teuthology.orchestra.run.smithi055.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T03:29:55.784 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T03:29:55.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:55.785 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-10T03:29:55.834 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T03:29:55.834 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T03:29:55.839 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-10T03:29:55.904 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T03:29:55.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T03:29:55.971 INFO:teuthology.orchestra.run.smithi055.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T03:29:55.973 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T03:29:55.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:55.974 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-10T03:29:56.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:2024-05-10T03:29:56.117+0000 7fa7d9c0d480 -1 init, newargv = 0x556034b19ff0 newargc=15 2024-05-10T03:29:56.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[41476]: starting ceph client 2024-05-10T03:29:56.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[41476]: starting fuse 2024-05-10T03:29:57.124 DEBUG:teuthology.orchestra.run.smithi055:> sudo modprobe fuse 2024-05-10T03:29:57.220 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T03:29:57.221 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T03:29:57.309 INFO:teuthology.orchestra.run.smithi055.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T03:29:57.313 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T03:29:57.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:57.313 DEBUG:teuthology.orchestra.run.smithi055:> ls /sys/fs/fuse/connections 2024-05-10T03:29:57.370 INFO:teuthology.orchestra.run.smithi055.stdout:41 2024-05-10T03:29:57.371 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> import glob 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> import re 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> import os 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> import subprocess 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> def _find_admin_socket(client_name): 2024-05-10T03:29:57.371 DEBUG:teuthology.orchestra.run.smithi055:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> files = glob.glob(asok_path) 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> # Given a non-glob path, it better be there 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> if "*" not in asok_path: 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> assert(len(files) == 1) 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> return files[0] 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> for f in files: 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> contents = proc_f.read() 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> if mountpoint in contents: 2024-05-10T03:29:57.372 DEBUG:teuthology.orchestra.run.smithi055:> return f 2024-05-10T03:29:57.373 DEBUG:teuthology.orchestra.run.smithi055:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T03:29:57.373 DEBUG:teuthology.orchestra.run.smithi055:> 2024-05-10T03:29:57.373 DEBUG:teuthology.orchestra.run.smithi055:> print(_find_admin_socket("client.admin")) 2024-05-10T03:29:57.373 DEBUG:teuthology.orchestra.run.smithi055:> ' 2024-05-10T03:29:57.545 INFO:teuthology.orchestra.run.smithi055.stdout:/var/run/ceph/ceph-client.admin.41476.asok 2024-05-10T03:29:57.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41476.asok 2024-05-10T03:29:57.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:57.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41476.asok status 2024-05-10T03:29:58.033 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:29:58.033 INFO:teuthology.orchestra.run.smithi055.stdout: "metadata": { 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_sha1": "50174015a7e79f951b988949abe236791f52d4da", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "admin", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "pid": "41476", 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_count": 0, 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "dentry_pinned_count": 0, 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4404, 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": { 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-10T03:29:58.034 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 4404 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 936082892 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "192.168.0.1:0", 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 936082892 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:29:58.035 INFO:teuthology.orchestra.run.smithi055.stdout: "inst_str": "client.4404 192.168.0.1:0/936082892", 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "addr_str": "192.168.0.1:0/936082892", 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "inode_count": 1, 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_epoch": 7, 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch": 25, 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_epoch_barrier": 0, 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "blocklisted": false, 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs" 2024-05-10T03:29:58.036 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:29:58.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:58.055 DEBUG:teuthology.orchestra.run.smithi055:> stat --file-system '--printf=%T 2024-05-10T03:29:58.055 DEBUG:teuthology.orchestra.run.smithi055:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:58.109 INFO:teuthology.orchestra.run.smithi055.stdout:fuseblk 2024-05-10T03:29:58.110 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:58.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:58.110 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:29:58.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:29:58.197 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T03:29:58.289 INFO:teuthology.orchestra.run.smithi055.stdout:# file: . 2024-05-10T03:29:58.289 INFO:teuthology.orchestra.run.smithi055.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T03:29:58.290 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:58.291 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:29:58.291 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:29:58.353 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:29:58.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.635+0000 7fc5cba00640 1 Processor -- start 2024-05-10T03:29:58.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.636+0000 7fc5cba00640 1 -- start start 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.638+0000 7fc5cba00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5c40b2140 0x7fc5c4142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.638+0000 7fc5cba00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c406ad20 con 0x7fc5c40b2550 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.638+0000 7fc5cba00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c406a110 con 0x7fc5c415fe50 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.638+0000 7fc5cba00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c406a3f0 con 0x7fc5c40b2140 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.639+0000 7fc5c9a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc5c40b2550 0x7fc5c406b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45028/0 (socket says 172.21.15.55:45028) 2024-05-10T03:29:58.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.639+0000 7fc5c9a00640 1 -- 172.21.15.55:0/4127066019 learned_addr learned my addr 172.21.15.55:0/4127066019 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:58.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.639+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/4127066019 >> v2:172.21.15.162:3300/0 conn(0x7fc5c40b2140 0x7fc5c4142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.639+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630940849 0 0) 0x7fc5c406ad20 con 0x7fc5c40b2550 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.639+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5ac003520 con 0x7fc5c40b2550 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246575568 0 0) 0x7fc5ac003520 con 0x7fc5c40b2550 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 >> v2:172.21.15.162:3300/0 conn(0x7fc5c40b2140 msgr2=0x7fc5c4142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 --2- 172.21.15.55:0/4127066019 >> v2:172.21.15.162:3300/0 conn(0x7fc5c40b2140 0x7fc5c4142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 >> v1:172.21.15.162:6789/0 conn(0x7fc5c415fe50 legacy=0x7fc5c4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c406a6d0 con 0x7fc5c40b2550 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fc5bc0019f0 con 0x7fc5c40b2550 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 >> v1:172.21.15.55:6789/0 conn(0x7fc5c40b2550 legacy=0x7fc5c406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac003970 0x7fc5ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 0x7fc5ac004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac005180 0x7fc5ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c406a3f0 con 0x7fc5ac0047c0 2024-05-10T03:29:58.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac00c110 con 0x7fc5ac005180 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac00c2d0 con 0x7fc5ac003970 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 0x7fc5ac004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.640+0000 7fc5c9000640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac003970 0x7fc5ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c3e00640 1 -- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac003970 msgr2=0x7fc5ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c9a00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac005180 0x7fc5ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5ac003970 0x7fc5ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c3e00640 1 -- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac005180 msgr2=0x7fc5ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac005180 0x7fc5ac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c9a00640 1 -- 172.21.15.55:0/4127066019 reap_dead start 2024-05-10T03:29:58.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.641+0000 7fc5c3e00640 1 -- 172.21.15.55:0/4127066019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac003520 con 0x7fc5ac0047c0 2024-05-10T03:29:58.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.642+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 0x7fc5ac004bd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009ac0 tx=0x7fc5ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ad60f8cac03565 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:58.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.642+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0031400 con 0x7fc5ac0047c0 2024-05-10T03:29:58.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.642+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b002d050 con 0x7fc5ac0047c0 2024-05-10T03:29:58.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.642+0000 7fc5c3400640 1 -- 172.21.15.55:0/4127066019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0031780 con 0x7fc5ac0047c0 2024-05-10T03:29:58.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 -- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 msgr2=0x7fc5ac004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 0x7fc5ac004bd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009ac0 tx=0x7fc5ac0057f0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 -- 172.21.15.55:0/4127066019 shutdown_connections 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 --2- 172.21.15.55:0/4127066019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5ac0047c0 0x7fc5ac004bd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 -- 172.21.15.55:0/4127066019 >> 172.21.15.55:0/4127066019 conn(0x7fc5c40cbb90 msgr2=0x7fc5c40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 -- 172.21.15.55:0/4127066019 shutdown_connections 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.643+0000 7fc5cba00640 1 -- 172.21.15.55:0/4127066019 wait complete. 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.646+0000 7fc5cba00640 1 Processor -- start 2024-05-10T03:29:58.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.646+0000 7fc5cba00640 1 -- start start 2024-05-10T03:29:58.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0047c0 0x7fc5c415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5c41e6c00 0x7fc5c4143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41f5000 con 0x7fc5c4144400 2024-05-10T03:29:58.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41e51c0 con 0x7fc5ac0047c0 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.647+0000 7fc5cba00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4144810 con 0x7fc5c41e6c00 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37340/0 (socket says 172.21.15.55:37340) 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 -- 172.21.15.55:0/2065504635 learned_addr learned my addr 172.21.15.55:0/2065504635 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c3e00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5c41e6c00 0x7fc5c4143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9000640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0047c0 0x7fc5c415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 -- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5c41e6c00 msgr2=0x7fc5c4143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5c41e6c00 0x7fc5c4143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 -- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0047c0 msgr2=0x7fc5c415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0047c0 0x7fc5c415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 -- 172.21.15.55:0/2065504635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c41e68b0 con 0x7fc5c4144400 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.648+0000 7fc5c9a00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc000f80 tx=0x7fc5bc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:58.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.649+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc004d60 con 0x7fc5c4144400 2024-05-10T03:29:58.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.649+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc00bb80 con 0x7fc5c4144400 2024-05-10T03:29:58.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.649+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc007070 con 0x7fc5c4144400 2024-05-10T03:29:58.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.649+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c41e4f70 con 0x7fc5c4144400 2024-05-10T03:29:58.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.649+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c41e56f0 con 0x7fc5c4144400 2024-05-10T03:29:58.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.650+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c4066f60 con 0x7fc5c4144400 2024-05-10T03:29:58.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.651+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc5bc0073e0 con 0x7fc5c4144400 2024-05-10T03:29:58.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.652+0000 7fc5c0c00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 0x7fc5a406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:58.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.652+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc5bc089a90 con 0x7fc5c4144400 2024-05-10T03:29:58.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.653+0000 7fc5c9000640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 0x7fc5a406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:58.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.654+0000 7fc5c9000640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 0x7fc5a406d110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8007fd0 tx=0x7fc5b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.658+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc5bc0548e0 con 0x7fc5c4144400 2024-05-10T03:29:58.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.859+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5c4067220 con 0x7fc5c4144400 2024-05-10T03:29:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.861+0000 7fc5c0c00640 1 -- 172.21.15.55:0/2065504635 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fc5bc057f00 con 0x7fc5c4144400 2024-05-10T03:29:58.866 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:29:58.866 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:29:58.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 msgr2=0x7fc5a406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 0x7fc5a406d110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8007fd0 tx=0x7fc5b8009040 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 msgr2=0x7fc5c415bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc000f80 tx=0x7fc5bc00b4a0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 shutdown_connections 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5c4144400 0x7fc5c415bd70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5c41e6c00 0x7fc5c4143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc5a406ac00 0x7fc5a406d110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 --2- 172.21.15.55:0/2065504635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5ac0047c0 0x7fc5c415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 >> 172.21.15.55:0/2065504635 conn(0x7fc5c40cbb90 msgr2=0x7fc5c40aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 shutdown_connections 2024-05-10T03:29:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:58.864+0000 7fc5cba00640 1 -- 172.21.15.55:0/2065504635 wait complete. 2024-05-10T03:29:58.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:29:58.897 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:29:59.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.175+0000 7f1747e00640 1 Processor -- start 2024-05-10T03:29:59.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.176+0000 7f1747e00640 1 -- start start 2024-05-10T03:29:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.178+0000 7f1747e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f17480b2110 0x7f174815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.178+0000 7f1747e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.178+0000 7f1747e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17480ceb00 con 0x7f17480ce090 2024-05-10T03:29:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.178+0000 7f1747e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174815dcd0 con 0x7f17480b2110 2024-05-10T03:29:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.178+0000 7f1747e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480d2150 con 0x7f17480b1bc0 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.179+0000 7f173fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.179+0000 7f173fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:58304/0 (socket says 172.21.15.55:58304) 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.179+0000 7f173fe00640 1 -- 172.21.15.55:0/392812396 learned_addr learned my addr 172.21.15.55:0/392812396 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.179+0000 7f1746a00640 1 --2- 172.21.15.55:0/392812396 >> v2:172.21.15.162:3300/0 conn(0x7f17480b2110 0x7f174815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.180+0000 7f173fe00640 1 -- 172.21.15.55:0/392812396 >> v2:172.21.15.162:3300/0 conn(0x7f17480b2110 msgr2=0x7f174815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.180+0000 7f173fe00640 1 --2- 172.21.15.55:0/392812396 >> v2:172.21.15.162:3300/0 conn(0x7f17480b2110 0x7f174815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.180+0000 7f173fe00640 1 -- 172.21.15.55:0/392812396 >> v1:172.21.15.162:6789/0 conn(0x7f17480ce090 legacy=0x7f1748161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.180+0000 7f173fe00640 1 -- 172.21.15.55:0/392812396 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17480b2910 con 0x7f17480b1bc0 2024-05-10T03:29:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.181+0000 7f173fe00640 1 --2- 172.21.15.55:0/392812396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1730009870 tx=0x7f173002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2bd71fa9c2a14f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.182+0000 7f1746000640 1 -- 172.21.15.55:0/392812396 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730032440 con 0x7f17480b1bc0 2024-05-10T03:29:59.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.182+0000 7f1746000640 1 -- 172.21.15.55:0/392812396 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17300325a0 con 0x7f17480b1bc0 2024-05-10T03:29:59.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.182+0000 7f1746000640 1 -- 172.21.15.55:0/392812396 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730032930 con 0x7f17480b1bc0 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 -- 172.21.15.55:0/392812396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 msgr2=0x7f17480cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 --2- 172.21.15.55:0/392812396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1730009870 tx=0x7f173002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 -- 172.21.15.55:0/392812396 shutdown_connections 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 --2- 172.21.15.55:0/392812396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480b1bc0 0x7f17480cdb50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 --2- 172.21.15.55:0/392812396 >> v2:172.21.15.162:3300/0 conn(0x7f17480b2110 0x7f174815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 -- 172.21.15.55:0/392812396 >> 172.21.15.55:0/392812396 conn(0x7f17480d30d0 msgr2=0x7f17480d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 -- 172.21.15.55:0/392812396 shutdown_connections 2024-05-10T03:29:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.183+0000 7f1747e00640 1 -- 172.21.15.55:0/392812396 wait complete. 2024-05-10T03:29:59.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.186+0000 7f1747e00640 1 Processor -- start 2024-05-10T03:29:59.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.186+0000 7f1747e00640 1 -- start start 2024-05-10T03:29:59.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.188+0000 7f1747e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17480b2110 0x7f174815e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 0x7f174806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748160530 con 0x7f17480b1bc0 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481531e0 con 0x7f17480b2110 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17481e93b0 con 0x7f17480ce090 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1746a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1746a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37346/0 (socket says 172.21.15.55:37346) 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1746a00640 1 -- 172.21.15.55:0/2665746794 learned_addr learned my addr 172.21.15.55:0/2665746794 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f1747400640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 0x7f174806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.189+0000 7f173fe00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17480b2110 0x7f174815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 -- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 msgr2=0x7f174806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 0x7f174806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 -- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17480b2110 msgr2=0x7f174815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17480b2110 0x7f174815e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 -- 172.21.15.55:0/2665746794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17300094d0 con 0x7f17480b1bc0 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1747400640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 0x7f174806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1746a00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f173800e8a0 tx=0x7f173800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380094f0 con 0x7f17480b1bc0 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173800c940 con 0x7f17480b1bc0 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800ccf0 con 0x7f17480b1bc0 2024-05-10T03:29:59.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.190+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174815f660 con 0x7f17480b1bc0 2024-05-10T03:29:59.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.191+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174815fb90 con 0x7f17480b1bc0 2024-05-10T03:29:59.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.192+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17480c9340 con 0x7f17480b1bc0 2024-05-10T03:29:59.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.193+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1738013470 con 0x7f17480b1bc0 2024-05-10T03:29:59.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.194+0000 7f173ea00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 0x7f172406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.194+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1738089b00 con 0x7f17480b1bc0 2024-05-10T03:29:59.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.195+0000 7f173fe00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 0x7f172406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.196+0000 7f173fe00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 0x7f172406d140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f173002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.200+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f173808be60 con 0x7f17480b1bc0 2024-05-10T03:29:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.396+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f174804e290 con 0x7f17480b1bc0 2024-05-10T03:29:59.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.397+0000 7f173ea00640 1 -- 172.21.15.55:0/2665746794 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17380170a0 con 0x7f17480b1bc0 2024-05-10T03:29:59.402 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 msgr2=0x7f172406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 0x7f172406d140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1730034000 tx=0x7f173002e000 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 msgr2=0x7f1748152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f173800e8a0 tx=0x7f173800ed60 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 shutdown_connections 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17480ce090 0x7f174806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f172406ac30 0x7f172406d140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17480b2110 0x7f174815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 --2- 172.21.15.55:0/2665746794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17480b1bc0 0x7f1748152ca0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 >> 172.21.15.55:0/2665746794 conn(0x7f17480d30d0 msgr2=0x7f174806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 shutdown_connections 2024-05-10T03:29:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.400+0000 7f1747e00640 1 -- 172.21.15.55:0/2665746794 wait complete. 2024-05-10T03:29:59.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:29:59.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.715+0000 7f3539c00640 1 Processor -- start 2024-05-10T03:29:59.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.715+0000 7f3539c00640 1 -- start start 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.717+0000 7f3539c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f353406b120 0x7f353406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.717+0000 7f3539c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35340adb20 0x7f35340adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.717+0000 7f3539c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35340b3590 con 0x7f35340b4450 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.717+0000 7f3539c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353406ba70 con 0x7f353406b120 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.717+0000 7f3539c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35340b29c0 con 0x7f35340adb20 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f352be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f353406b120 0x7f353406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f3532a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f35340b4450 0x7f35340b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45032/0 (socket says 172.21.15.55:45032) 2024-05-10T03:29:59.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f3532a00640 1 -- 172.21.15.55:0/322592081 learned_addr learned my addr 172.21.15.55:0/322592081 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:29:59.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f352be00640 1 -- 172.21.15.55:0/322592081 >> v2:172.21.15.162:3300/0 conn(0x7f35340adb20 msgr2=0x7f35340adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:29:59.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f352be00640 1 --2- 172.21.15.55:0/322592081 >> v2:172.21.15.162:3300/0 conn(0x7f35340adb20 0x7f35340adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f352be00640 1 -- 172.21.15.55:0/322592081 >> v1:172.21.15.55:6789/0 conn(0x7f35340b4450 legacy=0x7f35340b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f352be00640 1 -- 172.21.15.55:0/322592081 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340b2c60 con 0x7f353406b120 2024-05-10T03:29:59.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.718+0000 7f3532000640 1 -- 172.21.15.55:0/322592081 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266980795 0 0) 0x7f35340b3590 con 0x7f35340b4450 2024-05-10T03:29:59.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.719+0000 7f352be00640 1 --2- 172.21.15.55:0/322592081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f353406b530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35340706d0 tx=0x7f351c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d11a40a4e0c2030 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.720+0000 7f3532000640 1 -- 172.21.15.55:0/322592081 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c02ae60 con 0x7f353406b120 2024-05-10T03:29:59.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.720+0000 7f3532000640 1 -- 172.21.15.55:0/322592081 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c032440 con 0x7f353406b120 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.720+0000 7f3532000640 1 -- 172.21.15.55:0/322592081 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0327d0 con 0x7f353406b120 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 -- 172.21.15.55:0/322592081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 msgr2=0x7f353406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 --2- 172.21.15.55:0/322592081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f353406b530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35340706d0 tx=0x7f351c02a110 comp rx=0 tx=0).stop 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 -- 172.21.15.55:0/322592081 shutdown_connections 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 --2- 172.21.15.55:0/322592081 >> v2:172.21.15.162:3300/0 conn(0x7f35340adb20 0x7f35340adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 --2- 172.21.15.55:0/322592081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f353406b530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 -- 172.21.15.55:0/322592081 >> 172.21.15.55:0/322592081 conn(0x7f3534070900 msgr2=0x7f3534070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:59.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 -- 172.21.15.55:0/322592081 shutdown_connections 2024-05-10T03:29:59.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.721+0000 7f3539c00640 1 -- 172.21.15.55:0/322592081 wait complete. 2024-05-10T03:29:59.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.723+0000 7f3539c00640 1 Processor -- start 2024-05-10T03:29:59.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.723+0000 7f3539c00640 1 -- start start 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35340adb20 0x7f35340cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35340b4450 0x7f35341f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353415f380 con 0x7f353406b120 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35341f8060 con 0x7f35340b4450 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3539c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35340ceeb0 con 0x7f35340adb20 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3532a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3532a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37366/0 (socket says 172.21.15.55:37366) 2024-05-10T03:29:59.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.725+0000 7f3532a00640 1 -- 172.21.15.55:0/3319936577 learned_addr learned my addr 172.21.15.55:0/3319936577 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:29:59.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f352be00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35340adb20 0x7f35340cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3533400640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35340b4450 0x7f35341f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 -- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35340adb20 msgr2=0x7f35340cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35340adb20 0x7f35340cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 -- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35340b4450 msgr2=0x7f35341f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35340b4450 0x7f35341f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 -- 172.21.15.55:0/3319936577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c009400 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.726+0000 7f3532a00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f352400c410 tx=0x7f352400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.727+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240074e0 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.727+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524007660 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.727+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524004a20 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.727+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35340ce640 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.727+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35340ce0a0 con 0x7f353406b120 2024-05-10T03:29:59.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.728+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3534067220 con 0x7f353406b120 2024-05-10T03:29:59.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.729+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3524004b80 con 0x7f353406b120 2024-05-10T03:29:59.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.731+0000 7f352aa00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 0x7f351006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:29:59.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.731+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3524089590 con 0x7f353406b120 2024-05-10T03:29:59.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.731+0000 7f352be00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 0x7f351006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:29:59.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.732+0000 7f352be00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 0x7f351006d110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f351c034000 tx=0x7f351c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:29:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.735+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3524057e70 con 0x7f353406b120 2024-05-10T03:29:59.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.938+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35341445f0 con 0x7f353406b120 2024-05-10T03:29:59.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.939+0000 7f352aa00640 1 -- 172.21.15.55:0/3319936577 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3524057a00 con 0x7f353406b120 2024-05-10T03:29:59.943 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 msgr2=0x7f351006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 0x7f351006d110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f351c034000 tx=0x7f351c02e000 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 msgr2=0x7f35340cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f352400c410 tx=0x7f352400c8d0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 shutdown_connections 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f351006ac00 0x7f351006d110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35340b4450 0x7f35341f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35340adb20 0x7f35340cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 --2- 172.21.15.55:0/3319936577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f353406b120 0x7f35340cbb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:29:59.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 >> 172.21.15.55:0/3319936577 conn(0x7f3534070900 msgr2=0x7f35340adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:29:59.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 shutdown_connections 2024-05-10T03:29:59.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:29:59.942+0000 7f3539c00640 1 -- 172.21.15.55:0/3319936577 wait complete. 2024-05-10T03:29:59.974 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:00.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.261+0000 7fe706e00640 1 Processor -- start 2024-05-10T03:30:00.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.261+0000 7fe706e00640 1 -- start start 2024-05-10T03:30:00.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.263+0000 7fe706e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7000b2140 0x7fe700142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.263+0000 7fe706e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7000b2550 0x7fe700142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe706e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70006ad20 con 0x7fe70015fe50 2024-05-10T03:30:00.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe706e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70006a110 con 0x7fe7000b2140 2024-05-10T03:30:00.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe706e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70006a3f0 con 0x7fe7000b2550 2024-05-10T03:30:00.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ff400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7000b2140 0x7fe700142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ffe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe70015fe50 0x7fe700141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45040/0 (socket says 172.21.15.55:45040) 2024-05-10T03:30:00.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ffe00640 1 -- 172.21.15.55:0/562002397 learned_addr learned my addr 172.21.15.55:0/562002397 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:00.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe704e00640 1 --2- 172.21.15.55:0/562002397 >> v2:172.21.15.162:3300/0 conn(0x7fe7000b2550 0x7fe700142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ff400640 1 -- 172.21.15.55:0/562002397 >> v2:172.21.15.162:3300/0 conn(0x7fe7000b2550 msgr2=0x7fe700142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:00.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ff400640 1 --2- 172.21.15.55:0/562002397 >> v2:172.21.15.162:3300/0 conn(0x7fe7000b2550 0x7fe700142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.264+0000 7fe6ff400640 1 -- 172.21.15.55:0/562002397 >> v1:172.21.15.55:6789/0 conn(0x7fe70015fe50 legacy=0x7fe700141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.265+0000 7fe6ff400640 1 -- 172.21.15.55:0/562002397 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70006a6d0 con 0x7fe7000b2140 2024-05-10T03:30:00.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.266+0000 7fe6ff400640 1 --2- 172.21.15.55:0/562002397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7000b2140 0x7fe700142440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe70006d130 tx=0x7fe6ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8362ae69f9c8faf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.266+0000 7fe6fea00640 1 -- 172.21.15.55:0/562002397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec032440 con 0x7fe7000b2140 2024-05-10T03:30:00.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.266+0000 7fe6fea00640 1 -- 172.21.15.55:0/562002397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec0325a0 con 0x7fe7000b2140 2024-05-10T03:30:00.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.266+0000 7fe6fea00640 1 -- 172.21.15.55:0/562002397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec032930 con 0x7fe7000b2140 2024-05-10T03:30:00.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 -- 172.21.15.55:0/562002397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7000b2140 msgr2=0x7fe700142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 --2- 172.21.15.55:0/562002397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7000b2140 0x7fe700142440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe70006d130 tx=0x7fe6ec02a230 comp rx=0 tx=0).stop 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 -- 172.21.15.55:0/562002397 shutdown_connections 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 --2- 172.21.15.55:0/562002397 >> v2:172.21.15.162:3300/0 conn(0x7fe7000b2550 0x7fe700142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 --2- 172.21.15.55:0/562002397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7000b2140 0x7fe700142440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 -- 172.21.15.55:0/562002397 >> 172.21.15.55:0/562002397 conn(0x7fe7000cbb90 msgr2=0x7fe7000cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 -- 172.21.15.55:0/562002397 shutdown_connections 2024-05-10T03:30:00.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.267+0000 7fe706e00640 1 -- 172.21.15.55:0/562002397 wait complete. 2024-05-10T03:30:00.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.269+0000 7fe706e00640 1 Processor -- start 2024-05-10T03:30:00.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.270+0000 7fe706e00640 1 -- start start 2024-05-10T03:30:00.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7000b2140 0x7fe7000aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7000b2550 0x7fe7001f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700071260 con 0x7fe70015fe50 2024-05-10T03:30:00.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000acbe0 con 0x7fe7000b2550 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe706e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000ada40 con 0x7fe7000b2140 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe704e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.271+0000 7fe704e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37388/0 (socket says 172.21.15.55:37388) 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 -- 172.21.15.55:0/1034097920 learned_addr learned my addr 172.21.15.55:0/1034097920 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe6ff400640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7000b2550 0x7fe7001f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe6ffe00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7000b2140 0x7fe7000aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 -- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7000b2140 msgr2=0x7fe7000aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7000b2140 0x7fe7000aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 -- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7000b2550 msgr2=0x7fe7001f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7000b2550 0x7fe7001f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 -- 172.21.15.55:0/1034097920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec009400 con 0x7fe70015fe50 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.272+0000 7fe704e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007e60 tx=0x7fe6f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.273+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8014420 con 0x7fe70015fe50 2024-05-10T03:30:00.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.273+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80145a0 con 0x7fe70015fe50 2024-05-10T03:30:00.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.273+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8014950 con 0x7fe70015fe50 2024-05-10T03:30:00.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.273+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7000ac9c0 con 0x7fe70015fe50 2024-05-10T03:30:00.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.273+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7000ace30 con 0x7fe70015fe50 2024-05-10T03:30:00.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.275+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe700066f60 con 0x7fe70015fe50 2024-05-10T03:30:00.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.276+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe6f8014ab0 con 0x7fe70015fe50 2024-05-10T03:30:00.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.276+0000 7fe6f7e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 0x7fe6e006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.277+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe6f8089f50 con 0x7fe70015fe50 2024-05-10T03:30:00.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.277+0000 7fe6ffe00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 0x7fe6e006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.278+0000 7fe6ffe00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 0x7fe6e006d140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005880 tx=0x7fe6f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.281+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe6f8054e40 con 0x7fe70015fe50 2024-05-10T03:30:00.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.484+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe70006b870 con 0x7fe70015fe50 2024-05-10T03:30:00.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.485+0000 7fe6f7e00640 1 -- 172.21.15.55:0/1034097920 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6f8058460 con 0x7fe70015fe50 2024-05-10T03:30:00.490 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 msgr2=0x7fe6e006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 0x7fe6e006d140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0005880 tx=0x7fe6f0009040 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 msgr2=0x7fe700070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8007e60 tx=0x7fe6f800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 shutdown_connections 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe70015fe50 0x7fe700070540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe6e006ac30 0x7fe6e006d140 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7000b2550 0x7fe7001f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 --2- 172.21.15.55:0/1034097920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7000b2140 0x7fe7000aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 >> 172.21.15.55:0/1034097920 conn(0x7fe7000cbb90 msgr2=0x7fe7000cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:00.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 shutdown_connections 2024-05-10T03:30:00.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.488+0000 7fe706e00640 1 -- 172.21.15.55:0/1034097920 wait complete. 2024-05-10T03:30:00.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.802+0000 7fd999000640 1 Processor -- start 2024-05-10T03:30:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.802+0000 7fd999000640 1 -- start start 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.805+0000 7fd999000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9940719f0 0x7fd99406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.805+0000 7fd999000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd99406b840 0x7fd9940700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.805+0000 7fd999000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99415fdf0 con 0x7fd994071e00 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.805+0000 7fd999000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99415f1e0 con 0x7fd9940719f0 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.805+0000 7fd999000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99415f4c0 con 0x7fd99406b840 2024-05-10T03:30:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9940719f0 0x7fd99406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9940719f0 0x7fd99406b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37390/0 (socket says 172.21.15.55:37390) 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 -- 172.21.15.55:0/892642232 learned_addr learned my addr 172.21.15.55:0/892642232 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd98aa00640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd99406b840 0x7fd9940700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 -- 172.21.15.55:0/892642232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd99406b840 msgr2=0x7fd9940700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd99406b840 0x7fd9940700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 -- 172.21.15.55:0/892642232 >> v1:172.21.15.162:6789/0 conn(0x7fd994071e00 legacy=0x7fd994070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.806+0000 7fd993400640 1 -- 172.21.15.55:0/892642232 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99415f7a0 con 0x7fd9940719f0 2024-05-10T03:30:00.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.808+0000 7fd993400640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9940719f0 0x7fd99406b300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347a872bde974fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.808+0000 7fd992a00640 1 -- 172.21.15.55:0/892642232 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984032440 con 0x7fd9940719f0 2024-05-10T03:30:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.808+0000 7fd992a00640 1 -- 172.21.15.55:0/892642232 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9840325a0 con 0x7fd9940719f0 2024-05-10T03:30:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.808+0000 7fd992a00640 1 -- 172.21.15.55:0/892642232 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984032930 con 0x7fd9940719f0 2024-05-10T03:30:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 -- 172.21.15.55:0/892642232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9940719f0 msgr2=0x7fd99406b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9940719f0 0x7fd99406b300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 -- 172.21.15.55:0/892642232 shutdown_connections 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd99406b840 0x7fd9940700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 --2- 172.21.15.55:0/892642232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9940719f0 0x7fd99406b300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 -- 172.21.15.55:0/892642232 >> 172.21.15.55:0/892642232 conn(0x7fd99406c750 msgr2=0x7fd9940b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.809+0000 7fd999000640 1 -- 172.21.15.55:0/892642232 shutdown_connections 2024-05-10T03:30:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.810+0000 7fd999000640 1 -- 172.21.15.55:0/892642232 wait complete. 2024-05-10T03:30:00.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.812+0000 7fd999000640 1 Processor -- start 2024-05-10T03:30:00.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.812+0000 7fd999000640 1 -- start start 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9940719f0 0x7fd9940aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd994071e00 0x7fd9940af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940ab900 con 0x7fd99406b840 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940afd20 con 0x7fd9940719f0 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd999000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99413ff70 con 0x7fd994071e00 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37396/0 (socket says 172.21.15.55:37396) 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 -- 172.21.15.55:0/7720324 learned_addr learned my addr 172.21.15.55:0/7720324 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:00.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd98aa00640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9940719f0 0x7fd9940aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 -- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd994071e00 msgr2=0x7fd9940af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd994071e00 0x7fd9940af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 -- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9940719f0 msgr2=0x7fd9940aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9940719f0 0x7fd9940aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.814+0000 7fd993400640 1 -- 172.21.15.55:0/7720324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840094d0 con 0x7fd99406b840 2024-05-10T03:30:00.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd993400640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd9840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984032440 con 0x7fd99406b840 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984030980 con 0x7fd99406b840 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984030d30 con 0x7fd99406b840 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9940aac80 con 0x7fd99406b840 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.815+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940ab240 con 0x7fd99406b840 2024-05-10T03:30:00.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.816+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd994141d20 con 0x7fd99406b840 2024-05-10T03:30:00.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.818+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd984039040 con 0x7fd99406b840 2024-05-10T03:30:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.818+0000 7fd98be00640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 0x7fd97006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.819+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd9840a9fe0 con 0x7fd99406b840 2024-05-10T03:30:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.819+0000 7fd98aa00640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 0x7fd97006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.820+0000 7fd98aa00640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 0x7fd97006d110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd9941f6a00 tx=0x7fd97c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:00.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:00.824+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd984074ec0 con 0x7fd99406b840 2024-05-10T03:30:01.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.023+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd99404e290 con 0x7fd99406b840 2024-05-10T03:30:01.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.024+0000 7fd98be00640 1 -- 172.21.15.55:0/7720324 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd9840784e0 con 0x7fd99406b840 2024-05-10T03:30:01.028 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 msgr2=0x7fd97006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 0x7fd97006d110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd9941f6a00 tx=0x7fd97c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 msgr2=0x7fd9940abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd9840046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 shutdown_connections 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd994071e00 0x7fd9940af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd97006ac00 0x7fd97006d110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9940719f0 0x7fd9940aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 --2- 172.21.15.55:0/7720324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd99406b840 0x7fd9940abd20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 >> 172.21.15.55:0/7720324 conn(0x7fd99406c750 msgr2=0x7fd99406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 shutdown_connections 2024-05-10T03:30:01.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.027+0000 7fd999000640 1 -- 172.21.15.55:0/7720324 wait complete. 2024-05-10T03:30:01.061 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:30:01.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.336+0000 7ff9b3e00640 1 Processor -- start 2024-05-10T03:30:01.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.337+0000 7ff9b3e00640 1 -- start start 2024-05-10T03:30:01.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.339+0000 7ff9b3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9b406b830 0x7ff9b4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.339+0000 7ff9b3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40cdbe0 con 0x7ff9b41435d0 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b415e8b0 con 0x7ff9b40b2040 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40ce140 con 0x7ff9b406b830 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9b406b830 0x7ff9b4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b2000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff9b41435d0 0x7ff9b406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45042/0 (socket says 172.21.15.55:45042) 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b2000640 1 -- 172.21.15.55:0/505373819 learned_addr learned my addr 172.21.15.55:0/505373819 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b3400640 1 -- 172.21.15.55:0/505373819 >> v1:172.21.15.162:6789/0 conn(0x7ff9b40b2040 legacy=0x7ff9b40ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b3400640 1 -- 172.21.15.55:0/505373819 >> v1:172.21.15.55:6789/0 conn(0x7ff9b41435d0 legacy=0x7ff9b406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.341+0000 7ff9b3400640 1 -- 172.21.15.55:0/505373819 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b415ede0 con 0x7ff9b406b830 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.340+0000 7ff9b1600640 1 -- 172.21.15.55:0/505373819 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472560447 0 0) 0x7ff9b40cdbe0 con 0x7ff9b41435d0 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.342+0000 7ff9b3400640 1 --2- 172.21.15.55:0/505373819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b406b830 0x7ff9b4143020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009dc0 tx=0x7ff9ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0f6ff7f6c5a98b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.342+0000 7ff9b1600640 1 -- 172.21.15.55:0/505373819 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac007740 con 0x7ff9b406b830 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.342+0000 7ff9b1600640 1 -- 172.21.15.55:0/505373819 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac02d050 con 0x7ff9b406b830 2024-05-10T03:30:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.342+0000 7ff9b1600640 1 -- 172.21.15.55:0/505373819 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac007ac0 con 0x7ff9b406b830 2024-05-10T03:30:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 -- 172.21.15.55:0/505373819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b406b830 msgr2=0x7ff9b4143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/505373819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b406b830 0x7ff9b4143020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009dc0 tx=0x7ff9ac02a6a0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 -- 172.21.15.55:0/505373819 shutdown_connections 2024-05-10T03:30:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/505373819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b406b830 0x7ff9b4143020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 -- 172.21.15.55:0/505373819 >> 172.21.15.55:0/505373819 conn(0x7ff9b406c730 msgr2=0x7ff9b40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:01.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 -- 172.21.15.55:0/505373819 shutdown_connections 2024-05-10T03:30:01.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.343+0000 7ff9b3e00640 1 -- 172.21.15.55:0/505373819 wait complete. 2024-05-10T03:30:01.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.345+0000 7ff9b3e00640 1 Processor -- start 2024-05-10T03:30:01.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.345+0000 7ff9b3e00640 1 -- start start 2024-05-10T03:30:01.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9b41435d0 0x7ff9b40ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9b415d580 0x7ff9b415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b412da90 con 0x7ff9b40b2040 2024-05-10T03:30:01.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40aaef0 con 0x7ff9b41435d0 2024-05-10T03:30:01.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.347+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b412dec0 con 0x7ff9b415d580 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37418/0 (socket says 172.21.15.55:37418) 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 -- 172.21.15.55:0/3246050850 learned_addr learned my addr 172.21.15.55:0/3246050850 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b3400640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9b415d580 0x7ff9b415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2000640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9b41435d0 0x7ff9b40ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 -- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9b415d580 msgr2=0x7ff9b415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9b415d580 0x7ff9b415d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 -- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9b41435d0 msgr2=0x7ff9b40ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9b41435d0 0x7ff9b40ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 -- 172.21.15.55:0/3246050850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac009a70 con 0x7ff9b40b2040 2024-05-10T03:30:01.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.348+0000 7ff9b2a00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c430 tx=0x7ff9a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.349+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40052d0 con 0x7ff9b40b2040 2024-05-10T03:30:01.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.349+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a4005430 con 0x7ff9b40b2040 2024-05-10T03:30:01.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.349+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4003830 con 0x7ff9b40b2040 2024-05-10T03:30:01.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.349+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40ab9b0 con 0x7ff9b40b2040 2024-05-10T03:30:01.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.349+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b415c7c0 con 0x7ff9b40b2040 2024-05-10T03:30:01.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.350+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b40b1680 con 0x7ff9b40b2040 2024-05-10T03:30:01.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.351+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9a4003990 con 0x7ff9b40b2040 2024-05-10T03:30:01.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.352+0000 7ff9aaa00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 0x7ff99406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.352+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff9a4088ab0 con 0x7ff9b40b2040 2024-05-10T03:30:01.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.353+0000 7ff9b2000640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 0x7ff99406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.354+0000 7ff9b2000640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 0x7ff99406d110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9b406bd00 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.357+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff9a4053900 con 0x7ff9b40b2040 2024-05-10T03:30:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.574+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff9b40ab9b0 con 0x7ff9b40b2040 2024-05-10T03:30:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.575+0000 7ff9aaa00640 1 -- 172.21.15.55:0/3246050850 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff9b40ab9b0 con 0x7ff9b40b2040 2024-05-10T03:30:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 msgr2=0x7ff99406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 0x7ff99406d110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9b406bd00 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 msgr2=0x7ff9b412d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c430 tx=0x7ff9a400c8f0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 shutdown_connections 2024-05-10T03:30:01.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9b415d580 0x7ff9b415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff99406ac00 0x7ff99406d110 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9b41435d0 0x7ff9b40ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 --2- 172.21.15.55:0/3246050850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b40b2040 0x7ff9b412d550 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.578+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 >> 172.21.15.55:0/3246050850 conn(0x7ff9b406c730 msgr2=0x7ff9b4142280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.579+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 shutdown_connections 2024-05-10T03:30:01.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.579+0000 7ff9b3e00640 1 -- 172.21.15.55:0/3246050850 wait complete. 2024-05-10T03:30:01.611 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:01.611 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:01.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:01.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.953+0000 7f1399200640 1 Processor -- start 2024-05-10T03:30:01.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.953+0000 7f1399200640 1 -- start start 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.955+0000 7f1399200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13940b6680 0x7f139415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.955+0000 7f1399200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 0x7f13941611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.956+0000 7f1399200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139406bac0 con 0x7f13941616e0 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.956+0000 7f1399200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13940d2b20 con 0x7f13940b6680 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.956+0000 7f1399200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139415db20 con 0x7f13940b2170 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.956+0000 7f1392a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 0x7f13941611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1393400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13940b6680 0x7f139415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1393e00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f13941616e0 0x7f13940cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:38924/0 (socket says 172.21.15.55:38924) 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1393e00640 1 -- 172.21.15.55:0/2790168843 learned_addr learned my addr 172.21.15.55:0/2790168843 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1392a00640 1 -- 172.21.15.55:0/2790168843 >> v2:172.21.15.162:3300/0 conn(0x7f13940b6680 msgr2=0x7f139415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1392a00640 1 --2- 172.21.15.55:0/2790168843 >> v2:172.21.15.162:3300/0 conn(0x7f13940b6680 0x7f139415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.957+0000 7f1392a00640 1 -- 172.21.15.55:0/2790168843 >> v1:172.21.15.162:6789/0 conn(0x7f13941616e0 legacy=0x7f13940cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.958+0000 7f1392a00640 1 -- 172.21.15.55:0/2790168843 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13940d25a0 con 0x7f13940b2170 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.959+0000 7f1392a00640 1 --2- 172.21.15.55:0/2790168843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 0x7f13941611a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1380009d90 tx=0x7f138002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2f76bb4d7cb387 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.959+0000 7f1392000640 1 -- 172.21.15.55:0/2790168843 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380007740 con 0x7f13940b2170 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.959+0000 7f1392000640 1 -- 172.21.15.55:0/2790168843 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138002d050 con 0x7f13940b2170 2024-05-10T03:30:01.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.959+0000 7f1392000640 1 -- 172.21.15.55:0/2790168843 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380007ac0 con 0x7f13940b2170 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 -- 172.21.15.55:0/2790168843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 msgr2=0x7f13941611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 --2- 172.21.15.55:0/2790168843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 0x7f13941611a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1380009d90 tx=0x7f138002a660 comp rx=0 tx=0).stop 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 -- 172.21.15.55:0/2790168843 shutdown_connections 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 --2- 172.21.15.55:0/2790168843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13940b2170 0x7f13941611a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 --2- 172.21.15.55:0/2790168843 >> v2:172.21.15.162:3300/0 conn(0x7f13940b6680 0x7f139415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.960+0000 7f1399200640 1 -- 172.21.15.55:0/2790168843 >> 172.21.15.55:0/2790168843 conn(0x7f13940b5730 msgr2=0x7f13940d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.961+0000 7f1399200640 1 -- 172.21.15.55:0/2790168843 shutdown_connections 2024-05-10T03:30:01.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.961+0000 7f1399200640 1 -- 172.21.15.55:0/2790168843 wait complete. 2024-05-10T03:30:01.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.964+0000 7f1399200640 1 Processor -- start 2024-05-10T03:30:01.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.964+0000 7f1399200640 1 -- start start 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13940b2170 0x7f1394158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 0x7f139415e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394158d80 con 0x7f13940b6680 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394160710 con 0x7f13940b2170 2024-05-10T03:30:01.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.966+0000 7f1399200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139415fb30 con 0x7f13941616e0 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37424/0 (socket says 172.21.15.55:37424) 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 -- 172.21.15.55:0/1718667237 learned_addr learned my addr 172.21.15.55:0/1718667237 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1393e00640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 0x7f139415e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1393400640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13940b2170 0x7f1394158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 -- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 msgr2=0x7f139415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 0x7f139415e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 -- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13940b2170 msgr2=0x7f1394158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13940b2170 0x7f1394158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.967+0000 7f1392a00640 1 -- 172.21.15.55:0/1718667237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009a70 con 0x7f13940b6680 2024-05-10T03:30:01.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f1393e00640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 0x7f139415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f1392a00640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1380009eb0 tx=0x7f1380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138003b820 con 0x7f13940b6680 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138003b980 con 0x7f13940b6680 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138003bcf0 con 0x7f13940b6680 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.968+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394158f90 con 0x7f13940b6680 2024-05-10T03:30:01.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.969+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139415e360 con 0x7f13940b6680 2024-05-10T03:30:01.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.970+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139404e290 con 0x7f13940b6680 2024-05-10T03:30:01.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.971+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1380044460 con 0x7f13940b6680 2024-05-10T03:30:01.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.972+0000 7f138b400640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 0x7f137406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:01.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.972+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f13800aaf00 con 0x7f13940b6680 2024-05-10T03:30:01.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.973+0000 7f1393400640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 0x7f137406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:01.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.974+0000 7f1393400640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 0x7f137406d110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1384009a30 tx=0x7f1384008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:01.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:01.977+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1380075d50 con 0x7f13940b6680 2024-05-10T03:30:02.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.178+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13940d1510 con 0x7f13940b6680 2024-05-10T03:30:02.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.180+0000 7f138b400640 1 -- 172.21.15.55:0/1718667237 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f1380049020 con 0x7f13940b6680 2024-05-10T03:30:02.185 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:02.186 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 msgr2=0x7f137406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 0x7f137406d110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1384009a30 tx=0x7f1384008040 comp rx=0 tx=0).stop 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 msgr2=0x7f13941601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1380009eb0 tx=0x7f1380004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 shutdown_connections 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13941616e0 0x7f139415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13940b6680 0x7f13941601d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f137406ac00 0x7f137406d110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 --2- 172.21.15.55:0/1718667237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13940b2170 0x7f1394158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.183+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 >> 172.21.15.55:0/1718667237 conn(0x7f13940b5730 msgr2=0x7f13940ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.184+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 shutdown_connections 2024-05-10T03:30:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.184+0000 7f1399200640 1 -- 172.21.15.55:0/1718667237 wait complete. 2024-05-10T03:30:02.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:02.216 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:02.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.499+0000 7ff2ed200640 1 Processor -- start 2024-05-10T03:30:02.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.499+0000 7ff2ed200640 1 -- start start 2024-05-10T03:30:02.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.501+0000 7ff2ed200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff2e8070050 0x7ff2e8141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.501+0000 7ff2ed200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff2e815bce0 0x7ff2e815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.501+0000 7ff2ed200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e8071ce0 con 0x7ff2e80cd750 2024-05-10T03:30:02.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.501+0000 7ff2ed200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e8070f40 con 0x7ff2e815bce0 2024-05-10T03:30:02.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.501+0000 7ff2ed200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e815f280 con 0x7ff2e8070050 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff2e815bce0 0x7ff2e815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff2e80cd750 0x7ff2e815f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45050/0 (socket says 172.21.15.55:45050) 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e5400640 1 -- 172.21.15.55:0/2718603799 learned_addr learned my addr 172.21.15.55:0/2718603799 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e5e00640 1 --2- 172.21.15.55:0/2718603799 >> v2:172.21.15.162:3300/0 conn(0x7ff2e8070050 0x7ff2e8141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e6800640 1 -- 172.21.15.55:0/2718603799 >> v2:172.21.15.162:3300/0 conn(0x7ff2e8070050 msgr2=0x7ff2e8141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e6800640 1 --2- 172.21.15.55:0/2718603799 >> v2:172.21.15.162:3300/0 conn(0x7ff2e8070050 0x7ff2e8141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e6800640 1 -- 172.21.15.55:0/2718603799 >> v1:172.21.15.55:6789/0 conn(0x7ff2e80cd750 legacy=0x7ff2e815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.502+0000 7ff2e6800640 1 -- 172.21.15.55:0/2718603799 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8070460 con 0x7ff2e815bce0 2024-05-10T03:30:02.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.503+0000 7ff2e6800640 1 --2- 172.21.15.55:0/2718603799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e815c0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2e806f540 tx=0x7ff2e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82146f120ef94e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:02.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.504+0000 7ff2e4a00640 1 -- 172.21.15.55:0/2718603799 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e002aea0 con 0x7ff2e815bce0 2024-05-10T03:30:02.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.504+0000 7ff2e4a00640 1 -- 172.21.15.55:0/2718603799 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e002d050 con 0x7ff2e815bce0 2024-05-10T03:30:02.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.504+0000 7ff2e4a00640 1 -- 172.21.15.55:0/2718603799 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0007960 con 0x7ff2e815bce0 2024-05-10T03:30:02.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 -- 172.21.15.55:0/2718603799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 msgr2=0x7ff2e815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2718603799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e815c0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff2e806f540 tx=0x7ff2e002a680 comp rx=0 tx=0).stop 2024-05-10T03:30:02.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 -- 172.21.15.55:0/2718603799 shutdown_connections 2024-05-10T03:30:02.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2718603799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e815c0f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2718603799 >> v2:172.21.15.162:3300/0 conn(0x7ff2e8070050 0x7ff2e8141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 -- 172.21.15.55:0/2718603799 >> 172.21.15.55:0/2718603799 conn(0x7ff2e806c120 msgr2=0x7ff2e806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:02.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 -- 172.21.15.55:0/2718603799 shutdown_connections 2024-05-10T03:30:02.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.505+0000 7ff2ed200640 1 -- 172.21.15.55:0/2718603799 wait complete. 2024-05-10T03:30:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.507+0000 7ff2ed200640 1 Processor -- start 2024-05-10T03:30:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.507+0000 7ff2ed200640 1 -- start start 2024-05-10T03:30:02.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2e8070050 0x7ff2e8149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 0x7ff2e81eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8148340 con 0x7ff2e815bce0 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e80ad260 con 0x7ff2e80cd750 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2ed200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e814a390 con 0x7ff2e8070050 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2e6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2e6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37450/0 (socket says 172.21.15.55:37450) 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.509+0000 7ff2e6800640 1 -- 172.21.15.55:0/2426563157 learned_addr learned my addr 172.21.15.55:0/2426563157 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e5e00640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2e8070050 0x7ff2e8149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e5400640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 0x7ff2e81eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 -- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2e8070050 msgr2=0x7ff2e8149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2e8070050 0x7ff2e8149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 -- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 msgr2=0x7ff2e81eedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 0x7ff2e81eedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 -- 172.21.15.55:0/2426563157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009a70 con 0x7ff2e815bce0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e5400640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 0x7ff2e81eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2e6800640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2e002ac70 tx=0x7ff2e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.510+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e003b820 con 0x7ff2e815bce0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.511+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e003b980 con 0x7ff2e815bce0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.511+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e80ad4b0 con 0x7ff2e815bce0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.511+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e003bcf0 con 0x7ff2e815bce0 2024-05-10T03:30:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.511+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e80ada70 con 0x7ff2e815bce0 2024-05-10T03:30:02.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.512+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e8066f60 con 0x7ff2e815bce0 2024-05-10T03:30:02.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.513+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff2e0031400 con 0x7ff2e815bce0 2024-05-10T03:30:02.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.514+0000 7ff2de000640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 0x7ff2c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:02.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.514+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff2e00aad40 con 0x7ff2e815bce0 2024-05-10T03:30:02.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.514+0000 7ff2e5e00640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 0x7ff2c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:02.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.516+0000 7ff2e5e00640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 0x7ff2c806d140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:02.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.519+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2e0075b90 con 0x7ff2e815bce0 2024-05-10T03:30:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.716+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2e8070460 con 0x7ff2e815bce0 2024-05-10T03:30:02.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.717+0000 7ff2de000640 1 -- 172.21.15.55:0/2426563157 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2e8070460 con 0x7ff2e815bce0 2024-05-10T03:30:02.721 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 msgr2=0x7ff2c806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 0x7ff2c806d140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006fd0 tx=0x7ff2d8008040 comp rx=0 tx=0).stop 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 msgr2=0x7ff2e80af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2e002ac70 tx=0x7ff2e0004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 shutdown_connections 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2e815bce0 0x7ff2e80af9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff2c806ac30 0x7ff2c806d140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e80cd750 0x7ff2e81eedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 --2- 172.21.15.55:0/2426563157 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2e8070050 0x7ff2e8149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 >> 172.21.15.55:0/2426563157 conn(0x7ff2e806c120 msgr2=0x7ff2e813fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 shutdown_connections 2024-05-10T03:30:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:02.720+0000 7ff2ed200640 1 -- 172.21.15.55:0/2426563157 wait complete. 2024-05-10T03:30:02.750 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:03.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.031+0000 7fbd68a00640 1 Processor -- start 2024-05-10T03:30:03.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.031+0000 7fbd68a00640 1 -- start start 2024-05-10T03:30:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd68a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd640b2110 0x7fbd6415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd68a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd640b1bc0 0x7fbd640cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd68a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640ceb00 con 0x7fbd640ce090 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd68a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6415dcd0 con 0x7fbd640b2110 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd68a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640d2150 con 0x7fbd640b1bc0 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd63400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd640b2110 0x7fbd6415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd63e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbd640ce090 0x7fbd64161310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58214/0 (socket says 172.21.15.55:58214) 2024-05-10T03:30:03.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.033+0000 7fbd63e00640 1 -- 172.21.15.55:0/2780136399 learned_addr learned my addr 172.21.15.55:0/2780136399 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:03.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.034+0000 7fbd5be00640 1 --2- 172.21.15.55:0/2780136399 >> v2:172.21.15.162:3300/0 conn(0x7fbd640b1bc0 0x7fbd640cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.034+0000 7fbd63400640 1 -- 172.21.15.55:0/2780136399 >> v2:172.21.15.162:3300/0 conn(0x7fbd640b1bc0 msgr2=0x7fbd640cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:03.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.034+0000 7fbd63400640 1 --2- 172.21.15.55:0/2780136399 >> v2:172.21.15.162:3300/0 conn(0x7fbd640b1bc0 0x7fbd640cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.034+0000 7fbd63400640 1 -- 172.21.15.55:0/2780136399 >> v1:172.21.15.55:6789/0 conn(0x7fbd640ce090 legacy=0x7fbd64161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.034+0000 7fbd63400640 1 -- 172.21.15.55:0/2780136399 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640b2910 con 0x7fbd640b2110 2024-05-10T03:30:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.035+0000 7fbd63400640 1 --2- 172.21.15.55:0/2780136399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b2110 0x7fbd6415d790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd64068570 tx=0x7fbd5402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83da57e3d819250d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.035+0000 7fbd62a00640 1 -- 172.21.15.55:0/2780136399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5402ae60 con 0x7fbd640b2110 2024-05-10T03:30:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.035+0000 7fbd62a00640 1 -- 172.21.15.55:0/2780136399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54032440 con 0x7fbd640b2110 2024-05-10T03:30:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.035+0000 7fbd62a00640 1 -- 172.21.15.55:0/2780136399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540327d0 con 0x7fbd640b2110 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 -- 172.21.15.55:0/2780136399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b2110 msgr2=0x7fbd6415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 --2- 172.21.15.55:0/2780136399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b2110 0x7fbd6415d790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd64068570 tx=0x7fbd5402a110 comp rx=0 tx=0).stop 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 -- 172.21.15.55:0/2780136399 shutdown_connections 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 --2- 172.21.15.55:0/2780136399 >> v2:172.21.15.162:3300/0 conn(0x7fbd640b1bc0 0x7fbd640cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 --2- 172.21.15.55:0/2780136399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b2110 0x7fbd6415d790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 -- 172.21.15.55:0/2780136399 >> 172.21.15.55:0/2780136399 conn(0x7fbd640d30d0 msgr2=0x7fbd640d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 -- 172.21.15.55:0/2780136399 shutdown_connections 2024-05-10T03:30:03.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.036+0000 7fbd68a00640 1 -- 172.21.15.55:0/2780136399 wait complete. 2024-05-10T03:30:03.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.039+0000 7fbd68a00640 1 Processor -- start 2024-05-10T03:30:03.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.039+0000 7fbd68a00640 1 -- start start 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.040+0000 7fbd68a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd68a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd640b2110 0x7fbd6414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd68a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd640ce090 0x7fbd6415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd68a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641f4940 con 0x7fbd640b1bc0 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd68a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640cbc50 con 0x7fbd640ce090 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd68a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640ccab0 con 0x7fbd640b2110 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45802/0 (socket says 172.21.15.55:45802) 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 -- 172.21.15.55:0/956211310 learned_addr learned my addr 172.21.15.55:0/956211310 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:03.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 -- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd640b2110 msgr2=0x7fbd6414c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd640b2110 0x7fbd6414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 -- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd640ce090 msgr2=0x7fbd6415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd640ce090 0x7fbd6415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.041+0000 7fbd63400640 1 -- 172.21.15.55:0/956211310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54009400 con 0x7fbd640b1bc0 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd63400640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54030990 con 0x7fbd640b1bc0 2024-05-10T03:30:03.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54030b10 con 0x7fbd640b1bc0 2024-05-10T03:30:03.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5403d420 con 0x7fbd640b1bc0 2024-05-10T03:30:03.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640cbea0 con 0x7fbd640b1bc0 2024-05-10T03:30:03.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.042+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd640cc460 con 0x7fbd640b1bc0 2024-05-10T03:30:03.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.044+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6404e290 con 0x7fbd640b1bc0 2024-05-10T03:30:03.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.046+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd54039040 con 0x7fbd640b1bc0 2024-05-10T03:30:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.047+0000 7fbd5b400640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 0x7fbd4006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.047+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd540aa8a0 con 0x7fbd640b1bc0 2024-05-10T03:30:03.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.048+0000 7fbd5be00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 0x7fbd4006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.049+0000 7fbd5be00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 0x7fbd4006d110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.051+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbd540756e0 con 0x7fbd640b1bc0 2024-05-10T03:30:03.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.251+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd640cd490 con 0x7fbd640b1bc0 2024-05-10T03:30:03.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.253+0000 7fbd5b400640 1 -- 172.21.15.55:0/956211310 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd54078d00 con 0x7fbd640b1bc0 2024-05-10T03:30:03.257 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 msgr2=0x7fbd4006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 0x7fbd4006d110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 msgr2=0x7fbd640cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 shutdown_connections 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd4006ac00 0x7fbd4006d110 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd640ce090 0x7fbd6415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd640b2110 0x7fbd6414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 --2- 172.21.15.55:0/956211310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd640b1bc0 0x7fbd640cb150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 >> 172.21.15.55:0/956211310 conn(0x7fbd640d30d0 msgr2=0x7fbd640c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:03.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 shutdown_connections 2024-05-10T03:30:03.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.256+0000 7fbd68a00640 1 -- 172.21.15.55:0/956211310 wait complete. 2024-05-10T03:30:03.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:03.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.561+0000 7f8200e00640 1 Processor -- start 2024-05-10T03:30:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.561+0000 7f8200e00640 1 -- start start 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81fc0ad7b0 0x7f81fc161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81fc161c70 0x7f81fc0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 0x7f81fc071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc1623a0 con 0x7f81fc0ad7b0 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0cdd60 con 0x7f81fc161c70 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f8200e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0ce040 con 0x7f81fc0adbc0 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f81f9e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81fc0ad7b0 0x7f81fc161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f81f9e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81fc0ad7b0 0x7f81fc161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45804/0 (socket says 172.21.15.55:45804) 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.564+0000 7f81f9e00640 1 -- 172.21.15.55:0/1382531930 learned_addr learned my addr 172.21.15.55:0/1382531930 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81fa800640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 0x7f81fc071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81f9e00640 1 -- 172.21.15.55:0/1382531930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 msgr2=0x7f81fc071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81f9e00640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 0x7f81fc071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81f9e00640 1 -- 172.21.15.55:0/1382531930 >> v2:172.21.15.162:3300/0 conn(0x7f81fc161c70 msgr2=0x7f81fc0719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81f9e00640 1 --2- 172.21.15.55:0/1382531930 >> v2:172.21.15.162:3300/0 conn(0x7f81fc161c70 0x7f81fc0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81f9e00640 1 -- 172.21.15.55:0/1382531930 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0ce320 con 0x7f81fc0ad7b0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.565+0000 7f81fa800640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 0x7f81fc071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.566+0000 7f81f9e00640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc0ad7b0 0x7f81fc161730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009870 tx=0x7f81ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa78801690645d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.566+0000 7f81f8a00640 1 -- 172.21.15.55:0/1382531930 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec02ae90 con 0x7f81fc0ad7b0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.566+0000 7f81f8a00640 1 -- 172.21.15.55:0/1382531930 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec032440 con 0x7f81fc0ad7b0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.566+0000 7f81f8a00640 1 -- 172.21.15.55:0/1382531930 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0327d0 con 0x7f81fc0ad7b0 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 -- 172.21.15.55:0/1382531930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc0ad7b0 msgr2=0x7f81fc161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc0ad7b0 0x7f81fc161730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009870 tx=0x7f81ec02a180 comp rx=0 tx=0).stop 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 -- 172.21.15.55:0/1382531930 shutdown_connections 2024-05-10T03:30:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc0adbc0 0x7f81fc071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 --2- 172.21.15.55:0/1382531930 >> v2:172.21.15.162:3300/0 conn(0x7f81fc161c70 0x7f81fc0719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 --2- 172.21.15.55:0/1382531930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc0ad7b0 0x7f81fc161730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 -- 172.21.15.55:0/1382531930 >> 172.21.15.55:0/1382531930 conn(0x7f81fc0cbb90 msgr2=0x7f81fc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 -- 172.21.15.55:0/1382531930 shutdown_connections 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.567+0000 7f8200e00640 1 -- 172.21.15.55:0/1382531930 wait complete. 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.570+0000 7f8200e00640 1 Processor -- start 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.570+0000 7f8200e00640 1 -- start start 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81fc0adbc0 0x7f81fc1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc161c70 0x7f81fc071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc1e7400 con 0x7f81fc1600e0 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc1f3a10 con 0x7f81fc0adbc0 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.571+0000 7f8200e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc160a30 con 0x7f81fc161c70 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45810/0 (socket says 172.21.15.55:45810) 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 -- 172.21.15.55:0/216708125 learned_addr learned my addr 172.21.15.55:0/216708125 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81f9e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81fc0adbc0 0x7f81fc1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81f9400640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc161c70 0x7f81fc071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 -- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc161c70 msgr2=0x7f81fc071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc161c70 0x7f81fc071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 -- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81fc0adbc0 msgr2=0x7f81fc1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81fc0adbc0 0x7f81fc1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.572+0000 7f81fa800640 1 -- 172.21.15.55:0/216708125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0094d0 con 0x7f81fc1600e0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f81fa800640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81f400cd80 tx=0x7f81f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400f680 con 0x7f81fc1600e0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f400f800 con 0x7f81fc1600e0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400fbb0 con 0x7f81fc1600e0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc1e7610 con 0x7f81fc1600e0 2024-05-10T03:30:03.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.573+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc1e7bd0 con 0x7f81fc1600e0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.574+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc066f60 con 0x7f81fc1600e0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.575+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f81f400b040 con 0x7f81fc1600e0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.577+0000 7f81f2000640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 0x7f81dc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.577+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f81f408a190 con 0x7f81fc1600e0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.578+0000 7f81f9e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 0x7f81dc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.579+0000 7f81f9e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 0x7f81dc06d110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81ec034000 tx=0x7f81ec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:03.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.582+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f81f4055000 con 0x7f81fc1600e0 2024-05-10T03:30:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.774+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81fc067220 con 0x7f81fc1600e0 2024-05-10T03:30:03.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.774+0000 7f81f2000640 1 -- 172.21.15.55:0/216708125 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81f401d0a0 con 0x7f81fc1600e0 2024-05-10T03:30:03.779 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:03.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.777+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 msgr2=0x7f81dc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.777+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 0x7f81dc06d110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81ec034000 tx=0x7f81ec02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:03.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.777+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 msgr2=0x7f81fc1604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:03.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.777+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81f400cd80 tx=0x7f81f400a650 comp rx=0 tx=0).stop 2024-05-10T03:30:03.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 shutdown_connections 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81fc1600e0 0x7f81fc1604f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81fc161c70 0x7f81fc071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f81dc06ac00 0x7f81dc06d110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 --2- 172.21.15.55:0/216708125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81fc0adbc0 0x7f81fc1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 >> 172.21.15.55:0/216708125 conn(0x7f81fc0cbb90 msgr2=0x7f81fc06b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 shutdown_connections 2024-05-10T03:30:03.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:03.778+0000 7f8200e00640 1 -- 172.21.15.55:0/216708125 wait complete. 2024-05-10T03:30:03.809 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:04.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.085+0000 7f2bf4a00640 1 Processor -- start 2024-05-10T03:30:04.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.085+0000 7f2bf4a00640 1 -- start start 2024-05-10T03:30:04.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bf4a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2bf0144900 0x7f2bf006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bf4a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2bf00b4d90 0x7f2bf00cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bf4a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf006edd0 con 0x7f2bf01419e0 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bf4a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf006ddf0 con 0x7f2bf0144900 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bf4a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf00ce3e0 con 0x7f2bf00b4d90 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2beea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2bf01419e0 0x7f2bf00b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58226/0 (socket says 172.21.15.55:58226) 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2beea00640 1 -- 172.21.15.55:0/4164546794 learned_addr learned my addr 172.21.15.55:0/4164546794 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.087+0000 7f2bef400640 1 --2- 172.21.15.55:0/4164546794 >> v2:172.21.15.55:3300/0 conn(0x7f2bf0144900 0x7f2bf006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.088+0000 7f2befe00640 1 --2- 172.21.15.55:0/4164546794 >> v2:172.21.15.162:3300/0 conn(0x7f2bf00b4d90 0x7f2bf00cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.088+0000 7f2bef400640 1 -- 172.21.15.55:0/4164546794 >> v2:172.21.15.162:3300/0 conn(0x7f2bf00b4d90 msgr2=0x7f2bf00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:04.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.088+0000 7f2bef400640 1 --2- 172.21.15.55:0/4164546794 >> v2:172.21.15.162:3300/0 conn(0x7f2bf00b4d90 0x7f2bf00cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.088+0000 7f2bef400640 1 -- 172.21.15.55:0/4164546794 >> v1:172.21.15.55:6789/0 conn(0x7f2bf01419e0 legacy=0x7f2bf00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.088+0000 7f2bef400640 1 -- 172.21.15.55:0/4164546794 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf00ce6a0 con 0x7f2bf0144900 2024-05-10T03:30:04.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.089+0000 7f2bef400640 1 --2- 172.21.15.55:0/4164546794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf0144900 0x7f2bf006e890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00ce000 tx=0x7f2be002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3848ed7d969c9bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.089+0000 7f2bee000640 1 -- 172.21.15.55:0/4164546794 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0032440 con 0x7f2bf0144900 2024-05-10T03:30:04.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.089+0000 7f2bee000640 1 -- 172.21.15.55:0/4164546794 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00325a0 con 0x7f2bf0144900 2024-05-10T03:30:04.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.089+0000 7f2bee000640 1 -- 172.21.15.55:0/4164546794 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0032930 con 0x7f2bf0144900 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 -- 172.21.15.55:0/4164546794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf0144900 msgr2=0x7f2bf006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/4164546794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf0144900 0x7f2bf006e890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00ce000 tx=0x7f2be002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 -- 172.21.15.55:0/4164546794 shutdown_connections 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/4164546794 >> v2:172.21.15.162:3300/0 conn(0x7f2bf00b4d90 0x7f2bf00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/4164546794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf0144900 0x7f2bf006e890 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 -- 172.21.15.55:0/4164546794 >> 172.21.15.55:0/4164546794 conn(0x7f2bf00b6680 msgr2=0x7f2bf00b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 -- 172.21.15.55:0/4164546794 shutdown_connections 2024-05-10T03:30:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.090+0000 7f2bf4a00640 1 -- 172.21.15.55:0/4164546794 wait complete. 2024-05-10T03:30:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.092+0000 7f2bf4a00640 1 Processor -- start 2024-05-10T03:30:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.093+0000 7f2bf4a00640 1 -- start start 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 0x7f2bf015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 0x7f2bf00afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0143fb0 con 0x7f2bf015df70 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf013bc10 con 0x7f2bf01419e0 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2bf4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf013e970 con 0x7f2bf00b4d90 2024-05-10T03:30:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.094+0000 7f2befe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45836/0 (socket says 172.21.15.55:45836) 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 -- 172.21.15.55:0/1739132565 learned_addr learned my addr 172.21.15.55:0/1739132565 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2bef400640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 0x7f2bf015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2beea00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 0x7f2bf00afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 -- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 msgr2=0x7f2bf015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 0x7f2bf015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 -- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 msgr2=0x7f2bf00afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 0x7f2bf00afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 -- 172.21.15.55:0/1739132565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009400 con 0x7f2bf015df70 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2bef400640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 0x7f2bf015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2beea00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 0x7f2bf00afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.095+0000 7f2befe00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2be8007e60 tx=0x7f2be800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.096+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8014420 con 0x7f2bf015df70 2024-05-10T03:30:04.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.096+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80145a0 con 0x7f2bf015df70 2024-05-10T03:30:04.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.096+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8014950 con 0x7f2bf015df70 2024-05-10T03:30:04.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.096+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf0142aa0 con 0x7f2bf015df70 2024-05-10T03:30:04.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.096+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf0141e90 con 0x7f2bf015df70 2024-05-10T03:30:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.097+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf0142c30 con 0x7f2bf015df70 2024-05-10T03:30:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.098+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2be8014ab0 con 0x7f2bf015df70 2024-05-10T03:30:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.099+0000 7f2be7400640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 0x7f2bd006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.099+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2be808a0a0 con 0x7f2bf015df70 2024-05-10T03:30:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.100+0000 7f2bef400640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 0x7f2bd006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.101+0000 7f2bef400640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 0x7f2bd006d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2be0034000 tx=0x7f2be002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.104+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2be8054ef0 con 0x7f2bf015df70 2024-05-10T03:30:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.299+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2bf00b3350 con 0x7f2bf015df70 2024-05-10T03:30:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.300+0000 7f2be7400640 1 -- 172.21.15.55:0/1739132565 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2be8058510 con 0x7f2bf015df70 2024-05-10T03:30:04.305 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 msgr2=0x7f2bd006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 0x7f2bd006d110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2be0034000 tx=0x7f2be002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 msgr2=0x7f2bf015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2be8007e60 tx=0x7f2be800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 shutdown_connections 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bf015df70 0x7f2bf015e380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2bd006ac00 0x7f2bd006d110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf01419e0 0x7f2bf00afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 --2- 172.21.15.55:0/1739132565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf00b4d90 0x7f2bf015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.303+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 >> 172.21.15.55:0/1739132565 conn(0x7f2bf00b6680 msgr2=0x7f2bf0066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.304+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 shutdown_connections 2024-05-10T03:30:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.304+0000 7f2bf4a00640 1 -- 172.21.15.55:0/1739132565 wait complete. 2024-05-10T03:30:04.334 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:30:04.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.612+0000 7fd72ca00640 1 Processor -- start 2024-05-10T03:30:04.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.613+0000 7fd72ca00640 1 -- start start 2024-05-10T03:30:04.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.615+0000 7fd72ca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281435d0 0x7fd728143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.615+0000 7fd72ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72806b120 0x7fd72806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.615+0000 7fd72ca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7280cdbe0 con 0x7fd7280b2040 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.615+0000 7fd72ca00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72815e8b0 con 0x7fd7281435d0 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.615+0000 7fd72ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280ce140 con 0x7fd72806b120 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281435d0 0x7fd728143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281435d0 0x7fd728143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45838/0 (socket says 172.21.15.55:45838) 2024-05-10T03:30:04.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 -- 172.21.15.55:0/2774019237 learned_addr learned my addr 172.21.15.55:0/2774019237 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:04.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd727200640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72806b120 0x7fd72806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 -- 172.21.15.55:0/2774019237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72806b120 msgr2=0x7fd72806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:04.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72806b120 0x7fd72806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 -- 172.21.15.55:0/2774019237 >> v1:172.21.15.162:6789/0 conn(0x7fd7280b2040 legacy=0x7fd7280ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.616+0000 7fd71fe00640 1 -- 172.21.15.55:0/2774019237 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72806ba70 con 0x7fd7281435d0 2024-05-10T03:30:04.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.618+0000 7fd71fe00640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd728143020 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71000baf0 tx=0x7fd71002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab94d7b3c369f193 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.618+0000 7fd725e00640 1 -- 172.21.15.55:0/2774019237 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710004230 con 0x7fd7281435d0 2024-05-10T03:30:04.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.618+0000 7fd725e00640 1 -- 172.21.15.55:0/2774019237 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710004390 con 0x7fd7281435d0 2024-05-10T03:30:04.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.618+0000 7fd725e00640 1 -- 172.21.15.55:0/2774019237 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710003810 con 0x7fd7281435d0 2024-05-10T03:30:04.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 -- 172.21.15.55:0/2774019237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 msgr2=0x7fd728143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd728143020 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd71000baf0 tx=0x7fd71002c870 comp rx=0 tx=0).stop 2024-05-10T03:30:04.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 -- 172.21.15.55:0/2774019237 shutdown_connections 2024-05-10T03:30:04.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd72806b120 0x7fd72806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 --2- 172.21.15.55:0/2774019237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd728143020 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 -- 172.21.15.55:0/2774019237 >> 172.21.15.55:0/2774019237 conn(0x7fd72806c730 msgr2=0x7fd7280b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:04.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 -- 172.21.15.55:0/2774019237 shutdown_connections 2024-05-10T03:30:04.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.619+0000 7fd72ca00640 1 -- 172.21.15.55:0/2774019237 wait complete. 2024-05-10T03:30:04.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.621+0000 7fd72ca00640 1 Processor -- start 2024-05-10T03:30:04.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.621+0000 7fd72ca00640 1 -- start start 2024-05-10T03:30:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd72806b120 0x7fd72813e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7280b2040 0x7fd7280adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728157470 con 0x7fd7281435d0 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72815a690 con 0x7fd72806b120 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd72ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd728156c60 con 0x7fd7280b2040 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd727200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd727200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45846/0 (socket says 172.21.15.55:45846) 2024-05-10T03:30:04.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.623+0000 7fd727200640 1 -- 172.21.15.55:0/31193075 learned_addr learned my addr 172.21.15.55:0/31193075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:04.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd71fe00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7280b2040 0x7fd7280adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd726800640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd72806b120 0x7fd72813e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 -- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7280b2040 msgr2=0x7fd7280adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7280b2040 0x7fd7280adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 -- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd72806b120 msgr2=0x7fd72813e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd72806b120 0x7fd72813e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 -- 172.21.15.55:0/31193075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71000b700 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd727200640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd720007d70 tx=0x7fd72000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720014420 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200145a0 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720014950 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.624+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd728158ec0 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.625+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7281ef830 con 0x7fd7281435d0 2024-05-10T03:30:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.626+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280b1680 con 0x7fd7281435d0 2024-05-10T03:30:04.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.627+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd720014ab0 con 0x7fd7281435d0 2024-05-10T03:30:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.628+0000 7fd71ea00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 0x7fd70406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.628+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd720089e40 con 0x7fd7281435d0 2024-05-10T03:30:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.628+0000 7fd726800640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 0x7fd70406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:04.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.630+0000 7fd726800640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 0x7fd70406d140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd718000f60 tx=0x7fd718009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:04.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.632+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd720054dc0 con 0x7fd7281435d0 2024-05-10T03:30:04.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.857+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd728159050 con 0x7fd7281435d0 2024-05-10T03:30:04.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.857+0000 7fd71ea00640 1 -- 172.21.15.55:0/31193075 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd7200583e0 con 0x7fd7281435d0 2024-05-10T03:30:04.862 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:30:04.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 msgr2=0x7fd70406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 0x7fd70406d140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd718000f60 tx=0x7fd718009040 comp rx=0 tx=0).stop 2024-05-10T03:30:04.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 msgr2=0x7fd72815bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:04.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd720007d70 tx=0x7fd72000e4b0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 shutdown_connections 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281435d0 0x7fd72815bbf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7280b2040 0x7fd7280adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd70406ac30 0x7fd70406d140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 --2- 172.21.15.55:0/31193075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd72806b120 0x7fd72813e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 >> 172.21.15.55:0/31193075 conn(0x7fd72806c730 msgr2=0x7fd7280b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.860+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 shutdown_connections 2024-05-10T03:30:04.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:04.861+0000 7fd72ca00640 1 -- 172.21.15.55:0/31193075 wait complete. 2024-05-10T03:30:04.890 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi055.front.sepia.ceph.com... 2024-05-10T03:30:04.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:04.890 DEBUG:teuthology.orchestra.run.smithi055:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T03:30:04.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi055.stderr:ceph-fuse[41476]: fuse finished with error 0 and tester_r 0 2024-05-10T03:30:04.981 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:30:05.049 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:30:05.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:05.050 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T03:30:05.124 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.409+0000 7f813fe00640 1 Processor -- start 2024-05-10T03:30:05.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.410+0000 7f813fe00640 1 -- start start 2024-05-10T03:30:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f8140144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f814006d7e0 0x7f81400b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81400b2fc0 0x7f814015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814006d350 con 0x7f81400b2fc0 2024-05-10T03:30:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814006c740 con 0x7f814006d7e0 2024-05-10T03:30:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.412+0000 7f813fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814006ca20 con 0x7f8140144710 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81400b2fc0 0x7f814015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81400b2fc0 0x7f814015eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45860/0 (socket says 172.21.15.55:45860) 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 -- 172.21.15.55:0/3764509644 learned_addr learned my addr 172.21.15.55:0/3764509644 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813ea00640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f8140144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f8137e00640 1 --2- 172.21.15.55:0/3764509644 >> v2:172.21.15.162:3300/0 conn(0x7f814006d7e0 0x7f81400b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 -- 172.21.15.55:0/3764509644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 msgr2=0x7f8140144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f8140144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 -- 172.21.15.55:0/3764509644 >> v2:172.21.15.162:3300/0 conn(0x7f814006d7e0 msgr2=0x7f81400b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:05.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 --2- 172.21.15.55:0/3764509644 >> v2:172.21.15.162:3300/0 conn(0x7f814006d7e0 0x7f81400b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.413+0000 7f813f400640 1 -- 172.21.15.55:0/3764509644 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814006cd00 con 0x7f81400b2fc0 2024-05-10T03:30:05.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.415+0000 7f813f400640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81400b2fc0 0x7f814015eff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8140072270 tx=0x7f813802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfa4b1714c72b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.415+0000 7f813e000640 1 -- 172.21.15.55:0/3764509644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138031600 con 0x7f81400b2fc0 2024-05-10T03:30:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.415+0000 7f813e000640 1 -- 172.21.15.55:0/3764509644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813802d050 con 0x7f81400b2fc0 2024-05-10T03:30:05.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.415+0000 7f813e000640 1 -- 172.21.15.55:0/3764509644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138031980 con 0x7f81400b2fc0 2024-05-10T03:30:05.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 -- 172.21.15.55:0/3764509644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81400b2fc0 msgr2=0x7f814015eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81400b2fc0 0x7f814015eff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8140072270 tx=0x7f813802a610 comp rx=0 tx=0).stop 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 -- 172.21.15.55:0/3764509644 shutdown_connections 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81400b2fc0 0x7f814015eff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 --2- 172.21.15.55:0/3764509644 >> v2:172.21.15.162:3300/0 conn(0x7f814006d7e0 0x7f81400b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 --2- 172.21.15.55:0/3764509644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f8140144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 -- 172.21.15.55:0/3764509644 >> 172.21.15.55:0/3764509644 conn(0x7f814006a110 msgr2=0x7f814006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 -- 172.21.15.55:0/3764509644 shutdown_connections 2024-05-10T03:30:05.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.416+0000 7f813fe00640 1 -- 172.21.15.55:0/3764509644 wait complete. 2024-05-10T03:30:05.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.419+0000 7f813fe00640 1 Processor -- start 2024-05-10T03:30:05.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.419+0000 7f813fe00640 1 -- start start 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.420+0000 7f813fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 0x7f814013cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f814013d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814015e940 con 0x7f814006d7e0 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140071e00 con 0x7f81400b2fc0 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814015e6f0 con 0x7f8140144710 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45866/0 (socket says 172.21.15.55:45866) 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 -- 172.21.15.55:0/2468041301 learned_addr learned my addr 172.21.15.55:0/2468041301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:05.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813f400640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f814013d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f8137e00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 0x7f814013cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 -- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 msgr2=0x7f814013d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f814013d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 -- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 msgr2=0x7f814013cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 0x7f814013cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f813ea00640 1 -- 172.21.15.55:0/2468041301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138009b30 con 0x7f814006d7e0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.421+0000 7f8137e00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 0x7f814013cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f813ea00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f813000e870 tx=0x7f813000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81300094f0 con 0x7f814006d7e0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000c940 con 0x7f814006d7e0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81400705c0 con 0x7f814006d7e0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000ccd0 con 0x7f814006d7e0 2024-05-10T03:30:05.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.422+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814015cf50 con 0x7f814006d7e0 2024-05-10T03:30:05.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.424+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8140066ce0 con 0x7f814006d7e0 2024-05-10T03:30:05.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.425+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8130013470 con 0x7f814006d7e0 2024-05-10T03:30:05.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.426+0000 7f8137400640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 0x7f811c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.426+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8130089670 con 0x7f814006d7e0 2024-05-10T03:30:05.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.426+0000 7f8137e00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 0x7f811c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.428+0000 7f8137e00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 0x7f811c06d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f814013ab70 tx=0x7f8128008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.430+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8130054550 con 0x7f814006d7e0 2024-05-10T03:30:05.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.632+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f814015d140 con 0x7f814006d7e0 2024-05-10T03:30:05.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.636+0000 7f8137400640 1 -- 172.21.15.55:0/2468041301 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f8130057b70 con 0x7f814006d7e0 2024-05-10T03:30:05.641 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:05.641 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 msgr2=0x7f811c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 0x7f811c06d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f814013ab70 tx=0x7f8128008040 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 msgr2=0x7f8140152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f813000e870 tx=0x7f813000ed30 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 shutdown_connections 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8140144710 0x7f814013d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f811c06ac00 0x7f811c06d110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81400b2fc0 0x7f814013cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 --2- 172.21.15.55:0/2468041301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f814006d7e0 0x7f8140152770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 >> 172.21.15.55:0/2468041301 conn(0x7f814006a110 msgr2=0x7f81400ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:05.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 shutdown_connections 2024-05-10T03:30:05.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.639+0000 7f813fe00640 1 -- 172.21.15.55:0/2468041301 wait complete. 2024-05-10T03:30:05.670 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:05.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.950+0000 7fb038a00640 1 Processor -- start 2024-05-10T03:30:05.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.950+0000 7fb038a00640 1 -- start start 2024-05-10T03:30:05.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.952+0000 7fb038a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb034141b00 0x7fb0340b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.952+0000 7fb038a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03415f1f0 0x7fb03415f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.952+0000 7fb038a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0340b4b90 con 0x7fb03406d430 2024-05-10T03:30:05.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.952+0000 7fb038a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340d0e80 con 0x7fb034141b00 2024-05-10T03:30:05.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.952+0000 7fb038a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03406d840 con 0x7fb03415f1f0 2024-05-10T03:30:05.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.953+0000 7fb031e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb034141b00 0x7fb0340b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.953+0000 7fb032800640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb03406d430 0x7fb0340b5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:47450/0 (socket says 172.21.15.55:47450) 2024-05-10T03:30:05.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.953+0000 7fb032800640 1 -- 172.21.15.55:0/2454730194 learned_addr learned my addr 172.21.15.55:0/2454730194 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:05.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.953+0000 7fb033200640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03415f1f0 0x7fb03415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.954+0000 7fb031e00640 1 -- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03415f1f0 msgr2=0x7fb03415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:05.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.954+0000 7fb031e00640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03415f1f0 0x7fb03415f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.954+0000 7fb031e00640 1 -- 172.21.15.55:0/2454730194 >> v1:172.21.15.162:6789/0 conn(0x7fb03406d430 legacy=0x7fb0340b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.954+0000 7fb031e00640 1 -- 172.21.15.55:0/2454730194 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034141f10 con 0x7fb034141b00 2024-05-10T03:30:05.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.956+0000 7fb031e00640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb034141b00 0x7fb0340b5730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb024009d90 tx=0x7fb02402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82764b3b3eb3416b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.956+0000 7fb031400640 1 -- 172.21.15.55:0/2454730194 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024007740 con 0x7fb034141b00 2024-05-10T03:30:05.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.957+0000 7fb031400640 1 -- 172.21.15.55:0/2454730194 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02402d050 con 0x7fb034141b00 2024-05-10T03:30:05.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.957+0000 7fb031400640 1 -- 172.21.15.55:0/2454730194 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024007ac0 con 0x7fb034141b00 2024-05-10T03:30:05.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.957+0000 7fb038a00640 1 -- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb034141b00 msgr2=0x7fb0340b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.957+0000 7fb038a00640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb034141b00 0x7fb0340b5730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb024009d90 tx=0x7fb02402a660 comp rx=0 tx=0).stop 2024-05-10T03:30:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 -- 172.21.15.55:0/2454730194 shutdown_connections 2024-05-10T03:30:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03415f1f0 0x7fb03415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 --2- 172.21.15.55:0/2454730194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb034141b00 0x7fb0340b5730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 -- 172.21.15.55:0/2454730194 >> 172.21.15.55:0/2454730194 conn(0x7fb0340b6680 msgr2=0x7fb0340b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:05.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 -- 172.21.15.55:0/2454730194 shutdown_connections 2024-05-10T03:30:05.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.958+0000 7fb038a00640 1 -- 172.21.15.55:0/2454730194 wait complete. 2024-05-10T03:30:05.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.961+0000 7fb038a00640 1 Processor -- start 2024-05-10T03:30:05.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.961+0000 7fb038a00640 1 -- start start 2024-05-10T03:30:05.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb034141b00 0x7fb0341397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03415f1f0 0x7fb03406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340ae650 con 0x7fb03406d430 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034152270 con 0x7fb03415f1f0 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb038a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb034154580 con 0x7fb034141b00 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb032800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb032800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45872/0 (socket says 172.21.15.55:45872) 2024-05-10T03:30:05.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb032800640 1 -- 172.21.15.55:0/1831835305 learned_addr learned my addr 172.21.15.55:0/1831835305 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.963+0000 7fb033200640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03415f1f0 0x7fb03406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb031e00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb034141b00 0x7fb0341397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 -- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb034141b00 msgr2=0x7fb0341397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb034141b00 0x7fb0341397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 -- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03415f1f0 msgr2=0x7fb03406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03415f1f0 0x7fb03406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 -- 172.21.15.55:0/1831835305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009a70 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb032800640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c3c0 tx=0x7fb01c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c005ce0 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c005e40 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0072d0 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.964+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034152000 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.965+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0341524c0 con 0x7fb03406d430 2024-05-10T03:30:05.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.966+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0341f7e00 con 0x7fb03406d430 2024-05-10T03:30:05.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.967+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb01c007430 con 0x7fb03406d430 2024-05-10T03:30:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.968+0000 7fb02aa00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 0x7fb01006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.969+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb01c088bd0 con 0x7fb03406d430 2024-05-10T03:30:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.969+0000 7fb031e00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 0x7fb01006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:05.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.970+0000 7fb031e00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 0x7fb01006d110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb024034000 tx=0x7fb024033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:05.973+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb01c053a40 con 0x7fb03406d430 2024-05-10T03:30:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.175+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb03404e290 con 0x7fb03406d430 2024-05-10T03:30:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.179+0000 7fb02aa00640 1 -- 172.21.15.55:0/1831835305 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fb01c017070 con 0x7fb03406d430 2024-05-10T03:30:06.184 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:06.184 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:06.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 msgr2=0x7fb01006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 0x7fb01006d110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb024034000 tx=0x7fb024033040 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 msgr2=0x7fb0341f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c3c0 tx=0x7fb01c00c880 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 shutdown_connections 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb01006ac00 0x7fb01006d110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03415f1f0 0x7fb03406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb034141b00 0x7fb0341397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 --2- 172.21.15.55:0/1831835305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03406d430 0x7fb0341f78c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 >> 172.21.15.55:0/1831835305 conn(0x7fb0340b6680 msgr2=0x7fb0340b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 shutdown_connections 2024-05-10T03:30:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.182+0000 7fb038a00640 1 -- 172.21.15.55:0/1831835305 wait complete. 2024-05-10T03:30:06.215 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:30:06.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.500+0000 7feefa200640 1 Processor -- start 2024-05-10T03:30:06.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.501+0000 7feefa200640 1 -- start start 2024-05-10T03:30:06.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.503+0000 7feefa200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feef40719f0 0x7feef406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.503+0000 7feefa200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feef406b840 0x7feef40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.503+0000 7feefa200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef415fdf0 con 0x7feef4071e00 2024-05-10T03:30:06.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.503+0000 7feefa200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef415f1e0 con 0x7feef406b840 2024-05-10T03:30:06.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.503+0000 7feefa200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef415f4c0 con 0x7feef40719f0 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.504+0000 7feef3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feef406b840 0x7feef40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.504+0000 7feef3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feef406b840 0x7feef40700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45878/0 (socket says 172.21.15.55:45878) 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.504+0000 7feef3e00640 1 -- 172.21.15.55:0/3834099055 learned_addr learned my addr 172.21.15.55:0/3834099055 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.504+0000 7feef8e00640 1 --2- 172.21.15.55:0/3834099055 >> v2:172.21.15.162:3300/0 conn(0x7feef40719f0 0x7feef406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.505+0000 7feef3e00640 1 -- 172.21.15.55:0/3834099055 >> v2:172.21.15.162:3300/0 conn(0x7feef40719f0 msgr2=0x7feef406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.505+0000 7feef3e00640 1 --2- 172.21.15.55:0/3834099055 >> v2:172.21.15.162:3300/0 conn(0x7feef40719f0 0x7feef406b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.505+0000 7feef3e00640 1 -- 172.21.15.55:0/3834099055 >> v1:172.21.15.162:6789/0 conn(0x7feef4071e00 legacy=0x7feef4070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.505+0000 7feef3e00640 1 -- 172.21.15.55:0/3834099055 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef415f7a0 con 0x7feef406b840 2024-05-10T03:30:06.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.506+0000 7feef3e00640 1 --2- 172.21.15.55:0/3834099055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef406b840 0x7feef40700e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feef40b41f0 tx=0x7feee002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38a787521ae6c5b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:06.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.506+0000 7feef3400640 1 -- 172.21.15.55:0/3834099055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0032440 con 0x7feef406b840 2024-05-10T03:30:06.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.506+0000 7feef3400640 1 -- 172.21.15.55:0/3834099055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee00325a0 con 0x7feef406b840 2024-05-10T03:30:06.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.506+0000 7feef3400640 1 -- 172.21.15.55:0/3834099055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0032930 con 0x7feef406b840 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 -- 172.21.15.55:0/3834099055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef406b840 msgr2=0x7feef40700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 --2- 172.21.15.55:0/3834099055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef406b840 0x7feef40700e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feef40b41f0 tx=0x7feee002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 -- 172.21.15.55:0/3834099055 shutdown_connections 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 --2- 172.21.15.55:0/3834099055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef406b840 0x7feef40700e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 --2- 172.21.15.55:0/3834099055 >> v2:172.21.15.162:3300/0 conn(0x7feef40719f0 0x7feef406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 -- 172.21.15.55:0/3834099055 >> 172.21.15.55:0/3834099055 conn(0x7feef406c750 msgr2=0x7feef40b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 -- 172.21.15.55:0/3834099055 shutdown_connections 2024-05-10T03:30:06.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.507+0000 7feefa200640 1 -- 172.21.15.55:0/3834099055 wait complete. 2024-05-10T03:30:06.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.510+0000 7feefa200640 1 Processor -- start 2024-05-10T03:30:06.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.510+0000 7feefa200640 1 -- start start 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 0x7feef415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feef4071e00 0x7feef40b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef41581a0 con 0x7feef40719f0 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40ade40 con 0x7feef4071e00 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.511+0000 7feefa200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef41583b0 con 0x7feef406b840 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45880/0 (socket says 172.21.15.55:45880) 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 -- 172.21.15.55:0/3631958076 learned_addr learned my addr 172.21.15.55:0/3631958076 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef8e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 0x7feef415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef9800640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feef4071e00 0x7feef40b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 -- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 msgr2=0x7feef415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 0x7feef415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 -- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feef4071e00 msgr2=0x7feef40b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:06.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feef4071e00 0x7feef40b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:06.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 -- 172.21.15.55:0/3631958076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00094d0 con 0x7feef40719f0 2024-05-10T03:30:06.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef8e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 0x7feef415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:06.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.512+0000 7feef3e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feee002a220 tx=0x7feee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.513+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0032440 con 0x7feef40719f0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.513+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0030ba0 con 0x7feef40719f0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.513+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee003d440 con 0x7feef40719f0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.513+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4158690 con 0x7feef40719f0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.513+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef4158c50 con 0x7feef40719f0 2024-05-10T03:30:06.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.514+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef404e290 con 0x7feef40719f0 2024-05-10T03:30:06.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.515+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feee0039040 con 0x7feef40719f0 2024-05-10T03:30:06.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.516+0000 7feef0c00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 0x7feed406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.516+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feee00aa2e0 con 0x7feef40719f0 2024-05-10T03:30:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.517+0000 7feef8e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 0x7feed406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:06.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.518+0000 7feef8e00640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 0x7feed406d140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feee4006fd0 tx=0x7feee4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:06.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.521+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feee00750a0 con 0x7feef40719f0 2024-05-10T03:30:06.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:06.762+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feef40b3f70 con 0x7feef40719f0 2024-05-10T03:30:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.092+0000 7feef0c00640 1 -- 172.21.15.55:0/3631958076 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7feee00410b0 con 0x7feef40719f0 2024-05-10T03:30:07.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 msgr2=0x7feed406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 0x7feed406d140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feee4006fd0 tx=0x7feee4008040 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 msgr2=0x7feef4137420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feee002a220 tx=0x7feee00046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 shutdown_connections 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feed406ac30 0x7feed406d140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feef4071e00 0x7feef40b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef40719f0 0x7feef4137420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 --2- 172.21.15.55:0/3631958076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feef406b840 0x7feef415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 >> 172.21.15.55:0/3631958076 conn(0x7feef406c750 msgr2=0x7feef4142320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 shutdown_connections 2024-05-10T03:30:07.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.096+0000 7feefa200640 1 -- 172.21.15.55:0/3631958076 wait complete. 2024-05-10T03:30:07.132 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-10T03:30:07.133 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:30:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.407+0000 7fce3e600640 1 Processor -- start 2024-05-10T03:30:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.407+0000 7fce3e600640 1 -- start start 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.409+0000 7fce3e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce38141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.409+0000 7fce3e600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce380cd750 0x7fce3815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.409+0000 7fce3e600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce38071ce0 con 0x7fce3815fed0 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce3e600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce38070f40 con 0x7fce380cd750 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce3e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3815f280 con 0x7fce38070050 2024-05-10T03:30:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce37e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fce3815fed0 0x7fce3815bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58242/0 (socket says 172.21.15.55:58242) 2024-05-10T03:30:07.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce37e00640 1 -- 172.21.15.55:0/3656290343 learned_addr learned my addr 172.21.15.55:0/3656290343 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce37400640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce38141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce36a00640 1 --2- 172.21.15.55:0/3656290343 >> v2:172.21.15.162:3300/0 conn(0x7fce380cd750 0x7fce3815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.410+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151525533 0 0) 0x7fce30001490 con 0x7fce3815fed0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c003520 con 0x7fce3815fed0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129495800 0 0) 0x7fce1c003520 con 0x7fce3815fed0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 msgr2=0x7fce38141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce38141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 >> v2:172.21.15.162:3300/0 conn(0x7fce380cd750 msgr2=0x7fce3815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 --2- 172.21.15.55:0/3656290343 >> v2:172.21.15.162:3300/0 conn(0x7fce380cd750 0x7fce3815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38070460 con 0x7fce3815fed0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fce30001b80 con 0x7fce3815fed0 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 >> v1:172.21.15.55:6789/0 conn(0x7fce3815fed0 legacy=0x7fce3815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce1c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c003ef0 0x7fce1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce1c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce38070f40 con 0x7fce1c004d00 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c00b980 con 0x7fce1c003ef0 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.411+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c00bb10 con 0x7fce38070050 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce1c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce36a00640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c003ef0 0x7fce1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37400640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce1c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 -- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 msgr2=0x7fce1c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce38070050 0x7fce1c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 -- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c003ef0 msgr2=0x7fce1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1c003ef0 0x7fce1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.412+0000 7fce37e00640 1 -- 172.21.15.55:0/3656290343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c003520 con 0x7fce1c004d00 2024-05-10T03:30:07.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.413+0000 7fce37e00640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce1c005110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce30002230 tx=0x7fce1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40af97150d12b405 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.413+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3002b9f0 con 0x7fce1c004d00 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.413+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3002bb70 con 0x7fce1c004d00 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.413+0000 7fce36000640 1 -- 172.21.15.55:0/3656290343 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30034470 con 0x7fce1c004d00 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 -- 172.21.15.55:0/3656290343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 msgr2=0x7fce1c005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 --2- 172.21.15.55:0/3656290343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce1c005110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce30002230 tx=0x7fce1c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce37e00640 1 -- 172.21.15.55:0/3656290343 reap_dead start 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 -- 172.21.15.55:0/3656290343 shutdown_connections 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 -- 172.21.15.55:0/3656290343 >> 172.21.15.55:0/3656290343 conn(0x7fce3806c120 msgr2=0x7fce3806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 -- 172.21.15.55:0/3656290343 shutdown_connections 2024-05-10T03:30:07.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.414+0000 7fce3e600640 1 -- 172.21.15.55:0/3656290343 wait complete. 2024-05-10T03:30:07.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.417+0000 7fce3e600640 1 Processor -- start 2024-05-10T03:30:07.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.417+0000 7fce3e600640 1 -- start start 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.418+0000 7fce3e600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce3e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce381f46d0 0x7fce3814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce3e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce3815d420 0x7fce3815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce3e600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce380abca0 con 0x7fce1c004d00 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce3e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce381f33e0 con 0x7fce3815d420 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce3e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce381f4240 con 0x7fce381f46d0 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45890/0 (socket says 172.21.15.55:45890) 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 -- 172.21.15.55:0/4057091090 learned_addr learned my addr 172.21.15.55:0/4057091090 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce36a00640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce381f46d0 0x7fce3814b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37e00640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce3815d420 0x7fce3815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 -- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce381f46d0 msgr2=0x7fce3814b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce381f46d0 0x7fce3814b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 -- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce3815d420 msgr2=0x7fce3815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce3815d420 0x7fce3815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.419+0000 7fce37400640 1 -- 172.21.15.55:0/4057091090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3815dd70 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce37400640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce28011600 tx=0x7fce28011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800f8c0 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2800fa20 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800fdd0 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381f3630 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.420+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce381f3bf0 con 0x7fce1c004d00 2024-05-10T03:30:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.422+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38066f60 con 0x7fce1c004d00 2024-05-10T03:30:07.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.423+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fce28015460 con 0x7fce1c004d00 2024-05-10T03:30:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.424+0000 7fce2f400640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 0x7fce1806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.424+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce28089fc0 con 0x7fce1c004d00 2024-05-10T03:30:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.424+0000 7fce36a00640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 0x7fce1806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:07.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.426+0000 7fce36a00640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 0x7fce1806d110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce24009b00 tx=0x7fce24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:07.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.428+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fce28054d80 con 0x7fce1c004d00 2024-05-10T03:30:07.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:07.665+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fce38067220 con 0x7fce1c004d00 2024-05-10T03:30:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.115+0000 7fce2f400640 1 -- 172.21.15.55:0/4057091090 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce28022020 con 0x7fce1c004d00 2024-05-10T03:30:08.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.119+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 msgr2=0x7fce1806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.119+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 0x7fce1806d110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce24009b00 tx=0x7fce24008040 comp rx=0 tx=0).stop 2024-05-10T03:30:08.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.119+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 msgr2=0x7fce380a9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.119+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fce28011600 tx=0x7fce28011ac0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 shutdown_connections 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1806ac00 0x7fce1806d110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce3815d420 0x7fce3815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce381f46d0 0x7fce3814b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 --2- 172.21.15.55:0/4057091090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce1c004d00 0x7fce380a9190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 >> 172.21.15.55:0/4057091090 conn(0x7fce3806c120 msgr2=0x7fce380af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 shutdown_connections 2024-05-10T03:30:08.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.120+0000 7fce3e600640 1 -- 172.21.15.55:0/4057091090 wait complete. 2024-05-10T03:30:08.158 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:30:08.158 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:30:08.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.447+0000 7f5cc2a00640 1 Processor -- start 2024-05-10T03:30:08.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.447+0000 7f5cc2a00640 1 -- start start 2024-05-10T03:30:08.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cc2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:08.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cc2a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc0708b0 con 0x7f5cbc071db0 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cc2a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc15b940 con 0x7f5cbc0b3860 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cc2a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc0b2490 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cbbe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5cbc071db0 0x7f5cbc0ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58252/0 (socket says 172.21.15.55:58252) 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cbbe00640 1 -- 172.21.15.55:0/2847967315 learned_addr learned my addr 172.21.15.55:0/2847967315 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.450+0000 7f5cb3400640 1 --2- 172.21.15.55:0/2847967315 >> v2:172.21.15.55:3300/0 conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:08.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.451+0000 7f5cbb400640 1 -- 172.21.15.55:0/2847967315 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277834814 0 0) 0x7f5cbc0708b0 con 0x7f5cbc071db0 2024-05-10T03:30:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.451+0000 7f5cbb400640 1 -- 172.21.15.55:0/2847967315 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca0005730 con 0x7f5cbc071db0 2024-05-10T03:30:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.451+0000 7f5cb3400640 1 -- 172.21.15.55:0/2847967315 >> v1:172.21.15.162:6789/0 conn(0x7f5cbc0b3860 legacy=0x7f5cbc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.451+0000 7f5cb3400640 1 -- 172.21.15.55:0/2847967315 >> v1:172.21.15.55:6789/0 conn(0x7f5cbc071db0 legacy=0x7f5cbc0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.451+0000 7f5cb3400640 1 -- 172.21.15.55:0/2847967315 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc070260 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.452+0000 7f5cb3400640 1 --2- 172.21.15.55:0/2847967315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40097a0 tx=0x7f5ca402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2fa52c074eb510 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:08.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.452+0000 7f5cbb400640 1 -- 172.21.15.55:0/2847967315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca402ae60 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.452+0000 7f5cbb400640 1 -- 172.21.15.55:0/2847967315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4032440 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.452+0000 7f5cbb400640 1 -- 172.21.15.55:0/2847967315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40327d0 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 -- 172.21.15.55:0/2847967315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc06a4b0 msgr2=0x7f5cbc158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/2847967315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40097a0 tx=0x7f5ca402a110 comp rx=0 tx=0).stop 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 -- 172.21.15.55:0/2847967315 shutdown_connections 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/2847967315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 -- 172.21.15.55:0/2847967315 >> 172.21.15.55:0/2847967315 conn(0x7f5cbc0cbb90 msgr2=0x7f5cbc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 -- 172.21.15.55:0/2847967315 shutdown_connections 2024-05-10T03:30:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.453+0000 7f5cc2a00640 1 -- 172.21.15.55:0/2847967315 wait complete. 2024-05-10T03:30:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.456+0000 7f5cc2a00640 1 Processor -- start 2024-05-10T03:30:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.456+0000 7f5cc2a00640 1 -- start start 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.457+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.457+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc071db0 0x7f5cbc156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc2a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc1f3c70 con 0x7f5cbc0b3860 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc2a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc158950 con 0x7f5cbc071db0 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc2a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc158690 con 0x7f5cbc06a4b0 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45910/0 (socket says 172.21.15.55:45910) 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 -- 172.21.15.55:0/777950501 learned_addr learned my addr 172.21.15.55:0/777950501 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cb3400640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc071db0 0x7f5cbc156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cbbe00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 -- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 msgr2=0x7f5cbc158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 -- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc071db0 msgr2=0x7f5cbc156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc071db0 0x7f5cbc156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.458+0000 7f5cc0a00640 1 -- 172.21.15.55:0/777950501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4009400 con 0x7f5cbc0b3860 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cbbe00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cc0a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c390 tx=0x7f5cb400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40052d0 con 0x7f5cbc0b3860 2024-05-10T03:30:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb4005430 con 0x7f5cbc0b3860 2024-05-10T03:30:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4003810 con 0x7f5cbc0b3860 2024-05-10T03:30:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc158ba0 con 0x7f5cbc0b3860 2024-05-10T03:30:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.459+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc159160 con 0x7f5cbc0b3860 2024-05-10T03:30:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.460+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc066f60 con 0x7f5cbc0b3860 2024-05-10T03:30:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.462+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5cb4003970 con 0x7f5cbc0b3860 2024-05-10T03:30:08.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.463+0000 7f5cb8c00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 0x7f5c9806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:08.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.463+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5cb4088c40 con 0x7f5cbc0b3860 2024-05-10T03:30:08.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.464+0000 7f5cbbe00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 0x7f5c9806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.465+0000 7f5cbbe00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 0x7f5c9806d140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001560 tx=0x7f5cac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:08.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.468+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5cb4053a00 con 0x7f5cbc0b3860 2024-05-10T03:30:08.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.707+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cbc067220 con 0x7f5cbc0b3860 2024-05-10T03:30:08.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.708+0000 7f5cb8c00640 1 -- 172.21.15.55:0/777950501 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f5cb4017020 con 0x7f5cbc0b3860 2024-05-10T03:30:08.713 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:08.713 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:30:08.713 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:30:08.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.710+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 msgr2=0x7f5c9806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 0x7f5c9806d140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001560 tx=0x7f5cac009040 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 msgr2=0x7f5cbc0cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c390 tx=0x7f5cb400c850 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 shutdown_connections 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc0b3860 0x7f5cbc0cd420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c9806ac30 0x7f5c9806d140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc071db0 0x7f5cbc156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 --2- 172.21.15.55:0/777950501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc06a4b0 0x7f5cbc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 >> 172.21.15.55:0/777950501 conn(0x7f5cbc0cbb90 msgr2=0x7f5cbc15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 shutdown_connections 2024-05-10T03:30:08.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:08.711+0000 7f5cc2a00640 1 -- 172.21.15.55:0/777950501 wait complete. 2024-05-10T03:30:08.744 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.015+0000 7fb644200640 1 Processor -- start 2024-05-10T03:30:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.015+0000 7fb644200640 1 -- start start 2024-05-10T03:30:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb63c06b6e0 0x7fb63c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb63c1659c0 0x7fb63c165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c162380 0x7fb63c162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0711c0 con 0x7fb63c1659c0 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c06baf0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb644200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0715e0 con 0x7fb63c162380 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb640e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb63c1659c0 0x7fb63c165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb640e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb63c1659c0 0x7fb63c165dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45912/0 (socket says 172.21.15.55:45912) 2024-05-10T03:30:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.018+0000 7fb640e00640 1 -- 172.21.15.55:0/872792691 learned_addr learned my addr 172.21.15.55:0/872792691 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb642200640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c162380 0x7fb63c162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb641800640 1 --2- 172.21.15.55:0/872792691 >> v2:172.21.15.162:3300/0 conn(0x7fb63c06b6e0 0x7fb63c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb640e00640 1 -- 172.21.15.55:0/872792691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c162380 msgr2=0x7fb63c162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb640e00640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c162380 0x7fb63c162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb640e00640 1 -- 172.21.15.55:0/872792691 >> v2:172.21.15.162:3300/0 conn(0x7fb63c06b6e0 msgr2=0x7fb63c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb640e00640 1 --2- 172.21.15.55:0/872792691 >> v2:172.21.15.162:3300/0 conn(0x7fb63c06b6e0 0x7fb63c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.019+0000 7fb640e00640 1 -- 172.21.15.55:0/872792691 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c166310 con 0x7fb63c1659c0 2024-05-10T03:30:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.020+0000 7fb640e00640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1659c0 0x7fb63c165dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb63c06bcb0 tx=0x7fb62802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377084bf9a751a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.020+0000 7fb63be00640 1 -- 172.21.15.55:0/872792691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62802ae90 con 0x7fb63c1659c0 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.020+0000 7fb63be00640 1 -- 172.21.15.55:0/872792691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628032440 con 0x7fb63c1659c0 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.020+0000 7fb63be00640 1 -- 172.21.15.55:0/872792691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6280327d0 con 0x7fb63c1659c0 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 -- 172.21.15.55:0/872792691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1659c0 msgr2=0x7fb63c165dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1659c0 0x7fb63c165dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb63c06bcb0 tx=0x7fb62802a140 comp rx=0 tx=0).stop 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 -- 172.21.15.55:0/872792691 shutdown_connections 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c162380 0x7fb63c162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 --2- 172.21.15.55:0/872792691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c1659c0 0x7fb63c165dd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 --2- 172.21.15.55:0/872792691 >> v2:172.21.15.162:3300/0 conn(0x7fb63c06b6e0 0x7fb63c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 -- 172.21.15.55:0/872792691 >> 172.21.15.55:0/872792691 conn(0x7fb63c0cbb90 msgr2=0x7fb63c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 -- 172.21.15.55:0/872792691 shutdown_connections 2024-05-10T03:30:09.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.021+0000 7fb644200640 1 -- 172.21.15.55:0/872792691 wait complete. 2024-05-10T03:30:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.024+0000 7fb644200640 1 Processor -- start 2024-05-10T03:30:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.024+0000 7fb644200640 1 -- start start 2024-05-10T03:30:09.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb63c162380 0x7fb63c1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c1659c0 0x7fb63c1f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c149940 con 0x7fb63c06b6e0 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c1fa070 con 0x7fb63c162380 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb644200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c148720 con 0x7fb63c1659c0 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb641800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb641800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45920/0 (socket says 172.21.15.55:45920) 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.026+0000 7fb641800640 1 -- 172.21.15.55:0/4206691004 learned_addr learned my addr 172.21.15.55:0/4206691004 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 -- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c1659c0 msgr2=0x7fb63c1f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb640e00640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb63c162380 0x7fb63c1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb642200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c1659c0 0x7fb63c1f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c1659c0 0x7fb63c1f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 -- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb63c162380 msgr2=0x7fb63c1f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb63c162380 0x7fb63c1f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 -- 172.21.15.55:0/4206691004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280094d0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.027+0000 7fb641800640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb63000d5f0 tx=0x7fb63000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.028+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630018070 con 0x7fb63c06b6e0 2024-05-10T03:30:09.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.028+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630004240 con 0x7fb63c06b6e0 2024-05-10T03:30:09.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.028+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300043f0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.028+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c1faed0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.028+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c1fa5a0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.029+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c066f60 con 0x7fb63c06b6e0 2024-05-10T03:30:09.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.030+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb630003610 con 0x7fb63c06b6e0 2024-05-10T03:30:09.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.031+0000 7fb639600640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 0x7fb61c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.031+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb630089670 con 0x7fb63c06b6e0 2024-05-10T03:30:09.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.032+0000 7fb640e00640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 0x7fb61c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.033+0000 7fb640e00640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 0x7fb61c06d110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb628034000 tx=0x7fb62802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.037+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb6300543b0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.241+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63c15c810 con 0x7fb63c06b6e0 2024-05-10T03:30:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.244+0000 7fb639600640 1 -- 172.21.15.55:0/4206691004 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fb6300579d0 con 0x7fb63c06b6e0 2024-05-10T03:30:09.249 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:09.249 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 msgr2=0x7fb61c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 0x7fb61c06d110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb628034000 tx=0x7fb62802e000 comp rx=0 tx=0).stop 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 msgr2=0x7fb63c1481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb63000d5f0 tx=0x7fb63000dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 shutdown_connections 2024-05-10T03:30:09.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63c1659c0 0x7fb63c1f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb61c06ac00 0x7fb61c06d110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb63c162380 0x7fb63c1f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 --2- 172.21.15.55:0/4206691004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb63c06b6e0 0x7fb63c1481e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 >> 172.21.15.55:0/4206691004 conn(0x7fb63c0cbb90 msgr2=0x7fb63c0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 shutdown_connections 2024-05-10T03:30:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.247+0000 7fb644200640 1 -- 172.21.15.55:0/4206691004 wait complete. 2024-05-10T03:30:09.278 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:30:09.279 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:30:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.560+0000 7fa87ae00640 1 Processor -- start 2024-05-10T03:30:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.560+0000 7fa87ae00640 1 -- start start 2024-05-10T03:30:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.562+0000 7fa87ae00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.562+0000 7fa87ae00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa874161c70 0x7fa8740719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.562+0000 7fa87ae00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8741623a0 con 0x7fa8740adbc0 2024-05-10T03:30:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.562+0000 7fa87ae00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8740cdd60 con 0x7fa874161c70 2024-05-10T03:30:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.562+0000 7fa87ae00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740ce040 con 0x7fa8740ad7b0 2024-05-10T03:30:09.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.563+0000 7fa873e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.563+0000 7fa873e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37200/0 (socket says 172.21.15.55:37200) 2024-05-10T03:30:09.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.563+0000 7fa873e00640 1 -- 172.21.15.55:0/2691495231 learned_addr learned my addr 172.21.15.55:0/2691495231 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:09.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.563+0000 7fa86be00640 1 --2- 172.21.15.55:0/2691495231 >> v2:172.21.15.162:3300/0 conn(0x7fa874161c70 0x7fa8740719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.564+0000 7fa873400640 1 -- 172.21.15.55:0/2691495231 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570556113 0 0) 0x7fa8741623a0 con 0x7fa8740adbc0 2024-05-10T03:30:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.564+0000 7fa873e00640 1 -- 172.21.15.55:0/2691495231 >> v2:172.21.15.162:3300/0 conn(0x7fa874161c70 msgr2=0x7fa8740719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.564+0000 7fa873e00640 1 --2- 172.21.15.55:0/2691495231 >> v2:172.21.15.162:3300/0 conn(0x7fa874161c70 0x7fa8740719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.564+0000 7fa873e00640 1 -- 172.21.15.55:0/2691495231 >> v1:172.21.15.162:6789/0 conn(0x7fa8740adbc0 legacy=0x7fa8741541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.564+0000 7fa873e00640 1 -- 172.21.15.55:0/2691495231 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8740ce320 con 0x7fa8740ad7b0 2024-05-10T03:30:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.566+0000 7fa873e00640 1 --2- 172.21.15.55:0/2691495231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa864009870 tx=0x7fa86402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65e2799a57e6b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.566+0000 7fa873400640 1 -- 172.21.15.55:0/2691495231 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864032440 con 0x7fa8740ad7b0 2024-05-10T03:30:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.566+0000 7fa873400640 1 -- 172.21.15.55:0/2691495231 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8640325a0 con 0x7fa8740ad7b0 2024-05-10T03:30:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.566+0000 7fa873400640 1 -- 172.21.15.55:0/2691495231 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864032930 con 0x7fa8740ad7b0 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 -- 172.21.15.55:0/2691495231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 msgr2=0x7fa874161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 --2- 172.21.15.55:0/2691495231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa864009870 tx=0x7fa86402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 -- 172.21.15.55:0/2691495231 shutdown_connections 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 --2- 172.21.15.55:0/2691495231 >> v2:172.21.15.162:3300/0 conn(0x7fa874161c70 0x7fa8740719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 --2- 172.21.15.55:0/2691495231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740ad7b0 0x7fa874161730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 -- 172.21.15.55:0/2691495231 >> 172.21.15.55:0/2691495231 conn(0x7fa8740cbb90 msgr2=0x7fa8740cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 -- 172.21.15.55:0/2691495231 shutdown_connections 2024-05-10T03:30:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.567+0000 7fa87ae00640 1 -- 172.21.15.55:0/2691495231 wait complete. 2024-05-10T03:30:09.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.570+0000 7fa87ae00640 1 Processor -- start 2024-05-10T03:30:09.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.570+0000 7fa87ae00640 1 -- start start 2024-05-10T03:30:09.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740adbc0 0x7fa874160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 0x7fa8741525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8741f48f0 con 0x7fa8740ad7b0 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740b2a70 con 0x7fa874161c70 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa87ae00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740b2d50 con 0x7fa8740adbc0 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45924/0 (socket says 172.21.15.55:45924) 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 -- 172.21.15.55:0/1969055962 learned_addr learned my addr 172.21.15.55:0/1969055962 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa86be00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740adbc0 0x7fa874160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 -- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740adbc0 msgr2=0x7fa874160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740adbc0 0x7fa874160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa878e00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 0x7fa8741525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.572+0000 7fa873e00640 1 -- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 msgr2=0x7fa8741525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa873e00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 0x7fa8741525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa873e00640 1 -- 172.21.15.55:0/1969055962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8640094d0 con 0x7fa8740ad7b0 2024-05-10T03:30:09.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa878e00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 0x7fa8741525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:09.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa873e00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa864009870 tx=0x7fa8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864032440 con 0x7fa8740ad7b0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864004030 con 0x7fa8740ad7b0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640043e0 con 0x7fa8740ad7b0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8740b3030 con 0x7fa8740ad7b0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.573+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8740b35f0 con 0x7fa8740ad7b0 2024-05-10T03:30:09.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.575+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa874066f60 con 0x7fa8740ad7b0 2024-05-10T03:30:09.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.576+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa864030980 con 0x7fa8740ad7b0 2024-05-10T03:30:09.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.577+0000 7fa870c00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 0x7fa85006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:09.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.577+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8640aa940 con 0x7fa8740ad7b0 2024-05-10T03:30:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.577+0000 7fa86be00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 0x7fa85006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.578+0000 7fa86be00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 0x7fa85006d140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa874151540 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:09.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.581+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa864075790 con 0x7fa8740ad7b0 2024-05-10T03:30:09.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.777+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8740b2ec0 con 0x7fa85006ac30 2024-05-10T03:30:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.811+0000 7fa870c00640 1 -- 172.21.15.55:0/1969055962 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345071 (secure 0 0 0) 0x7fa8740b2ec0 con 0x7fa85006ac30 2024-05-10T03:30:09.821 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:09.823 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:30:09.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 msgr2=0x7fa85006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 0x7fa85006d140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa874151540 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 msgr2=0x7fa874146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa864009870 tx=0x7fa8640046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 shutdown_connections 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa85006ac30 0x7fa85006d140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874161c70 0x7fa8741525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8740adbc0 0x7fa874160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 --2- 172.21.15.55:0/1969055962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8740ad7b0 0x7fa874146a60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.819+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 >> 172.21.15.55:0/1969055962 conn(0x7fa8740cbb90 msgr2=0x7fa8740b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.820+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 shutdown_connections 2024-05-10T03:30:09.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:09.820+0000 7fa87ae00640 1 -- 172.21.15.55:0/1969055962 wait complete. 2024-05-10T03:30:09.855 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-10T03:30:08.952613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231796,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754742924,"statfs":{"total":773094113280,"available":772856754176,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005627"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.796272+0000","last_change":"2024-05-10T03:29:43.770873+0000","last_active":"2024-05-10T03:29:48.796272+0000","last_peered":"2024-05-10T03:29:48.796272+0000","last_clean":"2024-05-10T03:29:48.796272+0000","last_became_active":"2024-05-10T03:29:43.770443+0000","last_became_peered":"2024-05-10T03:29:43.770443+0000","last_unstale":"2024-05-10T03:29:48.796272+0000","last_undegraded":"2024-05-10T03:29:48.796272+0000","last_fullsized":"2024-05-10T03:29:48.796272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:55.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.814450+0000","last_change":"2024-05-10T03:29:44.352733+0000","last_active":"2024-05-10T03:29:48.814450+0000","last_peered":"2024-05-10T03:29:48.814450+0000","last_clean":"2024-05-10T03:29:48.814450+0000","last_became_active":"2024-05-10T03:29:44.351354+0000","last_became_peered":"2024-05-10T03:29:44.351354+0000","last_unstale":"2024-05-10T03:29:48.814450+0000","last_undegraded":"2024-05-10T03:29:48.814450+0000","last_fullsized":"2024-05-10T03:29:48.814450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367124+0000","last_change":"2024-05-10T03:29:49.255640+0000","last_active":"2024-05-10T03:29:49.367124+0000","last_peered":"2024-05-10T03:29:49.367124+0000","last_clean":"2024-05-10T03:29:49.367124+0000","last_became_active":"2024-05-10T03:29:49.254068+0000","last_became_peered":"2024-05-10T03:29:49.254068+0000","last_unstale":"2024-05-10T03:29:49.367124+0000","last_undegraded":"2024-05-10T03:29:49.367124+0000","last_fullsized":"2024-05-10T03:29:49.367124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:09.608357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813230+0000","last_change":"2024-05-10T03:29:43.768485+0000","last_active":"2024-05-10T03:29:48.813230+0000","last_peered":"2024-05-10T03:29:48.813230+0000","last_clean":"2024-05-10T03:29:48.813230+0000","last_became_active":"2024-05-10T03:29:43.767506+0000","last_became_peered":"2024-05-10T03:29:43.767506+0000","last_unstale":"2024-05-10T03:29:48.813230+0000","last_undegraded":"2024-05-10T03:29:48.813230+0000","last_fullsized":"2024-05-10T03:29:48.813230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795311+0000","last_change":"2024-05-10T03:29:44.339118+0000","last_active":"2024-05-10T03:29:48.795311+0000","last_peered":"2024-05-10T03:29:48.795311+0000","last_clean":"2024-05-10T03:29:48.795311+0000","last_became_active":"2024-05-10T03:29:44.338690+0000","last_became_peered":"2024-05-10T03:29:44.338690+0000","last_unstale":"2024-05-10T03:29:48.795311+0000","last_undegraded":"2024-05-10T03:29:48.795311+0000","last_fullsized":"2024-05-10T03:29:48.795311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:23.150055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909491+0000","last_change":"2024-05-10T03:29:48.819825+0000","last_active":"2024-05-10T03:29:49.909491+0000","last_peered":"2024-05-10T03:29:49.909491+0000","last_clean":"2024-05-10T03:29:49.909491+0000","last_became_active":"2024-05-10T03:29:48.818817+0000","last_became_peered":"2024-05-10T03:29:48.818817+0000","last_unstale":"2024-05-10T03:29:49.909491+0000","last_undegraded":"2024-05-10T03:29:49.909491+0000","last_fullsized":"2024-05-10T03:29:49.909491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:10.138375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247614+0000","last_change":"2024-05-10T03:29:43.777967+0000","last_active":"2024-05-10T03:29:49.247614+0000","last_peered":"2024-05-10T03:29:49.247614+0000","last_clean":"2024-05-10T03:29:49.247614+0000","last_became_active":"2024-05-10T03:29:43.777585+0000","last_became_peered":"2024-05-10T03:29:43.777585+0000","last_unstale":"2024-05-10T03:29:49.247614+0000","last_undegraded":"2024-05-10T03:29:49.247614+0000","last_fullsized":"2024-05-10T03:29:49.247614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797875+0000","last_change":"2024-05-10T03:29:44.343238+0000","last_active":"2024-05-10T03:29:48.797875+0000","last_peered":"2024-05-10T03:29:48.797875+0000","last_clean":"2024-05-10T03:29:48.797875+0000","last_became_active":"2024-05-10T03:29:44.339308+0000","last_became_peered":"2024-05-10T03:29:44.339308+0000","last_unstale":"2024-05-10T03:29:48.797875+0000","last_undegraded":"2024-05-10T03:29:48.797875+0000","last_fullsized":"2024-05-10T03:29:48.797875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910600+0000","last_change":"2024-05-10T03:29:48.821031+0000","last_active":"2024-05-10T03:29:49.910600+0000","last_peered":"2024-05-10T03:29:49.910600+0000","last_clean":"2024-05-10T03:29:49.910600+0000","last_became_active":"2024-05-10T03:29:48.818081+0000","last_became_peered":"2024-05-10T03:29:48.818081+0000","last_unstale":"2024-05-10T03:29:49.910600+0000","last_undegraded":"2024-05-10T03:29:49.910600+0000","last_fullsized":"2024-05-10T03:29:49.910600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243722+0000","last_change":"2024-05-10T03:29:43.776394+0000","last_active":"2024-05-10T03:29:49.243722+0000","last_peered":"2024-05-10T03:29:49.243722+0000","last_clean":"2024-05-10T03:29:49.243722+0000","last_became_active":"2024-05-10T03:29:43.774971+0000","last_became_peered":"2024-05-10T03:29:43.774971+0000","last_unstale":"2024-05-10T03:29:49.243722+0000","last_undegraded":"2024-05-10T03:29:49.243722+0000","last_fullsized":"2024-05-10T03:29:49.243722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798050+0000","last_change":"2024-05-10T03:29:44.336984+0000","last_active":"2024-05-10T03:29:48.798050+0000","last_peered":"2024-05-10T03:29:48.798050+0000","last_clean":"2024-05-10T03:29:48.798050+0000","last_became_active":"2024-05-10T03:29:44.335925+0000","last_became_peered":"2024-05-10T03:29:44.335925+0000","last_unstale":"2024-05-10T03:29:48.798050+0000","last_undegraded":"2024-05-10T03:29:48.798050+0000","last_fullsized":"2024-05-10T03:29:48.798050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:25.887533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909455+0000","last_change":"2024-05-10T03:29:48.818620+0000","last_active":"2024-05-10T03:29:49.909455+0000","last_peered":"2024-05-10T03:29:49.909455+0000","last_clean":"2024-05-10T03:29:49.909455+0000","last_became_active":"2024-05-10T03:29:48.817907+0000","last_became_peered":"2024-05-10T03:29:48.817907+0000","last_unstale":"2024-05-10T03:29:49.909455+0000","last_undegraded":"2024-05-10T03:29:49.909455+0000","last_fullsized":"2024-05-10T03:29:49.909455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808538+0000","last_change":"2024-05-10T03:29:44.347407+0000","last_active":"2024-05-10T03:29:48.808538+0000","last_peered":"2024-05-10T03:29:48.808538+0000","last_clean":"2024-05-10T03:29:48.808538+0000","last_became_active":"2024-05-10T03:29:44.347072+0000","last_became_peered":"2024-05-10T03:29:44.347072+0000","last_unstale":"2024-05-10T03:29:48.808538+0000","last_undegraded":"2024-05-10T03:29:48.808538+0000","last_fullsized":"2024-05-10T03:29:48.808538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243215+0000","last_change":"2024-05-10T03:29:43.775422+0000","last_active":"2024-05-10T03:29:49.243215+0000","last_peered":"2024-05-10T03:29:49.243215+0000","last_clean":"2024-05-10T03:29:49.243215+0000","last_became_active":"2024-05-10T03:29:43.775083+0000","last_became_peered":"2024-05-10T03:29:43.775083+0000","last_unstale":"2024-05-10T03:29:49.243215+0000","last_undegraded":"2024-05-10T03:29:49.243215+0000","last_fullsized":"2024-05-10T03:29:49.243215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910633+0000","last_change":"2024-05-10T03:29:48.812082+0000","last_active":"2024-05-10T03:29:49.910633+0000","last_peered":"2024-05-10T03:29:49.910633+0000","last_clean":"2024-05-10T03:29:49.910633+0000","last_became_active":"2024-05-10T03:29:48.809401+0000","last_became_peered":"2024-05-10T03:29:48.809401+0000","last_unstale":"2024-05-10T03:29:49.910633+0000","last_undegraded":"2024-05-10T03:29:49.910633+0000","last_fullsized":"2024-05-10T03:29:49.910633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794221+0000","last_change":"2024-05-10T03:29:44.355438+0000","last_active":"2024-05-10T03:29:48.794221+0000","last_peered":"2024-05-10T03:29:48.794221+0000","last_clean":"2024-05-10T03:29:48.794221+0000","last_became_active":"2024-05-10T03:29:44.354716+0000","last_became_peered":"2024-05-10T03:29:44.354716+0000","last_unstale":"2024-05-10T03:29:48.794221+0000","last_undegraded":"2024-05-10T03:29:48.794221+0000","last_fullsized":"2024-05-10T03:29:48.794221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797885+0000","last_change":"2024-05-10T03:29:43.761646+0000","last_active":"2024-05-10T03:29:48.797885+0000","last_peered":"2024-05-10T03:29:48.797885+0000","last_clean":"2024-05-10T03:29:48.797885+0000","last_became_active":"2024-05-10T03:29:43.759174+0000","last_became_peered":"2024-05-10T03:29:43.759174+0000","last_unstale":"2024-05-10T03:29:48.797885+0000","last_undegraded":"2024-05-10T03:29:48.797885+0000","last_fullsized":"2024-05-10T03:29:48.797885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910230+0000","last_change":"2024-05-10T03:29:48.822517+0000","last_active":"2024-05-10T03:29:49.910230+0000","last_peered":"2024-05-10T03:29:49.910230+0000","last_clean":"2024-05-10T03:29:49.910230+0000","last_became_active":"2024-05-10T03:29:48.821950+0000","last_became_peered":"2024-05-10T03:29:48.821950+0000","last_unstale":"2024-05-10T03:29:49.910230+0000","last_undegraded":"2024-05-10T03:29:49.910230+0000","last_fullsized":"2024-05-10T03:29:49.910230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:56:50.388163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808692+0000","last_change":"2024-05-10T03:29:44.324359+0000","last_active":"2024-05-10T03:29:48.808692+0000","last_peered":"2024-05-10T03:29:48.808692+0000","last_clean":"2024-05-10T03:29:48.808692+0000","last_became_active":"2024-05-10T03:29:44.323910+0000","last_became_peered":"2024-05-10T03:29:44.323910+0000","last_unstale":"2024-05-10T03:29:48.808692+0000","last_undegraded":"2024-05-10T03:29:48.808692+0000","last_fullsized":"2024-05-10T03:29:48.808692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794312+0000","last_change":"2024-05-10T03:29:43.771353+0000","last_active":"2024-05-10T03:29:48.794312+0000","last_peered":"2024-05-10T03:29:48.794312+0000","last_clean":"2024-05-10T03:29:48.794312+0000","last_became_active":"2024-05-10T03:29:43.769654+0000","last_became_peered":"2024-05-10T03:29:43.769654+0000","last_unstale":"2024-05-10T03:29:48.794312+0000","last_undegraded":"2024-05-10T03:29:48.794312+0000","last_fullsized":"2024-05-10T03:29:48.794312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494881+0000","last_change":"2024-05-10T03:29:49.254990+0000","last_active":"2024-05-10T03:29:49.494881+0000","last_peered":"2024-05-10T03:29:49.494881+0000","last_clean":"2024-05-10T03:29:49.494881+0000","last_became_active":"2024-05-10T03:29:49.253789+0000","last_became_peered":"2024-05-10T03:29:49.253789+0000","last_unstale":"2024-05-10T03:29:49.494881+0000","last_undegraded":"2024-05-10T03:29:49.494881+0000","last_fullsized":"2024-05-10T03:29:49.494881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790993+0000","last_change":"2024-05-10T03:29:44.347866+0000","last_active":"2024-05-10T03:29:48.790993+0000","last_peered":"2024-05-10T03:29:48.790993+0000","last_clean":"2024-05-10T03:29:48.790993+0000","last_became_active":"2024-05-10T03:29:44.346222+0000","last_became_peered":"2024-05-10T03:29:44.346222+0000","last_unstale":"2024-05-10T03:29:48.790993+0000","last_undegraded":"2024-05-10T03:29:48.790993+0000","last_fullsized":"2024-05-10T03:29:48.790993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241506+0000","last_change":"2024-05-10T03:29:43.775452+0000","last_active":"2024-05-10T03:29:49.241506+0000","last_peered":"2024-05-10T03:29:49.241506+0000","last_clean":"2024-05-10T03:29:49.241506+0000","last_became_active":"2024-05-10T03:29:43.774420+0000","last_became_peered":"2024-05-10T03:29:43.774420+0000","last_unstale":"2024-05-10T03:29:49.241506+0000","last_undegraded":"2024-05-10T03:29:49.241506+0000","last_fullsized":"2024-05-10T03:29:49.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910708+0000","last_change":"2024-05-10T03:29:48.815642+0000","last_active":"2024-05-10T03:29:49.910708+0000","last_peered":"2024-05-10T03:29:49.910708+0000","last_clean":"2024-05-10T03:29:49.910708+0000","last_became_active":"2024-05-10T03:29:48.815170+0000","last_became_peered":"2024-05-10T03:29:48.815170+0000","last_unstale":"2024-05-10T03:29:49.910708+0000","last_undegraded":"2024-05-10T03:29:49.910708+0000","last_fullsized":"2024-05-10T03:29:49.910708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.338211+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.337821+0000","last_became_peered":"2024-05-10T03:29:44.337821+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811440+0000","last_change":"2024-05-10T03:29:43.774006+0000","last_active":"2024-05-10T03:29:48.811440+0000","last_peered":"2024-05-10T03:29:48.811440+0000","last_clean":"2024-05-10T03:29:48.811440+0000","last_became_active":"2024-05-10T03:29:43.773614+0000","last_became_peered":"2024-05-10T03:29:43.773614+0000","last_unstale":"2024-05-10T03:29:48.811440+0000","last_undegraded":"2024-05-10T03:29:48.811440+0000","last_fullsized":"2024-05-10T03:29:48.811440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910293+0000","last_change":"2024-05-10T03:29:49.254777+0000","last_active":"2024-05-10T03:29:49.910293+0000","last_peered":"2024-05-10T03:29:49.910293+0000","last_clean":"2024-05-10T03:29:49.910293+0000","last_became_active":"2024-05-10T03:29:49.253585+0000","last_became_peered":"2024-05-10T03:29:49.253585+0000","last_unstale":"2024-05-10T03:29:49.910293+0000","last_undegraded":"2024-05-10T03:29:49.910293+0000","last_fullsized":"2024-05-10T03:29:49.910293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:58.544334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813189+0000","last_change":"2024-05-10T03:29:44.344906+0000","last_active":"2024-05-10T03:29:48.813189+0000","last_peered":"2024-05-10T03:29:48.813189+0000","last_clean":"2024-05-10T03:29:48.813189+0000","last_became_active":"2024-05-10T03:29:44.344600+0000","last_became_peered":"2024-05-10T03:29:44.344600+0000","last_unstale":"2024-05-10T03:29:48.813189+0000","last_undegraded":"2024-05-10T03:29:48.813189+0000","last_fullsized":"2024-05-10T03:29:48.813189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795841+0000","last_change":"2024-05-10T03:29:43.768246+0000","last_active":"2024-05-10T03:29:48.795841+0000","last_peered":"2024-05-10T03:29:48.795841+0000","last_clean":"2024-05-10T03:29:48.795841+0000","last_became_active":"2024-05-10T03:29:43.767828+0000","last_became_peered":"2024-05-10T03:29:43.767828+0000","last_unstale":"2024-05-10T03:29:48.795841+0000","last_undegraded":"2024-05-10T03:29:48.795841+0000","last_fullsized":"2024-05-10T03:29:48.795841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366780+0000","last_change":"2024-05-10T03:29:49.246734+0000","last_active":"2024-05-10T03:29:49.366780+0000","last_peered":"2024-05-10T03:29:49.366780+0000","last_clean":"2024-05-10T03:29:49.366780+0000","last_became_active":"2024-05-10T03:29:49.246242+0000","last_became_peered":"2024-05-10T03:29:49.246242+0000","last_unstale":"2024-05-10T03:29:49.366780+0000","last_undegraded":"2024-05-10T03:29:49.366780+0000","last_fullsized":"2024-05-10T03:29:49.366780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797404+0000","last_change":"2024-05-10T03:29:44.342815+0000","last_active":"2024-05-10T03:29:48.797404+0000","last_peered":"2024-05-10T03:29:48.797404+0000","last_clean":"2024-05-10T03:29:48.797404+0000","last_became_active":"2024-05-10T03:29:44.340796+0000","last_became_peered":"2024-05-10T03:29:44.340796+0000","last_unstale":"2024-05-10T03:29:48.797404+0000","last_undegraded":"2024-05-10T03:29:48.797404+0000","last_fullsized":"2024-05-10T03:29:48.797404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:08.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.248230+0000","last_change":"2024-05-10T03:29:43.775843+0000","last_active":"2024-05-10T03:29:49.248230+0000","last_peered":"2024-05-10T03:29:49.248230+0000","last_clean":"2024-05-10T03:29:49.248230+0000","last_became_active":"2024-05-10T03:29:43.775096+0000","last_became_peered":"2024-05-10T03:29:43.775096+0000","last_unstale":"2024-05-10T03:29:49.248230+0000","last_undegraded":"2024-05-10T03:29:49.248230+0000","last_fullsized":"2024-05-10T03:29:49.248230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:22.134113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909045+0000","last_change":"2024-05-10T03:29:49.250836+0000","last_active":"2024-05-10T03:29:49.909045+0000","last_peered":"2024-05-10T03:29:49.909045+0000","last_clean":"2024-05-10T03:29:49.909045+0000","last_became_active":"2024-05-10T03:29:49.249127+0000","last_became_peered":"2024-05-10T03:29:49.249127+0000","last_unstale":"2024-05-10T03:29:49.909045+0000","last_undegraded":"2024-05-10T03:29:49.909045+0000","last_fullsized":"2024-05-10T03:29:49.909045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798180+0000","last_change":"2024-05-10T03:29:44.349438+0000","last_active":"2024-05-10T03:29:48.798180+0000","last_peered":"2024-05-10T03:29:48.798180+0000","last_clean":"2024-05-10T03:29:48.798180+0000","last_became_active":"2024-05-10T03:29:44.348987+0000","last_became_peered":"2024-05-10T03:29:44.348987+0000","last_unstale":"2024-05-10T03:29:48.798180+0000","last_undegraded":"2024-05-10T03:29:48.798180+0000","last_fullsized":"2024-05-10T03:29:48.798180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797443+0000","last_change":"2024-05-10T03:29:43.765358+0000","last_active":"2024-05-10T03:29:48.797443+0000","last_peered":"2024-05-10T03:29:48.797443+0000","last_clean":"2024-05-10T03:29:48.797443+0000","last_became_active":"2024-05-10T03:29:43.764855+0000","last_became_peered":"2024-05-10T03:29:43.764855+0000","last_unstale":"2024-05-10T03:29:48.797443+0000","last_undegraded":"2024-05-10T03:29:48.797443+0000","last_fullsized":"2024-05-10T03:29:48.797443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494037+0000","last_change":"2024-05-10T03:29:48.807909+0000","last_active":"2024-05-10T03:29:49.494037+0000","last_peered":"2024-05-10T03:29:49.494037+0000","last_clean":"2024-05-10T03:29:49.494037+0000","last_became_active":"2024-05-10T03:29:48.807403+0000","last_became_peered":"2024-05-10T03:29:48.807403+0000","last_unstale":"2024-05-10T03:29:49.494037+0000","last_undegraded":"2024-05-10T03:29:49.494037+0000","last_fullsized":"2024-05-10T03:29:49.494037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791871+0000","last_change":"2024-05-10T03:29:44.352509+0000","last_active":"2024-05-10T03:29:48.791871+0000","last_peered":"2024-05-10T03:29:48.791871+0000","last_clean":"2024-05-10T03:29:48.791871+0000","last_became_active":"2024-05-10T03:29:44.349456+0000","last_became_peered":"2024-05-10T03:29:44.349456+0000","last_unstale":"2024-05-10T03:29:48.791871+0000","last_undegraded":"2024-05-10T03:29:48.791871+0000","last_fullsized":"2024-05-10T03:29:48.791871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:10.785937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792782+0000","last_change":"2024-05-10T03:29:43.754724+0000","last_active":"2024-05-10T03:29:48.792782+0000","last_peered":"2024-05-10T03:29:48.792782+0000","last_clean":"2024-05-10T03:29:48.792782+0000","last_became_active":"2024-05-10T03:29:43.754159+0000","last_became_peered":"2024-05-10T03:29:43.754159+0000","last_unstale":"2024-05-10T03:29:48.792782+0000","last_undegraded":"2024-05-10T03:29:48.792782+0000","last_fullsized":"2024-05-10T03:29:48.792782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494653+0000","last_change":"2024-05-10T03:29:48.822295+0000","last_active":"2024-05-10T03:29:49.494653+0000","last_peered":"2024-05-10T03:29:49.494653+0000","last_clean":"2024-05-10T03:29:49.494653+0000","last_became_active":"2024-05-10T03:29:48.821883+0000","last_became_peered":"2024-05-10T03:29:48.821883+0000","last_unstale":"2024-05-10T03:29:49.494653+0000","last_undegraded":"2024-05-10T03:29:49.494653+0000","last_fullsized":"2024-05-10T03:29:49.494653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799865+0000","last_change":"2024-05-10T03:29:44.330889+0000","last_active":"2024-05-10T03:29:48.799865+0000","last_peered":"2024-05-10T03:29:48.799865+0000","last_clean":"2024-05-10T03:29:48.799865+0000","last_became_active":"2024-05-10T03:29:44.327692+0000","last_became_peered":"2024-05-10T03:29:44.327692+0000","last_unstale":"2024-05-10T03:29:48.799865+0000","last_undegraded":"2024-05-10T03:29:48.799865+0000","last_fullsized":"2024-05-10T03:29:48.799865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780603+0000","last_change":"2024-05-10T03:29:43.760419+0000","last_active":"2024-05-10T03:29:48.780603+0000","last_peered":"2024-05-10T03:29:48.780603+0000","last_clean":"2024-05-10T03:29:48.780603+0000","last_became_active":"2024-05-10T03:29:43.760082+0000","last_became_peered":"2024-05-10T03:29:43.760082+0000","last_unstale":"2024-05-10T03:29:48.780603+0000","last_undegraded":"2024-05-10T03:29:48.780603+0000","last_fullsized":"2024-05-10T03:29:48.780603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494952+0000","last_change":"2024-05-10T03:29:48.819719+0000","last_active":"2024-05-10T03:29:49.494952+0000","last_peered":"2024-05-10T03:29:49.494952+0000","last_clean":"2024-05-10T03:29:49.494952+0000","last_became_active":"2024-05-10T03:29:48.818741+0000","last_became_peered":"2024-05-10T03:29:48.818741+0000","last_unstale":"2024-05-10T03:29:49.494952+0000","last_undegraded":"2024-05-10T03:29:49.494952+0000","last_fullsized":"2024-05-10T03:29:49.494952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.787035+0000","last_change":"2024-05-10T03:29:44.354949+0000","last_active":"2024-05-10T03:29:48.787035+0000","last_peered":"2024-05-10T03:29:48.787035+0000","last_clean":"2024-05-10T03:29:48.787035+0000","last_became_active":"2024-05-10T03:29:44.354451+0000","last_became_peered":"2024-05-10T03:29:44.354451+0000","last_unstale":"2024-05-10T03:29:48.787035+0000","last_undegraded":"2024-05-10T03:29:48.787035+0000","last_fullsized":"2024-05-10T03:29:48.787035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241517+0000","last_change":"2024-05-10T03:29:43.776046+0000","last_active":"2024-05-10T03:29:49.241517+0000","last_peered":"2024-05-10T03:29:49.241517+0000","last_clean":"2024-05-10T03:29:49.241517+0000","last_became_active":"2024-05-10T03:29:43.775739+0000","last_became_peered":"2024-05-10T03:29:43.775739+0000","last_unstale":"2024-05-10T03:29:49.241517+0000","last_undegraded":"2024-05-10T03:29:49.241517+0000","last_fullsized":"2024-05-10T03:29:49.241517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909517+0000","last_change":"2024-05-10T03:29:49.247763+0000","last_active":"2024-05-10T03:29:49.909517+0000","last_peered":"2024-05-10T03:29:49.909517+0000","last_clean":"2024-05-10T03:29:49.909517+0000","last_became_active":"2024-05-10T03:29:49.246481+0000","last_became_peered":"2024-05-10T03:29:49.246481+0000","last_unstale":"2024-05-10T03:29:49.909517+0000","last_undegraded":"2024-05-10T03:29:49.909517+0000","last_fullsized":"2024-05-10T03:29:49.909517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.793334+0000","last_change":"2024-05-10T03:29:44.337125+0000","last_active":"2024-05-10T03:29:48.793334+0000","last_peered":"2024-05-10T03:29:48.793334+0000","last_clean":"2024-05-10T03:29:48.793334+0000","last_became_active":"2024-05-10T03:29:44.336773+0000","last_became_peered":"2024-05-10T03:29:44.336773+0000","last_unstale":"2024-05-10T03:29:48.793334+0000","last_undegraded":"2024-05-10T03:29:48.793334+0000","last_fullsized":"2024-05-10T03:29:48.793334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797167+0000","last_change":"2024-05-10T03:29:43.759632+0000","last_active":"2024-05-10T03:29:48.797167+0000","last_peered":"2024-05-10T03:29:48.797167+0000","last_clean":"2024-05-10T03:29:48.797167+0000","last_became_active":"2024-05-10T03:29:43.758627+0000","last_became_peered":"2024-05-10T03:29:43.758627+0000","last_unstale":"2024-05-10T03:29:48.797167+0000","last_undegraded":"2024-05-10T03:29:48.797167+0000","last_fullsized":"2024-05-10T03:29:48.797167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241817+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:29:49.241817+0000","last_peered":"2024-05-10T03:29:49.241817+0000","last_clean":"2024-05-10T03:29:49.241817+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:29:49.241817+0000","last_undegraded":"2024-05-10T03:29:49.241817+0000","last_fullsized":"2024-05-10T03:29:49.241817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373998+0000","last_change":"2024-05-10T03:29:49.258186+0000","last_active":"2024-05-10T03:29:49.373998+0000","last_peered":"2024-05-10T03:29:49.373998+0000","last_clean":"2024-05-10T03:29:49.373998+0000","last_became_active":"2024-05-10T03:29:49.256940+0000","last_became_peered":"2024-05-10T03:29:49.256940+0000","last_unstale":"2024-05-10T03:29:49.373998+0000","last_undegraded":"2024-05-10T03:29:49.373998+0000","last_fullsized":"2024-05-10T03:29:49.373998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247185+0000","last_change":"2024-05-10T03:29:44.333742+0000","last_active":"2024-05-10T03:29:49.247185+0000","last_peered":"2024-05-10T03:29:49.247185+0000","last_clean":"2024-05-10T03:29:49.247185+0000","last_became_active":"2024-05-10T03:29:44.333369+0000","last_became_peered":"2024-05-10T03:29:44.333369+0000","last_unstale":"2024-05-10T03:29:49.247185+0000","last_undegraded":"2024-05-10T03:29:49.247185+0000","last_fullsized":"2024-05-10T03:29:49.247185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.782735+0000","last_change":"2024-05-10T03:29:43.759722+0000","last_active":"2024-05-10T03:29:48.782735+0000","last_peered":"2024-05-10T03:29:48.782735+0000","last_clean":"2024-05-10T03:29:48.782735+0000","last_became_active":"2024-05-10T03:29:43.759323+0000","last_became_peered":"2024-05-10T03:29:43.759323+0000","last_unstale":"2024-05-10T03:29:48.782735+0000","last_undegraded":"2024-05-10T03:29:48.782735+0000","last_fullsized":"2024-05-10T03:29:48.782735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.374329+0000","last_change":"2024-05-10T03:29:49.246585+0000","last_active":"2024-05-10T03:29:49.374329+0000","last_peered":"2024-05-10T03:29:49.374329+0000","last_clean":"2024-05-10T03:29:49.374329+0000","last_became_active":"2024-05-10T03:29:49.245297+0000","last_became_peered":"2024-05-10T03:29:49.245297+0000","last_unstale":"2024-05-10T03:29:49.374329+0000","last_undegraded":"2024-05-10T03:29:49.374329+0000","last_fullsized":"2024-05-10T03:29:49.374329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797859+0000","last_change":"2024-05-10T03:29:44.341567+0000","last_active":"2024-05-10T03:29:48.797859+0000","last_peered":"2024-05-10T03:29:48.797859+0000","last_clean":"2024-05-10T03:29:48.797859+0000","last_became_active":"2024-05-10T03:29:44.341235+0000","last_became_peered":"2024-05-10T03:29:44.341235+0000","last_unstale":"2024-05-10T03:29:48.797859+0000","last_undegraded":"2024-05-10T03:29:48.797859+0000","last_fullsized":"2024-05-10T03:29:48.797859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795601+0000","last_change":"2024-05-10T03:29:43.756510+0000","last_active":"2024-05-10T03:29:48.795601+0000","last_peered":"2024-05-10T03:29:48.795601+0000","last_clean":"2024-05-10T03:29:48.795601+0000","last_became_active":"2024-05-10T03:29:43.756205+0000","last_became_peered":"2024-05-10T03:29:43.756205+0000","last_unstale":"2024-05-10T03:29:48.795601+0000","last_undegraded":"2024-05-10T03:29:48.795601+0000","last_fullsized":"2024-05-10T03:29:48.795601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.908687+0000","last_change":"2024-05-10T03:29:48.806390+0000","last_active":"2024-05-10T03:29:49.908687+0000","last_peered":"2024-05-10T03:29:49.908687+0000","last_clean":"2024-05-10T03:29:49.908687+0000","last_became_active":"2024-05-10T03:29:48.805905+0000","last_became_peered":"2024-05-10T03:29:48.805905+0000","last_unstale":"2024-05-10T03:29:49.908687+0000","last_undegraded":"2024-05-10T03:29:49.908687+0000","last_fullsized":"2024-05-10T03:29:49.908687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249530+0000","last_change":"2024-05-10T03:29:44.348672+0000","last_active":"2024-05-10T03:29:49.249530+0000","last_peered":"2024-05-10T03:29:49.249530+0000","last_clean":"2024-05-10T03:29:49.249530+0000","last_became_active":"2024-05-10T03:29:44.348328+0000","last_became_peered":"2024-05-10T03:29:44.348328+0000","last_unstale":"2024-05-10T03:29:49.249530+0000","last_undegraded":"2024-05-10T03:29:49.249530+0000","last_fullsized":"2024-05-10T03:29:49.249530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781197+0000","last_change":"2024-05-10T03:29:43.760410+0000","last_active":"2024-05-10T03:29:48.781197+0000","last_peered":"2024-05-10T03:29:48.781197+0000","last_clean":"2024-05-10T03:29:48.781197+0000","last_became_active":"2024-05-10T03:29:43.760137+0000","last_became_peered":"2024-05-10T03:29:43.760137+0000","last_unstale":"2024-05-10T03:29:48.781197+0000","last_undegraded":"2024-05-10T03:29:48.781197+0000","last_fullsized":"2024-05-10T03:29:48.781197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.370957+0000","last_change":"2024-05-10T03:29:49.256630+0000","last_active":"2024-05-10T03:29:49.370957+0000","last_peered":"2024-05-10T03:29:49.370957+0000","last_clean":"2024-05-10T03:29:49.370957+0000","last_became_active":"2024-05-10T03:29:49.246648+0000","last_became_peered":"2024-05-10T03:29:49.246648+0000","last_unstale":"2024-05-10T03:29:49.370957+0000","last_undegraded":"2024-05-10T03:29:49.370957+0000","last_fullsized":"2024-05-10T03:29:49.370957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.244841+0000","last_change":"2024-05-10T03:29:44.347464+0000","last_active":"2024-05-10T03:29:49.244841+0000","last_peered":"2024-05-10T03:29:49.244841+0000","last_clean":"2024-05-10T03:29:49.244841+0000","last_became_active":"2024-05-10T03:29:44.347135+0000","last_became_peered":"2024-05-10T03:29:44.347135+0000","last_unstale":"2024-05-10T03:29:49.244841+0000","last_undegraded":"2024-05-10T03:29:49.244841+0000","last_fullsized":"2024-05-10T03:29:49.244841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792883+0000","last_change":"2024-05-10T03:29:43.758164+0000","last_active":"2024-05-10T03:29:48.792883+0000","last_peered":"2024-05-10T03:29:48.792883+0000","last_clean":"2024-05-10T03:29:48.792883+0000","last_became_active":"2024-05-10T03:29:43.757793+0000","last_became_peered":"2024-05-10T03:29:43.757793+0000","last_unstale":"2024-05-10T03:29:48.792883+0000","last_undegraded":"2024-05-10T03:29:48.792883+0000","last_fullsized":"2024-05-10T03:29:48.792883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:39.499394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909165+0000","last_change":"2024-05-10T03:29:48.821419+0000","last_active":"2024-05-10T03:29:49.909165+0000","last_peered":"2024-05-10T03:29:49.909165+0000","last_clean":"2024-05-10T03:29:49.909165+0000","last_became_active":"2024-05-10T03:29:48.820977+0000","last_became_peered":"2024-05-10T03:29:48.820977+0000","last_unstale":"2024-05-10T03:29:49.909165+0000","last_undegraded":"2024-05-10T03:29:49.909165+0000","last_fullsized":"2024-05-10T03:29:49.909165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:17.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241985+0000","last_change":"2024-05-10T03:29:44.349656+0000","last_active":"2024-05-10T03:29:49.241985+0000","last_peered":"2024-05-10T03:29:49.241985+0000","last_clean":"2024-05-10T03:29:49.241985+0000","last_became_active":"2024-05-10T03:29:44.349360+0000","last_became_peered":"2024-05-10T03:29:44.349360+0000","last_unstale":"2024-05-10T03:29:49.241985+0000","last_undegraded":"2024-05-10T03:29:49.241985+0000","last_fullsized":"2024-05-10T03:29:49.241985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781272+0000","last_change":"2024-05-10T03:29:43.755515+0000","last_active":"2024-05-10T03:29:48.781272+0000","last_peered":"2024-05-10T03:29:48.781272+0000","last_clean":"2024-05-10T03:29:48.781272+0000","last_became_active":"2024-05-10T03:29:43.755202+0000","last_became_peered":"2024-05-10T03:29:43.755202+0000","last_unstale":"2024-05-10T03:29:48.781272+0000","last_undegraded":"2024-05-10T03:29:48.781272+0000","last_fullsized":"2024-05-10T03:29:48.781272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909234+0000","last_change":"2024-05-10T03:29:48.816001+0000","last_active":"2024-05-10T03:29:49.909234+0000","last_peered":"2024-05-10T03:29:49.909234+0000","last_clean":"2024-05-10T03:29:49.909234+0000","last_became_active":"2024-05-10T03:29:48.815291+0000","last_became_peered":"2024-05-10T03:29:48.815291+0000","last_unstale":"2024-05-10T03:29:49.909234+0000","last_undegraded":"2024-05-10T03:29:49.909234+0000","last_fullsized":"2024-05-10T03:29:49.909234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812139+0000","last_change":"2024-05-10T03:29:44.334673+0000","last_active":"2024-05-10T03:29:48.812139+0000","last_peered":"2024-05-10T03:29:48.812139+0000","last_clean":"2024-05-10T03:29:48.812139+0000","last_became_active":"2024-05-10T03:29:44.333855+0000","last_became_peered":"2024-05-10T03:29:44.333855+0000","last_unstale":"2024-05-10T03:29:48.812139+0000","last_undegraded":"2024-05-10T03:29:48.812139+0000","last_fullsized":"2024-05-10T03:29:48.812139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.807723+0000","last_change":"2024-05-10T03:29:43.768430+0000","last_active":"2024-05-10T03:29:48.807723+0000","last_peered":"2024-05-10T03:29:48.807723+0000","last_clean":"2024-05-10T03:29:48.807723+0000","last_became_active":"2024-05-10T03:29:43.767491+0000","last_became_peered":"2024-05-10T03:29:43.767491+0000","last_unstale":"2024-05-10T03:29:48.807723+0000","last_undegraded":"2024-05-10T03:29:48.807723+0000","last_fullsized":"2024-05-10T03:29:48.807723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.368290+0000","last_change":"2024-05-10T03:29:49.247514+0000","last_active":"2024-05-10T03:29:49.368290+0000","last_peered":"2024-05-10T03:29:49.368290+0000","last_clean":"2024-05-10T03:29:49.368290+0000","last_became_active":"2024-05-10T03:29:49.246239+0000","last_became_peered":"2024-05-10T03:29:49.246239+0000","last_unstale":"2024-05-10T03:29:49.368290+0000","last_undegraded":"2024-05-10T03:29:49.368290+0000","last_fullsized":"2024-05-10T03:29:49.368290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:20:20.616507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242291+0000","last_change":"2024-05-10T03:29:44.348653+0000","last_active":"2024-05-10T03:29:49.242291+0000","last_peered":"2024-05-10T03:29:49.242291+0000","last_clean":"2024-05-10T03:29:49.242291+0000","last_became_active":"2024-05-10T03:29:44.348301+0000","last_became_peered":"2024-05-10T03:29:44.348301+0000","last_unstale":"2024-05-10T03:29:49.242291+0000","last_undegraded":"2024-05-10T03:29:49.242291+0000","last_fullsized":"2024-05-10T03:29:49.242291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792608+0000","last_change":"2024-05-10T03:29:43.760877+0000","last_active":"2024-05-10T03:29:48.792608+0000","last_peered":"2024-05-10T03:29:48.792608+0000","last_clean":"2024-05-10T03:29:48.792608+0000","last_became_active":"2024-05-10T03:29:43.760572+0000","last_became_peered":"2024-05-10T03:29:43.760572+0000","last_unstale":"2024-05-10T03:29:48.792608+0000","last_undegraded":"2024-05-10T03:29:48.792608+0000","last_fullsized":"2024-05-10T03:29:48.792608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367784+0000","last_change":"2024-05-10T03:29:49.246283+0000","last_active":"2024-05-10T03:29:49.367784+0000","last_peered":"2024-05-10T03:29:49.367784+0000","last_clean":"2024-05-10T03:29:49.367784+0000","last_became_active":"2024-05-10T03:29:49.243969+0000","last_became_peered":"2024-05-10T03:29:49.243969+0000","last_unstale":"2024-05-10T03:29:49.367784+0000","last_undegraded":"2024-05-10T03:29:49.367784+0000","last_fullsized":"2024-05-10T03:29:49.367784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:30.586336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.350944+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.350411+0000","last_became_peered":"2024-05-10T03:29:44.350411+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791490+0000","last_change":"2024-05-10T03:29:43.763410+0000","last_active":"2024-05-10T03:29:48.791490+0000","last_peered":"2024-05-10T03:29:48.791490+0000","last_clean":"2024-05-10T03:29:48.791490+0000","last_became_active":"2024-05-10T03:29:43.761799+0000","last_became_peered":"2024-05-10T03:29:43.761799+0000","last_unstale":"2024-05-10T03:29:48.791490+0000","last_undegraded":"2024-05-10T03:29:48.791490+0000","last_fullsized":"2024-05-10T03:29:48.791490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:27.283436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373121+0000","last_change":"2024-05-10T03:29:49.254044+0000","last_active":"2024-05-10T03:29:49.373121+0000","last_peered":"2024-05-10T03:29:49.373121+0000","last_clean":"2024-05-10T03:29:49.373121+0000","last_became_active":"2024-05-10T03:29:49.252991+0000","last_became_peered":"2024-05-10T03:29:49.252991+0000","last_unstale":"2024-05-10T03:29:49.373121+0000","last_undegraded":"2024-05-10T03:29:49.373121+0000","last_fullsized":"2024-05-10T03:29:49.373121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:18:14.955082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243191+0000","last_change":"2024-05-10T03:29:44.361247+0000","last_active":"2024-05-10T03:29:49.243191+0000","last_peered":"2024-05-10T03:29:49.243191+0000","last_clean":"2024-05-10T03:29:49.243191+0000","last_became_active":"2024-05-10T03:29:44.360953+0000","last_became_peered":"2024-05-10T03:29:44.360953+0000","last_unstale":"2024-05-10T03:29:49.243191+0000","last_undegraded":"2024-05-10T03:29:49.243191+0000","last_fullsized":"2024-05-10T03:29:49.243191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:30:04.976035+0000","last_change":"2024-05-10T03:29:43.773652+0000","last_active":"2024-05-10T03:30:04.976035+0000","last_peered":"2024-05-10T03:30:04.976035+0000","last_clean":"2024-05-10T03:30:04.976035+0000","last_became_active":"2024-05-10T03:29:43.773313+0000","last_became_peered":"2024-05-10T03:29:43.773313+0000","last_unstale":"2024-05-10T03:30:04.976035+0000","last_undegraded":"2024-05-10T03:30:04.976035+0000","last_fullsized":"2024-05-10T03:30:04.976035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494687+0000","last_change":"2024-05-10T03:29:48.814023+0000","last_active":"2024-05-10T03:29:49.494687+0000","last_peered":"2024-05-10T03:29:49.494687+0000","last_clean":"2024-05-10T03:29:49.494687+0000","last_became_active":"2024-05-10T03:29:48.813592+0000","last_became_peered":"2024-05-10T03:29:48.813592+0000","last_unstale":"2024-05-10T03:29:49.494687+0000","last_undegraded":"2024-05-10T03:29:49.494687+0000","last_fullsized":"2024-05-10T03:29:49.494687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791481+0000","last_change":"2024-05-10T03:29:44.337189+0000","last_active":"2024-05-10T03:29:48.791481+0000","last_peered":"2024-05-10T03:29:48.791481+0000","last_clean":"2024-05-10T03:29:48.791481+0000","last_became_active":"2024-05-10T03:29:44.336858+0000","last_became_peered":"2024-05-10T03:29:44.336858+0000","last_unstale":"2024-05-10T03:29:48.791481+0000","last_undegraded":"2024-05-10T03:29:48.791481+0000","last_fullsized":"2024-05-10T03:29:48.791481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:34.453268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795391+0000","last_change":"2024-05-10T03:29:43.763074+0000","last_active":"2024-05-10T03:29:48.795391+0000","last_peered":"2024-05-10T03:29:48.795391+0000","last_clean":"2024-05-10T03:29:48.795391+0000","last_became_active":"2024-05-10T03:29:43.762746+0000","last_became_peered":"2024-05-10T03:29:43.762746+0000","last_unstale":"2024-05-10T03:29:48.795391+0000","last_undegraded":"2024-05-10T03:29:48.795391+0000","last_fullsized":"2024-05-10T03:29:48.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910002+0000","last_change":"2024-05-10T03:29:48.811643+0000","last_active":"2024-05-10T03:29:49.910002+0000","last_peered":"2024-05-10T03:29:49.910002+0000","last_clean":"2024-05-10T03:29:49.910002+0000","last_became_active":"2024-05-10T03:29:48.811181+0000","last_became_peered":"2024-05-10T03:29:48.811181+0000","last_unstale":"2024-05-10T03:29:49.910002+0000","last_undegraded":"2024-05-10T03:29:49.910002+0000","last_fullsized":"2024-05-10T03:29:49.910002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795416+0000","last_change":"2024-05-10T03:29:44.339071+0000","last_active":"2024-05-10T03:29:48.795416+0000","last_peered":"2024-05-10T03:29:48.795416+0000","last_clean":"2024-05-10T03:29:48.795416+0000","last_became_active":"2024-05-10T03:29:44.338222+0000","last_became_peered":"2024-05-10T03:29:44.338222+0000","last_unstale":"2024-05-10T03:29:48.795416+0000","last_undegraded":"2024-05-10T03:29:48.795416+0000","last_fullsized":"2024-05-10T03:29:48.795416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798392+0000","last_change":"2024-05-10T03:29:43.769611+0000","last_active":"2024-05-10T03:29:48.798392+0000","last_peered":"2024-05-10T03:29:48.798392+0000","last_clean":"2024-05-10T03:29:48.798392+0000","last_became_active":"2024-05-10T03:29:43.769245+0000","last_became_peered":"2024-05-10T03:29:43.769245+0000","last_unstale":"2024-05-10T03:29:48.798392+0000","last_undegraded":"2024-05-10T03:29:48.798392+0000","last_fullsized":"2024-05-10T03:29:48.798392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367049+0000","last_change":"2024-05-10T03:29:48.823083+0000","last_active":"2024-05-10T03:29:49.367049+0000","last_peered":"2024-05-10T03:29:49.367049+0000","last_clean":"2024-05-10T03:29:49.367049+0000","last_became_active":"2024-05-10T03:29:48.817306+0000","last_became_peered":"2024-05-10T03:29:48.817306+0000","last_unstale":"2024-05-10T03:29:49.367049+0000","last_undegraded":"2024-05-10T03:29:49.367049+0000","last_fullsized":"2024-05-10T03:29:49.367049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:31:18.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798233+0000","last_change":"2024-05-10T03:29:44.330232+0000","last_active":"2024-05-10T03:29:48.798233+0000","last_peered":"2024-05-10T03:29:48.798233+0000","last_clean":"2024-05-10T03:29:48.798233+0000","last_became_active":"2024-05-10T03:29:44.329857+0000","last_became_peered":"2024-05-10T03:29:44.329857+0000","last_unstale":"2024-05-10T03:29:48.798233+0000","last_undegraded":"2024-05-10T03:29:48.798233+0000","last_fullsized":"2024-05-10T03:29:48.798233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811307+0000","last_change":"2024-05-10T03:29:43.771585+0000","last_active":"2024-05-10T03:29:48.811307+0000","last_peered":"2024-05-10T03:29:48.811307+0000","last_clean":"2024-05-10T03:29:48.811307+0000","last_became_active":"2024-05-10T03:29:43.771253+0000","last_became_peered":"2024-05-10T03:29:43.771253+0000","last_unstale":"2024-05-10T03:29:48.811307+0000","last_undegraded":"2024-05-10T03:29:48.811307+0000","last_fullsized":"2024-05-10T03:29:48.811307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366899+0000","last_change":"2024-05-10T03:29:49.245635+0000","last_active":"2024-05-10T03:29:49.366899+0000","last_peered":"2024-05-10T03:29:49.366899+0000","last_clean":"2024-05-10T03:29:49.366899+0000","last_became_active":"2024-05-10T03:29:49.245125+0000","last_became_peered":"2024-05-10T03:29:49.245125+0000","last_unstale":"2024-05-10T03:29:49.366899+0000","last_undegraded":"2024-05-10T03:29:49.366899+0000","last_fullsized":"2024-05-10T03:29:49.366899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909603+0000","last_change":"2024-05-10T03:29:49.253042+0000","last_active":"2024-05-10T03:29:49.909603+0000","last_peered":"2024-05-10T03:29:49.909603+0000","last_clean":"2024-05-10T03:29:49.909603+0000","last_became_active":"2024-05-10T03:29:49.252051+0000","last_became_peered":"2024-05-10T03:29:49.252051+0000","last_unstale":"2024-05-10T03:29:49.909603+0000","last_undegraded":"2024-05-10T03:29:49.909603+0000","last_fullsized":"2024-05-10T03:29:49.909603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:57:00.757246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794396+0000","last_change":"2024-05-10T03:29:44.346922+0000","last_active":"2024-05-10T03:29:48.794396+0000","last_peered":"2024-05-10T03:29:48.794396+0000","last_clean":"2024-05-10T03:29:48.794396+0000","last_became_active":"2024-05-10T03:29:44.346260+0000","last_became_peered":"2024-05-10T03:29:44.346260+0000","last_unstale":"2024-05-10T03:29:48.794396+0000","last_undegraded":"2024-05-10T03:29:48.794396+0000","last_fullsized":"2024-05-10T03:29:48.794396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786593+0000","last_change":"2024-05-10T03:29:43.761072+0000","last_active":"2024-05-10T03:29:48.786593+0000","last_peered":"2024-05-10T03:29:48.786593+0000","last_clean":"2024-05-10T03:29:48.786593+0000","last_became_active":"2024-05-10T03:29:43.760425+0000","last_became_peered":"2024-05-10T03:29:43.760425+0000","last_unstale":"2024-05-10T03:29:48.786593+0000","last_undegraded":"2024-05-10T03:29:48.786593+0000","last_fullsized":"2024-05-10T03:29:48.786593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.375073+0000","last_change":"2024-05-10T03:29:48.818277+0000","last_active":"2024-05-10T03:29:49.375073+0000","last_peered":"2024-05-10T03:29:49.375073+0000","last_clean":"2024-05-10T03:29:49.375073+0000","last_became_active":"2024-05-10T03:29:48.817666+0000","last_became_peered":"2024-05-10T03:29:48.817666+0000","last_unstale":"2024-05-10T03:29:49.375073+0000","last_undegraded":"2024-05-10T03:29:49.375073+0000","last_fullsized":"2024-05-10T03:29:49.375073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808591+0000","last_change":"2024-05-10T03:29:44.356141+0000","last_active":"2024-05-10T03:29:48.808591+0000","last_peered":"2024-05-10T03:29:48.808591+0000","last_clean":"2024-05-10T03:29:48.808591+0000","last_became_active":"2024-05-10T03:29:44.355691+0000","last_became_peered":"2024-05-10T03:29:44.355691+0000","last_unstale":"2024-05-10T03:29:48.808591+0000","last_undegraded":"2024-05-10T03:29:48.808591+0000","last_fullsized":"2024-05-10T03:29:48.808591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801357+0000","last_change":"2024-05-10T03:29:43.767957+0000","last_active":"2024-05-10T03:29:48.801357+0000","last_peered":"2024-05-10T03:29:48.801357+0000","last_clean":"2024-05-10T03:29:48.801357+0000","last_became_active":"2024-05-10T03:29:43.767083+0000","last_became_peered":"2024-05-10T03:29:43.767083+0000","last_unstale":"2024-05-10T03:29:48.801357+0000","last_undegraded":"2024-05-10T03:29:48.801357+0000","last_fullsized":"2024-05-10T03:29:48.801357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.809728+0000","last_change":"2024-05-10T03:29:43.767890+0000","last_active":"2024-05-10T03:29:48.809728+0000","last_peered":"2024-05-10T03:29:48.809728+0000","last_clean":"2024-05-10T03:29:48.809728+0000","last_became_active":"2024-05-10T03:29:43.767550+0000","last_became_peered":"2024-05-10T03:29:43.767550+0000","last_unstale":"2024-05-10T03:29:48.809728+0000","last_undegraded":"2024-05-10T03:29:48.809728+0000","last_fullsized":"2024-05-10T03:29:48.809728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792435+0000","last_change":"2024-05-10T03:29:44.346806+0000","last_active":"2024-05-10T03:29:48.792435+0000","last_peered":"2024-05-10T03:29:48.792435+0000","last_clean":"2024-05-10T03:29:48.792435+0000","last_became_active":"2024-05-10T03:29:44.346484+0000","last_became_peered":"2024-05-10T03:29:44.346484+0000","last_unstale":"2024-05-10T03:29:48.792435+0000","last_undegraded":"2024-05-10T03:29:48.792435+0000","last_fullsized":"2024-05-10T03:29:48.792435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:50.377829+0000","last_change":"2024-05-10T03:29:48.818619+0000","last_active":"2024-05-10T03:29:50.377829+0000","last_peered":"2024-05-10T03:29:50.377829+0000","last_clean":"2024-05-10T03:29:50.377829+0000","last_became_active":"2024-05-10T03:29:48.818125+0000","last_became_peered":"2024-05-10T03:29:48.818125+0000","last_unstale":"2024-05-10T03:29:50.377829+0000","last_undegraded":"2024-05-10T03:29:50.377829+0000","last_fullsized":"2024-05-10T03:29:50.377829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:58.075131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794286+0000","last_change":"2024-05-10T03:29:43.771030+0000","last_active":"2024-05-10T03:29:48.794286+0000","last_peered":"2024-05-10T03:29:48.794286+0000","last_clean":"2024-05-10T03:29:48.794286+0000","last_became_active":"2024-05-10T03:29:43.770630+0000","last_became_peered":"2024-05-10T03:29:43.770630+0000","last_unstale":"2024-05-10T03:29:48.794286+0000","last_undegraded":"2024-05-10T03:29:48.794286+0000","last_fullsized":"2024-05-10T03:29:48.794286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791595+0000","last_change":"2024-05-10T03:29:44.330198+0000","last_active":"2024-05-10T03:29:48.791595+0000","last_peered":"2024-05-10T03:29:48.791595+0000","last_clean":"2024-05-10T03:29:48.791595+0000","last_became_active":"2024-05-10T03:29:44.329871+0000","last_became_peered":"2024-05-10T03:29:44.329871+0000","last_unstale":"2024-05-10T03:29:48.791595+0000","last_undegraded":"2024-05-10T03:29:48.791595+0000","last_fullsized":"2024-05-10T03:29:48.791595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367463+0000","last_change":"2024-05-10T03:29:48.822151+0000","last_active":"2024-05-10T03:29:49.367463+0000","last_peered":"2024-05-10T03:29:49.367463+0000","last_clean":"2024-05-10T03:29:49.367463+0000","last_became_active":"2024-05-10T03:29:48.819369+0000","last_became_peered":"2024-05-10T03:29:48.819369+0000","last_unstale":"2024-05-10T03:29:49.367463+0000","last_undegraded":"2024-05-10T03:29:49.367463+0000","last_fullsized":"2024-05-10T03:29:49.367463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:58.725548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808987+0000","last_change":"2024-05-10T03:29:44.334873+0000","last_active":"2024-05-10T03:29:48.808987+0000","last_peered":"2024-05-10T03:29:48.808987+0000","last_clean":"2024-05-10T03:29:48.808987+0000","last_became_active":"2024-05-10T03:29:44.334365+0000","last_became_peered":"2024-05-10T03:29:44.334365+0000","last_unstale":"2024-05-10T03:29:48.808987+0000","last_undegraded":"2024-05-10T03:29:48.808987+0000","last_fullsized":"2024-05-10T03:29:48.808987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780300+0000","last_change":"2024-05-10T03:29:44.333950+0000","last_active":"2024-05-10T03:29:48.780300+0000","last_peered":"2024-05-10T03:29:48.780300+0000","last_clean":"2024-05-10T03:29:48.780300+0000","last_became_active":"2024-05-10T03:29:44.331498+0000","last_became_peered":"2024-05-10T03:29:44.331498+0000","last_unstale":"2024-05-10T03:29:48.780300+0000","last_undegraded":"2024-05-10T03:29:48.780300+0000","last_fullsized":"2024-05-10T03:29:48.780300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792455+0000","last_change":"2024-05-10T03:29:44.341676+0000","last_active":"2024-05-10T03:29:48.792455+0000","last_peered":"2024-05-10T03:29:48.792455+0000","last_clean":"2024-05-10T03:29:48.792455+0000","last_became_active":"2024-05-10T03:29:44.341296+0000","last_became_peered":"2024-05-10T03:29:44.341296+0000","last_unstale":"2024-05-10T03:29:48.792455+0000","last_undegraded":"2024-05-10T03:29:48.792455+0000","last_fullsized":"2024-05-10T03:29:48.792455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791287+0000","last_change":"2024-05-10T03:29:44.349694+0000","last_active":"2024-05-10T03:29:48.791287+0000","last_peered":"2024-05-10T03:29:48.791287+0000","last_clean":"2024-05-10T03:29:48.791287+0000","last_became_active":"2024-05-10T03:29:44.349396+0000","last_became_peered":"2024-05-10T03:29:44.349396+0000","last_unstale":"2024-05-10T03:29:48.791287+0000","last_undegraded":"2024-05-10T03:29:48.791287+0000","last_fullsized":"2024-05-10T03:29:48.791287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808295+0000","last_change":"2024-05-10T03:29:44.359086+0000","last_active":"2024-05-10T03:29:48.808295+0000","last_peered":"2024-05-10T03:29:48.808295+0000","last_clean":"2024-05-10T03:29:48.808295+0000","last_became_active":"2024-05-10T03:29:44.358776+0000","last_became_peered":"2024-05-10T03:29:44.358776+0000","last_unstale":"2024-05-10T03:29:48.808295+0000","last_undegraded":"2024-05-10T03:29:48.808295+0000","last_fullsized":"2024-05-10T03:29:48.808295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794937+0000","last_change":"2024-05-10T03:29:44.352293+0000","last_active":"2024-05-10T03:29:48.794937+0000","last_peered":"2024-05-10T03:29:48.794937+0000","last_clean":"2024-05-10T03:29:48.794937+0000","last_became_active":"2024-05-10T03:29:44.352017+0000","last_became_peered":"2024-05-10T03:29:44.352017+0000","last_unstale":"2024-05-10T03:29:48.794937+0000","last_undegraded":"2024-05-10T03:29:48.794937+0000","last_fullsized":"2024-05-10T03:29:48.794937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812361+0000","last_change":"2024-05-10T03:29:44.339931+0000","last_active":"2024-05-10T03:29:48.812361+0000","last_peered":"2024-05-10T03:29:48.812361+0000","last_clean":"2024-05-10T03:29:48.812361+0000","last_became_active":"2024-05-10T03:29:44.339638+0000","last_became_peered":"2024-05-10T03:29:44.339638+0000","last_unstale":"2024-05-10T03:29:48.812361+0000","last_undegraded":"2024-05-10T03:29:48.812361+0000","last_fullsized":"2024-05-10T03:29:48.812361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.815879+0000","last_change":"2024-05-10T03:29:44.343280+0000","last_active":"2024-05-10T03:29:48.815879+0000","last_peered":"2024-05-10T03:29:48.815879+0000","last_clean":"2024-05-10T03:29:48.815879+0000","last_became_active":"2024-05-10T03:29:44.342959+0000","last_became_peered":"2024-05-10T03:29:44.342959+0000","last_unstale":"2024-05-10T03:29:48.815879+0000","last_undegraded":"2024-05-10T03:29:48.815879+0000","last_fullsized":"2024-05-10T03:29:48.815879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786197+0000","last_change":"2024-05-10T03:29:44.351381+0000","last_active":"2024-05-10T03:29:48.786197+0000","last_peered":"2024-05-10T03:29:48.786197+0000","last_clean":"2024-05-10T03:29:48.786197+0000","last_became_active":"2024-05-10T03:29:44.351019+0000","last_became_peered":"2024-05-10T03:29:44.351019+0000","last_unstale":"2024-05-10T03:29:48.786197+0000","last_undegraded":"2024-05-10T03:29:48.786197+0000","last_fullsized":"2024-05-10T03:29:48.786197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801184+0000","last_change":"2024-05-10T03:29:44.344852+0000","last_active":"2024-05-10T03:29:48.801184+0000","last_peered":"2024-05-10T03:29:48.801184+0000","last_clean":"2024-05-10T03:29:48.801184+0000","last_became_active":"2024-05-10T03:29:44.344659+0000","last_became_peered":"2024-05-10T03:29:44.344659+0000","last_unstale":"2024-05-10T03:29:48.801184+0000","last_undegraded":"2024-05-10T03:29:48.801184+0000","last_fullsized":"2024-05-10T03:29:48.801184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792296+0000","last_change":"2024-05-10T03:29:44.335097+0000","last_active":"2024-05-10T03:29:48.792296+0000","last_peered":"2024-05-10T03:29:48.792296+0000","last_clean":"2024-05-10T03:29:48.792296+0000","last_became_active":"2024-05-10T03:29:44.332994+0000","last_became_peered":"2024-05-10T03:29:44.332994+0000","last_unstale":"2024-05-10T03:29:48.792296+0000","last_undegraded":"2024-05-10T03:29:48.792296+0000","last_fullsized":"2024-05-10T03:29:48.792296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.788203+0000","last_change":"2024-05-10T03:29:44.346157+0000","last_active":"2024-05-10T03:29:48.788203+0000","last_peered":"2024-05-10T03:29:48.788203+0000","last_clean":"2024-05-10T03:29:48.788203+0000","last_became_active":"2024-05-10T03:29:44.345819+0000","last_became_peered":"2024-05-10T03:29:44.345819+0000","last_unstale":"2024-05-10T03:29:48.788203+0000","last_undegraded":"2024-05-10T03:29:48.788203+0000","last_fullsized":"2024-05-10T03:29:48.788203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813266+0000","last_change":"2024-05-10T03:29:44.351136+0000","last_active":"2024-05-10T03:29:48.813266+0000","last_peered":"2024-05-10T03:29:48.813266+0000","last_clean":"2024-05-10T03:29:48.813266+0000","last_became_active":"2024-05-10T03:29:44.347556+0000","last_became_peered":"2024-05-10T03:29:44.347556+0000","last_unstale":"2024-05-10T03:29:48.813266+0000","last_undegraded":"2024-05-10T03:29:48.813266+0000","last_fullsized":"2024-05-10T03:29:48.813266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247953+0000","last_change":"2024-05-10T03:29:44.354749+0000","last_active":"2024-05-10T03:29:49.247953+0000","last_peered":"2024-05-10T03:29:49.247953+0000","last_clean":"2024-05-10T03:29:49.247953+0000","last_became_active":"2024-05-10T03:29:44.354474+0000","last_became_peered":"2024-05-10T03:29:44.354474+0000","last_unstale":"2024-05-10T03:29:49.247953+0000","last_undegraded":"2024-05-10T03:29:49.247953+0000","last_fullsized":"2024-05-10T03:29:49.247953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780909+0000","last_change":"2024-05-10T03:29:44.339173+0000","last_active":"2024-05-10T03:29:48.780909+0000","last_peered":"2024-05-10T03:29:48.780909+0000","last_clean":"2024-05-10T03:29:48.780909+0000","last_became_active":"2024-05-10T03:29:44.338815+0000","last_became_peered":"2024-05-10T03:29:44.338815+0000","last_unstale":"2024-05-10T03:29:48.780909+0000","last_undegraded":"2024-05-10T03:29:48.780909+0000","last_fullsized":"2024-05-10T03:29:48.780909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799655+0000","last_change":"2024-05-10T03:29:44.334824+0000","last_active":"2024-05-10T03:29:48.799655+0000","last_peered":"2024-05-10T03:29:48.799655+0000","last_clean":"2024-05-10T03:29:48.799655+0000","last_became_active":"2024-05-10T03:29:44.333616+0000","last_became_peered":"2024-05-10T03:29:44.333616+0000","last_unstale":"2024-05-10T03:29:48.799655+0000","last_undegraded":"2024-05-10T03:29:48.799655+0000","last_fullsized":"2024-05-10T03:29:48.799655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:38.441292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.785927+0000","last_change":"2024-05-10T03:29:44.339089+0000","last_active":"2024-05-10T03:29:48.785927+0000","last_peered":"2024-05-10T03:29:48.785927+0000","last_clean":"2024-05-10T03:29:48.785927+0000","last_became_active":"2024-05-10T03:29:44.336113+0000","last_became_peered":"2024-05-10T03:29:44.336113+0000","last_unstale":"2024-05-10T03:29:48.785927+0000","last_undegraded":"2024-05-10T03:29:48.785927+0000","last_fullsized":"2024-05-10T03:29:48.785927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794173+0000","last_change":"2024-05-10T03:29:44.340483+0000","last_active":"2024-05-10T03:29:48.794173+0000","last_peered":"2024-05-10T03:29:48.794173+0000","last_clean":"2024-05-10T03:29:48.794173+0000","last_became_active":"2024-05-10T03:29:44.340093+0000","last_became_peered":"2024-05-10T03:29:44.340093+0000","last_unstale":"2024-05-10T03:29:48.794173+0000","last_undegraded":"2024-05-10T03:29:48.794173+0000","last_fullsized":"2024-05-10T03:29:48.794173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790438+0000","last_change":"2024-05-10T03:29:44.345377+0000","last_active":"2024-05-10T03:29:48.790438+0000","last_peered":"2024-05-10T03:29:48.790438+0000","last_clean":"2024-05-10T03:29:48.790438+0000","last_became_active":"2024-05-10T03:29:44.345048+0000","last_became_peered":"2024-05-10T03:29:44.345048+0000","last_unstale":"2024-05-10T03:29:48.790438+0000","last_undegraded":"2024-05-10T03:29:48.790438+0000","last_fullsized":"2024-05-10T03:29:48.790438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:36.993303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794132+0000","last_change":"2024-05-10T03:29:44.349235+0000","last_active":"2024-05-10T03:29:48.794132+0000","last_peered":"2024-05-10T03:29:48.794132+0000","last_clean":"2024-05-10T03:29:48.794132+0000","last_became_active":"2024-05-10T03:29:44.348922+0000","last_became_peered":"2024-05-10T03:29:44.348922+0000","last_unstale":"2024-05-10T03:29:48.794132+0000","last_undegraded":"2024-05-10T03:29:48.794132+0000","last_fullsized":"2024-05-10T03:29:48.794132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791228+0000","last_change":"2024-05-10T03:29:44.342929+0000","last_active":"2024-05-10T03:29:48.791228+0000","last_peered":"2024-05-10T03:29:48.791228+0000","last_clean":"2024-05-10T03:29:48.791228+0000","last_became_active":"2024-05-10T03:29:44.342505+0000","last_became_peered":"2024-05-10T03:29:44.342505+0000","last_unstale":"2024-05-10T03:29:48.791228+0000","last_undegraded":"2024-05-10T03:29:48.791228+0000","last_fullsized":"2024-05-10T03:29:48.791228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811605+0000","last_change":"2024-05-10T03:29:44.338969+0000","last_active":"2024-05-10T03:29:48.811605+0000","last_peered":"2024-05-10T03:29:48.811605+0000","last_clean":"2024-05-10T03:29:48.811605+0000","last_became_active":"2024-05-10T03:29:44.337703+0000","last_became_peered":"2024-05-10T03:29:44.337703+0000","last_unstale":"2024-05-10T03:29:48.811605+0000","last_undegraded":"2024-05-10T03:29:48.811605+0000","last_fullsized":"2024-05-10T03:29:48.811605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812225+0000","last_change":"2024-05-10T03:29:44.348810+0000","last_active":"2024-05-10T03:29:48.812225+0000","last_peered":"2024-05-10T03:29:48.812225+0000","last_clean":"2024-05-10T03:29:48.812225+0000","last_became_active":"2024-05-10T03:29:44.348512+0000","last_became_peered":"2024-05-10T03:29:44.348512+0000","last_unstale":"2024-05-10T03:29:48.812225+0000","last_undegraded":"2024-05-10T03:29:48.812225+0000","last_fullsized":"2024-05-10T03:29:48.812225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249372+0000","last_change":"2024-05-10T03:29:44.344167+0000","last_active":"2024-05-10T03:29:49.249372+0000","last_peered":"2024-05-10T03:29:49.249372+0000","last_clean":"2024-05-10T03:29:49.249372+0000","last_became_active":"2024-05-10T03:29:44.343405+0000","last_became_peered":"2024-05-10T03:29:44.343405+0000","last_unstale":"2024-05-10T03:29:49.249372+0000","last_undegraded":"2024-05-10T03:29:49.249372+0000","last_fullsized":"2024-05-10T03:29:49.249372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808451+0000","last_change":"2024-05-10T03:29:44.337141+0000","last_active":"2024-05-10T03:29:48.808451+0000","last_peered":"2024-05-10T03:29:48.808451+0000","last_clean":"2024-05-10T03:29:48.808451+0000","last_became_active":"2024-05-10T03:29:44.336801+0000","last_became_peered":"2024-05-10T03:29:44.336801+0000","last_unstale":"2024-05-10T03:29:48.808451+0000","last_undegraded":"2024-05-10T03:29:48.808451+0000","last_fullsized":"2024-05-10T03:29:48.808451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242013+0000","last_change":"2024-05-10T03:29:44.339252+0000","last_active":"2024-05-10T03:29:49.242013+0000","last_peered":"2024-05-10T03:29:49.242013+0000","last_clean":"2024-05-10T03:29:49.242013+0000","last_became_active":"2024-05-10T03:29:44.338870+0000","last_became_peered":"2024-05-10T03:29:44.338870+0000","last_unstale":"2024-05-10T03:29:49.242013+0000","last_undegraded":"2024-05-10T03:29:49.242013+0000","last_fullsized":"2024-05-10T03:29:49.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792099+0000","last_change":"2024-05-10T03:29:44.341080+0000","last_active":"2024-05-10T03:29:48.792099+0000","last_peered":"2024-05-10T03:29:48.792099+0000","last_clean":"2024-05-10T03:29:48.792099+0000","last_became_active":"2024-05-10T03:29:44.340720+0000","last_became_peered":"2024-05-10T03:29:44.340720+0000","last_unstale":"2024-05-10T03:29:48.792099+0000","last_undegraded":"2024-05-10T03:29:48.792099+0000","last_fullsized":"2024-05-10T03:29:48.792099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241944+0000","last_change":"2024-05-10T03:29:44.345293+0000","last_active":"2024-05-10T03:29:49.241944+0000","last_peered":"2024-05-10T03:29:49.241944+0000","last_clean":"2024-05-10T03:29:49.241944+0000","last_became_active":"2024-05-10T03:29:44.344092+0000","last_became_peered":"2024-05-10T03:29:44.344092+0000","last_unstale":"2024-05-10T03:29:49.241944+0000","last_undegraded":"2024-05-10T03:29:49.241944+0000","last_fullsized":"2024-05-10T03:29:49.241944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797675+0000","last_change":"2024-05-10T03:29:44.359220+0000","last_active":"2024-05-10T03:29:48.797675+0000","last_peered":"2024-05-10T03:29:48.797675+0000","last_clean":"2024-05-10T03:29:48.797675+0000","last_became_active":"2024-05-10T03:29:44.358849+0000","last_became_peered":"2024-05-10T03:29:44.358849+0000","last_unstale":"2024-05-10T03:29:48.797675+0000","last_undegraded":"2024-05-10T03:29:48.797675+0000","last_fullsized":"2024-05-10T03:29:48.797675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:54.507386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795357+0000","last_change":"2024-05-10T03:29:44.349124+0000","last_active":"2024-05-10T03:29:48.795357+0000","last_peered":"2024-05-10T03:29:48.795357+0000","last_clean":"2024-05-10T03:29:48.795357+0000","last_became_active":"2024-05-10T03:29:44.348841+0000","last_became_peered":"2024-05-10T03:29:44.348841+0000","last_unstale":"2024-05-10T03:29:48.795357+0000","last_undegraded":"2024-05-10T03:29:48.795357+0000","last_fullsized":"2024-05-10T03:29:48.795357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795763+0000","last_change":"2024-05-10T03:29:44.342192+0000","last_active":"2024-05-10T03:29:48.795763+0000","last_peered":"2024-05-10T03:29:48.795763+0000","last_clean":"2024-05-10T03:29:48.795763+0000","last_became_active":"2024-05-10T03:29:44.341267+0000","last_became_peered":"2024-05-10T03:29:44.341267+0000","last_unstale":"2024-05-10T03:29:48.795763+0000","last_undegraded":"2024-05-10T03:29:48.795763+0000","last_fullsized":"2024-05-10T03:29:48.795763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249243+0000","last_change":"2024-05-10T03:29:44.344420+0000","last_active":"2024-05-10T03:29:49.249243+0000","last_peered":"2024-05-10T03:29:49.249243+0000","last_clean":"2024-05-10T03:29:49.249243+0000","last_became_active":"2024-05-10T03:29:44.343841+0000","last_became_peered":"2024-05-10T03:29:44.343841+0000","last_unstale":"2024-05-10T03:29:49.249243+0000","last_undegraded":"2024-05-10T03:29:49.249243+0000","last_fullsized":"2024-05-10T03:29:49.249243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.929}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.661}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7729999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:30:09.861 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:30:10.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.180+0000 7f706d600640 1 Processor -- start 2024-05-10T03:30:10.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.180+0000 7f706d600640 1 -- start start 2024-05-10T03:30:10.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f706d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f7068144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f706d600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70680b3190 0x7f70680cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f706d600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706806d350 con 0x7f706806d7e0 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f706d600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706806c740 con 0x7f70680b3190 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f706d600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706806ca20 con 0x7f7068144710 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f705fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f706806d7e0 0x7f70680b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58258/0 (socket says 172.21.15.55:58258) 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.183+0000 7f705fe00640 1 -- 172.21.15.55:0/2273021114 learned_addr learned my addr 172.21.15.55:0/2273021114 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067e00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f7068144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f706cc00640 1 --2- 172.21.15.55:0/2273021114 >> v2:172.21.15.162:3300/0 conn(0x7f70680b3190 0x7f70680cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038884885 0 0) 0x7f706806d350 con 0x7f706806d7e0 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704c003520 con 0x7f706806d7e0 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1052050207 0 0) 0x7f704c003520 con 0x7f706806d7e0 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 msgr2=0x7f7068144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f7068144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 >> v2:172.21.15.162:3300/0 conn(0x7f70680b3190 msgr2=0x7f70680cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 --2- 172.21.15.55:0/2273021114 >> v2:172.21.15.162:3300/0 conn(0x7f70680b3190 0x7f70680cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706806cd00 con 0x7f706806d7e0 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7050001670 con 0x7f706806d7e0 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 >> v1:172.21.15.55:6789/0 conn(0x7f706806d7e0 legacy=0x7f70680b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.184+0000 7f7067400640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0039e0 0x7f704c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067400640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0047c0 0x7f704c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067400640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f704c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706806c740 con 0x7f704c0047c0 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706806ca20 con 0x7f704c0039e0 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c00bb10 con 0x7f7068144710 2024-05-10T03:30:10.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0047c0 0x7f704c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067e00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0039e0 0x7f704c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 -- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 msgr2=0x7f704c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f706cc00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f704c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f704c0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 -- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0039e0 msgr2=0x7f704c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0039e0 0x7f704c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f705fe00640 1 -- 172.21.15.55:0/2273021114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c003520 con 0x7f704c0047c0 2024-05-10T03:30:10.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f7067e00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f704c0039e0 0x7f704c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.185+0000 7f706cc00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7068144710 0x7f704c0051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.186+0000 7f705fe00640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0047c0 0x7f704c004bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70680ae870 tx=0x7f704c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a566aa2d58d9f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050005ad0 con 0x7f704c0047c0 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705002d070 con 0x7f704c0047c0 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f7067400640 1 -- 172.21.15.55:0/2273021114 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050005e70 con 0x7f704c0047c0 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f706d600640 1 -- 172.21.15.55:0/2273021114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0047c0 msgr2=0x7f704c004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f706d600640 1 --2- 172.21.15.55:0/2273021114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0047c0 0x7f704c004bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70680ae870 tx=0x7f704c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.187+0000 7f706cc00640 1 -- 172.21.15.55:0/2273021114 reap_dead start 2024-05-10T03:30:10.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.188+0000 7f706d600640 1 -- 172.21.15.55:0/2273021114 shutdown_connections 2024-05-10T03:30:10.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.188+0000 7f706d600640 1 -- 172.21.15.55:0/2273021114 >> 172.21.15.55:0/2273021114 conn(0x7f706806a110 msgr2=0x7f706806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:10.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.188+0000 7f706d600640 1 -- 172.21.15.55:0/2273021114 shutdown_connections 2024-05-10T03:30:10.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.188+0000 7f706d600640 1 -- 172.21.15.55:0/2273021114 wait complete. 2024-05-10T03:30:10.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.190+0000 7f706d600640 1 Processor -- start 2024-05-10T03:30:10.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.190+0000 7f706d600640 1 -- start start 2024-05-10T03:30:10.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c0047c0 0x7f7068140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70681f6bf0 0x7f70681f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068070740 con 0x7f70680ad6d0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681f3ca0 con 0x7f70681f6bf0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706d600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068142800 con 0x7f704c0047c0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45946/0 (socket says 172.21.15.55:45946) 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.192+0000 7f706cc00640 1 -- 172.21.15.55:0/3578677172 learned_addr learned my addr 172.21.15.55:0/3578677172 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f7067e00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c0047c0 0x7f7068140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f705fe00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70681f6bf0 0x7f70681f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 -- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c0047c0 msgr2=0x7f7068140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c0047c0 0x7f7068140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 -- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70681f6bf0 msgr2=0x7f70681f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70681f6bf0 0x7f70681f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 -- 172.21.15.55:0/3578677172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70681f7540 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.193+0000 7f706cc00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f706806e070 tx=0x7f706000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.194+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060017400 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.194+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060011040 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.194+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060017780 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.194+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70681f6960 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.194+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068070950 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.195+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7068066ce0 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.196+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f70600203b0 con 0x7f70680ad6d0 2024-05-10T03:30:10.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.197+0000 7f7064c00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 0x7f704406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.197+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7060089a00 con 0x7f70680ad6d0 2024-05-10T03:30:10.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.198+0000 7f7067e00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 0x7f704406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.199+0000 7f7067e00640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 0x7f704406d110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70580090e0 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.202+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f70600547c0 con 0x7f70680ad6d0 2024-05-10T03:30:10.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.402+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f706806dbf0 con 0x7f704406ac00 2024-05-10T03:30:10.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.430+0000 7f7064c00640 1 -- 172.21.15.55:0/3578677172 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345071 (secure 0 0 0) 0x7f7058011970 con 0x7f704406ac00 2024-05-10T03:30:10.439 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:10.441 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.436+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 msgr2=0x7f704406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.436+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 0x7f704406d110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70580090e0 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.436+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 msgr2=0x7f70680adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f706806e070 tx=0x7f706000f4a0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 shutdown_connections 2024-05-10T03:30:10.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70680ad6d0 0x7f70680adae0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f704406ac00 0x7f704406d110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70681f6bf0 0x7f70681f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 --2- 172.21.15.55:0/3578677172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f704c0047c0 0x7f7068140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 >> 172.21.15.55:0/3578677172 conn(0x7f706806a110 msgr2=0x7f70680b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 shutdown_connections 2024-05-10T03:30:10.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.437+0000 7f706d600640 1 -- 172.21.15.55:0/3578677172 wait complete. 2024-05-10T03:30:10.468 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-10T03:30:08.952613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231796,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754742924,"statfs":{"total":773094113280,"available":772856754176,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005627"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.796272+0000","last_change":"2024-05-10T03:29:43.770873+0000","last_active":"2024-05-10T03:29:48.796272+0000","last_peered":"2024-05-10T03:29:48.796272+0000","last_clean":"2024-05-10T03:29:48.796272+0000","last_became_active":"2024-05-10T03:29:43.770443+0000","last_became_peered":"2024-05-10T03:29:43.770443+0000","last_unstale":"2024-05-10T03:29:48.796272+0000","last_undegraded":"2024-05-10T03:29:48.796272+0000","last_fullsized":"2024-05-10T03:29:48.796272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:55.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.814450+0000","last_change":"2024-05-10T03:29:44.352733+0000","last_active":"2024-05-10T03:29:48.814450+0000","last_peered":"2024-05-10T03:29:48.814450+0000","last_clean":"2024-05-10T03:29:48.814450+0000","last_became_active":"2024-05-10T03:29:44.351354+0000","last_became_peered":"2024-05-10T03:29:44.351354+0000","last_unstale":"2024-05-10T03:29:48.814450+0000","last_undegraded":"2024-05-10T03:29:48.814450+0000","last_fullsized":"2024-05-10T03:29:48.814450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367124+0000","last_change":"2024-05-10T03:29:49.255640+0000","last_active":"2024-05-10T03:29:49.367124+0000","last_peered":"2024-05-10T03:29:49.367124+0000","last_clean":"2024-05-10T03:29:49.367124+0000","last_became_active":"2024-05-10T03:29:49.254068+0000","last_became_peered":"2024-05-10T03:29:49.254068+0000","last_unstale":"2024-05-10T03:29:49.367124+0000","last_undegraded":"2024-05-10T03:29:49.367124+0000","last_fullsized":"2024-05-10T03:29:49.367124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:09.608357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813230+0000","last_change":"2024-05-10T03:29:43.768485+0000","last_active":"2024-05-10T03:29:48.813230+0000","last_peered":"2024-05-10T03:29:48.813230+0000","last_clean":"2024-05-10T03:29:48.813230+0000","last_became_active":"2024-05-10T03:29:43.767506+0000","last_became_peered":"2024-05-10T03:29:43.767506+0000","last_unstale":"2024-05-10T03:29:48.813230+0000","last_undegraded":"2024-05-10T03:29:48.813230+0000","last_fullsized":"2024-05-10T03:29:48.813230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795311+0000","last_change":"2024-05-10T03:29:44.339118+0000","last_active":"2024-05-10T03:29:48.795311+0000","last_peered":"2024-05-10T03:29:48.795311+0000","last_clean":"2024-05-10T03:29:48.795311+0000","last_became_active":"2024-05-10T03:29:44.338690+0000","last_became_peered":"2024-05-10T03:29:44.338690+0000","last_unstale":"2024-05-10T03:29:48.795311+0000","last_undegraded":"2024-05-10T03:29:48.795311+0000","last_fullsized":"2024-05-10T03:29:48.795311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:23.150055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909491+0000","last_change":"2024-05-10T03:29:48.819825+0000","last_active":"2024-05-10T03:29:49.909491+0000","last_peered":"2024-05-10T03:29:49.909491+0000","last_clean":"2024-05-10T03:29:49.909491+0000","last_became_active":"2024-05-10T03:29:48.818817+0000","last_became_peered":"2024-05-10T03:29:48.818817+0000","last_unstale":"2024-05-10T03:29:49.909491+0000","last_undegraded":"2024-05-10T03:29:49.909491+0000","last_fullsized":"2024-05-10T03:29:49.909491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:10.138375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247614+0000","last_change":"2024-05-10T03:29:43.777967+0000","last_active":"2024-05-10T03:29:49.247614+0000","last_peered":"2024-05-10T03:29:49.247614+0000","last_clean":"2024-05-10T03:29:49.247614+0000","last_became_active":"2024-05-10T03:29:43.777585+0000","last_became_peered":"2024-05-10T03:29:43.777585+0000","last_unstale":"2024-05-10T03:29:49.247614+0000","last_undegraded":"2024-05-10T03:29:49.247614+0000","last_fullsized":"2024-05-10T03:29:49.247614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797875+0000","last_change":"2024-05-10T03:29:44.343238+0000","last_active":"2024-05-10T03:29:48.797875+0000","last_peered":"2024-05-10T03:29:48.797875+0000","last_clean":"2024-05-10T03:29:48.797875+0000","last_became_active":"2024-05-10T03:29:44.339308+0000","last_became_peered":"2024-05-10T03:29:44.339308+0000","last_unstale":"2024-05-10T03:29:48.797875+0000","last_undegraded":"2024-05-10T03:29:48.797875+0000","last_fullsized":"2024-05-10T03:29:48.797875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910600+0000","last_change":"2024-05-10T03:29:48.821031+0000","last_active":"2024-05-10T03:29:49.910600+0000","last_peered":"2024-05-10T03:29:49.910600+0000","last_clean":"2024-05-10T03:29:49.910600+0000","last_became_active":"2024-05-10T03:29:48.818081+0000","last_became_peered":"2024-05-10T03:29:48.818081+0000","last_unstale":"2024-05-10T03:29:49.910600+0000","last_undegraded":"2024-05-10T03:29:49.910600+0000","last_fullsized":"2024-05-10T03:29:49.910600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243722+0000","last_change":"2024-05-10T03:29:43.776394+0000","last_active":"2024-05-10T03:29:49.243722+0000","last_peered":"2024-05-10T03:29:49.243722+0000","last_clean":"2024-05-10T03:29:49.243722+0000","last_became_active":"2024-05-10T03:29:43.774971+0000","last_became_peered":"2024-05-10T03:29:43.774971+0000","last_unstale":"2024-05-10T03:29:49.243722+0000","last_undegraded":"2024-05-10T03:29:49.243722+0000","last_fullsized":"2024-05-10T03:29:49.243722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798050+0000","last_change":"2024-05-10T03:29:44.336984+0000","last_active":"2024-05-10T03:29:48.798050+0000","last_peered":"2024-05-10T03:29:48.798050+0000","last_clean":"2024-05-10T03:29:48.798050+0000","last_became_active":"2024-05-10T03:29:44.335925+0000","last_became_peered":"2024-05-10T03:29:44.335925+0000","last_unstale":"2024-05-10T03:29:48.798050+0000","last_undegraded":"2024-05-10T03:29:48.798050+0000","last_fullsized":"2024-05-10T03:29:48.798050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:25.887533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909455+0000","last_change":"2024-05-10T03:29:48.818620+0000","last_active":"2024-05-10T03:29:49.909455+0000","last_peered":"2024-05-10T03:29:49.909455+0000","last_clean":"2024-05-10T03:29:49.909455+0000","last_became_active":"2024-05-10T03:29:48.817907+0000","last_became_peered":"2024-05-10T03:29:48.817907+0000","last_unstale":"2024-05-10T03:29:49.909455+0000","last_undegraded":"2024-05-10T03:29:49.909455+0000","last_fullsized":"2024-05-10T03:29:49.909455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808538+0000","last_change":"2024-05-10T03:29:44.347407+0000","last_active":"2024-05-10T03:29:48.808538+0000","last_peered":"2024-05-10T03:29:48.808538+0000","last_clean":"2024-05-10T03:29:48.808538+0000","last_became_active":"2024-05-10T03:29:44.347072+0000","last_became_peered":"2024-05-10T03:29:44.347072+0000","last_unstale":"2024-05-10T03:29:48.808538+0000","last_undegraded":"2024-05-10T03:29:48.808538+0000","last_fullsized":"2024-05-10T03:29:48.808538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243215+0000","last_change":"2024-05-10T03:29:43.775422+0000","last_active":"2024-05-10T03:29:49.243215+0000","last_peered":"2024-05-10T03:29:49.243215+0000","last_clean":"2024-05-10T03:29:49.243215+0000","last_became_active":"2024-05-10T03:29:43.775083+0000","last_became_peered":"2024-05-10T03:29:43.775083+0000","last_unstale":"2024-05-10T03:29:49.243215+0000","last_undegraded":"2024-05-10T03:29:49.243215+0000","last_fullsized":"2024-05-10T03:29:49.243215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910633+0000","last_change":"2024-05-10T03:29:48.812082+0000","last_active":"2024-05-10T03:29:49.910633+0000","last_peered":"2024-05-10T03:29:49.910633+0000","last_clean":"2024-05-10T03:29:49.910633+0000","last_became_active":"2024-05-10T03:29:48.809401+0000","last_became_peered":"2024-05-10T03:29:48.809401+0000","last_unstale":"2024-05-10T03:29:49.910633+0000","last_undegraded":"2024-05-10T03:29:49.910633+0000","last_fullsized":"2024-05-10T03:29:49.910633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794221+0000","last_change":"2024-05-10T03:29:44.355438+0000","last_active":"2024-05-10T03:29:48.794221+0000","last_peered":"2024-05-10T03:29:48.794221+0000","last_clean":"2024-05-10T03:29:48.794221+0000","last_became_active":"2024-05-10T03:29:44.354716+0000","last_became_peered":"2024-05-10T03:29:44.354716+0000","last_unstale":"2024-05-10T03:29:48.794221+0000","last_undegraded":"2024-05-10T03:29:48.794221+0000","last_fullsized":"2024-05-10T03:29:48.794221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797885+0000","last_change":"2024-05-10T03:29:43.761646+0000","last_active":"2024-05-10T03:29:48.797885+0000","last_peered":"2024-05-10T03:29:48.797885+0000","last_clean":"2024-05-10T03:29:48.797885+0000","last_became_active":"2024-05-10T03:29:43.759174+0000","last_became_peered":"2024-05-10T03:29:43.759174+0000","last_unstale":"2024-05-10T03:29:48.797885+0000","last_undegraded":"2024-05-10T03:29:48.797885+0000","last_fullsized":"2024-05-10T03:29:48.797885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910230+0000","last_change":"2024-05-10T03:29:48.822517+0000","last_active":"2024-05-10T03:29:49.910230+0000","last_peered":"2024-05-10T03:29:49.910230+0000","last_clean":"2024-05-10T03:29:49.910230+0000","last_became_active":"2024-05-10T03:29:48.821950+0000","last_became_peered":"2024-05-10T03:29:48.821950+0000","last_unstale":"2024-05-10T03:29:49.910230+0000","last_undegraded":"2024-05-10T03:29:49.910230+0000","last_fullsized":"2024-05-10T03:29:49.910230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:56:50.388163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808692+0000","last_change":"2024-05-10T03:29:44.324359+0000","last_active":"2024-05-10T03:29:48.808692+0000","last_peered":"2024-05-10T03:29:48.808692+0000","last_clean":"2024-05-10T03:29:48.808692+0000","last_became_active":"2024-05-10T03:29:44.323910+0000","last_became_peered":"2024-05-10T03:29:44.323910+0000","last_unstale":"2024-05-10T03:29:48.808692+0000","last_undegraded":"2024-05-10T03:29:48.808692+0000","last_fullsized":"2024-05-10T03:29:48.808692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794312+0000","last_change":"2024-05-10T03:29:43.771353+0000","last_active":"2024-05-10T03:29:48.794312+0000","last_peered":"2024-05-10T03:29:48.794312+0000","last_clean":"2024-05-10T03:29:48.794312+0000","last_became_active":"2024-05-10T03:29:43.769654+0000","last_became_peered":"2024-05-10T03:29:43.769654+0000","last_unstale":"2024-05-10T03:29:48.794312+0000","last_undegraded":"2024-05-10T03:29:48.794312+0000","last_fullsized":"2024-05-10T03:29:48.794312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494881+0000","last_change":"2024-05-10T03:29:49.254990+0000","last_active":"2024-05-10T03:29:49.494881+0000","last_peered":"2024-05-10T03:29:49.494881+0000","last_clean":"2024-05-10T03:29:49.494881+0000","last_became_active":"2024-05-10T03:29:49.253789+0000","last_became_peered":"2024-05-10T03:29:49.253789+0000","last_unstale":"2024-05-10T03:29:49.494881+0000","last_undegraded":"2024-05-10T03:29:49.494881+0000","last_fullsized":"2024-05-10T03:29:49.494881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790993+0000","last_change":"2024-05-10T03:29:44.347866+0000","last_active":"2024-05-10T03:29:48.790993+0000","last_peered":"2024-05-10T03:29:48.790993+0000","last_clean":"2024-05-10T03:29:48.790993+0000","last_became_active":"2024-05-10T03:29:44.346222+0000","last_became_peered":"2024-05-10T03:29:44.346222+0000","last_unstale":"2024-05-10T03:29:48.790993+0000","last_undegraded":"2024-05-10T03:29:48.790993+0000","last_fullsized":"2024-05-10T03:29:48.790993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241506+0000","last_change":"2024-05-10T03:29:43.775452+0000","last_active":"2024-05-10T03:29:49.241506+0000","last_peered":"2024-05-10T03:29:49.241506+0000","last_clean":"2024-05-10T03:29:49.241506+0000","last_became_active":"2024-05-10T03:29:43.774420+0000","last_became_peered":"2024-05-10T03:29:43.774420+0000","last_unstale":"2024-05-10T03:29:49.241506+0000","last_undegraded":"2024-05-10T03:29:49.241506+0000","last_fullsized":"2024-05-10T03:29:49.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910708+0000","last_change":"2024-05-10T03:29:48.815642+0000","last_active":"2024-05-10T03:29:49.910708+0000","last_peered":"2024-05-10T03:29:49.910708+0000","last_clean":"2024-05-10T03:29:49.910708+0000","last_became_active":"2024-05-10T03:29:48.815170+0000","last_became_peered":"2024-05-10T03:29:48.815170+0000","last_unstale":"2024-05-10T03:29:49.910708+0000","last_undegraded":"2024-05-10T03:29:49.910708+0000","last_fullsized":"2024-05-10T03:29:49.910708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.338211+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.337821+0000","last_became_peered":"2024-05-10T03:29:44.337821+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811440+0000","last_change":"2024-05-10T03:29:43.774006+0000","last_active":"2024-05-10T03:29:48.811440+0000","last_peered":"2024-05-10T03:29:48.811440+0000","last_clean":"2024-05-10T03:29:48.811440+0000","last_became_active":"2024-05-10T03:29:43.773614+0000","last_became_peered":"2024-05-10T03:29:43.773614+0000","last_unstale":"2024-05-10T03:29:48.811440+0000","last_undegraded":"2024-05-10T03:29:48.811440+0000","last_fullsized":"2024-05-10T03:29:48.811440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910293+0000","last_change":"2024-05-10T03:29:49.254777+0000","last_active":"2024-05-10T03:29:49.910293+0000","last_peered":"2024-05-10T03:29:49.910293+0000","last_clean":"2024-05-10T03:29:49.910293+0000","last_became_active":"2024-05-10T03:29:49.253585+0000","last_became_peered":"2024-05-10T03:29:49.253585+0000","last_unstale":"2024-05-10T03:29:49.910293+0000","last_undegraded":"2024-05-10T03:29:49.910293+0000","last_fullsized":"2024-05-10T03:29:49.910293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:58.544334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813189+0000","last_change":"2024-05-10T03:29:44.344906+0000","last_active":"2024-05-10T03:29:48.813189+0000","last_peered":"2024-05-10T03:29:48.813189+0000","last_clean":"2024-05-10T03:29:48.813189+0000","last_became_active":"2024-05-10T03:29:44.344600+0000","last_became_peered":"2024-05-10T03:29:44.344600+0000","last_unstale":"2024-05-10T03:29:48.813189+0000","last_undegraded":"2024-05-10T03:29:48.813189+0000","last_fullsized":"2024-05-10T03:29:48.813189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795841+0000","last_change":"2024-05-10T03:29:43.768246+0000","last_active":"2024-05-10T03:29:48.795841+0000","last_peered":"2024-05-10T03:29:48.795841+0000","last_clean":"2024-05-10T03:29:48.795841+0000","last_became_active":"2024-05-10T03:29:43.767828+0000","last_became_peered":"2024-05-10T03:29:43.767828+0000","last_unstale":"2024-05-10T03:29:48.795841+0000","last_undegraded":"2024-05-10T03:29:48.795841+0000","last_fullsized":"2024-05-10T03:29:48.795841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366780+0000","last_change":"2024-05-10T03:29:49.246734+0000","last_active":"2024-05-10T03:29:49.366780+0000","last_peered":"2024-05-10T03:29:49.366780+0000","last_clean":"2024-05-10T03:29:49.366780+0000","last_became_active":"2024-05-10T03:29:49.246242+0000","last_became_peered":"2024-05-10T03:29:49.246242+0000","last_unstale":"2024-05-10T03:29:49.366780+0000","last_undegraded":"2024-05-10T03:29:49.366780+0000","last_fullsized":"2024-05-10T03:29:49.366780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797404+0000","last_change":"2024-05-10T03:29:44.342815+0000","last_active":"2024-05-10T03:29:48.797404+0000","last_peered":"2024-05-10T03:29:48.797404+0000","last_clean":"2024-05-10T03:29:48.797404+0000","last_became_active":"2024-05-10T03:29:44.340796+0000","last_became_peered":"2024-05-10T03:29:44.340796+0000","last_unstale":"2024-05-10T03:29:48.797404+0000","last_undegraded":"2024-05-10T03:29:48.797404+0000","last_fullsized":"2024-05-10T03:29:48.797404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:08.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.248230+0000","last_change":"2024-05-10T03:29:43.775843+0000","last_active":"2024-05-10T03:29:49.248230+0000","last_peered":"2024-05-10T03:29:49.248230+0000","last_clean":"2024-05-10T03:29:49.248230+0000","last_became_active":"2024-05-10T03:29:43.775096+0000","last_became_peered":"2024-05-10T03:29:43.775096+0000","last_unstale":"2024-05-10T03:29:49.248230+0000","last_undegraded":"2024-05-10T03:29:49.248230+0000","last_fullsized":"2024-05-10T03:29:49.248230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:22.134113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909045+0000","last_change":"2024-05-10T03:29:49.250836+0000","last_active":"2024-05-10T03:29:49.909045+0000","last_peered":"2024-05-10T03:29:49.909045+0000","last_clean":"2024-05-10T03:29:49.909045+0000","last_became_active":"2024-05-10T03:29:49.249127+0000","last_became_peered":"2024-05-10T03:29:49.249127+0000","last_unstale":"2024-05-10T03:29:49.909045+0000","last_undegraded":"2024-05-10T03:29:49.909045+0000","last_fullsized":"2024-05-10T03:29:49.909045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798180+0000","last_change":"2024-05-10T03:29:44.349438+0000","last_active":"2024-05-10T03:29:48.798180+0000","last_peered":"2024-05-10T03:29:48.798180+0000","last_clean":"2024-05-10T03:29:48.798180+0000","last_became_active":"2024-05-10T03:29:44.348987+0000","last_became_peered":"2024-05-10T03:29:44.348987+0000","last_unstale":"2024-05-10T03:29:48.798180+0000","last_undegraded":"2024-05-10T03:29:48.798180+0000","last_fullsized":"2024-05-10T03:29:48.798180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797443+0000","last_change":"2024-05-10T03:29:43.765358+0000","last_active":"2024-05-10T03:29:48.797443+0000","last_peered":"2024-05-10T03:29:48.797443+0000","last_clean":"2024-05-10T03:29:48.797443+0000","last_became_active":"2024-05-10T03:29:43.764855+0000","last_became_peered":"2024-05-10T03:29:43.764855+0000","last_unstale":"2024-05-10T03:29:48.797443+0000","last_undegraded":"2024-05-10T03:29:48.797443+0000","last_fullsized":"2024-05-10T03:29:48.797443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494037+0000","last_change":"2024-05-10T03:29:48.807909+0000","last_active":"2024-05-10T03:29:49.494037+0000","last_peered":"2024-05-10T03:29:49.494037+0000","last_clean":"2024-05-10T03:29:49.494037+0000","last_became_active":"2024-05-10T03:29:48.807403+0000","last_became_peered":"2024-05-10T03:29:48.807403+0000","last_unstale":"2024-05-10T03:29:49.494037+0000","last_undegraded":"2024-05-10T03:29:49.494037+0000","last_fullsized":"2024-05-10T03:29:49.494037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791871+0000","last_change":"2024-05-10T03:29:44.352509+0000","last_active":"2024-05-10T03:29:48.791871+0000","last_peered":"2024-05-10T03:29:48.791871+0000","last_clean":"2024-05-10T03:29:48.791871+0000","last_became_active":"2024-05-10T03:29:44.349456+0000","last_became_peered":"2024-05-10T03:29:44.349456+0000","last_unstale":"2024-05-10T03:29:48.791871+0000","last_undegraded":"2024-05-10T03:29:48.791871+0000","last_fullsized":"2024-05-10T03:29:48.791871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:10.785937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792782+0000","last_change":"2024-05-10T03:29:43.754724+0000","last_active":"2024-05-10T03:29:48.792782+0000","last_peered":"2024-05-10T03:29:48.792782+0000","last_clean":"2024-05-10T03:29:48.792782+0000","last_became_active":"2024-05-10T03:29:43.754159+0000","last_became_peered":"2024-05-10T03:29:43.754159+0000","last_unstale":"2024-05-10T03:29:48.792782+0000","last_undegraded":"2024-05-10T03:29:48.792782+0000","last_fullsized":"2024-05-10T03:29:48.792782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494653+0000","last_change":"2024-05-10T03:29:48.822295+0000","last_active":"2024-05-10T03:29:49.494653+0000","last_peered":"2024-05-10T03:29:49.494653+0000","last_clean":"2024-05-10T03:29:49.494653+0000","last_became_active":"2024-05-10T03:29:48.821883+0000","last_became_peered":"2024-05-10T03:29:48.821883+0000","last_unstale":"2024-05-10T03:29:49.494653+0000","last_undegraded":"2024-05-10T03:29:49.494653+0000","last_fullsized":"2024-05-10T03:29:49.494653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799865+0000","last_change":"2024-05-10T03:29:44.330889+0000","last_active":"2024-05-10T03:29:48.799865+0000","last_peered":"2024-05-10T03:29:48.799865+0000","last_clean":"2024-05-10T03:29:48.799865+0000","last_became_active":"2024-05-10T03:29:44.327692+0000","last_became_peered":"2024-05-10T03:29:44.327692+0000","last_unstale":"2024-05-10T03:29:48.799865+0000","last_undegraded":"2024-05-10T03:29:48.799865+0000","last_fullsized":"2024-05-10T03:29:48.799865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780603+0000","last_change":"2024-05-10T03:29:43.760419+0000","last_active":"2024-05-10T03:29:48.780603+0000","last_peered":"2024-05-10T03:29:48.780603+0000","last_clean":"2024-05-10T03:29:48.780603+0000","last_became_active":"2024-05-10T03:29:43.760082+0000","last_became_peered":"2024-05-10T03:29:43.760082+0000","last_unstale":"2024-05-10T03:29:48.780603+0000","last_undegraded":"2024-05-10T03:29:48.780603+0000","last_fullsized":"2024-05-10T03:29:48.780603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494952+0000","last_change":"2024-05-10T03:29:48.819719+0000","last_active":"2024-05-10T03:29:49.494952+0000","last_peered":"2024-05-10T03:29:49.494952+0000","last_clean":"2024-05-10T03:29:49.494952+0000","last_became_active":"2024-05-10T03:29:48.818741+0000","last_became_peered":"2024-05-10T03:29:48.818741+0000","last_unstale":"2024-05-10T03:29:49.494952+0000","last_undegraded":"2024-05-10T03:29:49.494952+0000","last_fullsized":"2024-05-10T03:29:49.494952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.787035+0000","last_change":"2024-05-10T03:29:44.354949+0000","last_active":"2024-05-10T03:29:48.787035+0000","last_peered":"2024-05-10T03:29:48.787035+0000","last_clean":"2024-05-10T03:29:48.787035+0000","last_became_active":"2024-05-10T03:29:44.354451+0000","last_became_peered":"2024-05-10T03:29:44.354451+0000","last_unstale":"2024-05-10T03:29:48.787035+0000","last_undegraded":"2024-05-10T03:29:48.787035+0000","last_fullsized":"2024-05-10T03:29:48.787035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241517+0000","last_change":"2024-05-10T03:29:43.776046+0000","last_active":"2024-05-10T03:29:49.241517+0000","last_peered":"2024-05-10T03:29:49.241517+0000","last_clean":"2024-05-10T03:29:49.241517+0000","last_became_active":"2024-05-10T03:29:43.775739+0000","last_became_peered":"2024-05-10T03:29:43.775739+0000","last_unstale":"2024-05-10T03:29:49.241517+0000","last_undegraded":"2024-05-10T03:29:49.241517+0000","last_fullsized":"2024-05-10T03:29:49.241517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909517+0000","last_change":"2024-05-10T03:29:49.247763+0000","last_active":"2024-05-10T03:29:49.909517+0000","last_peered":"2024-05-10T03:29:49.909517+0000","last_clean":"2024-05-10T03:29:49.909517+0000","last_became_active":"2024-05-10T03:29:49.246481+0000","last_became_peered":"2024-05-10T03:29:49.246481+0000","last_unstale":"2024-05-10T03:29:49.909517+0000","last_undegraded":"2024-05-10T03:29:49.909517+0000","last_fullsized":"2024-05-10T03:29:49.909517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.793334+0000","last_change":"2024-05-10T03:29:44.337125+0000","last_active":"2024-05-10T03:29:48.793334+0000","last_peered":"2024-05-10T03:29:48.793334+0000","last_clean":"2024-05-10T03:29:48.793334+0000","last_became_active":"2024-05-10T03:29:44.336773+0000","last_became_peered":"2024-05-10T03:29:44.336773+0000","last_unstale":"2024-05-10T03:29:48.793334+0000","last_undegraded":"2024-05-10T03:29:48.793334+0000","last_fullsized":"2024-05-10T03:29:48.793334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797167+0000","last_change":"2024-05-10T03:29:43.759632+0000","last_active":"2024-05-10T03:29:48.797167+0000","last_peered":"2024-05-10T03:29:48.797167+0000","last_clean":"2024-05-10T03:29:48.797167+0000","last_became_active":"2024-05-10T03:29:43.758627+0000","last_became_peered":"2024-05-10T03:29:43.758627+0000","last_unstale":"2024-05-10T03:29:48.797167+0000","last_undegraded":"2024-05-10T03:29:48.797167+0000","last_fullsized":"2024-05-10T03:29:48.797167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241817+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:29:49.241817+0000","last_peered":"2024-05-10T03:29:49.241817+0000","last_clean":"2024-05-10T03:29:49.241817+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:29:49.241817+0000","last_undegraded":"2024-05-10T03:29:49.241817+0000","last_fullsized":"2024-05-10T03:29:49.241817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373998+0000","last_change":"2024-05-10T03:29:49.258186+0000","last_active":"2024-05-10T03:29:49.373998+0000","last_peered":"2024-05-10T03:29:49.373998+0000","last_clean":"2024-05-10T03:29:49.373998+0000","last_became_active":"2024-05-10T03:29:49.256940+0000","last_became_peered":"2024-05-10T03:29:49.256940+0000","last_unstale":"2024-05-10T03:29:49.373998+0000","last_undegraded":"2024-05-10T03:29:49.373998+0000","last_fullsized":"2024-05-10T03:29:49.373998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247185+0000","last_change":"2024-05-10T03:29:44.333742+0000","last_active":"2024-05-10T03:29:49.247185+0000","last_peered":"2024-05-10T03:29:49.247185+0000","last_clean":"2024-05-10T03:29:49.247185+0000","last_became_active":"2024-05-10T03:29:44.333369+0000","last_became_peered":"2024-05-10T03:29:44.333369+0000","last_unstale":"2024-05-10T03:29:49.247185+0000","last_undegraded":"2024-05-10T03:29:49.247185+0000","last_fullsized":"2024-05-10T03:29:49.247185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.782735+0000","last_change":"2024-05-10T03:29:43.759722+0000","last_active":"2024-05-10T03:29:48.782735+0000","last_peered":"2024-05-10T03:29:48.782735+0000","last_clean":"2024-05-10T03:29:48.782735+0000","last_became_active":"2024-05-10T03:29:43.759323+0000","last_became_peered":"2024-05-10T03:29:43.759323+0000","last_unstale":"2024-05-10T03:29:48.782735+0000","last_undegraded":"2024-05-10T03:29:48.782735+0000","last_fullsized":"2024-05-10T03:29:48.782735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.374329+0000","last_change":"2024-05-10T03:29:49.246585+0000","last_active":"2024-05-10T03:29:49.374329+0000","last_peered":"2024-05-10T03:29:49.374329+0000","last_clean":"2024-05-10T03:29:49.374329+0000","last_became_active":"2024-05-10T03:29:49.245297+0000","last_became_peered":"2024-05-10T03:29:49.245297+0000","last_unstale":"2024-05-10T03:29:49.374329+0000","last_undegraded":"2024-05-10T03:29:49.374329+0000","last_fullsized":"2024-05-10T03:29:49.374329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797859+0000","last_change":"2024-05-10T03:29:44.341567+0000","last_active":"2024-05-10T03:29:48.797859+0000","last_peered":"2024-05-10T03:29:48.797859+0000","last_clean":"2024-05-10T03:29:48.797859+0000","last_became_active":"2024-05-10T03:29:44.341235+0000","last_became_peered":"2024-05-10T03:29:44.341235+0000","last_unstale":"2024-05-10T03:29:48.797859+0000","last_undegraded":"2024-05-10T03:29:48.797859+0000","last_fullsized":"2024-05-10T03:29:48.797859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795601+0000","last_change":"2024-05-10T03:29:43.756510+0000","last_active":"2024-05-10T03:29:48.795601+0000","last_peered":"2024-05-10T03:29:48.795601+0000","last_clean":"2024-05-10T03:29:48.795601+0000","last_became_active":"2024-05-10T03:29:43.756205+0000","last_became_peered":"2024-05-10T03:29:43.756205+0000","last_unstale":"2024-05-10T03:29:48.795601+0000","last_undegraded":"2024-05-10T03:29:48.795601+0000","last_fullsized":"2024-05-10T03:29:48.795601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.908687+0000","last_change":"2024-05-10T03:29:48.806390+0000","last_active":"2024-05-10T03:29:49.908687+0000","last_peered":"2024-05-10T03:29:49.908687+0000","last_clean":"2024-05-10T03:29:49.908687+0000","last_became_active":"2024-05-10T03:29:48.805905+0000","last_became_peered":"2024-05-10T03:29:48.805905+0000","last_unstale":"2024-05-10T03:29:49.908687+0000","last_undegraded":"2024-05-10T03:29:49.908687+0000","last_fullsized":"2024-05-10T03:29:49.908687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249530+0000","last_change":"2024-05-10T03:29:44.348672+0000","last_active":"2024-05-10T03:29:49.249530+0000","last_peered":"2024-05-10T03:29:49.249530+0000","last_clean":"2024-05-10T03:29:49.249530+0000","last_became_active":"2024-05-10T03:29:44.348328+0000","last_became_peered":"2024-05-10T03:29:44.348328+0000","last_unstale":"2024-05-10T03:29:49.249530+0000","last_undegraded":"2024-05-10T03:29:49.249530+0000","last_fullsized":"2024-05-10T03:29:49.249530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781197+0000","last_change":"2024-05-10T03:29:43.760410+0000","last_active":"2024-05-10T03:29:48.781197+0000","last_peered":"2024-05-10T03:29:48.781197+0000","last_clean":"2024-05-10T03:29:48.781197+0000","last_became_active":"2024-05-10T03:29:43.760137+0000","last_became_peered":"2024-05-10T03:29:43.760137+0000","last_unstale":"2024-05-10T03:29:48.781197+0000","last_undegraded":"2024-05-10T03:29:48.781197+0000","last_fullsized":"2024-05-10T03:29:48.781197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.370957+0000","last_change":"2024-05-10T03:29:49.256630+0000","last_active":"2024-05-10T03:29:49.370957+0000","last_peered":"2024-05-10T03:29:49.370957+0000","last_clean":"2024-05-10T03:29:49.370957+0000","last_became_active":"2024-05-10T03:29:49.246648+0000","last_became_peered":"2024-05-10T03:29:49.246648+0000","last_unstale":"2024-05-10T03:29:49.370957+0000","last_undegraded":"2024-05-10T03:29:49.370957+0000","last_fullsized":"2024-05-10T03:29:49.370957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.244841+0000","last_change":"2024-05-10T03:29:44.347464+0000","last_active":"2024-05-10T03:29:49.244841+0000","last_peered":"2024-05-10T03:29:49.244841+0000","last_clean":"2024-05-10T03:29:49.244841+0000","last_became_active":"2024-05-10T03:29:44.347135+0000","last_became_peered":"2024-05-10T03:29:44.347135+0000","last_unstale":"2024-05-10T03:29:49.244841+0000","last_undegraded":"2024-05-10T03:29:49.244841+0000","last_fullsized":"2024-05-10T03:29:49.244841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792883+0000","last_change":"2024-05-10T03:29:43.758164+0000","last_active":"2024-05-10T03:29:48.792883+0000","last_peered":"2024-05-10T03:29:48.792883+0000","last_clean":"2024-05-10T03:29:48.792883+0000","last_became_active":"2024-05-10T03:29:43.757793+0000","last_became_peered":"2024-05-10T03:29:43.757793+0000","last_unstale":"2024-05-10T03:29:48.792883+0000","last_undegraded":"2024-05-10T03:29:48.792883+0000","last_fullsized":"2024-05-10T03:29:48.792883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:39.499394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909165+0000","last_change":"2024-05-10T03:29:48.821419+0000","last_active":"2024-05-10T03:29:49.909165+0000","last_peered":"2024-05-10T03:29:49.909165+0000","last_clean":"2024-05-10T03:29:49.909165+0000","last_became_active":"2024-05-10T03:29:48.820977+0000","last_became_peered":"2024-05-10T03:29:48.820977+0000","last_unstale":"2024-05-10T03:29:49.909165+0000","last_undegraded":"2024-05-10T03:29:49.909165+0000","last_fullsized":"2024-05-10T03:29:49.909165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:17.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241985+0000","last_change":"2024-05-10T03:29:44.349656+0000","last_active":"2024-05-10T03:29:49.241985+0000","last_peered":"2024-05-10T03:29:49.241985+0000","last_clean":"2024-05-10T03:29:49.241985+0000","last_became_active":"2024-05-10T03:29:44.349360+0000","last_became_peered":"2024-05-10T03:29:44.349360+0000","last_unstale":"2024-05-10T03:29:49.241985+0000","last_undegraded":"2024-05-10T03:29:49.241985+0000","last_fullsized":"2024-05-10T03:29:49.241985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781272+0000","last_change":"2024-05-10T03:29:43.755515+0000","last_active":"2024-05-10T03:29:48.781272+0000","last_peered":"2024-05-10T03:29:48.781272+0000","last_clean":"2024-05-10T03:29:48.781272+0000","last_became_active":"2024-05-10T03:29:43.755202+0000","last_became_peered":"2024-05-10T03:29:43.755202+0000","last_unstale":"2024-05-10T03:29:48.781272+0000","last_undegraded":"2024-05-10T03:29:48.781272+0000","last_fullsized":"2024-05-10T03:29:48.781272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909234+0000","last_change":"2024-05-10T03:29:48.816001+0000","last_active":"2024-05-10T03:29:49.909234+0000","last_peered":"2024-05-10T03:29:49.909234+0000","last_clean":"2024-05-10T03:29:49.909234+0000","last_became_active":"2024-05-10T03:29:48.815291+0000","last_became_peered":"2024-05-10T03:29:48.815291+0000","last_unstale":"2024-05-10T03:29:49.909234+0000","last_undegraded":"2024-05-10T03:29:49.909234+0000","last_fullsized":"2024-05-10T03:29:49.909234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812139+0000","last_change":"2024-05-10T03:29:44.334673+0000","last_active":"2024-05-10T03:29:48.812139+0000","last_peered":"2024-05-10T03:29:48.812139+0000","last_clean":"2024-05-10T03:29:48.812139+0000","last_became_active":"2024-05-10T03:29:44.333855+0000","last_became_peered":"2024-05-10T03:29:44.333855+0000","last_unstale":"2024-05-10T03:29:48.812139+0000","last_undegraded":"2024-05-10T03:29:48.812139+0000","last_fullsized":"2024-05-10T03:29:48.812139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.807723+0000","last_change":"2024-05-10T03:29:43.768430+0000","last_active":"2024-05-10T03:29:48.807723+0000","last_peered":"2024-05-10T03:29:48.807723+0000","last_clean":"2024-05-10T03:29:48.807723+0000","last_became_active":"2024-05-10T03:29:43.767491+0000","last_became_peered":"2024-05-10T03:29:43.767491+0000","last_unstale":"2024-05-10T03:29:48.807723+0000","last_undegraded":"2024-05-10T03:29:48.807723+0000","last_fullsized":"2024-05-10T03:29:48.807723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.368290+0000","last_change":"2024-05-10T03:29:49.247514+0000","last_active":"2024-05-10T03:29:49.368290+0000","last_peered":"2024-05-10T03:29:49.368290+0000","last_clean":"2024-05-10T03:29:49.368290+0000","last_became_active":"2024-05-10T03:29:49.246239+0000","last_became_peered":"2024-05-10T03:29:49.246239+0000","last_unstale":"2024-05-10T03:29:49.368290+0000","last_undegraded":"2024-05-10T03:29:49.368290+0000","last_fullsized":"2024-05-10T03:29:49.368290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:20:20.616507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242291+0000","last_change":"2024-05-10T03:29:44.348653+0000","last_active":"2024-05-10T03:29:49.242291+0000","last_peered":"2024-05-10T03:29:49.242291+0000","last_clean":"2024-05-10T03:29:49.242291+0000","last_became_active":"2024-05-10T03:29:44.348301+0000","last_became_peered":"2024-05-10T03:29:44.348301+0000","last_unstale":"2024-05-10T03:29:49.242291+0000","last_undegraded":"2024-05-10T03:29:49.242291+0000","last_fullsized":"2024-05-10T03:29:49.242291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792608+0000","last_change":"2024-05-10T03:29:43.760877+0000","last_active":"2024-05-10T03:29:48.792608+0000","last_peered":"2024-05-10T03:29:48.792608+0000","last_clean":"2024-05-10T03:29:48.792608+0000","last_became_active":"2024-05-10T03:29:43.760572+0000","last_became_peered":"2024-05-10T03:29:43.760572+0000","last_unstale":"2024-05-10T03:29:48.792608+0000","last_undegraded":"2024-05-10T03:29:48.792608+0000","last_fullsized":"2024-05-10T03:29:48.792608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367784+0000","last_change":"2024-05-10T03:29:49.246283+0000","last_active":"2024-05-10T03:29:49.367784+0000","last_peered":"2024-05-10T03:29:49.367784+0000","last_clean":"2024-05-10T03:29:49.367784+0000","last_became_active":"2024-05-10T03:29:49.243969+0000","last_became_peered":"2024-05-10T03:29:49.243969+0000","last_unstale":"2024-05-10T03:29:49.367784+0000","last_undegraded":"2024-05-10T03:29:49.367784+0000","last_fullsized":"2024-05-10T03:29:49.367784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:30.586336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.350944+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.350411+0000","last_became_peered":"2024-05-10T03:29:44.350411+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791490+0000","last_change":"2024-05-10T03:29:43.763410+0000","last_active":"2024-05-10T03:29:48.791490+0000","last_peered":"2024-05-10T03:29:48.791490+0000","last_clean":"2024-05-10T03:29:48.791490+0000","last_became_active":"2024-05-10T03:29:43.761799+0000","last_became_peered":"2024-05-10T03:29:43.761799+0000","last_unstale":"2024-05-10T03:29:48.791490+0000","last_undegraded":"2024-05-10T03:29:48.791490+0000","last_fullsized":"2024-05-10T03:29:48.791490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:27.283436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373121+0000","last_change":"2024-05-10T03:29:49.254044+0000","last_active":"2024-05-10T03:29:49.373121+0000","last_peered":"2024-05-10T03:29:49.373121+0000","last_clean":"2024-05-10T03:29:49.373121+0000","last_became_active":"2024-05-10T03:29:49.252991+0000","last_became_peered":"2024-05-10T03:29:49.252991+0000","last_unstale":"2024-05-10T03:29:49.373121+0000","last_undegraded":"2024-05-10T03:29:49.373121+0000","last_fullsized":"2024-05-10T03:29:49.373121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:18:14.955082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243191+0000","last_change":"2024-05-10T03:29:44.361247+0000","last_active":"2024-05-10T03:29:49.243191+0000","last_peered":"2024-05-10T03:29:49.243191+0000","last_clean":"2024-05-10T03:29:49.243191+0000","last_became_active":"2024-05-10T03:29:44.360953+0000","last_became_peered":"2024-05-10T03:29:44.360953+0000","last_unstale":"2024-05-10T03:29:49.243191+0000","last_undegraded":"2024-05-10T03:29:49.243191+0000","last_fullsized":"2024-05-10T03:29:49.243191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:30:04.976035+0000","last_change":"2024-05-10T03:29:43.773652+0000","last_active":"2024-05-10T03:30:04.976035+0000","last_peered":"2024-05-10T03:30:04.976035+0000","last_clean":"2024-05-10T03:30:04.976035+0000","last_became_active":"2024-05-10T03:29:43.773313+0000","last_became_peered":"2024-05-10T03:29:43.773313+0000","last_unstale":"2024-05-10T03:30:04.976035+0000","last_undegraded":"2024-05-10T03:30:04.976035+0000","last_fullsized":"2024-05-10T03:30:04.976035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494687+0000","last_change":"2024-05-10T03:29:48.814023+0000","last_active":"2024-05-10T03:29:49.494687+0000","last_peered":"2024-05-10T03:29:49.494687+0000","last_clean":"2024-05-10T03:29:49.494687+0000","last_became_active":"2024-05-10T03:29:48.813592+0000","last_became_peered":"2024-05-10T03:29:48.813592+0000","last_unstale":"2024-05-10T03:29:49.494687+0000","last_undegraded":"2024-05-10T03:29:49.494687+0000","last_fullsized":"2024-05-10T03:29:49.494687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791481+0000","last_change":"2024-05-10T03:29:44.337189+0000","last_active":"2024-05-10T03:29:48.791481+0000","last_peered":"2024-05-10T03:29:48.791481+0000","last_clean":"2024-05-10T03:29:48.791481+0000","last_became_active":"2024-05-10T03:29:44.336858+0000","last_became_peered":"2024-05-10T03:29:44.336858+0000","last_unstale":"2024-05-10T03:29:48.791481+0000","last_undegraded":"2024-05-10T03:29:48.791481+0000","last_fullsized":"2024-05-10T03:29:48.791481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:34.453268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795391+0000","last_change":"2024-05-10T03:29:43.763074+0000","last_active":"2024-05-10T03:29:48.795391+0000","last_peered":"2024-05-10T03:29:48.795391+0000","last_clean":"2024-05-10T03:29:48.795391+0000","last_became_active":"2024-05-10T03:29:43.762746+0000","last_became_peered":"2024-05-10T03:29:43.762746+0000","last_unstale":"2024-05-10T03:29:48.795391+0000","last_undegraded":"2024-05-10T03:29:48.795391+0000","last_fullsized":"2024-05-10T03:29:48.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910002+0000","last_change":"2024-05-10T03:29:48.811643+0000","last_active":"2024-05-10T03:29:49.910002+0000","last_peered":"2024-05-10T03:29:49.910002+0000","last_clean":"2024-05-10T03:29:49.910002+0000","last_became_active":"2024-05-10T03:29:48.811181+0000","last_became_peered":"2024-05-10T03:29:48.811181+0000","last_unstale":"2024-05-10T03:29:49.910002+0000","last_undegraded":"2024-05-10T03:29:49.910002+0000","last_fullsized":"2024-05-10T03:29:49.910002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795416+0000","last_change":"2024-05-10T03:29:44.339071+0000","last_active":"2024-05-10T03:29:48.795416+0000","last_peered":"2024-05-10T03:29:48.795416+0000","last_clean":"2024-05-10T03:29:48.795416+0000","last_became_active":"2024-05-10T03:29:44.338222+0000","last_became_peered":"2024-05-10T03:29:44.338222+0000","last_unstale":"2024-05-10T03:29:48.795416+0000","last_undegraded":"2024-05-10T03:29:48.795416+0000","last_fullsized":"2024-05-10T03:29:48.795416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798392+0000","last_change":"2024-05-10T03:29:43.769611+0000","last_active":"2024-05-10T03:29:48.798392+0000","last_peered":"2024-05-10T03:29:48.798392+0000","last_clean":"2024-05-10T03:29:48.798392+0000","last_became_active":"2024-05-10T03:29:43.769245+0000","last_became_peered":"2024-05-10T03:29:43.769245+0000","last_unstale":"2024-05-10T03:29:48.798392+0000","last_undegraded":"2024-05-10T03:29:48.798392+0000","last_fullsized":"2024-05-10T03:29:48.798392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367049+0000","last_change":"2024-05-10T03:29:48.823083+0000","last_active":"2024-05-10T03:29:49.367049+0000","last_peered":"2024-05-10T03:29:49.367049+0000","last_clean":"2024-05-10T03:29:49.367049+0000","last_became_active":"2024-05-10T03:29:48.817306+0000","last_became_peered":"2024-05-10T03:29:48.817306+0000","last_unstale":"2024-05-10T03:29:49.367049+0000","last_undegraded":"2024-05-10T03:29:49.367049+0000","last_fullsized":"2024-05-10T03:29:49.367049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:31:18.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798233+0000","last_change":"2024-05-10T03:29:44.330232+0000","last_active":"2024-05-10T03:29:48.798233+0000","last_peered":"2024-05-10T03:29:48.798233+0000","last_clean":"2024-05-10T03:29:48.798233+0000","last_became_active":"2024-05-10T03:29:44.329857+0000","last_became_peered":"2024-05-10T03:29:44.329857+0000","last_unstale":"2024-05-10T03:29:48.798233+0000","last_undegraded":"2024-05-10T03:29:48.798233+0000","last_fullsized":"2024-05-10T03:29:48.798233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811307+0000","last_change":"2024-05-10T03:29:43.771585+0000","last_active":"2024-05-10T03:29:48.811307+0000","last_peered":"2024-05-10T03:29:48.811307+0000","last_clean":"2024-05-10T03:29:48.811307+0000","last_became_active":"2024-05-10T03:29:43.771253+0000","last_became_peered":"2024-05-10T03:29:43.771253+0000","last_unstale":"2024-05-10T03:29:48.811307+0000","last_undegraded":"2024-05-10T03:29:48.811307+0000","last_fullsized":"2024-05-10T03:29:48.811307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366899+0000","last_change":"2024-05-10T03:29:49.245635+0000","last_active":"2024-05-10T03:29:49.366899+0000","last_peered":"2024-05-10T03:29:49.366899+0000","last_clean":"2024-05-10T03:29:49.366899+0000","last_became_active":"2024-05-10T03:29:49.245125+0000","last_became_peered":"2024-05-10T03:29:49.245125+0000","last_unstale":"2024-05-10T03:29:49.366899+0000","last_undegraded":"2024-05-10T03:29:49.366899+0000","last_fullsized":"2024-05-10T03:29:49.366899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909603+0000","last_change":"2024-05-10T03:29:49.253042+0000","last_active":"2024-05-10T03:29:49.909603+0000","last_peered":"2024-05-10T03:29:49.909603+0000","last_clean":"2024-05-10T03:29:49.909603+0000","last_became_active":"2024-05-10T03:29:49.252051+0000","last_became_peered":"2024-05-10T03:29:49.252051+0000","last_unstale":"2024-05-10T03:29:49.909603+0000","last_undegraded":"2024-05-10T03:29:49.909603+0000","last_fullsized":"2024-05-10T03:29:49.909603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:57:00.757246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794396+0000","last_change":"2024-05-10T03:29:44.346922+0000","last_active":"2024-05-10T03:29:48.794396+0000","last_peered":"2024-05-10T03:29:48.794396+0000","last_clean":"2024-05-10T03:29:48.794396+0000","last_became_active":"2024-05-10T03:29:44.346260+0000","last_became_peered":"2024-05-10T03:29:44.346260+0000","last_unstale":"2024-05-10T03:29:48.794396+0000","last_undegraded":"2024-05-10T03:29:48.794396+0000","last_fullsized":"2024-05-10T03:29:48.794396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786593+0000","last_change":"2024-05-10T03:29:43.761072+0000","last_active":"2024-05-10T03:29:48.786593+0000","last_peered":"2024-05-10T03:29:48.786593+0000","last_clean":"2024-05-10T03:29:48.786593+0000","last_became_active":"2024-05-10T03:29:43.760425+0000","last_became_peered":"2024-05-10T03:29:43.760425+0000","last_unstale":"2024-05-10T03:29:48.786593+0000","last_undegraded":"2024-05-10T03:29:48.786593+0000","last_fullsized":"2024-05-10T03:29:48.786593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.375073+0000","last_change":"2024-05-10T03:29:48.818277+0000","last_active":"2024-05-10T03:29:49.375073+0000","last_peered":"2024-05-10T03:29:49.375073+0000","last_clean":"2024-05-10T03:29:49.375073+0000","last_became_active":"2024-05-10T03:29:48.817666+0000","last_became_peered":"2024-05-10T03:29:48.817666+0000","last_unstale":"2024-05-10T03:29:49.375073+0000","last_undegraded":"2024-05-10T03:29:49.375073+0000","last_fullsized":"2024-05-10T03:29:49.375073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808591+0000","last_change":"2024-05-10T03:29:44.356141+0000","last_active":"2024-05-10T03:29:48.808591+0000","last_peered":"2024-05-10T03:29:48.808591+0000","last_clean":"2024-05-10T03:29:48.808591+0000","last_became_active":"2024-05-10T03:29:44.355691+0000","last_became_peered":"2024-05-10T03:29:44.355691+0000","last_unstale":"2024-05-10T03:29:48.808591+0000","last_undegraded":"2024-05-10T03:29:48.808591+0000","last_fullsized":"2024-05-10T03:29:48.808591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801357+0000","last_change":"2024-05-10T03:29:43.767957+0000","last_active":"2024-05-10T03:29:48.801357+0000","last_peered":"2024-05-10T03:29:48.801357+0000","last_clean":"2024-05-10T03:29:48.801357+0000","last_became_active":"2024-05-10T03:29:43.767083+0000","last_became_peered":"2024-05-10T03:29:43.767083+0000","last_unstale":"2024-05-10T03:29:48.801357+0000","last_undegraded":"2024-05-10T03:29:48.801357+0000","last_fullsized":"2024-05-10T03:29:48.801357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.809728+0000","last_change":"2024-05-10T03:29:43.767890+0000","last_active":"2024-05-10T03:29:48.809728+0000","last_peered":"2024-05-10T03:29:48.809728+0000","last_clean":"2024-05-10T03:29:48.809728+0000","last_became_active":"2024-05-10T03:29:43.767550+0000","last_became_peered":"2024-05-10T03:29:43.767550+0000","last_unstale":"2024-05-10T03:29:48.809728+0000","last_undegraded":"2024-05-10T03:29:48.809728+0000","last_fullsized":"2024-05-10T03:29:48.809728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792435+0000","last_change":"2024-05-10T03:29:44.346806+0000","last_active":"2024-05-10T03:29:48.792435+0000","last_peered":"2024-05-10T03:29:48.792435+0000","last_clean":"2024-05-10T03:29:48.792435+0000","last_became_active":"2024-05-10T03:29:44.346484+0000","last_became_peered":"2024-05-10T03:29:44.346484+0000","last_unstale":"2024-05-10T03:29:48.792435+0000","last_undegraded":"2024-05-10T03:29:48.792435+0000","last_fullsized":"2024-05-10T03:29:48.792435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:50.377829+0000","last_change":"2024-05-10T03:29:48.818619+0000","last_active":"2024-05-10T03:29:50.377829+0000","last_peered":"2024-05-10T03:29:50.377829+0000","last_clean":"2024-05-10T03:29:50.377829+0000","last_became_active":"2024-05-10T03:29:48.818125+0000","last_became_peered":"2024-05-10T03:29:48.818125+0000","last_unstale":"2024-05-10T03:29:50.377829+0000","last_undegraded":"2024-05-10T03:29:50.377829+0000","last_fullsized":"2024-05-10T03:29:50.377829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:58.075131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794286+0000","last_change":"2024-05-10T03:29:43.771030+0000","last_active":"2024-05-10T03:29:48.794286+0000","last_peered":"2024-05-10T03:29:48.794286+0000","last_clean":"2024-05-10T03:29:48.794286+0000","last_became_active":"2024-05-10T03:29:43.770630+0000","last_became_peered":"2024-05-10T03:29:43.770630+0000","last_unstale":"2024-05-10T03:29:48.794286+0000","last_undegraded":"2024-05-10T03:29:48.794286+0000","last_fullsized":"2024-05-10T03:29:48.794286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791595+0000","last_change":"2024-05-10T03:29:44.330198+0000","last_active":"2024-05-10T03:29:48.791595+0000","last_peered":"2024-05-10T03:29:48.791595+0000","last_clean":"2024-05-10T03:29:48.791595+0000","last_became_active":"2024-05-10T03:29:44.329871+0000","last_became_peered":"2024-05-10T03:29:44.329871+0000","last_unstale":"2024-05-10T03:29:48.791595+0000","last_undegraded":"2024-05-10T03:29:48.791595+0000","last_fullsized":"2024-05-10T03:29:48.791595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367463+0000","last_change":"2024-05-10T03:29:48.822151+0000","last_active":"2024-05-10T03:29:49.367463+0000","last_peered":"2024-05-10T03:29:49.367463+0000","last_clean":"2024-05-10T03:29:49.367463+0000","last_became_active":"2024-05-10T03:29:48.819369+0000","last_became_peered":"2024-05-10T03:29:48.819369+0000","last_unstale":"2024-05-10T03:29:49.367463+0000","last_undegraded":"2024-05-10T03:29:49.367463+0000","last_fullsized":"2024-05-10T03:29:49.367463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:58.725548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808987+0000","last_change":"2024-05-10T03:29:44.334873+0000","last_active":"2024-05-10T03:29:48.808987+0000","last_peered":"2024-05-10T03:29:48.808987+0000","last_clean":"2024-05-10T03:29:48.808987+0000","last_became_active":"2024-05-10T03:29:44.334365+0000","last_became_peered":"2024-05-10T03:29:44.334365+0000","last_unstale":"2024-05-10T03:29:48.808987+0000","last_undegraded":"2024-05-10T03:29:48.808987+0000","last_fullsized":"2024-05-10T03:29:48.808987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780300+0000","last_change":"2024-05-10T03:29:44.333950+0000","last_active":"2024-05-10T03:29:48.780300+0000","last_peered":"2024-05-10T03:29:48.780300+0000","last_clean":"2024-05-10T03:29:48.780300+0000","last_became_active":"2024-05-10T03:29:44.331498+0000","last_became_peered":"2024-05-10T03:29:44.331498+0000","last_unstale":"2024-05-10T03:29:48.780300+0000","last_undegraded":"2024-05-10T03:29:48.780300+0000","last_fullsized":"2024-05-10T03:29:48.780300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792455+0000","last_change":"2024-05-10T03:29:44.341676+0000","last_active":"2024-05-10T03:29:48.792455+0000","last_peered":"2024-05-10T03:29:48.792455+0000","last_clean":"2024-05-10T03:29:48.792455+0000","last_became_active":"2024-05-10T03:29:44.341296+0000","last_became_peered":"2024-05-10T03:29:44.341296+0000","last_unstale":"2024-05-10T03:29:48.792455+0000","last_undegraded":"2024-05-10T03:29:48.792455+0000","last_fullsized":"2024-05-10T03:29:48.792455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791287+0000","last_change":"2024-05-10T03:29:44.349694+0000","last_active":"2024-05-10T03:29:48.791287+0000","last_peered":"2024-05-10T03:29:48.791287+0000","last_clean":"2024-05-10T03:29:48.791287+0000","last_became_active":"2024-05-10T03:29:44.349396+0000","last_became_peered":"2024-05-10T03:29:44.349396+0000","last_unstale":"2024-05-10T03:29:48.791287+0000","last_undegraded":"2024-05-10T03:29:48.791287+0000","last_fullsized":"2024-05-10T03:29:48.791287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808295+0000","last_change":"2024-05-10T03:29:44.359086+0000","last_active":"2024-05-10T03:29:48.808295+0000","last_peered":"2024-05-10T03:29:48.808295+0000","last_clean":"2024-05-10T03:29:48.808295+0000","last_became_active":"2024-05-10T03:29:44.358776+0000","last_became_peered":"2024-05-10T03:29:44.358776+0000","last_unstale":"2024-05-10T03:29:48.808295+0000","last_undegraded":"2024-05-10T03:29:48.808295+0000","last_fullsized":"2024-05-10T03:29:48.808295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794937+0000","last_change":"2024-05-10T03:29:44.352293+0000","last_active":"2024-05-10T03:29:48.794937+0000","last_peered":"2024-05-10T03:29:48.794937+0000","last_clean":"2024-05-10T03:29:48.794937+0000","last_became_active":"2024-05-10T03:29:44.352017+0000","last_became_peered":"2024-05-10T03:29:44.352017+0000","last_unstale":"2024-05-10T03:29:48.794937+0000","last_undegraded":"2024-05-10T03:29:48.794937+0000","last_fullsized":"2024-05-10T03:29:48.794937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812361+0000","last_change":"2024-05-10T03:29:44.339931+0000","last_active":"2024-05-10T03:29:48.812361+0000","last_peered":"2024-05-10T03:29:48.812361+0000","last_clean":"2024-05-10T03:29:48.812361+0000","last_became_active":"2024-05-10T03:29:44.339638+0000","last_became_peered":"2024-05-10T03:29:44.339638+0000","last_unstale":"2024-05-10T03:29:48.812361+0000","last_undegraded":"2024-05-10T03:29:48.812361+0000","last_fullsized":"2024-05-10T03:29:48.812361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.815879+0000","last_change":"2024-05-10T03:29:44.343280+0000","last_active":"2024-05-10T03:29:48.815879+0000","last_peered":"2024-05-10T03:29:48.815879+0000","last_clean":"2024-05-10T03:29:48.815879+0000","last_became_active":"2024-05-10T03:29:44.342959+0000","last_became_peered":"2024-05-10T03:29:44.342959+0000","last_unstale":"2024-05-10T03:29:48.815879+0000","last_undegraded":"2024-05-10T03:29:48.815879+0000","last_fullsized":"2024-05-10T03:29:48.815879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786197+0000","last_change":"2024-05-10T03:29:44.351381+0000","last_active":"2024-05-10T03:29:48.786197+0000","last_peered":"2024-05-10T03:29:48.786197+0000","last_clean":"2024-05-10T03:29:48.786197+0000","last_became_active":"2024-05-10T03:29:44.351019+0000","last_became_peered":"2024-05-10T03:29:44.351019+0000","last_unstale":"2024-05-10T03:29:48.786197+0000","last_undegraded":"2024-05-10T03:29:48.786197+0000","last_fullsized":"2024-05-10T03:29:48.786197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801184+0000","last_change":"2024-05-10T03:29:44.344852+0000","last_active":"2024-05-10T03:29:48.801184+0000","last_peered":"2024-05-10T03:29:48.801184+0000","last_clean":"2024-05-10T03:29:48.801184+0000","last_became_active":"2024-05-10T03:29:44.344659+0000","last_became_peered":"2024-05-10T03:29:44.344659+0000","last_unstale":"2024-05-10T03:29:48.801184+0000","last_undegraded":"2024-05-10T03:29:48.801184+0000","last_fullsized":"2024-05-10T03:29:48.801184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792296+0000","last_change":"2024-05-10T03:29:44.335097+0000","last_active":"2024-05-10T03:29:48.792296+0000","last_peered":"2024-05-10T03:29:48.792296+0000","last_clean":"2024-05-10T03:29:48.792296+0000","last_became_active":"2024-05-10T03:29:44.332994+0000","last_became_peered":"2024-05-10T03:29:44.332994+0000","last_unstale":"2024-05-10T03:29:48.792296+0000","last_undegraded":"2024-05-10T03:29:48.792296+0000","last_fullsized":"2024-05-10T03:29:48.792296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.788203+0000","last_change":"2024-05-10T03:29:44.346157+0000","last_active":"2024-05-10T03:29:48.788203+0000","last_peered":"2024-05-10T03:29:48.788203+0000","last_clean":"2024-05-10T03:29:48.788203+0000","last_became_active":"2024-05-10T03:29:44.345819+0000","last_became_peered":"2024-05-10T03:29:44.345819+0000","last_unstale":"2024-05-10T03:29:48.788203+0000","last_undegraded":"2024-05-10T03:29:48.788203+0000","last_fullsized":"2024-05-10T03:29:48.788203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813266+0000","last_change":"2024-05-10T03:29:44.351136+0000","last_active":"2024-05-10T03:29:48.813266+0000","last_peered":"2024-05-10T03:29:48.813266+0000","last_clean":"2024-05-10T03:29:48.813266+0000","last_became_active":"2024-05-10T03:29:44.347556+0000","last_became_peered":"2024-05-10T03:29:44.347556+0000","last_unstale":"2024-05-10T03:29:48.813266+0000","last_undegraded":"2024-05-10T03:29:48.813266+0000","last_fullsized":"2024-05-10T03:29:48.813266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247953+0000","last_change":"2024-05-10T03:29:44.354749+0000","last_active":"2024-05-10T03:29:49.247953+0000","last_peered":"2024-05-10T03:29:49.247953+0000","last_clean":"2024-05-10T03:29:49.247953+0000","last_became_active":"2024-05-10T03:29:44.354474+0000","last_became_peered":"2024-05-10T03:29:44.354474+0000","last_unstale":"2024-05-10T03:29:49.247953+0000","last_undegraded":"2024-05-10T03:29:49.247953+0000","last_fullsized":"2024-05-10T03:29:49.247953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780909+0000","last_change":"2024-05-10T03:29:44.339173+0000","last_active":"2024-05-10T03:29:48.780909+0000","last_peered":"2024-05-10T03:29:48.780909+0000","last_clean":"2024-05-10T03:29:48.780909+0000","last_became_active":"2024-05-10T03:29:44.338815+0000","last_became_peered":"2024-05-10T03:29:44.338815+0000","last_unstale":"2024-05-10T03:29:48.780909+0000","last_undegraded":"2024-05-10T03:29:48.780909+0000","last_fullsized":"2024-05-10T03:29:48.780909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799655+0000","last_change":"2024-05-10T03:29:44.334824+0000","last_active":"2024-05-10T03:29:48.799655+0000","last_peered":"2024-05-10T03:29:48.799655+0000","last_clean":"2024-05-10T03:29:48.799655+0000","last_became_active":"2024-05-10T03:29:44.333616+0000","last_became_peered":"2024-05-10T03:29:44.333616+0000","last_unstale":"2024-05-10T03:29:48.799655+0000","last_undegraded":"2024-05-10T03:29:48.799655+0000","last_fullsized":"2024-05-10T03:29:48.799655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:38.441292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.785927+0000","last_change":"2024-05-10T03:29:44.339089+0000","last_active":"2024-05-10T03:29:48.785927+0000","last_peered":"2024-05-10T03:29:48.785927+0000","last_clean":"2024-05-10T03:29:48.785927+0000","last_became_active":"2024-05-10T03:29:44.336113+0000","last_became_peered":"2024-05-10T03:29:44.336113+0000","last_unstale":"2024-05-10T03:29:48.785927+0000","last_undegraded":"2024-05-10T03:29:48.785927+0000","last_fullsized":"2024-05-10T03:29:48.785927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794173+0000","last_change":"2024-05-10T03:29:44.340483+0000","last_active":"2024-05-10T03:29:48.794173+0000","last_peered":"2024-05-10T03:29:48.794173+0000","last_clean":"2024-05-10T03:29:48.794173+0000","last_became_active":"2024-05-10T03:29:44.340093+0000","last_became_peered":"2024-05-10T03:29:44.340093+0000","last_unstale":"2024-05-10T03:29:48.794173+0000","last_undegraded":"2024-05-10T03:29:48.794173+0000","last_fullsized":"2024-05-10T03:29:48.794173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790438+0000","last_change":"2024-05-10T03:29:44.345377+0000","last_active":"2024-05-10T03:29:48.790438+0000","last_peered":"2024-05-10T03:29:48.790438+0000","last_clean":"2024-05-10T03:29:48.790438+0000","last_became_active":"2024-05-10T03:29:44.345048+0000","last_became_peered":"2024-05-10T03:29:44.345048+0000","last_unstale":"2024-05-10T03:29:48.790438+0000","last_undegraded":"2024-05-10T03:29:48.790438+0000","last_fullsized":"2024-05-10T03:29:48.790438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:36.993303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794132+0000","last_change":"2024-05-10T03:29:44.349235+0000","last_active":"2024-05-10T03:29:48.794132+0000","last_peered":"2024-05-10T03:29:48.794132+0000","last_clean":"2024-05-10T03:29:48.794132+0000","last_became_active":"2024-05-10T03:29:44.348922+0000","last_became_peered":"2024-05-10T03:29:44.348922+0000","last_unstale":"2024-05-10T03:29:48.794132+0000","last_undegraded":"2024-05-10T03:29:48.794132+0000","last_fullsized":"2024-05-10T03:29:48.794132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791228+0000","last_change":"2024-05-10T03:29:44.342929+0000","last_active":"2024-05-10T03:29:48.791228+0000","last_peered":"2024-05-10T03:29:48.791228+0000","last_clean":"2024-05-10T03:29:48.791228+0000","last_became_active":"2024-05-10T03:29:44.342505+0000","last_became_peered":"2024-05-10T03:29:44.342505+0000","last_unstale":"2024-05-10T03:29:48.791228+0000","last_undegraded":"2024-05-10T03:29:48.791228+0000","last_fullsized":"2024-05-10T03:29:48.791228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811605+0000","last_change":"2024-05-10T03:29:44.338969+0000","last_active":"2024-05-10T03:29:48.811605+0000","last_peered":"2024-05-10T03:29:48.811605+0000","last_clean":"2024-05-10T03:29:48.811605+0000","last_became_active":"2024-05-10T03:29:44.337703+0000","last_became_peered":"2024-05-10T03:29:44.337703+0000","last_unstale":"2024-05-10T03:29:48.811605+0000","last_undegraded":"2024-05-10T03:29:48.811605+0000","last_fullsized":"2024-05-10T03:29:48.811605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812225+0000","last_change":"2024-05-10T03:29:44.348810+0000","last_active":"2024-05-10T03:29:48.812225+0000","last_peered":"2024-05-10T03:29:48.812225+0000","last_clean":"2024-05-10T03:29:48.812225+0000","last_became_active":"2024-05-10T03:29:44.348512+0000","last_became_peered":"2024-05-10T03:29:44.348512+0000","last_unstale":"2024-05-10T03:29:48.812225+0000","last_undegraded":"2024-05-10T03:29:48.812225+0000","last_fullsized":"2024-05-10T03:29:48.812225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249372+0000","last_change":"2024-05-10T03:29:44.344167+0000","last_active":"2024-05-10T03:29:49.249372+0000","last_peered":"2024-05-10T03:29:49.249372+0000","last_clean":"2024-05-10T03:29:49.249372+0000","last_became_active":"2024-05-10T03:29:44.343405+0000","last_became_peered":"2024-05-10T03:29:44.343405+0000","last_unstale":"2024-05-10T03:29:49.249372+0000","last_undegraded":"2024-05-10T03:29:49.249372+0000","last_fullsized":"2024-05-10T03:29:49.249372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808451+0000","last_change":"2024-05-10T03:29:44.337141+0000","last_active":"2024-05-10T03:29:48.808451+0000","last_peered":"2024-05-10T03:29:48.808451+0000","last_clean":"2024-05-10T03:29:48.808451+0000","last_became_active":"2024-05-10T03:29:44.336801+0000","last_became_peered":"2024-05-10T03:29:44.336801+0000","last_unstale":"2024-05-10T03:29:48.808451+0000","last_undegraded":"2024-05-10T03:29:48.808451+0000","last_fullsized":"2024-05-10T03:29:48.808451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242013+0000","last_change":"2024-05-10T03:29:44.339252+0000","last_active":"2024-05-10T03:29:49.242013+0000","last_peered":"2024-05-10T03:29:49.242013+0000","last_clean":"2024-05-10T03:29:49.242013+0000","last_became_active":"2024-05-10T03:29:44.338870+0000","last_became_peered":"2024-05-10T03:29:44.338870+0000","last_unstale":"2024-05-10T03:29:49.242013+0000","last_undegraded":"2024-05-10T03:29:49.242013+0000","last_fullsized":"2024-05-10T03:29:49.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792099+0000","last_change":"2024-05-10T03:29:44.341080+0000","last_active":"2024-05-10T03:29:48.792099+0000","last_peered":"2024-05-10T03:29:48.792099+0000","last_clean":"2024-05-10T03:29:48.792099+0000","last_became_active":"2024-05-10T03:29:44.340720+0000","last_became_peered":"2024-05-10T03:29:44.340720+0000","last_unstale":"2024-05-10T03:29:48.792099+0000","last_undegraded":"2024-05-10T03:29:48.792099+0000","last_fullsized":"2024-05-10T03:29:48.792099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241944+0000","last_change":"2024-05-10T03:29:44.345293+0000","last_active":"2024-05-10T03:29:49.241944+0000","last_peered":"2024-05-10T03:29:49.241944+0000","last_clean":"2024-05-10T03:29:49.241944+0000","last_became_active":"2024-05-10T03:29:44.344092+0000","last_became_peered":"2024-05-10T03:29:44.344092+0000","last_unstale":"2024-05-10T03:29:49.241944+0000","last_undegraded":"2024-05-10T03:29:49.241944+0000","last_fullsized":"2024-05-10T03:29:49.241944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797675+0000","last_change":"2024-05-10T03:29:44.359220+0000","last_active":"2024-05-10T03:29:48.797675+0000","last_peered":"2024-05-10T03:29:48.797675+0000","last_clean":"2024-05-10T03:29:48.797675+0000","last_became_active":"2024-05-10T03:29:44.358849+0000","last_became_peered":"2024-05-10T03:29:44.358849+0000","last_unstale":"2024-05-10T03:29:48.797675+0000","last_undegraded":"2024-05-10T03:29:48.797675+0000","last_fullsized":"2024-05-10T03:29:48.797675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:54.507386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795357+0000","last_change":"2024-05-10T03:29:44.349124+0000","last_active":"2024-05-10T03:29:48.795357+0000","last_peered":"2024-05-10T03:29:48.795357+0000","last_clean":"2024-05-10T03:29:48.795357+0000","last_became_active":"2024-05-10T03:29:44.348841+0000","last_became_peered":"2024-05-10T03:29:44.348841+0000","last_unstale":"2024-05-10T03:29:48.795357+0000","last_undegraded":"2024-05-10T03:29:48.795357+0000","last_fullsized":"2024-05-10T03:29:48.795357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795763+0000","last_change":"2024-05-10T03:29:44.342192+0000","last_active":"2024-05-10T03:29:48.795763+0000","last_peered":"2024-05-10T03:29:48.795763+0000","last_clean":"2024-05-10T03:29:48.795763+0000","last_became_active":"2024-05-10T03:29:44.341267+0000","last_became_peered":"2024-05-10T03:29:44.341267+0000","last_unstale":"2024-05-10T03:29:48.795763+0000","last_undegraded":"2024-05-10T03:29:48.795763+0000","last_fullsized":"2024-05-10T03:29:48.795763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249243+0000","last_change":"2024-05-10T03:29:44.344420+0000","last_active":"2024-05-10T03:29:49.249243+0000","last_peered":"2024-05-10T03:29:49.249243+0000","last_clean":"2024-05-10T03:29:49.249243+0000","last_became_active":"2024-05-10T03:29:44.343841+0000","last_became_peered":"2024-05-10T03:29:44.343841+0000","last_unstale":"2024-05-10T03:29:49.249243+0000","last_undegraded":"2024-05-10T03:29:49.249243+0000","last_fullsized":"2024-05-10T03:29:49.249243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.929}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.113}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.661}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7729999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:30:10.474 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:30:10.474 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T03:30:10.474 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-10T03:30:10.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-10T03:30:10.475 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-10T03:30:10.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.796+0000 7fe1c1400640 1 Processor -- start 2024-05-10T03:30:10.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.796+0000 7fe1c1400640 1 -- start start 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.798+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc0b2170 0x7fe1bc15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.798+0000 7fe1c1400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1bc1618b0 0x7fe1bc06aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.798+0000 7fe1c1400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0d25a0 con 0x7fe1bc0b6680 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1c1400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc0d2b20 con 0x7fe1bc1618b0 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1c1400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15db20 con 0x7fe1bc0b2170 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1bbe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1bc0b6680 0x7fe1bc1611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58266/0 (socket says 172.21.15.55:58266) 2024-05-10T03:30:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1bbe00640 1 -- 172.21.15.55:0/2258534631 learned_addr learned my addr 172.21.15.55:0/2258534631 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1bb400640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc0b2170 0x7fe1bc15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.799+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2258534631 >> v2:172.21.15.162:3300/0 conn(0x7fe1bc1618b0 0x7fe1bc06aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199429443 0 0) 0x7fe1bc0d25a0 con 0x7fe1bc0b6680 2024-05-10T03:30:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a0003520 con 0x7fe1bc0b6680 2024-05-10T03:30:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971001026 0 0) 0x7fe1a0003520 con 0x7fe1bc0b6680 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc0b2170 msgr2=0x7fe1bc15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc0b2170 0x7fe1bc15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 >> v2:172.21.15.162:3300/0 conn(0x7fe1bc1618b0 msgr2=0x7fe1bc06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 --2- 172.21.15.55:0/2258534631 >> v2:172.21.15.162:3300/0 conn(0x7fe1bc1618b0 0x7fe1bc06aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0b28d0 con 0x7fe1bc0b6680 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe1ac001670 con 0x7fe1bc0b6680 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 >> v1:172.21.15.55:6789/0 conn(0x7fe1bc0b6680 legacy=0x7fe1bc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1a00039e0 0x7fe1a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a00047c0 0x7fe1a0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1a0005110 0x7fe1a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0d2b20 con 0x7fe1a0005110 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15db20 con 0x7fe1a00047c0 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.800+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a000bfa0 con 0x7fe1a00039e0 2024-05-10T03:30:10.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1a0005110 0x7fe1a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1bbe00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1a00039e0 0x7fe1a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1bb400640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a00047c0 0x7fe1a0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1a00039e0 msgr2=0x7fe1a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1a00039e0 0x7fe1a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a00047c0 msgr2=0x7fe1a0004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a00047c0 0x7fe1a0004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.801+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2258534631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0003520 con 0x7fe1a0005110 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.802+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1a0005110 0x7fe1a000b980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc162280 tx=0x7fe1a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ef447725355976 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4031400 con 0x7fe1a0005110 2024-05-10T03:30:10.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b402d050 con 0x7fe1a0005110 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1baa00640 1 -- 172.21.15.55:0/2258534631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40317a0 con 0x7fe1a0005110 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1c1400640 1 -- 172.21.15.55:0/2258534631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1a0005110 msgr2=0x7fe1a000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2258534631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1a0005110 0x7fe1a000b980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc162280 tx=0x7fe1a00057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.803+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2258534631 reap_dead start 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.804+0000 7fe1c1400640 1 -- 172.21.15.55:0/2258534631 shutdown_connections 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.804+0000 7fe1c1400640 1 -- 172.21.15.55:0/2258534631 >> 172.21.15.55:0/2258534631 conn(0x7fe1bc0b5730 msgr2=0x7fe1bc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.804+0000 7fe1c1400640 1 -- 172.21.15.55:0/2258534631 shutdown_connections 2024-05-10T03:30:10.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.804+0000 7fe1c1400640 1 -- 172.21.15.55:0/2258534631 wait complete. 2024-05-10T03:30:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.806+0000 7fe1c1400640 1 Processor -- start 2024-05-10T03:30:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.806+0000 7fe1c1400640 1 -- start start 2024-05-10T03:30:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a0005110 0x7fe1bc1e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc15f540 0x7fe1bc15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc14f1d0 con 0x7fe1bc15fe90 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc151c50 con 0x7fe1a0005110 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c1400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc160880 con 0x7fe1bc15f540 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45964/0 (socket says 172.21.15.55:45964) 2024-05-10T03:30:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.808+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2124596323 learned_addr learned my addr 172.21.15.55:0/2124596323 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:10.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1bbe00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a0005110 0x7fe1bc1e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1bb400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc15f540 0x7fe1bc15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc15f540 msgr2=0x7fe1bc15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc15f540 0x7fe1bc15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a0005110 msgr2=0x7fe1bc1e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a0005110 0x7fe1bc1e75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 -- 172.21.15.55:0/2124596323 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc154700 con 0x7fe1bc15fe90 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c0a00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4034040 tx=0x7fe1b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b403a600 con 0x7fe1bc15fe90 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b403a7a0 con 0x7fe1bc15fe90 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.809+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc154950 con 0x7fe1bc15fe90 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.810+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b403ab50 con 0x7fe1bc15fe90 2024-05-10T03:30:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.810+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc15ed40 con 0x7fe1bc15fe90 2024-05-10T03:30:10.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.811+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc067740 con 0x7fe1bc15fe90 2024-05-10T03:30:10.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.812+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe1b4038040 con 0x7fe1bc15fe90 2024-05-10T03:30:10.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.813+0000 7fe1b3e00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 0x7fe19806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:10.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.813+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe1b40a9730 con 0x7fe1bc15fe90 2024-05-10T03:30:10.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.813+0000 7fe1bbe00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 0x7fe19806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:10.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.815+0000 7fe1bbe00640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 0x7fe19806d140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac000c00 tx=0x7fe1ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:10.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:10.818+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe1b4074580 con 0x7fe1bc15fe90 2024-05-10T03:30:11.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.073+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe1bc154e70 con 0x7fe1bc15fe90 2024-05-10T03:30:11.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.082+0000 7fe1b3e00640 1 -- 172.21.15.55:0/2124596323 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170943 (secure 0 0 0) 0x7fe1bc154e70 con 0x7fe1bc15fe90 2024-05-10T03:30:11.088 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 msgr2=0x7fe19806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 0x7fe19806d140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac000c00 tx=0x7fe1ac009040 comp rx=0 tx=0).stop 2024-05-10T03:30:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 msgr2=0x7fe1bc1534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4034040 tx=0x7fe1b4004730 comp rx=0 tx=0).stop 2024-05-10T03:30:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 shutdown_connections 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1bc15fe90 0x7fe1bc1534b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1bc15f540 0x7fe1bc15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe19806ac30 0x7fe19806d140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 --2- 172.21.15.55:0/2124596323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1a0005110 0x7fe1bc1e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 >> 172.21.15.55:0/2124596323 conn(0x7fe1bc0b5730 msgr2=0x7fe1bc0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 shutdown_connections 2024-05-10T03:30:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.088+0000 7fe1c1400640 1 -- 172.21.15.55:0/2124596323 wait complete. 2024-05-10T03:30:11.120 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":34944},{"type":"v1","addr":"172.21.15.162:6833","nonce":34944}]},"active_addr":"172.21.15.162:6833/34944","active_change":"2024-05-10T03:29:22.909091+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2077348968}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":167549326}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3816131784}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":4159359685}]}]} 2024-05-10T03:30:11.124 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-10T03:30:11.124 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-10T03:30:11.125 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:11.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.395+0000 7f8332000640 1 Processor -- start 2024-05-10T03:30:11.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.395+0000 7f8332000640 1 -- start start 2024-05-10T03:30:11.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f8332000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f832c0b2170 0x7f832c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f8332000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0d25a0 con 0x7f832c0b6680 2024-05-10T03:30:11.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f8332000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0b28d0 con 0x7f832c1618b0 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f8332000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0d2b20 con 0x7f832c0b2170 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f832be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f832c0b2170 0x7f832c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f832be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f832c0b2170 0x7f832c15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45968/0 (socket says 172.21.15.55:45968) 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.398+0000 7f832be00640 1 -- 172.21.15.55:0/554518191 learned_addr learned my addr 172.21.15.55:0/554518191 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.399+0000 7f832be00640 1 -- 172.21.15.55:0/554518191 >> v1:172.21.15.162:6789/0 conn(0x7f832c1618b0 legacy=0x7f832c06aab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.399+0000 7f832be00640 1 -- 172.21.15.55:0/554518191 >> v1:172.21.15.55:6789/0 conn(0x7f832c0b6680 legacy=0x7f832c1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.399+0000 7f832be00640 1 -- 172.21.15.55:0/554518191 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0d2840 con 0x7f832c0b2170 2024-05-10T03:30:11.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.400+0000 7f832be00640 1 --2- 172.21.15.55:0/554518191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c0b2170 0x7f832c15d5e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1deda07f3ffe1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.400+0000 7f832b400640 1 -- 172.21.15.55:0/554518191 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318032440 con 0x7f832c0b2170 2024-05-10T03:30:11.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.400+0000 7f832b400640 1 -- 172.21.15.55:0/554518191 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83180325a0 con 0x7f832c0b2170 2024-05-10T03:30:11.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.400+0000 7f832b400640 1 -- 172.21.15.55:0/554518191 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318032930 con 0x7f832c0b2170 2024-05-10T03:30:11.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 -- 172.21.15.55:0/554518191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c0b2170 msgr2=0x7f832c15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 --2- 172.21.15.55:0/554518191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c0b2170 0x7f832c15d5e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802a2a0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 -- 172.21.15.55:0/554518191 shutdown_connections 2024-05-10T03:30:11.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 --2- 172.21.15.55:0/554518191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c0b2170 0x7f832c15d5e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 -- 172.21.15.55:0/554518191 >> 172.21.15.55:0/554518191 conn(0x7f832c0b5730 msgr2=0x7f832c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:11.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 -- 172.21.15.55:0/554518191 shutdown_connections 2024-05-10T03:30:11.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.401+0000 7f8332000640 1 -- 172.21.15.55:0/554518191 wait complete. 2024-05-10T03:30:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.403+0000 7f8332000640 1 Processor -- start 2024-05-10T03:30:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.404+0000 7f8332000640 1 -- start start 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f832c0b2170 0x7f832c14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f832c0b6680 0x7f832c0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15db20 con 0x7f832c1618b0 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c8ae0 con 0x7f832c0b6680 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8332000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0cc810 con 0x7f832c0b2170 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8331600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8331600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45974/0 (socket says 172.21.15.55:45974) 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.405+0000 7f8331600640 1 -- 172.21.15.55:0/1278902464 learned_addr learned my addr 172.21.15.55:0/1278902464 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f832be00640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f832c0b6680 0x7f832c0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 -- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f832c0b2170 msgr2=0x7f832c14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f832c0b2170 0x7f832c14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 -- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f832c0b6680 msgr2=0x7f832c0cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f832c0b6680 0x7f832c0cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 -- 172.21.15.55:0/1278902464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180094d0 con 0x7f832c1618b0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.406+0000 7f8331600640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f832400b9c0 tx=0x7f832400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.407+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400cba0 con 0x7f832c1618b0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.407+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400cd00 con 0x7f832c1618b0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.407+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324014680 con 0x7f832c1618b0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.407+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0c7310 con 0x7f832c1618b0 2024-05-10T03:30:11.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.407+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0c6d00 con 0x7f832c1618b0 2024-05-10T03:30:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.408+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0c74a0 con 0x7f832c1618b0 2024-05-10T03:30:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.409+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f832401d3b0 con 0x7f832c1618b0 2024-05-10T03:30:11.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.410+0000 7f8328c00640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 0x7f830c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.410+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f83240895e0 con 0x7f832c1618b0 2024-05-10T03:30:11.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.410+0000 7f8330c00640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 0x7f830c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.412+0000 7f8330c00640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 0x7f830c06d140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8320000f60 tx=0x7f83200079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.415+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f83240544d0 con 0x7f832c1618b0 2024-05-10T03:30:11.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.618+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f832c14fc60 con 0x7f832c1618b0 2024-05-10T03:30:11.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.621+0000 7f8328c00640 1 -- 172.21.15.55:0/1278902464 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f8324057af0 con 0x7f832c1618b0 2024-05-10T03:30:11.626 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:11.626 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 msgr2=0x7f830c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 0x7f830c06d140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8320000f60 tx=0x7f83200079b0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 msgr2=0x7f832c160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f832400b9c0 tx=0x7f832400be80 comp rx=0 tx=0).stop 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 shutdown_connections 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f832c1618b0 0x7f832c160180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f830c06ac30 0x7f830c06d140 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f832c0b6680 0x7f832c0cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 --2- 172.21.15.55:0/1278902464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f832c0b2170 0x7f832c14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.624+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 >> 172.21.15.55:0/1278902464 conn(0x7f832c0b5730 msgr2=0x7f832c0cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:11.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.625+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 shutdown_connections 2024-05-10T03:30:11.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.625+0000 7f8332000640 1 -- 172.21.15.55:0/1278902464 wait complete. 2024-05-10T03:30:11.655 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-10T03:30:11.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:11.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.935+0000 7fa8d7200640 1 Processor -- start 2024-05-10T03:30:11.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.935+0000 7fa8d7200640 1 -- start start 2024-05-10T03:30:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8d815d970 0x7fa8d80b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d80b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d80b2aa0 con 0x7fa8d80b3d30 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d80b6190 con 0x7fa8d815d970 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80b6c30 con 0x7fa8d80ce3a0 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8d815d970 0x7fa8d80b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d6800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa8d80b3d30 0x7fa8d8161370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58286/0 (socket says 172.21.15.55:58286) 2024-05-10T03:30:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d6800640 1 -- 172.21.15.55:0/1754649271 learned_addr learned my addr 172.21.15.55:0/1754649271 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.938+0000 7fa8d5400640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d80b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d5e00640 1 -- 172.21.15.55:0/1754649271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 msgr2=0x7fa8d80b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d5e00640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d80b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d5e00640 1 -- 172.21.15.55:0/1754649271 >> v1:172.21.15.55:6789/0 conn(0x7fa8d80b3d30 legacy=0x7fa8d8161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d4a00640 1 -- 172.21.15.55:0/1754649271 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91448395 0 0) 0x7fa8d80b2aa0 con 0x7fa8d80b3d30 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d5e00640 1 -- 172.21.15.55:0/1754649271 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80b2d10 con 0x7fa8d815d970 2024-05-10T03:30:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.939+0000 7fa8d5400640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d80b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.940+0000 7fa8d5e00640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d80b66f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80097a0 tx=0x7fa8c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9059196351f9884 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.940+0000 7fa8d4a00640 1 -- 172.21.15.55:0/1754649271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8032440 con 0x7fa8d815d970 2024-05-10T03:30:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.940+0000 7fa8d4a00640 1 -- 172.21.15.55:0/1754649271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c80325a0 con 0x7fa8d815d970 2024-05-10T03:30:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.940+0000 7fa8d4a00640 1 -- 172.21.15.55:0/1754649271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8032930 con 0x7fa8d815d970 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 -- 172.21.15.55:0/1754649271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 msgr2=0x7fa8d80b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d80b66f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80097a0 tx=0x7fa8c802a180 comp rx=0 tx=0).stop 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 -- 172.21.15.55:0/1754649271 shutdown_connections 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d80b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 --2- 172.21.15.55:0/1754649271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d80b66f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 -- 172.21.15.55:0/1754649271 >> 172.21.15.55:0/1754649271 conn(0x7fa8d80d0370 msgr2=0x7fa8d80d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 -- 172.21.15.55:0/1754649271 shutdown_connections 2024-05-10T03:30:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.941+0000 7fa8d7200640 1 -- 172.21.15.55:0/1754649271 wait complete. 2024-05-10T03:30:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.944+0000 7fa8d7200640 1 Processor -- start 2024-05-10T03:30:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.944+0000 7fa8d7200640 1 -- start start 2024-05-10T03:30:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d80b3d30 0x7fa8d81534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d8160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80cab20 con 0x7fa8d815d970 2024-05-10T03:30:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80c6d00 con 0x7fa8d80b3d30 2024-05-10T03:30:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.945+0000 7fa8d7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80cac90 con 0x7fa8d80ce3a0 2024-05-10T03:30:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45986/0 (socket says 172.21.15.55:45986) 2024-05-10T03:30:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 -- 172.21.15.55:0/996427952 learned_addr learned my addr 172.21.15.55:0/996427952 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d5e00640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d80b3d30 0x7fa8d81534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d5400640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 -- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 msgr2=0x7fa8d8160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d8160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 -- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d80b3d30 msgr2=0x7fa8d81534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d80b3d30 0x7fa8d81534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.946+0000 7fa8d6800640 1 -- 172.21.15.55:0/996427952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009400 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.947+0000 7fa8d6800640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c410 tx=0x7fa8d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.947+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00074e0 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.947+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d0007640 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.947+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d80cd000 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.947+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004a20 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.948+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d81539f0 con 0x7fa8d815d970 2024-05-10T03:30:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.949+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d8162310 con 0x7fa8d815d970 2024-05-10T03:30:11.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.950+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa8d0004b80 con 0x7fa8d815d970 2024-05-10T03:30:11.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.951+0000 7fa8ce000640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 0x7fa8b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:11.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.951+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8d0089390 con 0x7fa8d815d970 2024-05-10T03:30:11.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.952+0000 7fa8d5e00640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 0x7fa8b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.953+0000 7fa8d5e00640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 0x7fa8b406d110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8034000 tx=0x7fa8c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:11.956+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa8d0054210 con 0x7fa8d815d970 2024-05-10T03:30:12.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.159+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d804e290 con 0x7fa8d815d970 2024-05-10T03:30:12.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.162+0000 7fa8ce000640 1 -- 172.21.15.55:0/996427952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7fa8d0057830 con 0x7fa8d815d970 2024-05-10T03:30:12.167 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:12.167 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:12.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 msgr2=0x7fa8b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 0x7fa8b406d110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8034000 tx=0x7fa8c802e000 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 msgr2=0x7fa8d81584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000c410 tx=0x7fa8d000c8d0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 shutdown_connections 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d815d970 0x7fa8d81584e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8d80ce3a0 0x7fa8d8160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8b406ac00 0x7fa8b406d110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 --2- 172.21.15.55:0/996427952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8d80b3d30 0x7fa8d81534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 >> 172.21.15.55:0/996427952 conn(0x7fa8d80d0370 msgr2=0x7fa8d80b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 shutdown_connections 2024-05-10T03:30:12.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.165+0000 7fa8d7200640 1 -- 172.21.15.55:0/996427952 wait complete. 2024-05-10T03:30:12.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-10T03:30:12.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-10T03:30:12.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-10T03:30:12.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-10T03:30:12.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-10T03:30:12.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-10T03:30:12.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-10T03:30:12.200 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-10T03:30:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.615+0000 7feaa4400640 1 Processor -- start 2024-05-10T03:30:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.615+0000 7fabc0c00640 1 Processor -- start 2024-05-10T03:30:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.616+0000 7fabc0c00640 1 -- start start 2024-05-10T03:30:12.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.616+0000 7fa907200640 1 Processor -- start 2024-05-10T03:30:12.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.616+0000 7feaa4400640 1 -- start start 2024-05-10T03:30:12.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.617+0000 7fa907200640 1 -- start start 2024-05-10T03:30:12.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fabc0c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fabb40e3850 0x7fabb4054060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fa907200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 0x7fa90815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7fa907200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9080b2170 0x7fa9081611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 0x7fabb403d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fabc0c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40549b0 con 0x7fabb40545a0 2024-05-10T03:30:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fabc0c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4057700 con 0x7fabb40e3850 2024-05-10T03:30:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.619+0000 7fabc0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40e3c60 con 0x7fabb4007420 2024-05-10T03:30:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7feaa4400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fea9c0ccad0 0x7fea9c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7fa907200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90806bac0 con 0x7fa9081616e0 2024-05-10T03:30:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7fabb9400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fabb40545a0 0x7fabb4006d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58292/0 (socket says 172.21.15.55:58292) 2024-05-10T03:30:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7fabb9400640 1 -- 172.21.15.55:0/2458418709 learned_addr learned my addr 172.21.15.55:0/2458418709 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fabba800640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 0x7fabb403d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fa905e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 0x7fa90815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fa905e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 0x7fa90815d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37332/0 (socket says 172.21.15.55:37332) 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fa905e00640 1 -- 172.21.15.55:0/42373836 learned_addr learned my addr 172.21.15.55:0/42373836 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7feaa4400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea9c0ce2b0 0x7fea9c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7feaa4400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c165680 con 0x7fea9c06b120 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fabb9e00640 1 --2- 172.21.15.55:0/2458418709 >> v2:172.21.15.55:3300/0 conn(0x7fabb40e3850 0x7fabb4054060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7feaa4400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c166300 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.620+0000 7feaa4400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c071670 con 0x7fea9c0ccad0 2024-05-10T03:30:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080d2b20 con 0x7fa9080b2170 2024-05-10T03:30:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815db20 con 0x7fa9080b6680 2024-05-10T03:30:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808455430 0 0) 0x7fabb40549b0 con 0x7fabb40545a0 2024-05-10T03:30:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba0003520 con 0x7fabb40545a0 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/42373836 >> v2:172.21.15.162:3300/0 conn(0x7fa9080b2170 0x7fa9081611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941864962 0 0) 0x7faba0003520 con 0x7fabb40545a0 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 msgr2=0x7fabb403d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 0x7fabb403d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 >> v2:172.21.15.55:3300/0 conn(0x7fabb40e3850 msgr2=0x7fabb4054060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 --2- 172.21.15.55:0/2458418709 >> v2:172.21.15.55:3300/0 conn(0x7fabb40e3850 0x7fabb4054060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4007950 con 0x7fabb40545a0 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7feaa1000640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fea9c06b120 0x7fea9c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:47510/0 (socket says 172.21.15.55:47510) 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7feaa1000640 1 -- 172.21.15.55:0/497598172 learned_addr learned my addr 172.21.15.55:0/497598172 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.621+0000 7feaa2400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea9c0ce2b0 0x7fea9c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7feaa1a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fea9c0ccad0 0x7fea9c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fa8ffe00640 1 -- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 msgr2=0x7fa90815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 0x7fa90815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fa8ffe00640 1 -- 172.21.15.55:0/42373836 >> v1:172.21.15.162:6789/0 conn(0x7fa9081616e0 legacy=0x7fa9080cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fabac001670 con 0x7fabb40545a0 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.622+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 >> v1:172.21.15.55:6789/0 conn(0x7fabb40545a0 legacy=0x7fabb4006d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faba0003970 0x7faba0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 0x7faba00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faba0004d50 0x7faba00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4057700 con 0x7faba0004d50 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40e3c60 con 0x7faba0003970 2024-05-10T03:30:12.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba000bc40 con 0x7fabb4007420 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7feaa2400640 1 -- 172.21.15.55:0/497598172 >> v2:172.21.15.162:3300/0 conn(0x7fea9c0ccad0 msgr2=0x7fea9c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7feaa2400640 1 --2- 172.21.15.55:0/497598172 >> v2:172.21.15.162:3300/0 conn(0x7fea9c0ccad0 0x7fea9c0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fabba800640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faba0004d50 0x7faba00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.624+0000 7fabb9e00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faba0003970 0x7faba0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7feaa2400640 1 -- 172.21.15.55:0/497598172 >> v1:172.21.15.162:6789/0 conn(0x7fea9c06b120 legacy=0x7fea9c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7feaa2400640 1 -- 172.21.15.55:0/497598172 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c165960 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fa905400640 1 -- 172.21.15.55:0/42373836 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917775743 0 0) 0x7fa90806bac0 con 0x7fa9081616e0 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.623+0000 7fa8ffe00640 1 -- 172.21.15.55:0/42373836 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080d25a0 con 0x7fa9080b2170 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabba800640 1 -- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 msgr2=0x7faba00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabba800640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabb4007420 0x7faba00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabba800640 1 -- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faba0003970 msgr2=0x7faba0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabba800640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faba0003970 0x7faba0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabba800640 1 -- 172.21.15.55:0/2458418709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0003520 con 0x7faba0004d50 2024-05-10T03:30:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.625+0000 7fabb9e00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faba0003970 0x7faba0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.626+0000 7feaa2400640 1 --2- 172.21.15.55:0/497598172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ce2b0 0x7fea9c0ce6c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea9c071240 tx=0x7fea9402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7885f9446549df2c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.626+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b2170 0x7fa9081611a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00097a0 tx=0x7fa8f002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff4d45ea5758716 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.626+0000 7fea9be00640 1 -- 172.21.15.55:0/497598172 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9402aea0 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.626+0000 7fea9be00640 1 -- 172.21.15.55:0/497598172 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9402d050 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.626+0000 7fea9be00640 1 -- 172.21.15.55:0/497598172 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94007960 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fa905400640 1 -- 172.21.15.55:0/42373836 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0032440 con 0x7fa9080b2170 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fa905400640 1 -- 172.21.15.55:0/42373836 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f00325a0 con 0x7fa9080b2170 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fabba800640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faba0004d50 0x7faba00051e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabb4007880 tx=0x7faba00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a24309eb36da40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005be0 con 0x7faba0004d50 2024-05-10T03:30:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8005d40 con 0x7faba0004d50 2024-05-10T03:30:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.627+0000 7fabb8a00640 1 -- 172.21.15.55:0/2458418709 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba802ab50 con 0x7faba0004d50 2024-05-10T03:30:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fa905400640 1 -- 172.21.15.55:0/42373836 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0032440 con 0x7fa9080b2170 2024-05-10T03:30:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b2170 msgr2=0x7fa9081611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fa907200640 1 --2- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b2170 0x7fa9081611a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00097a0 tx=0x7fa8f002a180 comp rx=0 tx=0).stop 2024-05-10T03:30:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7feaa4400640 1 -- 172.21.15.55:0/497598172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ce2b0 msgr2=0x7fea9c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7feaa4400640 1 --2- 172.21.15.55:0/497598172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ce2b0 0x7fea9c0ce6c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea9c071240 tx=0x7fea9402a680 comp rx=0 tx=0).stop 2024-05-10T03:30:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fabc0c00640 1 -- 172.21.15.55:0/2458418709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faba0004d50 msgr2=0x7faba00051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fabc0c00640 1 --2- 172.21.15.55:0/2458418709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faba0004d50 0x7faba00051e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabb4007880 tx=0x7faba00057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.628+0000 7fabba800640 1 -- 172.21.15.55:0/2458418709 reap_dead start 2024-05-10T03:30:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7fabc0c00640 1 -- 172.21.15.55:0/2458418709 shutdown_connections 2024-05-10T03:30:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7fabc0c00640 1 -- 172.21.15.55:0/2458418709 >> 172.21.15.55:0/2458418709 conn(0x7fabb403ac10 msgr2=0x7fabb403b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 -- 172.21.15.55:0/497598172 shutdown_connections 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 --2- 172.21.15.55:0/497598172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ce2b0 0x7fea9c0ce6c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 --2- 172.21.15.55:0/497598172 >> v2:172.21.15.162:3300/0 conn(0x7fea9c0ccad0 0x7fea9c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 -- 172.21.15.55:0/497598172 >> 172.21.15.55:0/497598172 conn(0x7fea9c06c730 msgr2=0x7fea9c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 -- 172.21.15.55:0/497598172 shutdown_connections 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7feaa4400640 1 -- 172.21.15.55:0/497598172 wait complete. 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.629+0000 7fabc0c00640 1 -- 172.21.15.55:0/2458418709 shutdown_connections 2024-05-10T03:30:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.630+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 shutdown_connections 2024-05-10T03:30:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.630+0000 7fa907200640 1 --2- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b2170 0x7fa9081611a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.630+0000 7fabc0c00640 1 -- 172.21.15.55:0/2458418709 wait complete. 2024-05-10T03:30:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.630+0000 7fa907200640 1 --2- 172.21.15.55:0/42373836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9080b6680 0x7fa90815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.630+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 >> 172.21.15.55:0/42373836 conn(0x7fa9080b5730 msgr2=0x7fa9080d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.631+0000 7f0812600640 1 Processor -- start 2024-05-10T03:30:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.632+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 shutdown_connections 2024-05-10T03:30:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.632+0000 7fa907200640 1 -- 172.21.15.55:0/42373836 wait complete. 2024-05-10T03:30:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.631+0000 7f0812600640 1 -- start start 2024-05-10T03:30:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.633+0000 7feaa4400640 1 Processor -- start 2024-05-10T03:30:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.633+0000 7feaa4400640 1 -- start start 2024-05-10T03:30:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.633+0000 7fabc0c00640 1 Processor -- start 2024-05-10T03:30:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.634+0000 7fabc0c00640 1 -- start start 2024-05-10T03:30:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.635+0000 7f0812600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08040e3a60 0x7f0804057b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.635+0000 7f0812600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0804058b60 0x7f08040380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.635+0000 7f3a2fa00640 1 Processor -- start 2024-05-10T03:30:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.635+0000 7f0812600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0804057700 con 0x7f08040073a0 2024-05-10T03:30:12.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7feaa4400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea9c06b120 0x7fea9c0a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.635+0000 7f3a2fa00640 1 -- start start 2024-05-10T03:30:12.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7fa907200640 1 Processor -- start 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7f0812600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08040385f0 con 0x7f0804058b60 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7f0812600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0804056dd0 con 0x7f08040e3a60 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7f080be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0804058b60 0x7f08040380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.636+0000 7f0811200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08040e3a60 0x7f0804057b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7fa907200640 1 -- start start 2024-05-10T03:30:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7f0811c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f08040073a0 0x7f08040077b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58306/0 (socket says 172.21.15.55:58306) 2024-05-10T03:30:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7f0811c00640 1 -- 172.21.15.55:0/1589521074 learned_addr learned my addr 172.21.15.55:0/1589521074 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7feaa4400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ccad0 0x7fea9c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faba0004d50 0x7fabb41896c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 0x7fabb400e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291019114 0 0) 0x7f0804057700 con 0x7f08040073a0 2024-05-10T03:30:12.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f4003520 con 0x7f08040073a0 2024-05-10T03:30:12.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7feaa4400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea9c0ce2b0 0x7fea9c0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb4185cb0 0x7fabb400ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7feaa4400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c1f8fe0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7feaa4400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0aad10 con 0x7fea9c06b120 2024-05-10T03:30:12.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7feaa4400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0abb70 con 0x7fea9c0ce2b0 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabc0c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40da200 con 0x7fabb41858a0 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabc0c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40db960 con 0x7fabb4185cb0 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabc0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4189fe0 con 0x7faba0004d50 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabb9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 0x7fabb400e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.637+0000 7feaa1a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea9c06b120 0x7fea9c0a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7feaa1a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea9c06b120 0x7fea9c0a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33820/0 (socket says 172.21.15.55:33820) 2024-05-10T03:30:12.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7feaa1a00640 1 -- 172.21.15.55:0/2742920010 learned_addr learned my addr 172.21.15.55:0/2742920010 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabb9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 0x7fabb400e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46020/0 (socket says 172.21.15.55:46020) 2024-05-10T03:30:12.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7feaa1000640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ccad0 0x7fea9c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7feaa2400640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea9c0ce2b0 0x7fea9c0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.638+0000 7fabb9400640 1 -- 172.21.15.55:0/891217263 learned_addr learned my addr 172.21.15.55:0/891217263 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7fabb9e00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faba0004d50 0x7fabb41896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7fabba800640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb4185cb0 0x7fabb400ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2fa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a2806b6e0 0x7f3a280ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2fa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a280b2700 0x7f3a280b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2fa00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a280715e0 con 0x7f3a281659c0 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2fa00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a280711c0 con 0x7f3a2806b6e0 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2fa00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2806baf0 con 0x7f3a280b2700 2024-05-10T03:30:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f3a2d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a2806b6e0 0x7f3a280ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.639+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1135021458 0 0) 0x7f07f4003520 con 0x7f08040073a0 2024-05-10T03:30:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08040e3a60 msgr2=0x7f0804057b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08040e3a60 0x7f0804057b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 >> v2:172.21.15.162:3300/0 conn(0x7f0804058b60 msgr2=0x7f08040380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 --2- 172.21.15.55:0/1589521074 >> v2:172.21.15.162:3300/0 conn(0x7f0804058b60 0x7f08040380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040570b0 con 0x7f08040073a0 2024-05-10T03:30:12.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f3a2d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a2806b6e0 0x7f3a280ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46036/0 (socket says 172.21.15.55:46036) 2024-05-10T03:30:12.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f3a2d000640 1 -- 172.21.15.55:0/189358658 learned_addr learned my addr 172.21.15.55:0/189358658 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f080c074030 con 0x7f08040073a0 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 >> v1:172.21.15.55:6789/0 conn(0x7f08040073a0 legacy=0x7f08040077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07f4003970 0x7f07f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f07f4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07f4004ff0 0x7f07f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040385f0 con 0x7f07f4004360 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0804056dd0 con 0x7f07f4003970 2024-05-10T03:30:12.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f3a2da00640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a280b2700 0x7f3a280b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f400bfa0 con 0x7f07f4004ff0 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fabb9400640 1 -- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faba0004d50 msgr2=0x7fabb41896c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fabb9400640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faba0004d50 0x7fabb41896c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fabb9400640 1 -- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb4185cb0 msgr2=0x7fabb400ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fabb9400640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabb4185cb0 0x7fabb400ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7feaa1000640 1 -- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea9c0ce2b0 msgr2=0x7fea9c0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fabb9400640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40d9800 con 0x7fabb41858a0 2024-05-10T03:30:12.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7fc8b3400640 1 Processor -- start 2024-05-10T03:30:12.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7feaa1000640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea9c0ce2b0 0x7fea9c0b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.640+0000 7feaa1000640 1 -- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea9c06b120 msgr2=0x7fea9c0a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7feaa1000640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea9c06b120 0x7fea9c0a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7feaa1000640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94009a70 con 0x7fea9c0ccad0 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7f3a2d000640 1 -- 172.21.15.55:0/189358658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a280b2700 msgr2=0x7f3a280b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7f3a2d000640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a280b2700 0x7f3a280b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7f3a2d000640 1 -- 172.21.15.55:0/189358658 >> v1:172.21.15.162:6789/0 conn(0x7f3a281659c0 legacy=0x7f3a28165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7f3a2d000640 1 -- 172.21.15.55:0/189358658 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a281667f0 con 0x7f3a2806b6e0 2024-05-10T03:30:12.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7fa907200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9080b2170 0x7fa9081ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7fa907200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 0x7fa90806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7fabb9400640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 0x7fabb400e7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c730 tx=0x7fabac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7feaa1000640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ccad0 0x7fea9c0aff60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c470 tx=0x7fea9000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.641+0000 7f080be00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f07f4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa907200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9081616e0 0x7fa90806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa907200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908200450 con 0x7fa9080b2170 2024-05-10T03:30:12.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0074e0 con 0x7fabb41858a0 2024-05-10T03:30:12.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb418a230 con 0x7fabb41858a0 2024-05-10T03:30:12.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa907200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908070bc0 con 0x7fa9080b6680 2024-05-10T03:30:12.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa907200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908159000 con 0x7fa9081616e0 2024-05-10T03:30:12.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 0x7fa90806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 0x7fa90806d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33856/0 (socket says 172.21.15.55:33856) 2024-05-10T03:30:12.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa8ffe00640 1 -- 172.21.15.55:0/387728847 learned_addr learned my addr 172.21.15.55:0/387728847 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fc8b3400640 1 -- start start 2024-05-10T03:30:12.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7f0811200640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07f4003970 0x7f07f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb40d9b40 con 0x7fabb41858a0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac007640 con 0x7fabb41858a0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0074e0 con 0x7fabb41858a0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7fa905e00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9080b2170 0x7fa9081ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90005ce0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea90005e40 con 0x7fea9c0ccad0 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea900072d0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7f0811c00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07f4004ff0 0x7f07f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.642+0000 7f080be00640 1 -- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07f4004ff0 msgr2=0x7f07f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f080be00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07f4004ff0 0x7f07f400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f080be00640 1 -- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07f4003970 msgr2=0x7f07f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f080be00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07f4003970 0x7f07f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f080be00640 1 -- 172.21.15.55:0/1589521074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040570b0 con 0x7f07f4004360 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f0811c00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07f4004ff0 0x7f07f400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:12.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7f0811200640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07f4003970 0x7f07f4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:12.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fa906800640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9081616e0 0x7fa90806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fa8ffe00640 1 -- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9081616e0 msgr2=0x7fa90806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9081616e0 0x7fa90806ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fa8ffe00640 1 -- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9080b2170 msgr2=0x7fa9081ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9080b2170 0x7fa9081ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.644+0000 7fa8ffe00640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0009400 con 0x7fa9080b6680 2024-05-10T03:30:12.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.644+0000 7fa8ffe00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 0x7fa90806d870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0000c00 tx=0x7fa8f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.644+0000 7f57a2c00640 1 Processor -- start 2024-05-10T03:30:12.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7f57a2c00640 1 -- start start 2024-05-10T03:30:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7feaa4400640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c0aa4f0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.643+0000 7feaa4400640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0ab240 con 0x7fea9c0ccad0 2024-05-10T03:30:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.644+0000 7f3a2d000640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a2806b6e0 0x7f3a280ce3b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009870 tx=0x7f3a1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c995a728107f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0003a20 con 0x7fa9080b6680 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f0003b80 con 0x7fa9080b6680 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0006e10 con 0x7fa9080b6680 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908070e10 con 0x7fa9080b6680 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.645+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081592c0 con 0x7fa9080b6680 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.644+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab94000fc0 con 0x7fabb41858a0 2024-05-10T03:30:12.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7f080be00640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f07f4004880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0800009a90 tx=0x7f07f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f32214f6b8b1a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fc8b3400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fc8b3400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8ac0b2700 0x7fc8ac0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fc8b3400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac0715e0 con 0x7fc8ac06b6e0 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fc8b3400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac0711c0 con 0x7fc8ac0b2700 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fc8b3400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac06baf0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea70000fc0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa90806a2b0 con 0x7fa9080b6680 2024-05-10T03:30:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fc8abe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fc8abe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37404/0 (socket says 172.21.15.55:37404) 2024-05-10T03:30:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fc8abe00640 1 -- 172.21.15.55:0/1582363410 learned_addr learned my addr 172.21.15.55:0/1582363410 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.646+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fabac0047e0 con 0x7fabb41858a0 2024-05-10T03:30:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7fc8b1400640 1 --2- 172.21.15.55:0/1582363410 >> v2:172.21.15.162:3300/0 conn(0x7fc8ac0b2700 0x7fc8ac0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.647+0000 7f8ccf200640 1 Processor -- start 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7f8ccf200640 1 -- start start 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fabb2000640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab9c06ac60 0x7fab9c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabac089180 con 0x7fabb41858a0 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fabb2000640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fab9c074270 0x7fab9c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab9c076dd0 con 0x7fab9c074270 2024-05-10T03:30:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabac0895d0 con 0x7fabb41858a0 2024-05-10T03:30:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fc8abe00640 1 -- 172.21.15.55:0/1582363410 >> v2:172.21.15.162:3300/0 conn(0x7fc8ac0b2700 msgr2=0x7fc8ac0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fc8abe00640 1 --2- 172.21.15.55:0/1582363410 >> v2:172.21.15.162:3300/0 conn(0x7fc8ac0b2700 0x7fc8ac0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fc8abe00640 1 -- 172.21.15.55:0/1582363410 >> v1:172.21.15.162:6789/0 conn(0x7fc8ac06b6e0 legacy=0x7fc8ac0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7fc8abe00640 1 -- 172.21.15.55:0/1582363410 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac166310 con 0x7fc8ac1659c0 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7f57a2c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f57a2c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c06ad20 con 0x7f579c15fe50 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f57a2c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c06a110 con 0x7f579c0b2550 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f57a2c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c06a3f0 con 0x7f579c0b2140 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f579b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f579be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f579c15fe50 0x7f579c141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58310/0 (socket says 172.21.15.55:58310) 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f579be00640 1 -- 172.21.15.55:0/3343831240 learned_addr learned my addr 172.21.15.55:0/3343831240 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:12.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7fabba800640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fab9c074270 0x7fab9c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7fabb9e00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab9c06ac60 0x7fab9c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa8f0039040 con 0x7fa9080b6680 2024-05-10T03:30:12.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f3a27400640 1 -- 172.21.15.55:0/189358658 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1402ae90 con 0x7f3a2806b6e0 2024-05-10T03:30:12.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031400 con 0x7f07f4004360 2024-05-10T03:30:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080002d050 con 0x7f07f4004360 2024-05-10T03:30:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.648+0000 7f080b400640 1 -- 172.21.15.55:0/1589521074 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031780 con 0x7f07f4004360 2024-05-10T03:30:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f0812600640 1 -- 172.21.15.55:0/1589521074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 msgr2=0x7f07f4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.649+0000 7f0812600640 1 --2- 172.21.15.55:0/1589521074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f07f4004880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0800009a90 tx=0x7f07f40057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7fc8abe00640 1 --2- 172.21.15.55:0/1582363410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc898009870 tx=0x7fc89802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb89f1464c84b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f3a27400640 1 -- 172.21.15.55:0/189358658 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1402ae90 con 0x7f3a2806b6e0 2024-05-10T03:30:12.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f3a27400640 1 -- 172.21.15.55:0/189358658 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a14032440 con 0x7f3a2806b6e0 2024-05-10T03:30:12.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fea90007430 con 0x7fea9c0ccad0 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f8ccf200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8cd00cdea0 0x7f8cd00ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f8ccf200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd00b6d70 con 0x7f8cd00b7200 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f8ccf200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd015dab0 con 0x7f8cd015d6a0 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f8ccf200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd00ce7f0 con 0x7f8cd00cdea0 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f579b400640 1 -- 172.21.15.55:0/3343831240 >> v1:172.21.15.162:6789/0 conn(0x7f579c0b2550 legacy=0x7f579c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f579b400640 1 -- 172.21.15.55:0/3343831240 >> v1:172.21.15.55:6789/0 conn(0x7f579c15fe50 legacy=0x7f579c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.650+0000 7f579b400640 1 -- 172.21.15.55:0/3343831240 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c06a6d0 con 0x7f579c0b2140 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f0811c00640 1 -- 172.21.15.55:0/1589521074 reap_dead start 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f0812600640 1 -- 172.21.15.55:0/1589521074 shutdown_connections 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f0812600640 1 -- 172.21.15.55:0/1589521074 >> 172.21.15.55:0/1589521074 conn(0x7f080403abc0 msgr2=0x7f080403afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f0812600640 1 -- 172.21.15.55:0/1589521074 shutdown_connections 2024-05-10T03:30:12.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f0812600640 1 -- 172.21.15.55:0/1589521074 wait complete. 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f8cce800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8cd00b7200 0x7f8cd00b7610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58314/0 (socket says 172.21.15.55:58314) 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f8cce800640 1 -- 172.21.15.55:0/3496604798 learned_addr learned my addr 172.21.15.55:0/3496604798 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f8ccde00640 1 --1- 172.21.15.55:0/3496604798 >> v1:172.21.15.162:6789/0 conn(0x7f8cd015d6a0 0x7f8cd006ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:47550/0 (socket says 172.21.15.55:47550) 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fabb9e00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab9c06ac60 0x7fab9c06d170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0798d0 tx=0x7fabbc04f270 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fabba800640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fab9c074270 0x7fab9c076700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f3a2fa00640 1 -- 172.21.15.55:0/189358658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a2806b6e0 msgr2=0x7f3a280ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a2806b6e0 0x7f3a280ce3b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009870 tx=0x7f3a1402a180 comp rx=0 tx=0).stop 2024-05-10T03:30:12.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 -- 172.21.15.55:0/189358658 shutdown_connections 2024-05-10T03:30:12.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.651+0000 7f8ccd400640 1 --2- 172.21.15.55:0/3496604798 >> v2:172.21.15.162:3300/0 conn(0x7f8cd00cdea0 0x7f8cd00ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a280b2700 0x7f3a280b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/189358658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a2806b6e0 0x7f3a280ce3b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 -- 172.21.15.55:0/189358658 >> 172.21.15.55:0/189358658 conn(0x7f3a280cbb90 msgr2=0x7f3a280cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f579b400640 1 --2- 172.21.15.55:0/3343831240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c142440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57880097d0 tx=0x7f578802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f09f2a7b0313a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 -- 172.21.15.55:0/189358658 shutdown_connections 2024-05-10T03:30:12.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766250080 0 0) 0x7f8cd00b6d70 con 0x7f8cd00b7200 2024-05-10T03:30:12.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb4003520 con 0x7f8cd00b7200 2024-05-10T03:30:12.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f3a2fa00640 1 -- 172.21.15.55:0/189358658 wait complete. 2024-05-10T03:30:12.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399872268 0 0) 0x7f8cd015dab0 con 0x7f8cd015d6a0 2024-05-10T03:30:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fa8fe000640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8e406ac50 0x7fa8e406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8f00aa700 con 0x7fa9080b6680 2024-05-10T03:30:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fa8fe000640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7fa8e40742e0 0x7fa8e4076770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8e4076e40 con 0x7fa8e40742e0 2024-05-10T03:30:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8f00aaa70 con 0x7fa9080b6680 2024-05-10T03:30:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd00b6d70 con 0x7f8cd015d6a0 2024-05-10T03:30:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.652+0000 7fc8ab400640 1 -- 172.21.15.55:0/1582363410 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89802ae90 con 0x7fc8ac1659c0 2024-05-10T03:30:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7fc8ab400640 1 -- 172.21.15.55:0/1582363410 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898032440 con 0x7fc8ac1659c0 2024-05-10T03:30:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7fc8ab400640 1 -- 172.21.15.55:0/1582363410 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980327d0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f579aa00640 1 -- 172.21.15.55:0/3343831240 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578802ae30 con 0x7f579c0b2140 2024-05-10T03:30:12.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f579aa00640 1 -- 172.21.15.55:0/3343831240 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788032440 con 0x7f579c0b2140 2024-05-10T03:30:12.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f579aa00640 1 -- 172.21.15.55:0/3343831240 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880327d0 con 0x7f579c0b2140 2024-05-10T03:30:12.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fa906800640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7fa8e40742e0 0x7fa8e4076770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f8ccd400640 1 -- 172.21.15.55:0/3496604798 >> v1:172.21.15.162:6789/0 conn(0x7f8cd015d6a0 legacy=0x7f8cd006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f8ccd400640 1 -- 172.21.15.55:0/3496604798 >> v1:172.21.15.55:6789/0 conn(0x7f8cd00b7200 legacy=0x7f8cd00b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.653+0000 7f8ccd400640 1 -- 172.21.15.55:0/3496604798 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd006b5a0 con 0x7f8cd00cdea0 2024-05-10T03:30:12.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fa905e00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8e406ac50 0x7fa8e406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fc8b3400640 1 -- 172.21.15.55:0/1582363410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 msgr2=0x7fc8ac165dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7f57a2c00640 1 -- 172.21.15.55:0/3343831240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 msgr2=0x7f579c142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7f57a2c00640 1 --2- 172.21.15.55:0/3343831240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c142440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57880097d0 tx=0x7f578802a1a0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fc8b3400640 1 --2- 172.21.15.55:0/1582363410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc898009870 tx=0x7fc89802a180 comp rx=0 tx=0).stop 2024-05-10T03:30:12.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fea99600640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea7c06acd0 0x7fea7c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7f57a2c00640 1 -- 172.21.15.55:0/3343831240 shutdown_connections 2024-05-10T03:30:12.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7f57a2c00640 1 --2- 172.21.15.55:0/3343831240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c142440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7f57a2c00640 1 -- 172.21.15.55:0/3343831240 >> 172.21.15.55:0/3343831240 conn(0x7f579c0cbb90 msgr2=0x7f579c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.654+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== osd.2 v2:172.21.15.55:6800/35475 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fab9c076dd0 con 0x7fab9c074270 2024-05-10T03:30:12.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea90088b10 con 0x7fea9c0ccad0 2024-05-10T03:30:12.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7f57a2c00640 1 -- 172.21.15.55:0/3343831240 shutdown_connections 2024-05-10T03:30:12.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7f57a2c00640 1 -- 172.21.15.55:0/3343831240 wait complete. 2024-05-10T03:30:12.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fa906800640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7fa8e40742e0 0x7fa8e4076770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fc8b3400640 1 -- 172.21.15.55:0/1582363410 shutdown_connections 2024-05-10T03:30:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fc8b3400640 1 --2- 172.21.15.55:0/1582363410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac1659c0 0x7fc8ac165dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fc8b3400640 1 --2- 172.21.15.55:0/1582363410 >> v2:172.21.15.162:3300/0 conn(0x7fc8ac0b2700 0x7fc8ac0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fc8b3400640 1 -- 172.21.15.55:0/1582363410 >> 172.21.15.55:0/1582363410 conn(0x7fc8ac0cbb90 msgr2=0x7fc8ac0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7feaa1a00640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea7c06acd0 0x7fea7c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fea99600640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7fea7c074360 0x7fea7c0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea7c076ec0 con 0x7fea7c074360 2024-05-10T03:30:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea9008d2a0 con 0x7fea9c0ccad0 2024-05-10T03:30:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7fc8b3400640 1 -- 172.21.15.55:0/1582363410 shutdown_connections 2024-05-10T03:30:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7fc8b3400640 1 -- 172.21.15.55:0/1582363410 wait complete. 2024-05-10T03:30:12.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.655+0000 7f8ccd400640 1 --2- 172.21.15.55:0/3496604798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00cdea0 0x7f8cd00ce2b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097a0 tx=0x7f8cbc02a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1aa100f61a01e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc02ae50 con 0x7f8cd00cdea0 2024-05-10T03:30:12.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc032440 con 0x7f8cd00cdea0 2024-05-10T03:30:12.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f8ccca00640 1 -- 172.21.15.55:0/3496604798 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc02ae50 con 0x7f8cd00cdea0 2024-05-10T03:30:12.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7feaa2400640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7fea7c074360 0x7fea7c0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f3a2fa00640 1 Processor -- start 2024-05-10T03:30:12.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.657+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== osd.0 v2:172.21.15.55:6816/35473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa8e4076e40 con 0x7fa8e40742e0 2024-05-10T03:30:12.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.657+0000 7f3a2fa00640 1 -- start start 2024-05-10T03:30:12.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f0812600640 1 Processor -- start 2024-05-10T03:30:12.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.656+0000 7f0812600640 1 -- start start 2024-05-10T03:30:12.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f57a2c00640 1 Processor -- start 2024-05-10T03:30:12.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.658+0000 7f0812600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f080416d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.658+0000 7f0812600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f080417a980 0x7f080417d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0812600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080417d950 0x7f08040caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0812600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040cbb20 con 0x7f07f4004360 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0812600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080416dbd0 con 0x7f080417d950 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0812600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080416ea30 con 0x7f080417a980 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f57a2c00640 1 -- start start 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0811c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080417d950 0x7f08040caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0811200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f080416d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0811c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080417d950 0x7f08040caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33882/0 (socket says 172.21.15.55:33882) 2024-05-10T03:30:12.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.659+0000 7f0811c00640 1 -- 172.21.15.55:0/1502190661 learned_addr learned my addr 172.21.15.55:0/1502190661 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7fc8b3400640 1 Processor -- start 2024-05-10T03:30:12.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f8ccf200640 1 -- 172.21.15.55:0/3496604798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00cdea0 msgr2=0x7f8cd00ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f0811200640 1 -- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f080417a980 msgr2=0x7f080417d410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:30:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f0811200640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f080417a980 0x7f080417d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f8ccf200640 1 --2- 172.21.15.55:0/3496604798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00cdea0 0x7f8cd00ce2b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097a0 tx=0x7f8cbc02a140 comp rx=0 tx=0).stop 2024-05-10T03:30:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f0811200640 1 -- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080417d950 msgr2=0x7f08040caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f0811200640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f080417d950 0x7f08040caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f3a2fa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a2806b6e0 0x7f3a281fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f0811200640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080416eec0 con 0x7f07f4004360 2024-05-10T03:30:12.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7fc8b3400640 1 -- start start 2024-05-10T03:30:12.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f3a2fa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a280b2700 0x7f3a28148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f8ccf200640 1 -- 172.21.15.55:0/3496604798 shutdown_connections 2024-05-10T03:30:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f8ccf200640 1 --2- 172.21.15.55:0/3496604798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00cdea0 0x7f8cd00ce2b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7f8ccf200640 1 -- 172.21.15.55:0/3496604798 >> 172.21.15.55:0/3496604798 conn(0x7f8cd00b56d0 msgr2=0x7f8cd00d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7fa905e00640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8e406ac50 0x7fa8e406d160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009ac0 tx=0x7fa8f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.658+0000 7feaa1a00640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea7c06acd0 0x7fea7c06d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea88007a40 tx=0x7fea88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.660+0000 7feaa2400640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7fea7c074360 0x7fea7c0767f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== osd.1 v2:172.21.15.55:6822/35474 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fea7c076ec0 con 0x7fea7c074360 2024-05-10T03:30:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f0811200640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f080416d4b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0804011ab0 tx=0x7f07f8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8010800 con 0x7f07f4004360 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8010960 con 0x7f07f4004360 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8010d10 con 0x7f07f4004360 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2fa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a281659c0 0x7f3a28164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2fa00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a281fbe60 con 0x7f3a280b2700 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2d000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a2806b6e0 0x7f3a281fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2d000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a2806b6e0 0x7f3a281fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37422/0 (socket says 172.21.15.55:37422) 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2d000640 1 -- 172.21.15.55:0/942529658 learned_addr learned my addr 172.21.15.55:0/942529658 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28149da0 con 0x7f3a281659c0 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2814f840 con 0x7f3a2806b6e0 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f3a27e00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a280b2700 0x7f3a28148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a2da00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a281659c0 0x7f3a28164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a27e00640 1 -- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a2806b6e0 msgr2=0x7f3a281fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a27e00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a2806b6e0 0x7f3a281fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a27e00640 1 -- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a281659c0 msgr2=0x7f3a28164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a27e00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a281659c0 0x7f3a28164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f3a27e00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140094d0 con 0x7f3a280b2700 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f3a27e00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a280b2700 0x7f3a28148de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000bd80 tx=0x7f3a2000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 0x7fc8ac156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac0b2700 0x7fc8ac157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8ac1659c0 0x7fc8ac15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac15fc80 con 0x7fc8ac1659c0 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac156690 con 0x7fc8ac06b6e0 2024-05-10T03:30:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7fc8b3400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac1568e0 con 0x7fc8ac0b2700 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f57a2c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c12f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.662+0000 7f57a2c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 0x7f579c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f57a2c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f579c15fe50 0x7f579c139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f57a2c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c12ffb0 con 0x7f579c15fe50 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f57a2c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c15cff0 con 0x7f579c0b2550 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f57a2c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c15de50 con 0x7f579c0b2140 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f579b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 0x7f579c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f579b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 0x7f579c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33902/0 (socket says 172.21.15.55:33902) 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f579b400640 1 -- 172.21.15.55:0/2478966944 learned_addr learned my addr 172.21.15.55:0/2478966944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.663+0000 7f57a0c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f579c15fe50 0x7f579c139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f8ccf200640 1 -- 172.21.15.55:0/3496604798 shutdown_connections 2024-05-10T03:30:12.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20014440 con 0x7f3a280b2700 2024-05-10T03:30:12.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a200145e0 con 0x7f3a280b2700 2024-05-10T03:30:12.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f579be00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c12f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20014990 con 0x7f3a280b2700 2024-05-10T03:30:12.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080416de20 con 0x7f07f4004360 2024-05-10T03:30:12.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2815eac0 con 0x7f3a280b2700 2024-05-10T03:30:12.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7fc8abe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac0b2700 0x7fc8ac157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080416e3e0 con 0x7f07f4004360 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7fc8b0a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 0x7fc8ac156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f57a0c00640 1 -- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 msgr2=0x7f579c12f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f57a0c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c12f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f57a0c00640 1 -- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 msgr2=0x7f579c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f57a0c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 0x7f579c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f57a0c00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009400 con 0x7f579c15fe50 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f579be00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f579c0b2140 0x7f579c12f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7f579b400640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f579c0b2550 0x7f579c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.664+0000 7fc8b0a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 0x7fc8ac156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33912/0 (socket says 172.21.15.55:33912) 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7fc8b0a00640 1 -- 172.21.15.55:0/4146840143 learned_addr learned my addr 172.21.15.55:0/4146840143 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.661+0000 7f8ccf200640 1 -- 172.21.15.55:0/3496604798 wait complete. 2024-05-10T03:30:12.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7fc8b1400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8ac1659c0 0x7fc8ac15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7f8ccf200640 1 Processor -- start 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b1400640 1 -- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac0b2700 msgr2=0x7fc8ac157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b1400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac0b2700 0x7fc8ac157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b1400640 1 -- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 msgr2=0x7fc8ac156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b1400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 0x7fc8ac156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0804037d40 con 0x7f07f4004360 2024-05-10T03:30:12.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b1400640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980094d0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7f57a0c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f579c15fe50 0x7f579c139040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f579400c730 tx=0x7f579400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7f8ccf200640 1 -- start start 2024-05-10T03:30:12.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.665+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2814b730 con 0x7f3a280b2700 2024-05-10T03:30:12.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a28070820 con 0x7f3a280b2700 2024-05-10T03:30:12.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.666+0000 7fc8b0a00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8ac06b6e0 0x7fc8ac156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:12.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7fc8abe00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8ac0b2700 0x7fc8ac157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:12.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794003b70 con 0x7f579c15fe50 2024-05-10T03:30:12.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794003cd0 con 0x7f579c15fe50 2024-05-10T03:30:12.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7fc8b1400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8ac1659c0 0x7fc8ac15f0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0cd1f0 tx=0x7fc89c00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794003b70 con 0x7f579c15fe50 2024-05-10T03:30:12.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c15d240 con 0x7f579c15fe50 2024-05-10T03:30:12.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.667+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c15d800 con 0x7f579c15fe50 2024-05-10T03:30:12.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.668+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f579c0aeb10 con 0x7f579c15fe50 2024-05-10T03:30:12.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 0x7f8cd00a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cd00cdea0 0x7f8cd0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cd015d6a0 0x7f8cd015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.670+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f07f8027480 con 0x7f07f4004360 2024-05-10T03:30:12.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00aa2b0 con 0x7f8cd015d6a0 2024-05-10T03:30:12.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01f4110 con 0x7f8cd00b7200 2024-05-10T03:30:12.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.669+0000 7f8ccf200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00aa060 con 0x7f8cd00cdea0 2024-05-10T03:30:12.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.670+0000 7f8ccde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 0x7f8cd00a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.670+0000 7f8ccde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 0x7f8cd00a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:33920/0 (socket says 172.21.15.55:33920) 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.670+0000 7f8ccde00640 1 -- 172.21.15.55:0/3537784413 learned_addr learned my addr 172.21.15.55:0/3537784413 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.670+0000 7f8ccd400640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cd00cdea0 0x7f8cd0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f8ccde00640 1 -- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cd00cdea0 msgr2=0x7f8cd0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a20014af0 con 0x7f3a280b2700 2024-05-10T03:30:12.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f0808c00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f07ec06ac80 0x7f07ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f8ccde00640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cd00cdea0 0x7f8cd0152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f8ccde00640 1 -- 172.21.15.55:0/3537784413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cd015d6a0 msgr2=0x7f8cd015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f8ccde00640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cd015d6a0 0x7f8cd015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.671+0000 7f8ccde00640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc009400 con 0x7f8cd00b7200 2024-05-10T03:30:12.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f8ccde00640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 0x7f8cd00a9a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c440 tx=0x7f8cc000c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f07f8089fa0 con 0x7f07f4004360 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f0808c00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f07ec074310 0x7f07ec0767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07ec076e70 con 0x7f07ec074310 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f808a310 con 0x7f07f4004360 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f0811c00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f07ec074310 0x7f07ec0767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c017070 con 0x7fc8ac1659c0 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c004020 con 0x7fc8ac1659c0 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0041b0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac156bc0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.672+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00074e0 con 0x7f8cd00b7200 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc0007640 con 0x7f8cd00b7200 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0004a20 con 0x7f8cd00b7200 2024-05-10T03:30:12.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7f3a24c00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0806ad00 0x7f3a0806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.674+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac157180 con 0x7fc8ac1659c0 2024-05-10T03:30:12.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.673+0000 7f080be00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f07ec06ac80 0x7f07ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.674+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd0155580 con 0x7f8cd00b7200 2024-05-10T03:30:12.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.674+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0154c50 con 0x7f8cd00b7200 2024-05-10T03:30:12.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.675+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8ac070260 con 0x7fc8ac1659c0 2024-05-10T03:30:12.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.675+0000 7f3a2d000640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0806ad00 0x7f3a0806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.675+0000 7f0811c00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f07ec074310 0x7f07ec0767a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.674+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3a2008a300 con 0x7f3a280b2700 2024-05-10T03:30:12.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.675+0000 7f3a24c00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f3a08074390 0x7f3a08076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.675+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cd00b6300 con 0x7f8cd00b7200 2024-05-10T03:30:12.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.676+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a08076ef0 con 0x7f3a08074390 2024-05-10T03:30:12.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.676+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a2008a6b0 con 0x7f3a280b2700 2024-05-10T03:30:12.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.676+0000 7f080be00640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f07ec06ac80 0x7f07ec06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0800033000 tx=0x7f0800008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.677+0000 7f3a2da00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f3a08074390 0x7f3a08076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.677+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== osd.3 v2:172.21.15.55:6808/35476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f07ec076e70 con 0x7f07ec074310 2024-05-10T03:30:12.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.677+0000 7f3a2d000640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0806ad00 0x7f3a0806d210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009160 tx=0x7f3a1402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.677+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc89c004310 con 0x7fc8ac1659c0 2024-05-10T03:30:12.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.678+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f57940052e0 con 0x7f579c15fe50 2024-05-10T03:30:12.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8cc0004b80 con 0x7f8cd00b7200 2024-05-10T03:30:12.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7fc8a8c00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88c06ac30 0x7fc88c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7f5793e00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f577c06ac90 0x7f577c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5794089bf0 con 0x7f579c15fe50 2024-05-10T03:30:12.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7f3a2da00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f3a08074390 0x7f3a08076820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc89c089210 con 0x7fc8ac1659c0 2024-05-10T03:30:12.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7fc8a8c00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7fc88c074240 0x7fc88c0766d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc88c076da0 con 0x7fc88c074240 2024-05-10T03:30:12.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.679+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc89c08b7f0 con 0x7fc8ac1659c0 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f5793e00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f577c074220 0x7f577c0766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f577c076d80 con 0x7f577c074220 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579408a040 con 0x7f579c15fe50 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f8cc6000640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cb006ac00 0x7f8cb006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8cc0088e70 con 0x7f8cd00b7200 2024-05-10T03:30:12.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7fc8abe00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7fc88c074240 0x7fc88c0766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f579be00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f577c06ac90 0x7f577c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.680+0000 7f8cc6000640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f8cb0074290 0x7f8cb0076720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:12.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.681+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb0076df0 con 0x7f8cb0074290 2024-05-10T03:30:12.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.681+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc0089220 con 0x7f8cd00b7200 2024-05-10T03:30:12.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.681+0000 7f579b400640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f577c074220 0x7f577c0766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.681+0000 7fc8b0a00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88c06ac30 0x7fc88c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.682+0000 7f8ccd400640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cb006ac00 0x7f8cb006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.682+0000 7fc8abe00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7fc88c074240 0x7fc88c0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.682+0000 7f579be00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f577c06ac90 0x7f577c06d1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f578c000f60 tx=0x7f578c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.682+0000 7f579b400640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f577c074220 0x7f577c0766b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.683+0000 7f8cce800640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f8cb0074290 0x7f8cb0076720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:12.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.684+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== osd.4 v2:172.21.15.162:6800/35542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3a08076ef0 con 0x7f3a08074390 2024-05-10T03:30:12.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.685+0000 7fc8b0a00640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88c06ac30 0x7fc88c06d140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c530 tx=0x7fc8a400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.685+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== osd.5 v2:172.21.15.162:6808/35543 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc88c076da0 con 0x7fc88c074240 2024-05-10T03:30:12.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.685+0000 7f8ccd400640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cb006ac00 0x7f8cb006d110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc034000 tx=0x7f8cbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.685+0000 7f8cce800640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f8cb0074290 0x7f8cb0076720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:12.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.685+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== osd.6 v2:172.21.15.162:6816/35544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f577c076d80 con 0x7f577c074220 2024-05-10T03:30:12.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.687+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== osd.7 v2:172.21.15.162:6824/35545 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8cb0076df0 con 0x7f8cb0074290 2024-05-10T03:30:12.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.692+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab94002cb0 con 0x7fab9c074270 2024-05-10T03:30:12.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.694+0000 7fabb2000640 1 -- 172.21.15.55:0/891217263 <== osd.2 v2:172.21.15.55:6800/35475 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab94002cb0 con 0x7fab9c074270 2024-05-10T03:30:12.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fab9c074270 msgr2=0x7fab9c076700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fab9c074270 0x7fab9c076700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab9c06ac60 msgr2=0x7fab9c06d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab9c06ac60 0x7fab9c06d170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0798d0 tx=0x7fabbc04f270 comp rx=0 tx=0).stop 2024-05-10T03:30:12.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 msgr2=0x7fabb400e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabc0c00640 1 --2- 172.21.15.55:0/891217263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabb41858a0 0x7fabb400e7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c730 tx=0x7fabac00cbf0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fabba800640 1 -- 172.21.15.55:0/891217263 reap_dead start 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.696+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 shutdown_connections 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.696+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 >> 172.21.15.55:0/891217263 conn(0x7fabb403ac10 msgr2=0x7fabb40e3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.695+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 --> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9080d2840 con 0x7fa8e40742e0 2024-05-10T03:30:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.697+0000 7fa8fe000640 1 -- 172.21.15.55:0/387728847 <== osd.0 v2:172.21.15.55:6816/35473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa900001900 con 0x7fa8e40742e0 2024-05-10T03:30:12.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.697+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 shutdown_connections 2024-05-10T03:30:12.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.697+0000 7fabc0c00640 1 -- 172.21.15.55:0/891217263 wait complete. 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.698+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7fa8e40742e0 msgr2=0x7fa8e4076770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.698+0000 7fa907200640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7fa8e40742e0 0x7fa8e4076770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8e406ac50 msgr2=0x7fa8e406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa8e406ac50 0x7fa8e406d160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009ac0 tx=0x7fa8f8008040 comp rx=0 tx=0).stop 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 msgr2=0x7fa90806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 --2- 172.21.15.55:0/387728847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9080b6680 0x7fa90806d870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0000c00 tx=0x7fa8f00046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa906800640 1 -- 172.21.15.55:0/387728847 reap_dead start 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 shutdown_connections 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.699+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 >> 172.21.15.55:0/387728847 conn(0x7fa9080b5730 msgr2=0x7fa90815c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.700+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 shutdown_connections 2024-05-10T03:30:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.700+0000 7fa907200640 1 -- 172.21.15.55:0/387728847 wait complete. 2024-05-10T03:30:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.701+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 --> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea70002cb0 con 0x7fea7c074360 2024-05-10T03:30:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.702+0000 7fea99600640 1 -- 172.21.15.55:0/2742920010 <== osd.1 v2:172.21.15.55:6822/35474 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea70002cb0 con 0x7fea7c074360 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7fea7c074360 msgr2=0x7fea7c0767f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7fea7c074360 0x7fea7c0767f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea7c06acd0 msgr2=0x7fea7c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea7c06acd0 0x7fea7c06d1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea88007a40 tx=0x7fea88008040 comp rx=0 tx=0).stop 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ccad0 msgr2=0x7fea9c0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7fea8ea00640 1 --2- 172.21.15.55:0/2742920010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea9c0ccad0 0x7fea9c0aff60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c470 tx=0x7fea9000c930 comp rx=0 tx=0).stop 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.703+0000 7feaa2400640 1 -- 172.21.15.55:0/2742920010 reap_dead start 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.706+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 shutdown_connections 2024-05-10T03:30:12.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.706+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 >> 172.21.15.55:0/2742920010 conn(0x7fea9c06c730 msgr2=0x7fea9c161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.705+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 --> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f579c142980 con 0x7f577c074220 2024-05-10T03:30:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.706+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 shutdown_connections 2024-05-10T03:30:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.706+0000 7fea8ea00640 1 -- 172.21.15.55:0/2742920010 wait complete. 2024-05-10T03:30:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.707+0000 7f5793e00640 1 -- 172.21.15.55:0/2478966944 <== osd.6 v2:172.21.15.162:6816/35544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5788004400 con 0x7f577c074220 2024-05-10T03:30:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.708+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f577c074220 msgr2=0x7f577c0766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.708+0000 7f57a2c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f577c074220 0x7f577c0766b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f577c06ac90 msgr2=0x7f577c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f577c06ac90 0x7f577c06d1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f578c000f60 tx=0x7f578c009040 comp rx=0 tx=0).stop 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f579c15fe50 msgr2=0x7f579c139040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 --2- 172.21.15.55:0/2478966944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f579c15fe50 0x7f579c139040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f579400c730 tx=0x7f579400cbf0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a0c00640 1 -- 172.21.15.55:0/2478966944 reap_dead start 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 shutdown_connections 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.709+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 >> 172.21.15.55:0/2478966944 conn(0x7f579c0cbb90 msgr2=0x7f579c135b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.710+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 --> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f080416eba0 con 0x7f07ec074310 2024-05-10T03:30:12.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.711+0000 7f0808c00640 1 -- 172.21.15.55:0/1502190661 <== osd.3 v2:172.21.15.55:6808/35476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f080416eba0 con 0x7f07ec074310 2024-05-10T03:30:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.711+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 shutdown_connections 2024-05-10T03:30:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.711+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f07ec074310 msgr2=0x7f07ec0767a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.711+0000 7f0812600640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f07ec074310 0x7f07ec0767a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.711+0000 7f57a2c00640 1 -- 172.21.15.55:0/2478966944 wait complete. 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.713+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f07ec06ac80 msgr2=0x7f07ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.713+0000 7f0812600640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f07ec06ac80 0x7f07ec06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0800033000 tx=0x7f0800008030 comp rx=0 tx=0).stop 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.713+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 msgr2=0x7f080416d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.713+0000 7f0812600640 1 --2- 172.21.15.55:0/1502190661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07f4004360 0x7f080416d4b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0804011ab0 tx=0x7f07f8003b20 comp rx=0 tx=0).stop 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.713+0000 7f0811c00640 1 -- 172.21.15.55:0/1502190661 reap_dead start 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.714+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 shutdown_connections 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.714+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 >> 172.21.15.55:0/1502190661 conn(0x7f080403abc0 msgr2=0x7f08040543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.715+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 shutdown_connections 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.715+0000 7f0812600640 1 -- 172.21.15.55:0/1502190661 wait complete. 2024-05-10T03:30:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.718+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 --> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a28070540 con 0x7f3a08074390 2024-05-10T03:30:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.719+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 --> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc8ac0ceac0 con 0x7fc88c074240 2024-05-10T03:30:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a24c00640 1 -- 172.21.15.55:0/942529658 <== osd.4 v2:172.21.15.162:6800/35542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a1800d4c0 con 0x7f3a08074390 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f3a08074390 msgr2=0x7f3a08076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f3a08074390 0x7f3a08076820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0806ad00 msgr2=0x7f3a0806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0806ad00 0x7f3a0806d210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009160 tx=0x7f3a1402e000 comp rx=0 tx=0).stop 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a280b2700 msgr2=0x7f3a28148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.720+0000 7f3a2fa00640 1 --2- 172.21.15.55:0/942529658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a280b2700 0x7f3a28148de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000bd80 tx=0x7f3a2000be70 comp rx=0 tx=0).stop 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.721+0000 7f3a2da00640 1 -- 172.21.15.55:0/942529658 reap_dead start 2024-05-10T03:30:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.721+0000 7fc8a8c00640 1 -- 172.21.15.55:0/4146840143 <== osd.5 v2:172.21.15.162:6808/35543 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc8980329f0 con 0x7fc88c074240 2024-05-10T03:30:12.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 shutdown_connections 2024-05-10T03:30:12.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 >> 172.21.15.55:0/942529658 conn(0x7f3a280cbb90 msgr2=0x7f3a2815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 shutdown_connections 2024-05-10T03:30:12.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7f3a2fa00640 1 -- 172.21.15.55:0/942529658 wait complete. 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7fc88c074240 msgr2=0x7fc88c0766d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7fc88c074240 0x7fc88c0766d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88c06ac30 msgr2=0x7fc88c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88c06ac30 0x7fc88c06d140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c530 tx=0x7fc8a400b040 comp rx=0 tx=0).stop 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8ac1659c0 msgr2=0x7fc8ac15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b3400640 1 --2- 172.21.15.55:0/4146840143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8ac1659c0 0x7fc8ac15f0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0cd1f0 tx=0x7fc89c00d8f0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.722+0000 7fc8b1400640 1 -- 172.21.15.55:0/4146840143 reap_dead start 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.723+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 shutdown_connections 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.723+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 >> 172.21.15.55:0/4146840143 conn(0x7fc8ac0cbb90 msgr2=0x7fc8ac15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.724+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 shutdown_connections 2024-05-10T03:30:12.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.725+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 --> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cd00b66b0 con 0x7f8cb0074290 2024-05-10T03:30:12.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.725+0000 7fc8b3400640 1 -- 172.21.15.55:0/4146840143 wait complete. 2024-05-10T03:30:12.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.726+0000 7f8cc6000640 1 -- 172.21.15.55:0/3537784413 <== osd.7 v2:172.21.15.162:6824/35545 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8cc80020b0 con 0x7f8cb0074290 2024-05-10T03:30:12.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.727+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f8cb0074290 msgr2=0x7f8cb0076720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.727+0000 7f8ccf200640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f8cb0074290 0x7f8cb0076720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cb006ac00 msgr2=0x7f8cb006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cb006ac00 0x7f8cb006d110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc034000 tx=0x7f8cbc02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 msgr2=0x7f8cd00a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 --2- 172.21.15.55:0/3537784413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cd00b7200 0x7f8cd00a9a60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c440 tx=0x7f8cc000c900 comp rx=0 tx=0).stop 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8cce800640 1 -- 172.21.15.55:0/3537784413 reap_dead start 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 shutdown_connections 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.728+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 >> 172.21.15.55:0/3537784413 conn(0x7f8cd00b56d0 msgr2=0x7f8cd015e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.730+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 shutdown_connections 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:12.730+0000 7f8ccf200640 1 -- 172.21.15.55:0/3537784413 wait complete. 2024-05-10T03:30:12.756 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-10T03:30:12.757 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-10T03:30:12.758 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-10T03:30:12.758 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-10T03:30:12.759 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:12.759 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-10T03:30:12.760 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:12.760 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-10T03:30:12.761 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:12.761 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-10T03:30:12.762 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:12.763 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-10T03:30:12.764 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:12.764 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-10T03:30:12.765 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-10T03:30:12.765 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-10T03:30:13.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.172+0000 7f6b02600640 1 Processor -- start 2024-05-10T03:30:13.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.172+0000 7f6b02600640 1 -- start start 2024-05-10T03:30:13.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.174+0000 7f3fc6a00640 1 Processor -- start 2024-05-10T03:30:13.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.174+0000 7f3ef7200640 1 Processor -- start 2024-05-10T03:30:13.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.174+0000 7f3fc6a00640 1 -- start start 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.175+0000 7f6b02600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6af40e3a60 0x7f6af4057b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.176+0000 7f6b02600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af4057700 con 0x7f6af40073a0 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.176+0000 7f6b02600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af4056dd0 con 0x7f6af4058b60 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.176+0000 7f6b02600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40570b0 con 0x7f6af40e3a60 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.177+0000 7f6b01c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6af40073a0 0x7f6af40077b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37992/0 (socket says 172.21.15.55:37992) 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.177+0000 7f6b01c00640 1 -- 172.21.15.55:0/2782673995 learned_addr learned my addr 172.21.15.55:0/2782673995 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.177+0000 7f6b01200640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6af40e3a60 0x7f6af4057b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.177+0000 7f5bbfe00640 1 Processor -- start 2024-05-10T03:30:13.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.176+0000 7f3ef7200640 1 -- start start 2024-05-10T03:30:13.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910124784 0 0) 0x7f6af4057700 con 0x7f6af40073a0 2024-05-10T03:30:13.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae4003520 con 0x7f6af40073a0 2024-05-10T03:30:13.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f3fc6a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3fc00b2d00 0x7f3fc01471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66192091 0 0) 0x7f6af4056dd0 con 0x7f6af4058b60 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af4057700 con 0x7f6af4058b60 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114293820 0 0) 0x7f6ae4003520 con 0x7f6af40073a0 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.178+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6af40e3a60 msgr2=0x7f6af4057b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6af40e3a60 0x7f6af4057b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 >> v1:172.21.15.162:6789/0 conn(0x7f6af4058b60 legacy=0x7f6af40380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4057390 con 0x7f6af40073a0 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f6afc074030 con 0x7f6af40073a0 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 >> v1:172.21.15.55:6789/0 conn(0x7f6af40073a0 legacy=0x7f6af40077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3ef7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ef00e7ec0 0x7f3ef0053620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3fc6a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc0147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3fc6a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc0162c50 con 0x7f3fc0144b60 2024-05-10T03:30:13.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3fc6a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc00b1f60 con 0x7f3fc00b2d00 2024-05-10T03:30:13.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3fc6a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0162ed0 con 0x7f3fc0162840 2024-05-10T03:30:13.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ae4003b30 0x7f6ae4003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ae40047c0 0x7f6ae4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae4005110 0x7f6ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40570b0 con 0x7f6ae4003b30 2024-05-10T03:30:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae400bff0 con 0x7f6ae40047c0 2024-05-10T03:30:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae400c1b0 con 0x7f6ae4005110 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3ef7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef0058990 con 0x7f3ef0007330 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3ef7200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef0058660 con 0x7f3ef003c290 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.179+0000 7f3ef7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef0058e20 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f3ef5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ef00e7ec0 0x7f3ef0053620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f3ef5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3ef0007330 0x7f3ef0007740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38006/0 (socket says 172.21.15.55:38006) 2024-05-10T03:30:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f3ef5400640 1 -- 172.21.15.55:0/1996116695 learned_addr learned my addr 172.21.15.55:0/1996116695 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:13.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f6b01c00640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae4005110 0x7f6ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f3fc5600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3fc00b2d00 0x7f3fc01471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f3fc6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc0147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.180+0000 7f6afbe00640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ae40047c0 0x7f6ae4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f3fc6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc0147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48780/0 (socket says 172.21.15.55:48780) 2024-05-10T03:30:13.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f6b01200640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ae4003b30 0x7f6ae4003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f3fc6000640 1 -- 172.21.15.55:0/390409268 learned_addr learned my addr 172.21.15.55:0/390409268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f6afbe00640 1 -- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae4005110 msgr2=0x7f6ae400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f6afbe00640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae4005110 0x7f6ae400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f6afbe00640 1 -- 172.21.15.55:0/2782673995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ae4003b30 msgr2=0x7f6ae4003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f6afbe00640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ae4003b30 0x7f6ae4003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f6afbe00640 1 -- 172.21.15.55:0/2782673995 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af4057390 con 0x7f6ae40047c0 2024-05-10T03:30:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.183+0000 7f3ef5e00640 1 -- 172.21.15.55:0/1996116695 >> v1:172.21.15.162:6789/0 conn(0x7f3ef003c290 legacy=0x7f3ef003c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f3ef4a00640 1 -- 172.21.15.55:0/1996116695 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912839366 0 0) 0x7f3ef0058660 con 0x7f3ef003c290 2024-05-10T03:30:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.183+0000 7f3ef5e00640 1 -- 172.21.15.55:0/1996116695 >> v1:172.21.15.55:6789/0 conn(0x7f3ef0007330 legacy=0x7f3ef0007740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.183+0000 7f3ef5e00640 1 -- 172.21.15.55:0/1996116695 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00541b0 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f3fc5600640 1 -- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 msgr2=0x7f3fc0147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f3fc5600640 1 --2- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc0147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f3fc5600640 1 -- 172.21.15.55:0/390409268 >> v1:172.21.15.55:6789/0 conn(0x7f3fc0144b60 legacy=0x7f3fc0162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.182+0000 7f3fc5600640 1 -- 172.21.15.55:0/390409268 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00b2220 con 0x7f3fc00b2d00 2024-05-10T03:30:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.183+0000 7f6b01200640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ae4003b30 0x7f6ae4003f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.181+0000 7f5bbfe00640 1 -- start start 2024-05-10T03:30:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5d06400640 1 Processor -- start 2024-05-10T03:30:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f3ef5e00640 1 --2- 172.21.15.55:0/1996116695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef0053620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ef804f040 tx=0x7f3ef80a4e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f826bc270d7e28f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5d06400640 1 -- start start 2024-05-10T03:30:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5bbfe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5bc00b2170 0x7f5bc015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5bbfe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5bc01618b0 0x7f5bc006aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5bbfe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00d25a0 con 0x7f5bc00b6680 2024-05-10T03:30:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5bbfe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00d2b20 con 0x7f5bc01618b0 2024-05-10T03:30:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.184+0000 7f5bbfe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc015db20 con 0x7f5bc00b2170 2024-05-10T03:30:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7fdbc3200640 1 Processor -- start 2024-05-10T03:30:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f3ef4a00640 1 -- 172.21.15.55:0/1996116695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80a5b70 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f3ef4a00640 1 -- 172.21.15.55:0/1996116695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80a5cd0 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f3ef4a00640 1 -- 172.21.15.55:0/1996116695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80a5b70 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7fdbc3200640 1 -- start start 2024-05-10T03:30:13.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f5bbf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5bc01618b0 0x7f5bc006aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f5bbf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5bc01618b0 0x7f5bc006aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33230/0 (socket says 172.21.15.55:33230) 2024-05-10T03:30:13.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f5bbf400640 1 -- 172.21.15.55:0/2163744954 learned_addr learned my addr 172.21.15.55:0/2163744954 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.185+0000 7f6afbe00640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ae40047c0 0x7f6ae4004bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80018d0 tx=0x7f6ae40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4961f69657b8bf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f3fc5600640 1 --2- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc00b2d00 0x7f3fc01471f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40097a0 tx=0x7f3fb402a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e2a60a4e7bb6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f3ef7200640 1 -- 172.21.15.55:0/1996116695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 msgr2=0x7f3ef0053620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f3ef7200640 1 --2- 172.21.15.55:0/1996116695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef0053620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ef804f040 tx=0x7f3ef80a4e20 comp rx=0 tx=0).stop 2024-05-10T03:30:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f5bbf400640 1 -- 172.21.15.55:0/2163744954 >> v2:172.21.15.162:3300/0 conn(0x7f5bc00b2170 msgr2=0x7f5bc015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f5bbf400640 1 --2- 172.21.15.55:0/2163744954 >> v2:172.21.15.162:3300/0 conn(0x7f5bc00b2170 0x7f5bc015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f5bbf400640 1 -- 172.21.15.55:0/2163744954 >> v1:172.21.15.162:6789/0 conn(0x7f5bc00b6680 legacy=0x7f5bc01611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f5bbf400640 1 -- 172.21.15.55:0/2163744954 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00b28d0 con 0x7f5bc01618b0 2024-05-10T03:30:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802ac80 con 0x7f6ae40047c0 2024-05-10T03:30:13.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae802ade0 con 0x7f6ae40047c0 2024-05-10T03:30:13.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.186+0000 7f6afb400640 1 -- 172.21.15.55:0/2782673995 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8032660 con 0x7f6ae40047c0 2024-05-10T03:30:13.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3ef7200640 1 -- 172.21.15.55:0/1996116695 shutdown_connections 2024-05-10T03:30:13.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3ef7200640 1 --2- 172.21.15.55:0/1996116695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef0053620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3ef7200640 1 -- 172.21.15.55:0/1996116695 >> 172.21.15.55:0/1996116695 conn(0x7f3ef003ab50 msgr2=0x7f3ef003af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3fbfe00640 1 -- 172.21.15.55:0/390409268 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402adc0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3fbfe00640 1 -- 172.21.15.55:0/390409268 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4032440 con 0x7f3fc00b2d00 2024-05-10T03:30:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3ef7200640 1 -- 172.21.15.55:0/1996116695 shutdown_connections 2024-05-10T03:30:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f5d06400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5d000b23e0 0x7f5d000ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f5d06400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d000b27f0 con 0x7f5d000d1880 2024-05-10T03:30:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f5d06400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d000ceba0 con 0x7f5d000ce790 2024-05-10T03:30:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f5d06400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0006b230 con 0x7f5d000b23e0 2024-05-10T03:30:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f6b02600640 1 -- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ae40047c0 msgr2=0x7f6ae4004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f6b02600640 1 --2- 172.21.15.55:0/2782673995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ae40047c0 0x7f6ae4004bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80018d0 tx=0x7f6ae40057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7f6b01c00640 1 -- 172.21.15.55:0/2782673995 reap_dead start 2024-05-10T03:30:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3ef7200640 1 -- 172.21.15.55:0/1996116695 wait complete. 2024-05-10T03:30:13.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.187+0000 7f3fbfe00640 1 -- 172.21.15.55:0/390409268 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40327d0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7f49e8a00640 1 Processor -- start 2024-05-10T03:30:13.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7f6b02600640 1 -- 172.21.15.55:0/2782673995 shutdown_connections 2024-05-10T03:30:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7fdbc3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdbbc0b6bf0 0x7fdbbc15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7fdbc3200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 0x7fdbbc0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7fdbc3200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0d2b70 con 0x7fdbbc0cd9a0 2024-05-10T03:30:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7fdbc3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc06c270 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7fdbc3200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc06bcb0 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.188+0000 7f6b02600640 1 -- 172.21.15.55:0/2782673995 >> 172.21.15.55:0/2782673995 conn(0x7f6af403abc0 msgr2=0x7f6af403afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f6b02600640 1 -- 172.21.15.55:0/2782673995 shutdown_connections 2024-05-10T03:30:13.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f5d05a00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5d000ce790 0x7f5d000b7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:38210/0 (socket says 172.21.15.55:38210) 2024-05-10T03:30:13.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f5d05a00640 1 -- 172.21.15.55:0/4271734408 learned_addr learned my addr 172.21.15.55:0/4271734408 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:13.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 -- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc00b2d00 msgr2=0x7f3fc01471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc00b2d00 0x7f3fc01471f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40097a0 tx=0x7f3fb402a0b0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 -- 172.21.15.55:0/390409268 shutdown_connections 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc0147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/390409268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc00b2d00 0x7f3fc01471f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f3fc6a00640 1 -- 172.21.15.55:0/390409268 >> 172.21.15.55:0/390409268 conn(0x7f3fc006a110 msgr2=0x7f3fc006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f6b02600640 1 -- 172.21.15.55:0/2782673995 wait complete. 2024-05-10T03:30:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f5cff400640 1 -- 172.21.15.55:0/4271734408 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440051462 0 0) 0x7f5d000ceba0 con 0x7f5d000ce790 2024-05-10T03:30:13.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f5cff400640 1 -- 172.21.15.55:0/4271734408 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce8003520 con 0x7f5d000ce790 2024-05-10T03:30:13.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f3fc6a00640 1 -- 172.21.15.55:0/390409268 shutdown_connections 2024-05-10T03:30:13.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f3fc6a00640 1 -- 172.21.15.55:0/390409268 wait complete. 2024-05-10T03:30:13.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.189+0000 7f49e8a00640 1 -- start start 2024-05-10T03:30:13.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7fdbbb400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 0x7fdbbc0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f5cffe00640 1 --2- 172.21.15.55:0/4271734408 >> v2:172.21.15.55:3300/0 conn(0x7f5d000b23e0 0x7f5d000ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7fdbbb400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 0x7fdbbc0ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46846/0 (socket says 172.21.15.55:46846) 2024-05-10T03:30:13.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7fdbbb400640 1 -- 172.21.15.55:0/1986872980 learned_addr learned my addr 172.21.15.55:0/1986872980 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f5cffe00640 1 -- 172.21.15.55:0/4271734408 >> v1:172.21.15.162:6789/0 conn(0x7f5d000ce790 legacy=0x7f5d000b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.190+0000 7f5cffe00640 1 -- 172.21.15.55:0/4271734408 >> v1:172.21.15.55:6789/0 conn(0x7f5d000d1880 legacy=0x7f5d0006ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.191+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/1986872980 >> v2:172.21.15.55:3300/0 conn(0x7fdbbc0b6bf0 0x7fdbbc15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.191+0000 7f5cffe00640 1 -- 172.21.15.55:0/4271734408 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0006b420 con 0x7f5d000b23e0 2024-05-10T03:30:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f49e8a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e40b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f5cffe00640 1 --2- 172.21.15.55:0/4271734408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d000ce250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d0008c5b0 tx=0x7f5cec02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e31361728ed4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f49e8a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49e40d2110 0x7f49e40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7fdbbbe00640 1 -- 172.21.15.55:0/1986872980 >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 msgr2=0x7fdbbc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/1986872980 >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 0x7fdbbc0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f49e8a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 0x7f49e40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f5cff400640 1 -- 172.21.15.55:0/4271734408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec02ae60 con 0x7f5d000b23e0 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f5cff400640 1 -- 172.21.15.55:0/4271734408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec032440 con 0x7f5d000b23e0 2024-05-10T03:30:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f5cff400640 1 -- 172.21.15.55:0/4271734408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0327d0 con 0x7f5d000b23e0 2024-05-10T03:30:13.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f49e2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 0x7f49e40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e1400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49e40d2110 0x7f49e40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 0x7f49e40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33260/0 (socket says 172.21.15.55:33260) 2024-05-10T03:30:13.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e2800640 1 -- 172.21.15.55:0/1744514432 learned_addr learned my addr 172.21.15.55:0/1744514432 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e40b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e40b3fa0 con 0x7f49e40ce470 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e40cedc0 con 0x7f49e40d2110 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40d31c0 con 0x7f49e40b4430 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7fdbbbe00640 1 -- 172.21.15.55:0/1986872980 >> v1:172.21.15.55:6789/0 conn(0x7fdbbc0cd9a0 legacy=0x7fdbbc161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f3fc6a00640 1 Processor -- start 2024-05-10T03:30:13.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f3ef7200640 1 Processor -- start 2024-05-10T03:30:13.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.192+0000 7f5bbf400640 1 --2- 172.21.15.55:0/2163744954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc006aab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cfaa0 tx=0x7f5bb802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb91bf2a6d1e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f5bbd600640 1 -- 172.21.15.55:0/2163744954 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8031600 con 0x7f5bc01618b0 2024-05-10T03:30:13.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f3ef7200640 1 -- start start 2024-05-10T03:30:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f5bbd600640 1 -- 172.21.15.55:0/2163744954 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb802d050 con 0x7f5bc01618b0 2024-05-10T03:30:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7f5bbd600640 1 -- 172.21.15.55:0/2163744954 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8031980 con 0x7f5bc01618b0 2024-05-10T03:30:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f3fc6a00640 1 -- start start 2024-05-10T03:30:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f49e1400640 1 -- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 msgr2=0x7f49e40b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f49e1400640 1 --2- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e40b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f49e1400640 1 -- 172.21.15.55:0/1744514432 >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 msgr2=0x7f49e40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f49e1400640 1 --2- 172.21.15.55:0/1744514432 >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 0x7f49e40ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f49e1400640 1 -- 172.21.15.55:0/1744514432 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e40b3390 con 0x7f49e40d2110 2024-05-10T03:30:13.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.194+0000 7f6b02600640 1 Processor -- start 2024-05-10T03:30:13.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.193+0000 7fdbbbe00640 1 -- 172.21.15.55:0/1986872980 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc06bf90 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f5d06400640 1 -- 172.21.15.55:0/4271734408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 msgr2=0x7f5d000ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f5d06400640 1 --2- 172.21.15.55:0/4271734408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d000ce250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d0008c5b0 tx=0x7f5cec02a150 comp rx=0 tx=0).stop 2024-05-10T03:30:13.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f6b02600640 1 -- start start 2024-05-10T03:30:13.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f49e1400640 1 --2- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e40b25f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49e40b4a70 tx=0x7f49cc02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174bc37f80f11ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f5d06400640 1 -- 172.21.15.55:0/4271734408 shutdown_connections 2024-05-10T03:30:13.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f5d06400640 1 --2- 172.21.15.55:0/4271734408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d000ce250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.195+0000 7f5d06400640 1 -- 172.21.15.55:0/4271734408 >> 172.21.15.55:0/4271734408 conn(0x7f5d000d31a0 msgr2=0x7f5d000d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/1986872980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0b6bf0 0x7fdbbc15dd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc08b840 tx=0x7fdbb402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06754bc7e3d84f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f5d06400640 1 -- 172.21.15.55:0/4271734408 shutdown_connections 2024-05-10T03:30:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f5d06400640 1 -- 172.21.15.55:0/4271734408 wait complete. 2024-05-10T03:30:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f5bbfe00640 1 -- 172.21.15.55:0/2163744954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 msgr2=0x7f5bc006aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/2163744954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc006aab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cfaa0 tx=0x7f5bb802a6f0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f49e0a00640 1 -- 172.21.15.55:0/1744514432 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc02ae40 con 0x7f49e40d2110 2024-05-10T03:30:13.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7fdbbaa00640 1 -- 172.21.15.55:0/1986872980 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4007740 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f49e0a00640 1 -- 172.21.15.55:0/1744514432 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc032440 con 0x7f49e40d2110 2024-05-10T03:30:13.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7fdbbaa00640 1 -- 172.21.15.55:0/1986872980 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb402d050 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7fdbbaa00640 1 -- 172.21.15.55:0/1986872980 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4007b00 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.196+0000 7f49e0a00640 1 -- 172.21.15.55:0/1744514432 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0327d0 con 0x7f49e40d2110 2024-05-10T03:30:13.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f60fce00640 1 Processor -- start 2024-05-10T03:30:13.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3fc6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 0x7f3fc013e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3fc6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3fc6a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3ef7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3ef7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ef003c290 0x7f3ef004cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3fc6a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0146800 con 0x7f3fc00b2d00 2024-05-10T03:30:13.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f3ef7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef017d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 msgr2=0x7f49e40b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 --2- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e40b25f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49e40b4a70 tx=0x7f49cc02a0f0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 shutdown_connections 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 --2- 172.21.15.55:0/1744514432 >> v2:172.21.15.55:3300/0 conn(0x7f49e40ce470 0x7f49e40ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 --2- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e40b25f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 --2- 172.21.15.55:0/1744514432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e40b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.197+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 >> 172.21.15.55:0/1744514432 conn(0x7f49e40d0720 msgr2=0x7f49e40d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 -- 172.21.15.55:0/2163744954 shutdown_connections 2024-05-10T03:30:13.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/2163744954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc006aab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/2163744954 >> v2:172.21.15.162:3300/0 conn(0x7f5bc00b2170 0x7f5bc015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 -- 172.21.15.55:0/2163744954 >> 172.21.15.55:0/2163744954 conn(0x7f5bc00b5730 msgr2=0x7f5bc00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 shutdown_connections 2024-05-10T03:30:13.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f49e8a00640 1 -- 172.21.15.55:0/1744514432 wait complete. 2024-05-10T03:30:13.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f60fce00640 1 -- start start 2024-05-10T03:30:13.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 -- 172.21.15.55:0/1986872980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0b6bf0 msgr2=0x7fdbbc15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 --2- 172.21.15.55:0/1986872980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0b6bf0 0x7fdbbc15dd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc08b840 tx=0x7fdbb402a730 comp rx=0 tx=0).stop 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae40047c0 0x7f6af40064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af4006a20 0x7f6af404cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af404d8e0 con 0x7f6af4006a20 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40dbe00 con 0x7f6af404d300 2024-05-10T03:30:13.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f6b02600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af404db30 con 0x7f6ae40047c0 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3ef5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3ef6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef017d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3ef5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46872/0 (socket says 172.21.15.55:46872) 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3ef5400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ef003c290 0x7f3ef004cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3ef7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00ca800 con 0x7f3ef00e7ec0 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3ef6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef017d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33274/0 (socket says 172.21.15.55:33274) 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3ef6800640 1 -- 172.21.15.55:0/175932823 learned_addr learned my addr 172.21.15.55:0/175932823 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 -- 172.21.15.55:0/2163744954 shutdown_connections 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f5bbfe00640 1 -- 172.21.15.55:0/2163744954 wait complete. 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 -- 172.21.15.55:0/1986872980 shutdown_connections 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 --2- 172.21.15.55:0/1986872980 >> v2:172.21.15.162:3300/0 conn(0x7fdbbc0cdef0 0x7fdbbc0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 --2- 172.21.15.55:0/1986872980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0b6bf0 0x7fdbbc15dd50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7fdbc3200640 1 -- 172.21.15.55:0/1986872980 >> 172.21.15.55:0/1986872980 conn(0x7fdbbc0d32c0 msgr2=0x7fdbbc0d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00e5280 con 0x7f3ef0007330 2024-05-10T03:30:13.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00e4fd0 con 0x7f3ef003c290 2024-05-10T03:30:13.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3fc4c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3fc4c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46860/0 (socket says 172.21.15.55:46860) 2024-05-10T03:30:13.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3fc4c00640 1 -- 172.21.15.55:0/4243135162 learned_addr learned my addr 172.21.15.55:0/4243135162 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.198+0000 7f3fc6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7fdbc3200640 1 -- 172.21.15.55:0/1986872980 shutdown_connections 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7fdbc3200640 1 -- 172.21.15.55:0/1986872980 wait complete. 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc013f890 con 0x7f3fc0144b60 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc013e8d0 con 0x7f3fc0162840 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f6b01c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f6b01c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46878/0 (socket says 172.21.15.55:46878) 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.199+0000 7f6b01c00640 1 -- 172.21.15.55:0/1048941428 learned_addr learned my addr 172.21.15.55:0/1048941428 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f6b01200640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae40047c0 0x7f6af40064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3fc5600640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 0x7f3fc013e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3ef5e00640 1 -- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ef003c290 msgr2=0x7f3ef004cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3ef5e00640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ef003c290 0x7f3ef004cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3ef5e00640 1 -- 172.21.15.55:0/175932823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 msgr2=0x7f3ef017d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3ef5e00640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef017d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f3ef5e00640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8074030 con 0x7f3ef0007330 2024-05-10T03:30:13.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f60fce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 0x7f60f80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f6afbe00640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af4006a20 0x7f6af404cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f60fce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60f806ab20 0x7f60f80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f6b01c00640 1 -- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae40047c0 msgr2=0x7f6af40064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f6b01c00640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae40047c0 0x7f6af40064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f6b01c00640 1 -- 172.21.15.55:0/1048941428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af4006a20 msgr2=0x7f6af404cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f6b01c00640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af4006a20 0x7f6af404cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f6b01c00640 1 -- 172.21.15.55:0/1048941428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af40db7b0 con 0x7f6af404d300 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f60fce00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60f806af30 0x7f60f80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.200+0000 7f60fce00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f806b420 con 0x7f60f806ab20 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60fce00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f806b700 con 0x7f60f806af30 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60fce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f806b9e0 con 0x7f60f80d1880 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 0x7f60f80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 0x7f60f80b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48812/0 (socket says 172.21.15.55:48812) 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60f7400640 1 -- 172.21.15.55:0/921673186 learned_addr learned my addr 172.21.15.55:0/921673186 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f3ef5e00640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80690b0 tx=0x7f3ef80a7070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.201+0000 7f60f6a00640 1 --2- 172.21.15.55:0/921673186 >> v2:172.21.15.55:3300/0 conn(0x7f60f806ab20 0x7f60f80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f7e00640 1 --2- 172.21.15.55:0/921673186 >> v2:172.21.15.162:3300/0 conn(0x7f60f806af30 0x7f60f80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc5600640 1 -- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 msgr2=0x7f3fc013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f6b01c00640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6afc064350 tx=0x7f6afc072680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f49e8a00640 1 Processor -- start 2024-05-10T03:30:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f5d06400640 1 Processor -- start 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f49e8a00640 1 -- start start 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc5600640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc013f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc5600640 1 -- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 msgr2=0x7f3fc013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc5600640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc5600640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4009400 con 0x7f3fc00b2d00 2024-05-10T03:30:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3fc4c00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80b6810 con 0x7f3ef0007330 2024-05-10T03:30:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f5d06400640 1 -- start start 2024-05-10T03:30:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc08f070 con 0x7f6af404d300 2024-05-10T03:30:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0727b0 con 0x7f6af404d300 2024-05-10T03:30:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc06d040 con 0x7f6af404d300 2024-05-10T03:30:13.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef00e54d0 con 0x7f3ef0007330 2024-05-10T03:30:13.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80b6970 con 0x7f3ef0007330 2024-05-10T03:30:13.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80b6ce0 con 0x7f3ef0007330 2024-05-10T03:30:13.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.203+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40dba90 con 0x7f6af404d300 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f6a00640 1 -- 172.21.15.55:0/921673186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 msgr2=0x7f60f80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f6a00640 1 --2- 172.21.15.55:0/921673186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 0x7f60f80b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f6a00640 1 -- 172.21.15.55:0/921673186 >> v2:172.21.15.162:3300/0 conn(0x7f60f806af30 msgr2=0x7f60f80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f6a00640 1 --2- 172.21.15.55:0/921673186 >> v2:172.21.15.162:3300/0 conn(0x7f60f806af30 0x7f60f80ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.202+0000 7f60f6a00640 1 -- 172.21.15.55:0/921673186 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80b6a50 con 0x7f60f806ab20 2024-05-10T03:30:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.203+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af4178f10 con 0x7f6af404d300 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f5bbfe00640 1 Processor -- start 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f5bbfe00640 1 -- start start 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00e5a90 con 0x7f3ef0007330 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f60f6a00640 1 --2- 172.21.15.55:0/921673186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f806ab20 0x7f60f80ce250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60f80d3150 tx=0x7f60e402a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e47dac0a5dbe0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f60f6000640 1 -- 172.21.15.55:0/921673186 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e402ad60 con 0x7f60f806ab20 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f60f6000640 1 -- 172.21.15.55:0/921673186 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e4032440 con 0x7f60f806ab20 2024-05-10T03:30:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.204+0000 7f60f6000640 1 -- 172.21.15.55:0/921673186 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4032810 con 0x7f60f806ab20 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 0x7f49e41574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41f8ec0 con 0x7f49e40ce470 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41561d0 con 0x7f49e40d2110 2024-05-10T03:30:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f49e8a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4157030 con 0x7f49e40b4430 2024-05-10T03:30:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7fdbc3200640 1 Processor -- start 2024-05-10T03:30:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f60fce00640 1 -- 172.21.15.55:0/921673186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f806ab20 msgr2=0x7f60f80ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f60fce00640 1 --2- 172.21.15.55:0/921673186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f806ab20 0x7f60f80ce250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60f80d3150 tx=0x7f60e402a010 comp rx=0 tx=0).stop 2024-05-10T03:30:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f3fc5600640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 0x7f3fc013e390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4031000 tx=0x7f3fb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.205+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb403c8d0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f60fce00640 1 -- 172.21.15.55:0/921673186 shutdown_connections 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f60fce00640 1 --2- 172.21.15.55:0/921673186 >> v2:172.21.15.162:3300/0 conn(0x7f60f806af30 0x7f60f80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f60fce00640 1 --2- 172.21.15.55:0/921673186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f806ab20 0x7f60f80ce250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f60fce00640 1 --2- 172.21.15.55:0/921673186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f80d1880 0x7f60f80b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f60fce00640 1 -- 172.21.15.55:0/921673186 >> 172.21.15.55:0/921673186 conn(0x7f60f80d31a0 msgr2=0x7f60f80d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7fdbc3200640 1 -- start start 2024-05-10T03:30:13.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0145030 con 0x7f3fc00b2d00 2024-05-10T03:30:13.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f49e2800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f49e1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 0x7f49e41574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef00054d0 con 0x7f3ef0007330 2024-05-10T03:30:13.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8005250 con 0x7f6af404d300 2024-05-10T03:30:13.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f60fce00640 1 -- 172.21.15.55:0/921673186 shutdown_connections 2024-05-10T03:30:13.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f60fce00640 1 -- 172.21.15.55:0/921673186 wait complete. 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f49e2800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46904/0 (socket says 172.21.15.55:46904) 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f49e1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48818/0 (socket says 172.21.15.55:48818) 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f49e1e00640 1 -- 172.21.15.55:0/2525458226 learned_addr learned my addr 172.21.15.55:0/2525458226 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb403ca30 con 0x7f3fc00b2d00 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb403c8d0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc013fdc0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f49e1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 0x7f49e41574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33312/0 (socket says 172.21.15.55:33312) 2024-05-10T03:30:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d000ce790 0x7f5d000c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d000d1880 0x7f5d000cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000c98e0 con 0x7f5d000b23e0 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cbf80 con 0x7f5d000d1880 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.206+0000 7f5d06400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00154c00 con 0x7f5d000ce790 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5d05000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5cffe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d000ce790 0x7f5d000c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5d05000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33316/0 (socket says 172.21.15.55:33316) 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5d05000640 1 -- 172.21.15.55:0/3539543813 learned_addr learned my addr 172.21.15.55:0/3539543813 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 -- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d000ce790 msgr2=0x7f5d000c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d000ce790 0x7f5d000c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 -- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d000d1880 msgr2=0x7f5d000cc410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d000d1880 0x7f5d000cc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 -- 172.21.15.55:0/3539543813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec009400 con 0x7f5d000b23e0 2024-05-10T03:30:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5d05000640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400b3e0 tx=0x7f5cf400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f49e1e00640 1 -- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 msgr2=0x7f49e4157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f49e1e00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e4157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f49e1e00640 1 -- 172.21.15.55:0/2525458226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 msgr2=0x7f49e41574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f49e1e00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 0x7f49e41574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7fdbc3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbbc0b6bf0 0x7fdbbc148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7fdbc3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 0x7fdbbc148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f49e1e00640 1 -- 172.21.15.55:0/2525458226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0093a0 con 0x7f49e40b4430 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 0x7f5bc015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 0x7f5bc014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc0070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c2c90 con 0x7f5bc01618b0 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc006eda0 con 0x7f5bc00b6680 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.207+0000 7f5bbfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c1dc0 con 0x7f5bc00b2170 2024-05-10T03:30:13.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.208+0000 7f5bbe000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 0x7f5bc014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5bbea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 0x7f5bc015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5bbe000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 0x7f5bc014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46920/0 (socket says 172.21.15.55:46920) 2024-05-10T03:30:13.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5bbe000640 1 -- 172.21.15.55:0/3347914396 learned_addr learned my addr 172.21.15.55:0/3347914396 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7fdbc3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 0x7fdbbc147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4011830 con 0x7f5d000b23e0 2024-05-10T03:30:13.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf40119b0 con 0x7f5d000b23e0 2024-05-10T03:30:13.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.209+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4011d60 con 0x7f5d000b23e0 2024-05-10T03:30:13.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbc3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1473f0 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbbb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 0x7fdbbc148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbbb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 0x7fdbbc148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46936/0 (socket says 172.21.15.55:46936) 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbbb400640 1 -- 172.21.15.55:0/3511202982 learned_addr learned my addr 172.21.15.55:0/3511202982 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbbc0b6bf0 0x7fdbbc148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc070b30 con 0x7fdbbc0cd9a0 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc159b00 con 0x7fdbbc0b6bf0 2024-05-10T03:30:13.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc01451c0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f60fce00640 1 Processor -- start 2024-05-10T03:30:13.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3ef80b4040 con 0x7f3ef0007330 2024-05-10T03:30:13.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f60fce00640 1 -- start start 2024-05-10T03:30:13.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbf400640 1 -- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 msgr2=0x7f5bc015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbf400640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 0x7f5bc015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbf400640 1 -- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 msgr2=0x7f5bc014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbf400640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 0x7f5bc014f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbf400640 1 -- 172.21.15.55:0/3347914396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009b30 con 0x7f5bc01618b0 2024-05-10T03:30:13.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f5bbea00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 0x7f5bc015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:13.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.211+0000 7f49e1e00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007e60 tx=0x7f49d400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7fdbc1200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 0x7fdbbc147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f5bbf400640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc0070570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00d40f0 tx=0x7f5bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d000c93e0 con 0x7f5d000b23e0 2024-05-10T03:30:13.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d000cb650 con 0x7f5d000b23e0 2024-05-10T03:30:13.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d000cc0f0 con 0x7f5d000b23e0 2024-05-10T03:30:13.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.210+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6afc092070 con 0x7f6af404d300 2024-05-10T03:30:13.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f6af8c00640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 0x7f6adc06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6afc103640 con 0x7f6af404d300 2024-05-10T03:30:13.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f806af30 0x7f60f8147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 0x7f60f8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8146390 con 0x7f60f80d1880 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f815a4a0 con 0x7f60f806af30 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f60fce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8159680 con 0x7f60f806ab20 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7fdbc1200640 1 -- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbbc0b6bf0 msgr2=0x7fdbbc148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7fdbc1200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbbc0b6bf0 0x7fdbbc148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7fdbc1200640 1 -- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 msgr2=0x7fdbbc148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7fdbc1200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 0x7fdbbc148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7fdbc1200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4009a70 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f3eee000640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 0x7f3ed806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb803c630 con 0x7f5bc01618b0 2024-05-10T03:30:13.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4014620 con 0x7f49e40b4430 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f60f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f60f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48856/0 (socket says 172.21.15.55:48856) 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f60f7400640 1 -- 172.21.15.55:0/1944522775 learned_addr learned my addr 172.21.15.55:0/1944522775 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f60f6a00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f806af30 0x7f60f8147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400ee90 con 0x7f49e40b4430 2024-05-10T03:30:13.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40149a0 con 0x7f49e40b4430 2024-05-10T03:30:13.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7fdbc1200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 0x7fdbbc147810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0b2cf0 tx=0x7fdbac00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc014f580 con 0x7f5bc01618b0 2024-05-10T03:30:13.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb803c7b0 con 0x7f5bc01618b0 2024-05-10T03:30:13.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.213+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb803cb60 con 0x7f5bc01618b0 2024-05-10T03:30:13.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f3ef5400640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 0x7f3ed806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc00c22f0 con 0x7f5bc01618b0 2024-05-10T03:30:13.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.212+0000 7f6b01200640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 0x7f6adc06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f3ef5400640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 0x7f3ed806d110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00e75f0 tx=0x7f3ee4007c40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7400640 1 -- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f806af30 msgr2=0x7f60f8147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7400640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f806af30 0x7f60f8147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7e00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 0x7f60f8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7400640 1 -- 172.21.15.55:0/1944522775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 msgr2=0x7f60f8148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7400640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 0x7f60f8148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7f60f7400640 1 -- 172.21.15.55:0/1944522775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4009400 con 0x7f60f806ab20 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60f7e00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 0x7f60f8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60f7400640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60e800e870 tx=0x7f60e800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800c8e0 con 0x7f60f806ab20 2024-05-10T03:30:13.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e800ca40 con 0x7f60f806ab20 2024-05-10T03:30:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e800cdb0 con 0x7f60f806ab20 2024-05-10T03:30:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f8145a80 con 0x7f60f806ab20 2024-05-10T03:30:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.216+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f81598d0 con 0x7f60f806ab20 2024-05-10T03:30:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4156420 con 0x7f49e40b4430 2024-05-10T03:30:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41569e0 con 0x7f49e40b4430 2024-05-10T03:30:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.215+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3fb403cbc0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0052d0 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac005430 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.214+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac003830 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.216+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc06dbd0 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.216+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc160790 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.216+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5cf401a400 con 0x7f5d000b23e0 2024-05-10T03:30:13.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f6b01200640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 0x7f6adc06d210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6af4011ab0 tx=0x7f6af0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.216+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc0067740 con 0x7f5bc01618b0 2024-05-10T03:30:13.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f804e290 con 0x7f60f806ab20 2024-05-10T03:30:13.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f3fbd600640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 0x7f3fa006d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e4067930 con 0x7f49e40b4430 2024-05-10T03:30:13.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5bb803ccc0 con 0x7f5bc01618b0 2024-05-10T03:30:13.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3fb40a93e0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.218+0000 7f5cfcc00640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 0x7f5ce006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.218+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5cf4089eb0 con 0x7f5d000b23e0 2024-05-10T03:30:13.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.217+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc147530 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.218+0000 7f3fc4c00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 0x7f3fa006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.219+0000 7f5cffe00640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 0x7f5ce006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.219+0000 7f5bb7400640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 0x7f5ba006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.219+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5bb80a9fd0 con 0x7f5bc01618b0 2024-05-10T03:30:13.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.219+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f60e8013470 con 0x7f60f806ab20 2024-05-10T03:30:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.220+0000 7f5bbea00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 0x7f5ba006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.220+0000 7f3fc4c00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 0x7f3fa006d170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0146f60 tx=0x7f3fac009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.220+0000 7f5cffe00640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 0x7f5ce006d210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5cec034000 tx=0x7f5cec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.220+0000 7f60ef400640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 0x7f60d806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.220+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f60e8089ee0 con 0x7f60f806ab20 2024-05-10T03:30:13.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.221+0000 7f5bbea00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 0x7f5ba006d160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.221+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f49d4014b00 con 0x7f49e40b4430 2024-05-10T03:30:13.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.221+0000 7f60f6a00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 0x7f60d806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.222+0000 7f49da000640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 0x7f49c006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.222+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49d408aca0 con 0x7f49e40b4430 2024-05-10T03:30:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.222+0000 7f49e1400640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 0x7f49c006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.223+0000 7f60f6a00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 0x7f60d806d110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60e4034000 tx=0x7f60e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.223+0000 7f49e1400640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 0x7f49c006d110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49cc034000 tx=0x7f49cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.224+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3fb40ae060 con 0x7f3fc00b2d00 2024-05-10T03:30:13.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.228+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6afc0ce510 con 0x7f6af404d300 2024-05-10T03:30:13.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.228+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ef80a5e40 con 0x7f3ef0007330 2024-05-10T03:30:13.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.231+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5cf4054cf0 con 0x7f5d000b23e0 2024-05-10T03:30:13.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.236+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5bb80750a0 con 0x7f5bc01618b0 2024-05-10T03:30:13.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.237+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49d4055be0 con 0x7f49e40b4430 2024-05-10T03:30:13.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.240+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdbac003990 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.241+0000 7fdbb3e00640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 0x7fdb9c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:13.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.241+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbac089010 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.242+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 0x7fdb9c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:13.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.243+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ef8129060 con 0x7f3ef0007330 2024-05-10T03:30:13.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.243+0000 7fdbbbe00640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 0x7fdb9c06d210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4034000 tx=0x7fdbb402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:13.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.245+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdbac054050 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.255+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f60e8054e90 con 0x7f60f806ab20 2024-05-10T03:30:13.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.675+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3fc0066ce0 con 0x7f3fc00b2d00 2024-05-10T03:30:13.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.675+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6ad80057d0 con 0x7f6af404d300 2024-05-10T03:30:13.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.676+0000 7f3fbd600640 1 -- 172.21.15.55:0/4243135162 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3fb4074260 con 0x7f3fc00b2d00 2024-05-10T03:30:13.683 INFO:teuthology.orchestra.run.smithi055.stdout:55834574859 2024-05-10T03:30:13.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.676+0000 7f6af8c00640 1 -- 172.21.15.55:0/1048941428 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6afc099020 con 0x7f6af404d300 2024-05-10T03:30:13.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.677+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5d0004e290 con 0x7f5d000b23e0 2024-05-10T03:30:13.684 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:13.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.678+0000 7f5cfcc00640 1 -- 172.21.15.55:0/3539543813 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cf4015020 con 0x7f5d000b23e0 2024-05-10T03:30:13.685 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:13.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.679+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 msgr2=0x7f3fa006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.679+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 0x7f3fa006d170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0146f60 tx=0x7f3fac009110 comp rx=0 tx=0).stop 2024-05-10T03:30:13.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.679+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 msgr2=0x7f3fc013e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.679+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 0x7f3fc013e390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4031000 tx=0x7f3fb40046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.679+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f49e406bcd0 con 0x7f49e40b4430 2024-05-10T03:30:13.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 shutdown_connections 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f49da000640 1 -- 172.21.15.55:0/2525458226 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49d4059200 con 0x7f49e40b4430 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5bc00d0f50 con 0x7f5bc01618b0 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0162840 0x7f3fc013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3fa006ac60 0x7f3fa006d170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fc0144b60 0x7f3fc013ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 --2- 172.21.15.55:0/4243135162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3fc00b2d00 0x7f3fc013e390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 >> 172.21.15.55:0/4243135162 conn(0x7f3fc006a110 msgr2=0x7f3fc0164410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.688 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 msgr2=0x7f6adc06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 0x7f6adc06d210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6af4011ab0 tx=0x7f6af0008040 comp rx=0 tx=0).stop 2024-05-10T03:30:13.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 msgr2=0x7f6af41789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6afc064350 tx=0x7f6afc072680 comp rx=0 tx=0).stop 2024-05-10T03:30:13.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.680+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 shutdown_connections 2024-05-10T03:30:13.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 shutdown_connections 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6adc06ad00 0x7f6adc06d210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6af404d300 0x7f6af41789d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af4006a20 0x7f6af404cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 --2- 172.21.15.55:0/1048941428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ae40047c0 0x7f6af40064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 >> 172.21.15.55:0/1048941428 conn(0x7f6af403abc0 msgr2=0x7f6af40e2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 shutdown_connections 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f6b02600640 1 -- 172.21.15.55:0/1048941428 wait complete. 2024-05-10T03:30:13.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f3fc6a00640 1 -- 172.21.15.55:0/4243135162 wait complete. 2024-05-10T03:30:13.691 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-10T03:30:13.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.681+0000 7f5bb7400640 1 -- 172.21.15.55:0/3347914396 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5bb8040090 con 0x7f5bc01618b0 2024-05-10T03:30:13.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.682+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 msgr2=0x7f5ce006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.682+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 0x7f5ce006d210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5cec034000 tx=0x7f5cec02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.682+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 msgr2=0x7f5d001546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.682+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400b3e0 tx=0x7f5cf400b8a0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.683+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 msgr2=0x7f49c006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.683+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 0x7f49c006d110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49cc034000 tx=0x7f49cc02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.683+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 msgr2=0x7f49e41556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.683+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49d4007e60 tx=0x7f49d400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 shutdown_connections 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 shutdown_connections 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ce006ad00 0x7f5ce006d210 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d000d1880 0x7f5d000cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d000ce790 0x7f5d000c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 --2- 172.21.15.55:0/3539543813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d000b23e0 0x7f5d001546c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 >> 172.21.15.55:0/3539543813 conn(0x7f5d000d31a0 msgr2=0x7f5d0015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c006ac00 0x7f49c006d110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e40d2110 0x7f49e4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e40ce470 0x7f49e41574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 --2- 172.21.15.55:0/2525458226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e40b4430 0x7f49e41556f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 >> 172.21.15.55:0/2525458226 conn(0x7f49e40d0720 msgr2=0x7f49e40b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 shutdown_connections 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f5d06400640 1 -- 172.21.15.55:0/3539543813 wait complete. 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 shutdown_connections 2024-05-10T03:30:13.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.685+0000 7f49e8a00640 1 -- 172.21.15.55:0/2525458226 wait complete. 2024-05-10T03:30:13.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.684+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3ef00054d0 con 0x7f3ef0007330 2024-05-10T03:30:13.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.685+0000 7f3eee000640 1 -- 172.21.15.55:0/175932823 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ef8129070 con 0x7f3ef0007330 2024-05-10T03:30:13.695 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.685+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 msgr2=0x7f5ba006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.685+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 0x7f5ba006d160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 msgr2=0x7f5bc0070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc0070570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00d40f0 tx=0x7f5bb8004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 shutdown_connections 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bc01618b0 0x7f5bc0070570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5ba006ac50 0x7f5ba006d160 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bc00b6680 0x7f5bc014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 --2- 172.21.15.55:0/3347914396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bc00b2170 0x7f5bc015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.686+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 >> 172.21.15.55:0/3347914396 conn(0x7f5bc00b5730 msgr2=0x7f5bc00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.687+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 shutdown_connections 2024-05-10T03:30:13.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.687+0000 7f5bbfe00640 1 -- 172.21.15.55:0/3347914396 wait complete. 2024-05-10T03:30:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdbbc147530 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 msgr2=0x7f3ed806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 0x7f3ed806d110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00e75f0 tx=0x7f3ee4007c40 comp rx=0 tx=0).stop 2024-05-10T03:30:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 msgr2=0x7f3ef00e6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80690b0 tx=0x7f3ef80a7070 comp rx=0 tx=0).stop 2024-05-10T03:30:13.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 shutdown_connections 2024-05-10T03:30:13.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ef00e7ec0 0x7f3ef017d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ef003c290 0x7f3ef004cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.690+0000 7fdbb3e00640 1 -- 172.21.15.55:0/3511202982 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdbbc147530 con 0x7fdbbc0cdef0 2024-05-10T03:30:13.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3ed806ac00 0x7f3ed806d110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 --2- 172.21.15.55:0/175932823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ef0007330 0x7f3ef00e6570 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80690b0 tx=0x7f3ef80a7070 comp rx=0 tx=0).stop 2024-05-10T03:30:13.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.689+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 >> 172.21.15.55:0/175932823 conn(0x7f3ef003ab50 msgr2=0x7f3ef003c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.690+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 shutdown_connections 2024-05-10T03:30:13.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.690+0000 7f3ef7200640 1 -- 172.21.15.55:0/175932823 wait complete. 2024-05-10T03:30:13.699 INFO:teuthology.orchestra.run.smithi055.stdout:55834574860 2024-05-10T03:30:13.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.694+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 msgr2=0x7fdb9c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.694+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 0x7fdb9c06d210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4034000 tx=0x7fdbb402f000 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.694+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 msgr2=0x7fdbbc147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.694+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 0x7fdbbc147810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0b2cf0 tx=0x7fdbac00c820 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 shutdown_connections 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbbc0cdef0 0x7fdbbc147810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb9c06ad00 0x7fdb9c06d210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbbc0cd9a0 0x7fdbbc148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 --2- 172.21.15.55:0/3511202982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbbc0b6bf0 0x7fdbbc148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 >> 172.21.15.55:0/3511202982 conn(0x7fdbbc0d32c0 msgr2=0x7fdbbc15b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 shutdown_connections 2024-05-10T03:30:13.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.695+0000 7fdbc3200640 1 -- 172.21.15.55:0/3511202982 wait complete. 2024-05-10T03:30:13.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.696+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f60f80b1d90 con 0x7f60f806ab20 2024-05-10T03:30:13.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.697+0000 7f60ef400640 1 -- 172.21.15.55:0/1944522775 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60e8021090 con 0x7f60f806ab20 2024-05-10T03:30:13.705 INFO:teuthology.orchestra.run.smithi055.stdout:55834574858 2024-05-10T03:30:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.701+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 msgr2=0x7f60d806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.701+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 0x7f60d806d110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60e4034000 tx=0x7f60e402e000 comp rx=0 tx=0).stop 2024-05-10T03:30:13.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.701+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 msgr2=0x7f60f814b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:13.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.701+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60e800e870 tx=0x7f60e800ed30 comp rx=0 tx=0).stop 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 shutdown_connections 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80d1880 0x7f60f8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60d806ac00 0x7f60d806d110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f806af30 0x7f60f8147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 --2- 172.21.15.55:0/1944522775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f806ab20 0x7f60f814b3a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:13.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.702+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 >> 172.21.15.55:0/1944522775 conn(0x7f60f80d31a0 msgr2=0x7f60f80b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:13.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.703+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 shutdown_connections 2024-05-10T03:30:13.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:13.703+0000 7f60fce00640 1 -- 172.21.15.55:0/1944522775 wait complete. 2024-05-10T03:30:13.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-05-10T03:30:13.730 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-05-10T03:30:13.731 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2024-05-10T03:30:13.734 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-05-10T03:30:13.736 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2024-05-10T03:30:13.737 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.1 2024-05-10T03:30:13.740 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2024-05-10T03:30:13.741 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-05-10T03:30:13.742 DEBUG:teuthology.parallel:result is None 2024-05-10T03:30:13.742 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T03:30:13.742 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:30:14.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.044+0000 7fa40f200640 1 Processor -- start 2024-05-10T03:30:14.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.044+0000 7fa40f200640 1 -- start start 2024-05-10T03:30:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.046+0000 7fa40f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa410144120 0x7fa41006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.046+0000 7fa40f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4100b3190 0x7fa4100cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.046+0000 7fa40f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41006d350 con 0x7fa410144710 2024-05-10T03:30:14.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.046+0000 7fa40f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa410144530 con 0x7fa4100b3190 2024-05-10T03:30:14.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.046+0000 7fa40f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41006dd20 con 0x7fa410144120 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4100b3190 0x7fa4100cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4100b3190 0x7fa4100cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33354/0 (socket says 172.21.15.55:33354) 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 -- 172.21.15.55:0/2160134715 learned_addr learned my addr 172.21.15.55:0/2160134715 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa405400640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa410144120 0x7fa41006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 -- 172.21.15.55:0/2160134715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa410144120 msgr2=0x7fa41006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa410144120 0x7fa41006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 -- 172.21.15.55:0/2160134715 >> v1:172.21.15.162:6789/0 conn(0x7fa410144710 legacy=0x7fa4100b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.047+0000 7fa40e800640 1 -- 172.21.15.55:0/2160134715 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41006c740 con 0x7fa4100b3190 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.048+0000 7fa40e800640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4100b3190 0x7fa4100cd9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9de73010f52534 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.048+0000 7fa40d400640 1 -- 172.21.15.55:0/2160134715 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408031600 con 0x7fa4100b3190 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40d400640 1 -- 172.21.15.55:0/2160134715 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40802d050 con 0x7fa4100b3190 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40d400640 1 -- 172.21.15.55:0/2160134715 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408031980 con 0x7fa4100b3190 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 -- 172.21.15.55:0/2160134715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4100b3190 msgr2=0x7fa4100cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4100b3190 0x7fa4100cd9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a5f0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 -- 172.21.15.55:0/2160134715 shutdown_connections 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4100b3190 0x7fa4100cd9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 --2- 172.21.15.55:0/2160134715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa410144120 0x7fa41006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.049+0000 7fa40f200640 1 -- 172.21.15.55:0/2160134715 >> 172.21.15.55:0/2160134715 conn(0x7fa41006a110 msgr2=0x7fa41006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.050+0000 7fa40f200640 1 -- 172.21.15.55:0/2160134715 shutdown_connections 2024-05-10T03:30:14.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.050+0000 7fa40f200640 1 -- 172.21.15.55:0/2160134715 wait complete. 2024-05-10T03:30:14.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.052+0000 7fa40f200640 1 Processor -- start 2024-05-10T03:30:14.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.052+0000 7fa40f200640 1 -- start start 2024-05-10T03:30:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.053+0000 7fa40f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4100b3190 0x7fa41015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.053+0000 7fa40f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.053+0000 7fa40f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa410144710 0x7fa41015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa40f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410071d70 con 0x7fa410144120 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa40f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015e9f0 con 0x7fa410144710 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa40f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100705a0 con 0x7fa4100b3190 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33362/0 (socket says 172.21.15.55:33362) 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 -- 172.21.15.55:0/3771406607 learned_addr learned my addr 172.21.15.55:0/3771406607 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa40de00640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4100b3190 0x7fa41015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa40e800640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa410144710 0x7fa41015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 -- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4100b3190 msgr2=0x7fa41015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4100b3190 0x7fa41015abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 -- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa410144710 msgr2=0x7fa41015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa410144710 0x7fa41015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.054+0000 7fa405400640 1 -- 172.21.15.55:0/3771406607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408009b30 con 0x7fa410144120 2024-05-10T03:30:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa405400640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d5a0 tx=0x7fa3f800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80125f0 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8012750 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa410139e00 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8012ae0 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.055+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa410139560 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.056+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa410066ce0 con 0x7fa410144120 2024-05-10T03:30:14.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.057+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa3f8012c40 con 0x7fa410144120 2024-05-10T03:30:14.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.058+0000 7fa406a00640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 0x7fa3ec06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.059+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3f8089cb0 con 0x7fa410144120 2024-05-10T03:30:14.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.060+0000 7fa40de00640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 0x7fa3ec06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.062+0000 7fa40de00640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 0x7fa3ec06d110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa400007fd0 tx=0x7fa400009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.063+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa3f8054a70 con 0x7fa410144120 2024-05-10T03:30:14.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.252+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa41006cfe0 con 0x7fa3ec06ac00 2024-05-10T03:30:14.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.279+0000 7fa406a00640 1 -- 172.21.15.55:0/3771406607 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345044 (secure 0 0 0) 0x7fa4000050d0 con 0x7fa3ec06ac00 2024-05-10T03:30:14.289 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 msgr2=0x7fa3ec06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 0x7fa3ec06d110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa400007fd0 tx=0x7fa400009040 comp rx=0 tx=0).stop 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 msgr2=0x7fa41015e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d5a0 tx=0x7fa3f800da60 comp rx=0 tx=0).stop 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 shutdown_connections 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa3ec06ac00 0x7fa3ec06d110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa410144710 0x7fa41015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa410144120 0x7fa41015e1f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 --2- 172.21.15.55:0/3771406607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4100b3190 0x7fa41015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 >> 172.21.15.55:0/3771406607 conn(0x7fa41006a110 msgr2=0x7fa4100b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 shutdown_connections 2024-05-10T03:30:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.287+0000 7fa40f200640 1 -- 172.21.15.55:0/3771406607 wait complete. 2024-05-10T03:30:14.321 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-05-10T03:30:12.954695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231796,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754742924,"statfs":{"total":773094113280,"available":772856754176,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005604"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.796272+0000","last_change":"2024-05-10T03:29:43.770873+0000","last_active":"2024-05-10T03:29:48.796272+0000","last_peered":"2024-05-10T03:29:48.796272+0000","last_clean":"2024-05-10T03:29:48.796272+0000","last_became_active":"2024-05-10T03:29:43.770443+0000","last_became_peered":"2024-05-10T03:29:43.770443+0000","last_unstale":"2024-05-10T03:29:48.796272+0000","last_undegraded":"2024-05-10T03:29:48.796272+0000","last_fullsized":"2024-05-10T03:29:48.796272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:55.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.814450+0000","last_change":"2024-05-10T03:29:44.352733+0000","last_active":"2024-05-10T03:29:48.814450+0000","last_peered":"2024-05-10T03:29:48.814450+0000","last_clean":"2024-05-10T03:29:48.814450+0000","last_became_active":"2024-05-10T03:29:44.351354+0000","last_became_peered":"2024-05-10T03:29:44.351354+0000","last_unstale":"2024-05-10T03:29:48.814450+0000","last_undegraded":"2024-05-10T03:29:48.814450+0000","last_fullsized":"2024-05-10T03:29:48.814450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367124+0000","last_change":"2024-05-10T03:29:49.255640+0000","last_active":"2024-05-10T03:29:49.367124+0000","last_peered":"2024-05-10T03:29:49.367124+0000","last_clean":"2024-05-10T03:29:49.367124+0000","last_became_active":"2024-05-10T03:29:49.254068+0000","last_became_peered":"2024-05-10T03:29:49.254068+0000","last_unstale":"2024-05-10T03:29:49.367124+0000","last_undegraded":"2024-05-10T03:29:49.367124+0000","last_fullsized":"2024-05-10T03:29:49.367124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:09.608357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813230+0000","last_change":"2024-05-10T03:29:43.768485+0000","last_active":"2024-05-10T03:29:48.813230+0000","last_peered":"2024-05-10T03:29:48.813230+0000","last_clean":"2024-05-10T03:29:48.813230+0000","last_became_active":"2024-05-10T03:29:43.767506+0000","last_became_peered":"2024-05-10T03:29:43.767506+0000","last_unstale":"2024-05-10T03:29:48.813230+0000","last_undegraded":"2024-05-10T03:29:48.813230+0000","last_fullsized":"2024-05-10T03:29:48.813230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795311+0000","last_change":"2024-05-10T03:29:44.339118+0000","last_active":"2024-05-10T03:29:48.795311+0000","last_peered":"2024-05-10T03:29:48.795311+0000","last_clean":"2024-05-10T03:29:48.795311+0000","last_became_active":"2024-05-10T03:29:44.338690+0000","last_became_peered":"2024-05-10T03:29:44.338690+0000","last_unstale":"2024-05-10T03:29:48.795311+0000","last_undegraded":"2024-05-10T03:29:48.795311+0000","last_fullsized":"2024-05-10T03:29:48.795311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:23.150055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909491+0000","last_change":"2024-05-10T03:29:48.819825+0000","last_active":"2024-05-10T03:29:49.909491+0000","last_peered":"2024-05-10T03:29:49.909491+0000","last_clean":"2024-05-10T03:29:49.909491+0000","last_became_active":"2024-05-10T03:29:48.818817+0000","last_became_peered":"2024-05-10T03:29:48.818817+0000","last_unstale":"2024-05-10T03:29:49.909491+0000","last_undegraded":"2024-05-10T03:29:49.909491+0000","last_fullsized":"2024-05-10T03:29:49.909491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:10.138375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247614+0000","last_change":"2024-05-10T03:29:43.777967+0000","last_active":"2024-05-10T03:29:49.247614+0000","last_peered":"2024-05-10T03:29:49.247614+0000","last_clean":"2024-05-10T03:29:49.247614+0000","last_became_active":"2024-05-10T03:29:43.777585+0000","last_became_peered":"2024-05-10T03:29:43.777585+0000","last_unstale":"2024-05-10T03:29:49.247614+0000","last_undegraded":"2024-05-10T03:29:49.247614+0000","last_fullsized":"2024-05-10T03:29:49.247614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797875+0000","last_change":"2024-05-10T03:29:44.343238+0000","last_active":"2024-05-10T03:29:48.797875+0000","last_peered":"2024-05-10T03:29:48.797875+0000","last_clean":"2024-05-10T03:29:48.797875+0000","last_became_active":"2024-05-10T03:29:44.339308+0000","last_became_peered":"2024-05-10T03:29:44.339308+0000","last_unstale":"2024-05-10T03:29:48.797875+0000","last_undegraded":"2024-05-10T03:29:48.797875+0000","last_fullsized":"2024-05-10T03:29:48.797875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910600+0000","last_change":"2024-05-10T03:29:48.821031+0000","last_active":"2024-05-10T03:29:49.910600+0000","last_peered":"2024-05-10T03:29:49.910600+0000","last_clean":"2024-05-10T03:29:49.910600+0000","last_became_active":"2024-05-10T03:29:48.818081+0000","last_became_peered":"2024-05-10T03:29:48.818081+0000","last_unstale":"2024-05-10T03:29:49.910600+0000","last_undegraded":"2024-05-10T03:29:49.910600+0000","last_fullsized":"2024-05-10T03:29:49.910600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243722+0000","last_change":"2024-05-10T03:29:43.776394+0000","last_active":"2024-05-10T03:29:49.243722+0000","last_peered":"2024-05-10T03:29:49.243722+0000","last_clean":"2024-05-10T03:29:49.243722+0000","last_became_active":"2024-05-10T03:29:43.774971+0000","last_became_peered":"2024-05-10T03:29:43.774971+0000","last_unstale":"2024-05-10T03:29:49.243722+0000","last_undegraded":"2024-05-10T03:29:49.243722+0000","last_fullsized":"2024-05-10T03:29:49.243722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798050+0000","last_change":"2024-05-10T03:29:44.336984+0000","last_active":"2024-05-10T03:29:48.798050+0000","last_peered":"2024-05-10T03:29:48.798050+0000","last_clean":"2024-05-10T03:29:48.798050+0000","last_became_active":"2024-05-10T03:29:44.335925+0000","last_became_peered":"2024-05-10T03:29:44.335925+0000","last_unstale":"2024-05-10T03:29:48.798050+0000","last_undegraded":"2024-05-10T03:29:48.798050+0000","last_fullsized":"2024-05-10T03:29:48.798050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:25.887533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909455+0000","last_change":"2024-05-10T03:29:48.818620+0000","last_active":"2024-05-10T03:29:49.909455+0000","last_peered":"2024-05-10T03:29:49.909455+0000","last_clean":"2024-05-10T03:29:49.909455+0000","last_became_active":"2024-05-10T03:29:48.817907+0000","last_became_peered":"2024-05-10T03:29:48.817907+0000","last_unstale":"2024-05-10T03:29:49.909455+0000","last_undegraded":"2024-05-10T03:29:49.909455+0000","last_fullsized":"2024-05-10T03:29:49.909455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808538+0000","last_change":"2024-05-10T03:29:44.347407+0000","last_active":"2024-05-10T03:29:48.808538+0000","last_peered":"2024-05-10T03:29:48.808538+0000","last_clean":"2024-05-10T03:29:48.808538+0000","last_became_active":"2024-05-10T03:29:44.347072+0000","last_became_peered":"2024-05-10T03:29:44.347072+0000","last_unstale":"2024-05-10T03:29:48.808538+0000","last_undegraded":"2024-05-10T03:29:48.808538+0000","last_fullsized":"2024-05-10T03:29:48.808538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243215+0000","last_change":"2024-05-10T03:29:43.775422+0000","last_active":"2024-05-10T03:29:49.243215+0000","last_peered":"2024-05-10T03:29:49.243215+0000","last_clean":"2024-05-10T03:29:49.243215+0000","last_became_active":"2024-05-10T03:29:43.775083+0000","last_became_peered":"2024-05-10T03:29:43.775083+0000","last_unstale":"2024-05-10T03:29:49.243215+0000","last_undegraded":"2024-05-10T03:29:49.243215+0000","last_fullsized":"2024-05-10T03:29:49.243215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910633+0000","last_change":"2024-05-10T03:29:48.812082+0000","last_active":"2024-05-10T03:29:49.910633+0000","last_peered":"2024-05-10T03:29:49.910633+0000","last_clean":"2024-05-10T03:29:49.910633+0000","last_became_active":"2024-05-10T03:29:48.809401+0000","last_became_peered":"2024-05-10T03:29:48.809401+0000","last_unstale":"2024-05-10T03:29:49.910633+0000","last_undegraded":"2024-05-10T03:29:49.910633+0000","last_fullsized":"2024-05-10T03:29:49.910633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794221+0000","last_change":"2024-05-10T03:29:44.355438+0000","last_active":"2024-05-10T03:29:48.794221+0000","last_peered":"2024-05-10T03:29:48.794221+0000","last_clean":"2024-05-10T03:29:48.794221+0000","last_became_active":"2024-05-10T03:29:44.354716+0000","last_became_peered":"2024-05-10T03:29:44.354716+0000","last_unstale":"2024-05-10T03:29:48.794221+0000","last_undegraded":"2024-05-10T03:29:48.794221+0000","last_fullsized":"2024-05-10T03:29:48.794221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797885+0000","last_change":"2024-05-10T03:29:43.761646+0000","last_active":"2024-05-10T03:29:48.797885+0000","last_peered":"2024-05-10T03:29:48.797885+0000","last_clean":"2024-05-10T03:29:48.797885+0000","last_became_active":"2024-05-10T03:29:43.759174+0000","last_became_peered":"2024-05-10T03:29:43.759174+0000","last_unstale":"2024-05-10T03:29:48.797885+0000","last_undegraded":"2024-05-10T03:29:48.797885+0000","last_fullsized":"2024-05-10T03:29:48.797885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910230+0000","last_change":"2024-05-10T03:29:48.822517+0000","last_active":"2024-05-10T03:29:49.910230+0000","last_peered":"2024-05-10T03:29:49.910230+0000","last_clean":"2024-05-10T03:29:49.910230+0000","last_became_active":"2024-05-10T03:29:48.821950+0000","last_became_peered":"2024-05-10T03:29:48.821950+0000","last_unstale":"2024-05-10T03:29:49.910230+0000","last_undegraded":"2024-05-10T03:29:49.910230+0000","last_fullsized":"2024-05-10T03:29:49.910230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:56:50.388163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808692+0000","last_change":"2024-05-10T03:29:44.324359+0000","last_active":"2024-05-10T03:29:48.808692+0000","last_peered":"2024-05-10T03:29:48.808692+0000","last_clean":"2024-05-10T03:29:48.808692+0000","last_became_active":"2024-05-10T03:29:44.323910+0000","last_became_peered":"2024-05-10T03:29:44.323910+0000","last_unstale":"2024-05-10T03:29:48.808692+0000","last_undegraded":"2024-05-10T03:29:48.808692+0000","last_fullsized":"2024-05-10T03:29:48.808692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794312+0000","last_change":"2024-05-10T03:29:43.771353+0000","last_active":"2024-05-10T03:29:48.794312+0000","last_peered":"2024-05-10T03:29:48.794312+0000","last_clean":"2024-05-10T03:29:48.794312+0000","last_became_active":"2024-05-10T03:29:43.769654+0000","last_became_peered":"2024-05-10T03:29:43.769654+0000","last_unstale":"2024-05-10T03:29:48.794312+0000","last_undegraded":"2024-05-10T03:29:48.794312+0000","last_fullsized":"2024-05-10T03:29:48.794312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494881+0000","last_change":"2024-05-10T03:29:49.254990+0000","last_active":"2024-05-10T03:29:49.494881+0000","last_peered":"2024-05-10T03:29:49.494881+0000","last_clean":"2024-05-10T03:29:49.494881+0000","last_became_active":"2024-05-10T03:29:49.253789+0000","last_became_peered":"2024-05-10T03:29:49.253789+0000","last_unstale":"2024-05-10T03:29:49.494881+0000","last_undegraded":"2024-05-10T03:29:49.494881+0000","last_fullsized":"2024-05-10T03:29:49.494881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790993+0000","last_change":"2024-05-10T03:29:44.347866+0000","last_active":"2024-05-10T03:29:48.790993+0000","last_peered":"2024-05-10T03:29:48.790993+0000","last_clean":"2024-05-10T03:29:48.790993+0000","last_became_active":"2024-05-10T03:29:44.346222+0000","last_became_peered":"2024-05-10T03:29:44.346222+0000","last_unstale":"2024-05-10T03:29:48.790993+0000","last_undegraded":"2024-05-10T03:29:48.790993+0000","last_fullsized":"2024-05-10T03:29:48.790993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241506+0000","last_change":"2024-05-10T03:29:43.775452+0000","last_active":"2024-05-10T03:29:49.241506+0000","last_peered":"2024-05-10T03:29:49.241506+0000","last_clean":"2024-05-10T03:29:49.241506+0000","last_became_active":"2024-05-10T03:29:43.774420+0000","last_became_peered":"2024-05-10T03:29:43.774420+0000","last_unstale":"2024-05-10T03:29:49.241506+0000","last_undegraded":"2024-05-10T03:29:49.241506+0000","last_fullsized":"2024-05-10T03:29:49.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910708+0000","last_change":"2024-05-10T03:29:48.815642+0000","last_active":"2024-05-10T03:29:49.910708+0000","last_peered":"2024-05-10T03:29:49.910708+0000","last_clean":"2024-05-10T03:29:49.910708+0000","last_became_active":"2024-05-10T03:29:48.815170+0000","last_became_peered":"2024-05-10T03:29:48.815170+0000","last_unstale":"2024-05-10T03:29:49.910708+0000","last_undegraded":"2024-05-10T03:29:49.910708+0000","last_fullsized":"2024-05-10T03:29:49.910708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.338211+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.337821+0000","last_became_peered":"2024-05-10T03:29:44.337821+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811440+0000","last_change":"2024-05-10T03:29:43.774006+0000","last_active":"2024-05-10T03:29:48.811440+0000","last_peered":"2024-05-10T03:29:48.811440+0000","last_clean":"2024-05-10T03:29:48.811440+0000","last_became_active":"2024-05-10T03:29:43.773614+0000","last_became_peered":"2024-05-10T03:29:43.773614+0000","last_unstale":"2024-05-10T03:29:48.811440+0000","last_undegraded":"2024-05-10T03:29:48.811440+0000","last_fullsized":"2024-05-10T03:29:48.811440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910293+0000","last_change":"2024-05-10T03:29:49.254777+0000","last_active":"2024-05-10T03:29:49.910293+0000","last_peered":"2024-05-10T03:29:49.910293+0000","last_clean":"2024-05-10T03:29:49.910293+0000","last_became_active":"2024-05-10T03:29:49.253585+0000","last_became_peered":"2024-05-10T03:29:49.253585+0000","last_unstale":"2024-05-10T03:29:49.910293+0000","last_undegraded":"2024-05-10T03:29:49.910293+0000","last_fullsized":"2024-05-10T03:29:49.910293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:58.544334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813189+0000","last_change":"2024-05-10T03:29:44.344906+0000","last_active":"2024-05-10T03:29:48.813189+0000","last_peered":"2024-05-10T03:29:48.813189+0000","last_clean":"2024-05-10T03:29:48.813189+0000","last_became_active":"2024-05-10T03:29:44.344600+0000","last_became_peered":"2024-05-10T03:29:44.344600+0000","last_unstale":"2024-05-10T03:29:48.813189+0000","last_undegraded":"2024-05-10T03:29:48.813189+0000","last_fullsized":"2024-05-10T03:29:48.813189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795841+0000","last_change":"2024-05-10T03:29:43.768246+0000","last_active":"2024-05-10T03:29:48.795841+0000","last_peered":"2024-05-10T03:29:48.795841+0000","last_clean":"2024-05-10T03:29:48.795841+0000","last_became_active":"2024-05-10T03:29:43.767828+0000","last_became_peered":"2024-05-10T03:29:43.767828+0000","last_unstale":"2024-05-10T03:29:48.795841+0000","last_undegraded":"2024-05-10T03:29:48.795841+0000","last_fullsized":"2024-05-10T03:29:48.795841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366780+0000","last_change":"2024-05-10T03:29:49.246734+0000","last_active":"2024-05-10T03:29:49.366780+0000","last_peered":"2024-05-10T03:29:49.366780+0000","last_clean":"2024-05-10T03:29:49.366780+0000","last_became_active":"2024-05-10T03:29:49.246242+0000","last_became_peered":"2024-05-10T03:29:49.246242+0000","last_unstale":"2024-05-10T03:29:49.366780+0000","last_undegraded":"2024-05-10T03:29:49.366780+0000","last_fullsized":"2024-05-10T03:29:49.366780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797404+0000","last_change":"2024-05-10T03:29:44.342815+0000","last_active":"2024-05-10T03:29:48.797404+0000","last_peered":"2024-05-10T03:29:48.797404+0000","last_clean":"2024-05-10T03:29:48.797404+0000","last_became_active":"2024-05-10T03:29:44.340796+0000","last_became_peered":"2024-05-10T03:29:44.340796+0000","last_unstale":"2024-05-10T03:29:48.797404+0000","last_undegraded":"2024-05-10T03:29:48.797404+0000","last_fullsized":"2024-05-10T03:29:48.797404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:08.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.248230+0000","last_change":"2024-05-10T03:29:43.775843+0000","last_active":"2024-05-10T03:29:49.248230+0000","last_peered":"2024-05-10T03:29:49.248230+0000","last_clean":"2024-05-10T03:29:49.248230+0000","last_became_active":"2024-05-10T03:29:43.775096+0000","last_became_peered":"2024-05-10T03:29:43.775096+0000","last_unstale":"2024-05-10T03:29:49.248230+0000","last_undegraded":"2024-05-10T03:29:49.248230+0000","last_fullsized":"2024-05-10T03:29:49.248230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:22.134113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909045+0000","last_change":"2024-05-10T03:29:49.250836+0000","last_active":"2024-05-10T03:29:49.909045+0000","last_peered":"2024-05-10T03:29:49.909045+0000","last_clean":"2024-05-10T03:29:49.909045+0000","last_became_active":"2024-05-10T03:29:49.249127+0000","last_became_peered":"2024-05-10T03:29:49.249127+0000","last_unstale":"2024-05-10T03:29:49.909045+0000","last_undegraded":"2024-05-10T03:29:49.909045+0000","last_fullsized":"2024-05-10T03:29:49.909045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798180+0000","last_change":"2024-05-10T03:29:44.349438+0000","last_active":"2024-05-10T03:29:48.798180+0000","last_peered":"2024-05-10T03:29:48.798180+0000","last_clean":"2024-05-10T03:29:48.798180+0000","last_became_active":"2024-05-10T03:29:44.348987+0000","last_became_peered":"2024-05-10T03:29:44.348987+0000","last_unstale":"2024-05-10T03:29:48.798180+0000","last_undegraded":"2024-05-10T03:29:48.798180+0000","last_fullsized":"2024-05-10T03:29:48.798180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797443+0000","last_change":"2024-05-10T03:29:43.765358+0000","last_active":"2024-05-10T03:29:48.797443+0000","last_peered":"2024-05-10T03:29:48.797443+0000","last_clean":"2024-05-10T03:29:48.797443+0000","last_became_active":"2024-05-10T03:29:43.764855+0000","last_became_peered":"2024-05-10T03:29:43.764855+0000","last_unstale":"2024-05-10T03:29:48.797443+0000","last_undegraded":"2024-05-10T03:29:48.797443+0000","last_fullsized":"2024-05-10T03:29:48.797443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494037+0000","last_change":"2024-05-10T03:29:48.807909+0000","last_active":"2024-05-10T03:29:49.494037+0000","last_peered":"2024-05-10T03:29:49.494037+0000","last_clean":"2024-05-10T03:29:49.494037+0000","last_became_active":"2024-05-10T03:29:48.807403+0000","last_became_peered":"2024-05-10T03:29:48.807403+0000","last_unstale":"2024-05-10T03:29:49.494037+0000","last_undegraded":"2024-05-10T03:29:49.494037+0000","last_fullsized":"2024-05-10T03:29:49.494037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791871+0000","last_change":"2024-05-10T03:29:44.352509+0000","last_active":"2024-05-10T03:29:48.791871+0000","last_peered":"2024-05-10T03:29:48.791871+0000","last_clean":"2024-05-10T03:29:48.791871+0000","last_became_active":"2024-05-10T03:29:44.349456+0000","last_became_peered":"2024-05-10T03:29:44.349456+0000","last_unstale":"2024-05-10T03:29:48.791871+0000","last_undegraded":"2024-05-10T03:29:48.791871+0000","last_fullsized":"2024-05-10T03:29:48.791871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:10.785937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792782+0000","last_change":"2024-05-10T03:29:43.754724+0000","last_active":"2024-05-10T03:29:48.792782+0000","last_peered":"2024-05-10T03:29:48.792782+0000","last_clean":"2024-05-10T03:29:48.792782+0000","last_became_active":"2024-05-10T03:29:43.754159+0000","last_became_peered":"2024-05-10T03:29:43.754159+0000","last_unstale":"2024-05-10T03:29:48.792782+0000","last_undegraded":"2024-05-10T03:29:48.792782+0000","last_fullsized":"2024-05-10T03:29:48.792782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494653+0000","last_change":"2024-05-10T03:29:48.822295+0000","last_active":"2024-05-10T03:29:49.494653+0000","last_peered":"2024-05-10T03:29:49.494653+0000","last_clean":"2024-05-10T03:29:49.494653+0000","last_became_active":"2024-05-10T03:29:48.821883+0000","last_became_peered":"2024-05-10T03:29:48.821883+0000","last_unstale":"2024-05-10T03:29:49.494653+0000","last_undegraded":"2024-05-10T03:29:49.494653+0000","last_fullsized":"2024-05-10T03:29:49.494653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799865+0000","last_change":"2024-05-10T03:29:44.330889+0000","last_active":"2024-05-10T03:29:48.799865+0000","last_peered":"2024-05-10T03:29:48.799865+0000","last_clean":"2024-05-10T03:29:48.799865+0000","last_became_active":"2024-05-10T03:29:44.327692+0000","last_became_peered":"2024-05-10T03:29:44.327692+0000","last_unstale":"2024-05-10T03:29:48.799865+0000","last_undegraded":"2024-05-10T03:29:48.799865+0000","last_fullsized":"2024-05-10T03:29:48.799865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780603+0000","last_change":"2024-05-10T03:29:43.760419+0000","last_active":"2024-05-10T03:29:48.780603+0000","last_peered":"2024-05-10T03:29:48.780603+0000","last_clean":"2024-05-10T03:29:48.780603+0000","last_became_active":"2024-05-10T03:29:43.760082+0000","last_became_peered":"2024-05-10T03:29:43.760082+0000","last_unstale":"2024-05-10T03:29:48.780603+0000","last_undegraded":"2024-05-10T03:29:48.780603+0000","last_fullsized":"2024-05-10T03:29:48.780603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494952+0000","last_change":"2024-05-10T03:29:48.819719+0000","last_active":"2024-05-10T03:29:49.494952+0000","last_peered":"2024-05-10T03:29:49.494952+0000","last_clean":"2024-05-10T03:29:49.494952+0000","last_became_active":"2024-05-10T03:29:48.818741+0000","last_became_peered":"2024-05-10T03:29:48.818741+0000","last_unstale":"2024-05-10T03:29:49.494952+0000","last_undegraded":"2024-05-10T03:29:49.494952+0000","last_fullsized":"2024-05-10T03:29:49.494952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.787035+0000","last_change":"2024-05-10T03:29:44.354949+0000","last_active":"2024-05-10T03:29:48.787035+0000","last_peered":"2024-05-10T03:29:48.787035+0000","last_clean":"2024-05-10T03:29:48.787035+0000","last_became_active":"2024-05-10T03:29:44.354451+0000","last_became_peered":"2024-05-10T03:29:44.354451+0000","last_unstale":"2024-05-10T03:29:48.787035+0000","last_undegraded":"2024-05-10T03:29:48.787035+0000","last_fullsized":"2024-05-10T03:29:48.787035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241517+0000","last_change":"2024-05-10T03:29:43.776046+0000","last_active":"2024-05-10T03:29:49.241517+0000","last_peered":"2024-05-10T03:29:49.241517+0000","last_clean":"2024-05-10T03:29:49.241517+0000","last_became_active":"2024-05-10T03:29:43.775739+0000","last_became_peered":"2024-05-10T03:29:43.775739+0000","last_unstale":"2024-05-10T03:29:49.241517+0000","last_undegraded":"2024-05-10T03:29:49.241517+0000","last_fullsized":"2024-05-10T03:29:49.241517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909517+0000","last_change":"2024-05-10T03:29:49.247763+0000","last_active":"2024-05-10T03:29:49.909517+0000","last_peered":"2024-05-10T03:29:49.909517+0000","last_clean":"2024-05-10T03:29:49.909517+0000","last_became_active":"2024-05-10T03:29:49.246481+0000","last_became_peered":"2024-05-10T03:29:49.246481+0000","last_unstale":"2024-05-10T03:29:49.909517+0000","last_undegraded":"2024-05-10T03:29:49.909517+0000","last_fullsized":"2024-05-10T03:29:49.909517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.793334+0000","last_change":"2024-05-10T03:29:44.337125+0000","last_active":"2024-05-10T03:29:48.793334+0000","last_peered":"2024-05-10T03:29:48.793334+0000","last_clean":"2024-05-10T03:29:48.793334+0000","last_became_active":"2024-05-10T03:29:44.336773+0000","last_became_peered":"2024-05-10T03:29:44.336773+0000","last_unstale":"2024-05-10T03:29:48.793334+0000","last_undegraded":"2024-05-10T03:29:48.793334+0000","last_fullsized":"2024-05-10T03:29:48.793334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797167+0000","last_change":"2024-05-10T03:29:43.759632+0000","last_active":"2024-05-10T03:29:48.797167+0000","last_peered":"2024-05-10T03:29:48.797167+0000","last_clean":"2024-05-10T03:29:48.797167+0000","last_became_active":"2024-05-10T03:29:43.758627+0000","last_became_peered":"2024-05-10T03:29:43.758627+0000","last_unstale":"2024-05-10T03:29:48.797167+0000","last_undegraded":"2024-05-10T03:29:48.797167+0000","last_fullsized":"2024-05-10T03:29:48.797167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241817+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:29:49.241817+0000","last_peered":"2024-05-10T03:29:49.241817+0000","last_clean":"2024-05-10T03:29:49.241817+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:29:49.241817+0000","last_undegraded":"2024-05-10T03:29:49.241817+0000","last_fullsized":"2024-05-10T03:29:49.241817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373998+0000","last_change":"2024-05-10T03:29:49.258186+0000","last_active":"2024-05-10T03:29:49.373998+0000","last_peered":"2024-05-10T03:29:49.373998+0000","last_clean":"2024-05-10T03:29:49.373998+0000","last_became_active":"2024-05-10T03:29:49.256940+0000","last_became_peered":"2024-05-10T03:29:49.256940+0000","last_unstale":"2024-05-10T03:29:49.373998+0000","last_undegraded":"2024-05-10T03:29:49.373998+0000","last_fullsized":"2024-05-10T03:29:49.373998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247185+0000","last_change":"2024-05-10T03:29:44.333742+0000","last_active":"2024-05-10T03:29:49.247185+0000","last_peered":"2024-05-10T03:29:49.247185+0000","last_clean":"2024-05-10T03:29:49.247185+0000","last_became_active":"2024-05-10T03:29:44.333369+0000","last_became_peered":"2024-05-10T03:29:44.333369+0000","last_unstale":"2024-05-10T03:29:49.247185+0000","last_undegraded":"2024-05-10T03:29:49.247185+0000","last_fullsized":"2024-05-10T03:29:49.247185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.782735+0000","last_change":"2024-05-10T03:29:43.759722+0000","last_active":"2024-05-10T03:29:48.782735+0000","last_peered":"2024-05-10T03:29:48.782735+0000","last_clean":"2024-05-10T03:29:48.782735+0000","last_became_active":"2024-05-10T03:29:43.759323+0000","last_became_peered":"2024-05-10T03:29:43.759323+0000","last_unstale":"2024-05-10T03:29:48.782735+0000","last_undegraded":"2024-05-10T03:29:48.782735+0000","last_fullsized":"2024-05-10T03:29:48.782735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.374329+0000","last_change":"2024-05-10T03:29:49.246585+0000","last_active":"2024-05-10T03:29:49.374329+0000","last_peered":"2024-05-10T03:29:49.374329+0000","last_clean":"2024-05-10T03:29:49.374329+0000","last_became_active":"2024-05-10T03:29:49.245297+0000","last_became_peered":"2024-05-10T03:29:49.245297+0000","last_unstale":"2024-05-10T03:29:49.374329+0000","last_undegraded":"2024-05-10T03:29:49.374329+0000","last_fullsized":"2024-05-10T03:29:49.374329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797859+0000","last_change":"2024-05-10T03:29:44.341567+0000","last_active":"2024-05-10T03:29:48.797859+0000","last_peered":"2024-05-10T03:29:48.797859+0000","last_clean":"2024-05-10T03:29:48.797859+0000","last_became_active":"2024-05-10T03:29:44.341235+0000","last_became_peered":"2024-05-10T03:29:44.341235+0000","last_unstale":"2024-05-10T03:29:48.797859+0000","last_undegraded":"2024-05-10T03:29:48.797859+0000","last_fullsized":"2024-05-10T03:29:48.797859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795601+0000","last_change":"2024-05-10T03:29:43.756510+0000","last_active":"2024-05-10T03:29:48.795601+0000","last_peered":"2024-05-10T03:29:48.795601+0000","last_clean":"2024-05-10T03:29:48.795601+0000","last_became_active":"2024-05-10T03:29:43.756205+0000","last_became_peered":"2024-05-10T03:29:43.756205+0000","last_unstale":"2024-05-10T03:29:48.795601+0000","last_undegraded":"2024-05-10T03:29:48.795601+0000","last_fullsized":"2024-05-10T03:29:48.795601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.908687+0000","last_change":"2024-05-10T03:29:48.806390+0000","last_active":"2024-05-10T03:29:49.908687+0000","last_peered":"2024-05-10T03:29:49.908687+0000","last_clean":"2024-05-10T03:29:49.908687+0000","last_became_active":"2024-05-10T03:29:48.805905+0000","last_became_peered":"2024-05-10T03:29:48.805905+0000","last_unstale":"2024-05-10T03:29:49.908687+0000","last_undegraded":"2024-05-10T03:29:49.908687+0000","last_fullsized":"2024-05-10T03:29:49.908687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249530+0000","last_change":"2024-05-10T03:29:44.348672+0000","last_active":"2024-05-10T03:29:49.249530+0000","last_peered":"2024-05-10T03:29:49.249530+0000","last_clean":"2024-05-10T03:29:49.249530+0000","last_became_active":"2024-05-10T03:29:44.348328+0000","last_became_peered":"2024-05-10T03:29:44.348328+0000","last_unstale":"2024-05-10T03:29:49.249530+0000","last_undegraded":"2024-05-10T03:29:49.249530+0000","last_fullsized":"2024-05-10T03:29:49.249530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781197+0000","last_change":"2024-05-10T03:29:43.760410+0000","last_active":"2024-05-10T03:29:48.781197+0000","last_peered":"2024-05-10T03:29:48.781197+0000","last_clean":"2024-05-10T03:29:48.781197+0000","last_became_active":"2024-05-10T03:29:43.760137+0000","last_became_peered":"2024-05-10T03:29:43.760137+0000","last_unstale":"2024-05-10T03:29:48.781197+0000","last_undegraded":"2024-05-10T03:29:48.781197+0000","last_fullsized":"2024-05-10T03:29:48.781197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.370957+0000","last_change":"2024-05-10T03:29:49.256630+0000","last_active":"2024-05-10T03:29:49.370957+0000","last_peered":"2024-05-10T03:29:49.370957+0000","last_clean":"2024-05-10T03:29:49.370957+0000","last_became_active":"2024-05-10T03:29:49.246648+0000","last_became_peered":"2024-05-10T03:29:49.246648+0000","last_unstale":"2024-05-10T03:29:49.370957+0000","last_undegraded":"2024-05-10T03:29:49.370957+0000","last_fullsized":"2024-05-10T03:29:49.370957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.244841+0000","last_change":"2024-05-10T03:29:44.347464+0000","last_active":"2024-05-10T03:29:49.244841+0000","last_peered":"2024-05-10T03:29:49.244841+0000","last_clean":"2024-05-10T03:29:49.244841+0000","last_became_active":"2024-05-10T03:29:44.347135+0000","last_became_peered":"2024-05-10T03:29:44.347135+0000","last_unstale":"2024-05-10T03:29:49.244841+0000","last_undegraded":"2024-05-10T03:29:49.244841+0000","last_fullsized":"2024-05-10T03:29:49.244841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792883+0000","last_change":"2024-05-10T03:29:43.758164+0000","last_active":"2024-05-10T03:29:48.792883+0000","last_peered":"2024-05-10T03:29:48.792883+0000","last_clean":"2024-05-10T03:29:48.792883+0000","last_became_active":"2024-05-10T03:29:43.757793+0000","last_became_peered":"2024-05-10T03:29:43.757793+0000","last_unstale":"2024-05-10T03:29:48.792883+0000","last_undegraded":"2024-05-10T03:29:48.792883+0000","last_fullsized":"2024-05-10T03:29:48.792883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:39.499394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909165+0000","last_change":"2024-05-10T03:29:48.821419+0000","last_active":"2024-05-10T03:29:49.909165+0000","last_peered":"2024-05-10T03:29:49.909165+0000","last_clean":"2024-05-10T03:29:49.909165+0000","last_became_active":"2024-05-10T03:29:48.820977+0000","last_became_peered":"2024-05-10T03:29:48.820977+0000","last_unstale":"2024-05-10T03:29:49.909165+0000","last_undegraded":"2024-05-10T03:29:49.909165+0000","last_fullsized":"2024-05-10T03:29:49.909165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:17.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241985+0000","last_change":"2024-05-10T03:29:44.349656+0000","last_active":"2024-05-10T03:29:49.241985+0000","last_peered":"2024-05-10T03:29:49.241985+0000","last_clean":"2024-05-10T03:29:49.241985+0000","last_became_active":"2024-05-10T03:29:44.349360+0000","last_became_peered":"2024-05-10T03:29:44.349360+0000","last_unstale":"2024-05-10T03:29:49.241985+0000","last_undegraded":"2024-05-10T03:29:49.241985+0000","last_fullsized":"2024-05-10T03:29:49.241985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781272+0000","last_change":"2024-05-10T03:29:43.755515+0000","last_active":"2024-05-10T03:29:48.781272+0000","last_peered":"2024-05-10T03:29:48.781272+0000","last_clean":"2024-05-10T03:29:48.781272+0000","last_became_active":"2024-05-10T03:29:43.755202+0000","last_became_peered":"2024-05-10T03:29:43.755202+0000","last_unstale":"2024-05-10T03:29:48.781272+0000","last_undegraded":"2024-05-10T03:29:48.781272+0000","last_fullsized":"2024-05-10T03:29:48.781272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909234+0000","last_change":"2024-05-10T03:29:48.816001+0000","last_active":"2024-05-10T03:29:49.909234+0000","last_peered":"2024-05-10T03:29:49.909234+0000","last_clean":"2024-05-10T03:29:49.909234+0000","last_became_active":"2024-05-10T03:29:48.815291+0000","last_became_peered":"2024-05-10T03:29:48.815291+0000","last_unstale":"2024-05-10T03:29:49.909234+0000","last_undegraded":"2024-05-10T03:29:49.909234+0000","last_fullsized":"2024-05-10T03:29:49.909234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812139+0000","last_change":"2024-05-10T03:29:44.334673+0000","last_active":"2024-05-10T03:29:48.812139+0000","last_peered":"2024-05-10T03:29:48.812139+0000","last_clean":"2024-05-10T03:29:48.812139+0000","last_became_active":"2024-05-10T03:29:44.333855+0000","last_became_peered":"2024-05-10T03:29:44.333855+0000","last_unstale":"2024-05-10T03:29:48.812139+0000","last_undegraded":"2024-05-10T03:29:48.812139+0000","last_fullsized":"2024-05-10T03:29:48.812139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.807723+0000","last_change":"2024-05-10T03:29:43.768430+0000","last_active":"2024-05-10T03:29:48.807723+0000","last_peered":"2024-05-10T03:29:48.807723+0000","last_clean":"2024-05-10T03:29:48.807723+0000","last_became_active":"2024-05-10T03:29:43.767491+0000","last_became_peered":"2024-05-10T03:29:43.767491+0000","last_unstale":"2024-05-10T03:29:48.807723+0000","last_undegraded":"2024-05-10T03:29:48.807723+0000","last_fullsized":"2024-05-10T03:29:48.807723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.368290+0000","last_change":"2024-05-10T03:29:49.247514+0000","last_active":"2024-05-10T03:29:49.368290+0000","last_peered":"2024-05-10T03:29:49.368290+0000","last_clean":"2024-05-10T03:29:49.368290+0000","last_became_active":"2024-05-10T03:29:49.246239+0000","last_became_peered":"2024-05-10T03:29:49.246239+0000","last_unstale":"2024-05-10T03:29:49.368290+0000","last_undegraded":"2024-05-10T03:29:49.368290+0000","last_fullsized":"2024-05-10T03:29:49.368290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:20:20.616507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242291+0000","last_change":"2024-05-10T03:29:44.348653+0000","last_active":"2024-05-10T03:29:49.242291+0000","last_peered":"2024-05-10T03:29:49.242291+0000","last_clean":"2024-05-10T03:29:49.242291+0000","last_became_active":"2024-05-10T03:29:44.348301+0000","last_became_peered":"2024-05-10T03:29:44.348301+0000","last_unstale":"2024-05-10T03:29:49.242291+0000","last_undegraded":"2024-05-10T03:29:49.242291+0000","last_fullsized":"2024-05-10T03:29:49.242291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792608+0000","last_change":"2024-05-10T03:29:43.760877+0000","last_active":"2024-05-10T03:29:48.792608+0000","last_peered":"2024-05-10T03:29:48.792608+0000","last_clean":"2024-05-10T03:29:48.792608+0000","last_became_active":"2024-05-10T03:29:43.760572+0000","last_became_peered":"2024-05-10T03:29:43.760572+0000","last_unstale":"2024-05-10T03:29:48.792608+0000","last_undegraded":"2024-05-10T03:29:48.792608+0000","last_fullsized":"2024-05-10T03:29:48.792608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367784+0000","last_change":"2024-05-10T03:29:49.246283+0000","last_active":"2024-05-10T03:29:49.367784+0000","last_peered":"2024-05-10T03:29:49.367784+0000","last_clean":"2024-05-10T03:29:49.367784+0000","last_became_active":"2024-05-10T03:29:49.243969+0000","last_became_peered":"2024-05-10T03:29:49.243969+0000","last_unstale":"2024-05-10T03:29:49.367784+0000","last_undegraded":"2024-05-10T03:29:49.367784+0000","last_fullsized":"2024-05-10T03:29:49.367784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:30.586336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.350944+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.350411+0000","last_became_peered":"2024-05-10T03:29:44.350411+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791490+0000","last_change":"2024-05-10T03:29:43.763410+0000","last_active":"2024-05-10T03:29:48.791490+0000","last_peered":"2024-05-10T03:29:48.791490+0000","last_clean":"2024-05-10T03:29:48.791490+0000","last_became_active":"2024-05-10T03:29:43.761799+0000","last_became_peered":"2024-05-10T03:29:43.761799+0000","last_unstale":"2024-05-10T03:29:48.791490+0000","last_undegraded":"2024-05-10T03:29:48.791490+0000","last_fullsized":"2024-05-10T03:29:48.791490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:27.283436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373121+0000","last_change":"2024-05-10T03:29:49.254044+0000","last_active":"2024-05-10T03:29:49.373121+0000","last_peered":"2024-05-10T03:29:49.373121+0000","last_clean":"2024-05-10T03:29:49.373121+0000","last_became_active":"2024-05-10T03:29:49.252991+0000","last_became_peered":"2024-05-10T03:29:49.252991+0000","last_unstale":"2024-05-10T03:29:49.373121+0000","last_undegraded":"2024-05-10T03:29:49.373121+0000","last_fullsized":"2024-05-10T03:29:49.373121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:18:14.955082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243191+0000","last_change":"2024-05-10T03:29:44.361247+0000","last_active":"2024-05-10T03:29:49.243191+0000","last_peered":"2024-05-10T03:29:49.243191+0000","last_clean":"2024-05-10T03:29:49.243191+0000","last_became_active":"2024-05-10T03:29:44.360953+0000","last_became_peered":"2024-05-10T03:29:44.360953+0000","last_unstale":"2024-05-10T03:29:49.243191+0000","last_undegraded":"2024-05-10T03:29:49.243191+0000","last_fullsized":"2024-05-10T03:29:49.243191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:30:04.976035+0000","last_change":"2024-05-10T03:29:43.773652+0000","last_active":"2024-05-10T03:30:04.976035+0000","last_peered":"2024-05-10T03:30:04.976035+0000","last_clean":"2024-05-10T03:30:04.976035+0000","last_became_active":"2024-05-10T03:29:43.773313+0000","last_became_peered":"2024-05-10T03:29:43.773313+0000","last_unstale":"2024-05-10T03:30:04.976035+0000","last_undegraded":"2024-05-10T03:30:04.976035+0000","last_fullsized":"2024-05-10T03:30:04.976035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494687+0000","last_change":"2024-05-10T03:29:48.814023+0000","last_active":"2024-05-10T03:29:49.494687+0000","last_peered":"2024-05-10T03:29:49.494687+0000","last_clean":"2024-05-10T03:29:49.494687+0000","last_became_active":"2024-05-10T03:29:48.813592+0000","last_became_peered":"2024-05-10T03:29:48.813592+0000","last_unstale":"2024-05-10T03:29:49.494687+0000","last_undegraded":"2024-05-10T03:29:49.494687+0000","last_fullsized":"2024-05-10T03:29:49.494687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791481+0000","last_change":"2024-05-10T03:29:44.337189+0000","last_active":"2024-05-10T03:29:48.791481+0000","last_peered":"2024-05-10T03:29:48.791481+0000","last_clean":"2024-05-10T03:29:48.791481+0000","last_became_active":"2024-05-10T03:29:44.336858+0000","last_became_peered":"2024-05-10T03:29:44.336858+0000","last_unstale":"2024-05-10T03:29:48.791481+0000","last_undegraded":"2024-05-10T03:29:48.791481+0000","last_fullsized":"2024-05-10T03:29:48.791481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:34.453268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795391+0000","last_change":"2024-05-10T03:29:43.763074+0000","last_active":"2024-05-10T03:29:48.795391+0000","last_peered":"2024-05-10T03:29:48.795391+0000","last_clean":"2024-05-10T03:29:48.795391+0000","last_became_active":"2024-05-10T03:29:43.762746+0000","last_became_peered":"2024-05-10T03:29:43.762746+0000","last_unstale":"2024-05-10T03:29:48.795391+0000","last_undegraded":"2024-05-10T03:29:48.795391+0000","last_fullsized":"2024-05-10T03:29:48.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910002+0000","last_change":"2024-05-10T03:29:48.811643+0000","last_active":"2024-05-10T03:29:49.910002+0000","last_peered":"2024-05-10T03:29:49.910002+0000","last_clean":"2024-05-10T03:29:49.910002+0000","last_became_active":"2024-05-10T03:29:48.811181+0000","last_became_peered":"2024-05-10T03:29:48.811181+0000","last_unstale":"2024-05-10T03:29:49.910002+0000","last_undegraded":"2024-05-10T03:29:49.910002+0000","last_fullsized":"2024-05-10T03:29:49.910002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795416+0000","last_change":"2024-05-10T03:29:44.339071+0000","last_active":"2024-05-10T03:29:48.795416+0000","last_peered":"2024-05-10T03:29:48.795416+0000","last_clean":"2024-05-10T03:29:48.795416+0000","last_became_active":"2024-05-10T03:29:44.338222+0000","last_became_peered":"2024-05-10T03:29:44.338222+0000","last_unstale":"2024-05-10T03:29:48.795416+0000","last_undegraded":"2024-05-10T03:29:48.795416+0000","last_fullsized":"2024-05-10T03:29:48.795416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798392+0000","last_change":"2024-05-10T03:29:43.769611+0000","last_active":"2024-05-10T03:29:48.798392+0000","last_peered":"2024-05-10T03:29:48.798392+0000","last_clean":"2024-05-10T03:29:48.798392+0000","last_became_active":"2024-05-10T03:29:43.769245+0000","last_became_peered":"2024-05-10T03:29:43.769245+0000","last_unstale":"2024-05-10T03:29:48.798392+0000","last_undegraded":"2024-05-10T03:29:48.798392+0000","last_fullsized":"2024-05-10T03:29:48.798392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367049+0000","last_change":"2024-05-10T03:29:48.823083+0000","last_active":"2024-05-10T03:29:49.367049+0000","last_peered":"2024-05-10T03:29:49.367049+0000","last_clean":"2024-05-10T03:29:49.367049+0000","last_became_active":"2024-05-10T03:29:48.817306+0000","last_became_peered":"2024-05-10T03:29:48.817306+0000","last_unstale":"2024-05-10T03:29:49.367049+0000","last_undegraded":"2024-05-10T03:29:49.367049+0000","last_fullsized":"2024-05-10T03:29:49.367049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:31:18.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798233+0000","last_change":"2024-05-10T03:29:44.330232+0000","last_active":"2024-05-10T03:29:48.798233+0000","last_peered":"2024-05-10T03:29:48.798233+0000","last_clean":"2024-05-10T03:29:48.798233+0000","last_became_active":"2024-05-10T03:29:44.329857+0000","last_became_peered":"2024-05-10T03:29:44.329857+0000","last_unstale":"2024-05-10T03:29:48.798233+0000","last_undegraded":"2024-05-10T03:29:48.798233+0000","last_fullsized":"2024-05-10T03:29:48.798233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811307+0000","last_change":"2024-05-10T03:29:43.771585+0000","last_active":"2024-05-10T03:29:48.811307+0000","last_peered":"2024-05-10T03:29:48.811307+0000","last_clean":"2024-05-10T03:29:48.811307+0000","last_became_active":"2024-05-10T03:29:43.771253+0000","last_became_peered":"2024-05-10T03:29:43.771253+0000","last_unstale":"2024-05-10T03:29:48.811307+0000","last_undegraded":"2024-05-10T03:29:48.811307+0000","last_fullsized":"2024-05-10T03:29:48.811307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366899+0000","last_change":"2024-05-10T03:29:49.245635+0000","last_active":"2024-05-10T03:29:49.366899+0000","last_peered":"2024-05-10T03:29:49.366899+0000","last_clean":"2024-05-10T03:29:49.366899+0000","last_became_active":"2024-05-10T03:29:49.245125+0000","last_became_peered":"2024-05-10T03:29:49.245125+0000","last_unstale":"2024-05-10T03:29:49.366899+0000","last_undegraded":"2024-05-10T03:29:49.366899+0000","last_fullsized":"2024-05-10T03:29:49.366899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909603+0000","last_change":"2024-05-10T03:29:49.253042+0000","last_active":"2024-05-10T03:29:49.909603+0000","last_peered":"2024-05-10T03:29:49.909603+0000","last_clean":"2024-05-10T03:29:49.909603+0000","last_became_active":"2024-05-10T03:29:49.252051+0000","last_became_peered":"2024-05-10T03:29:49.252051+0000","last_unstale":"2024-05-10T03:29:49.909603+0000","last_undegraded":"2024-05-10T03:29:49.909603+0000","last_fullsized":"2024-05-10T03:29:49.909603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:57:00.757246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794396+0000","last_change":"2024-05-10T03:29:44.346922+0000","last_active":"2024-05-10T03:29:48.794396+0000","last_peered":"2024-05-10T03:29:48.794396+0000","last_clean":"2024-05-10T03:29:48.794396+0000","last_became_active":"2024-05-10T03:29:44.346260+0000","last_became_peered":"2024-05-10T03:29:44.346260+0000","last_unstale":"2024-05-10T03:29:48.794396+0000","last_undegraded":"2024-05-10T03:29:48.794396+0000","last_fullsized":"2024-05-10T03:29:48.794396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786593+0000","last_change":"2024-05-10T03:29:43.761072+0000","last_active":"2024-05-10T03:29:48.786593+0000","last_peered":"2024-05-10T03:29:48.786593+0000","last_clean":"2024-05-10T03:29:48.786593+0000","last_became_active":"2024-05-10T03:29:43.760425+0000","last_became_peered":"2024-05-10T03:29:43.760425+0000","last_unstale":"2024-05-10T03:29:48.786593+0000","last_undegraded":"2024-05-10T03:29:48.786593+0000","last_fullsized":"2024-05-10T03:29:48.786593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.375073+0000","last_change":"2024-05-10T03:29:48.818277+0000","last_active":"2024-05-10T03:29:49.375073+0000","last_peered":"2024-05-10T03:29:49.375073+0000","last_clean":"2024-05-10T03:29:49.375073+0000","last_became_active":"2024-05-10T03:29:48.817666+0000","last_became_peered":"2024-05-10T03:29:48.817666+0000","last_unstale":"2024-05-10T03:29:49.375073+0000","last_undegraded":"2024-05-10T03:29:49.375073+0000","last_fullsized":"2024-05-10T03:29:49.375073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808591+0000","last_change":"2024-05-10T03:29:44.356141+0000","last_active":"2024-05-10T03:29:48.808591+0000","last_peered":"2024-05-10T03:29:48.808591+0000","last_clean":"2024-05-10T03:29:48.808591+0000","last_became_active":"2024-05-10T03:29:44.355691+0000","last_became_peered":"2024-05-10T03:29:44.355691+0000","last_unstale":"2024-05-10T03:29:48.808591+0000","last_undegraded":"2024-05-10T03:29:48.808591+0000","last_fullsized":"2024-05-10T03:29:48.808591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801357+0000","last_change":"2024-05-10T03:29:43.767957+0000","last_active":"2024-05-10T03:29:48.801357+0000","last_peered":"2024-05-10T03:29:48.801357+0000","last_clean":"2024-05-10T03:29:48.801357+0000","last_became_active":"2024-05-10T03:29:43.767083+0000","last_became_peered":"2024-05-10T03:29:43.767083+0000","last_unstale":"2024-05-10T03:29:48.801357+0000","last_undegraded":"2024-05-10T03:29:48.801357+0000","last_fullsized":"2024-05-10T03:29:48.801357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.809728+0000","last_change":"2024-05-10T03:29:43.767890+0000","last_active":"2024-05-10T03:29:48.809728+0000","last_peered":"2024-05-10T03:29:48.809728+0000","last_clean":"2024-05-10T03:29:48.809728+0000","last_became_active":"2024-05-10T03:29:43.767550+0000","last_became_peered":"2024-05-10T03:29:43.767550+0000","last_unstale":"2024-05-10T03:29:48.809728+0000","last_undegraded":"2024-05-10T03:29:48.809728+0000","last_fullsized":"2024-05-10T03:29:48.809728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792435+0000","last_change":"2024-05-10T03:29:44.346806+0000","last_active":"2024-05-10T03:29:48.792435+0000","last_peered":"2024-05-10T03:29:48.792435+0000","last_clean":"2024-05-10T03:29:48.792435+0000","last_became_active":"2024-05-10T03:29:44.346484+0000","last_became_peered":"2024-05-10T03:29:44.346484+0000","last_unstale":"2024-05-10T03:29:48.792435+0000","last_undegraded":"2024-05-10T03:29:48.792435+0000","last_fullsized":"2024-05-10T03:29:48.792435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:50.377829+0000","last_change":"2024-05-10T03:29:48.818619+0000","last_active":"2024-05-10T03:29:50.377829+0000","last_peered":"2024-05-10T03:29:50.377829+0000","last_clean":"2024-05-10T03:29:50.377829+0000","last_became_active":"2024-05-10T03:29:48.818125+0000","last_became_peered":"2024-05-10T03:29:48.818125+0000","last_unstale":"2024-05-10T03:29:50.377829+0000","last_undegraded":"2024-05-10T03:29:50.377829+0000","last_fullsized":"2024-05-10T03:29:50.377829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:58.075131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794286+0000","last_change":"2024-05-10T03:29:43.771030+0000","last_active":"2024-05-10T03:29:48.794286+0000","last_peered":"2024-05-10T03:29:48.794286+0000","last_clean":"2024-05-10T03:29:48.794286+0000","last_became_active":"2024-05-10T03:29:43.770630+0000","last_became_peered":"2024-05-10T03:29:43.770630+0000","last_unstale":"2024-05-10T03:29:48.794286+0000","last_undegraded":"2024-05-10T03:29:48.794286+0000","last_fullsized":"2024-05-10T03:29:48.794286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791595+0000","last_change":"2024-05-10T03:29:44.330198+0000","last_active":"2024-05-10T03:29:48.791595+0000","last_peered":"2024-05-10T03:29:48.791595+0000","last_clean":"2024-05-10T03:29:48.791595+0000","last_became_active":"2024-05-10T03:29:44.329871+0000","last_became_peered":"2024-05-10T03:29:44.329871+0000","last_unstale":"2024-05-10T03:29:48.791595+0000","last_undegraded":"2024-05-10T03:29:48.791595+0000","last_fullsized":"2024-05-10T03:29:48.791595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367463+0000","last_change":"2024-05-10T03:29:48.822151+0000","last_active":"2024-05-10T03:29:49.367463+0000","last_peered":"2024-05-10T03:29:49.367463+0000","last_clean":"2024-05-10T03:29:49.367463+0000","last_became_active":"2024-05-10T03:29:48.819369+0000","last_became_peered":"2024-05-10T03:29:48.819369+0000","last_unstale":"2024-05-10T03:29:49.367463+0000","last_undegraded":"2024-05-10T03:29:49.367463+0000","last_fullsized":"2024-05-10T03:29:49.367463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:58.725548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808987+0000","last_change":"2024-05-10T03:29:44.334873+0000","last_active":"2024-05-10T03:29:48.808987+0000","last_peered":"2024-05-10T03:29:48.808987+0000","last_clean":"2024-05-10T03:29:48.808987+0000","last_became_active":"2024-05-10T03:29:44.334365+0000","last_became_peered":"2024-05-10T03:29:44.334365+0000","last_unstale":"2024-05-10T03:29:48.808987+0000","last_undegraded":"2024-05-10T03:29:48.808987+0000","last_fullsized":"2024-05-10T03:29:48.808987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780300+0000","last_change":"2024-05-10T03:29:44.333950+0000","last_active":"2024-05-10T03:29:48.780300+0000","last_peered":"2024-05-10T03:29:48.780300+0000","last_clean":"2024-05-10T03:29:48.780300+0000","last_became_active":"2024-05-10T03:29:44.331498+0000","last_became_peered":"2024-05-10T03:29:44.331498+0000","last_unstale":"2024-05-10T03:29:48.780300+0000","last_undegraded":"2024-05-10T03:29:48.780300+0000","last_fullsized":"2024-05-10T03:29:48.780300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792455+0000","last_change":"2024-05-10T03:29:44.341676+0000","last_active":"2024-05-10T03:29:48.792455+0000","last_peered":"2024-05-10T03:29:48.792455+0000","last_clean":"2024-05-10T03:29:48.792455+0000","last_became_active":"2024-05-10T03:29:44.341296+0000","last_became_peered":"2024-05-10T03:29:44.341296+0000","last_unstale":"2024-05-10T03:29:48.792455+0000","last_undegraded":"2024-05-10T03:29:48.792455+0000","last_fullsized":"2024-05-10T03:29:48.792455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791287+0000","last_change":"2024-05-10T03:29:44.349694+0000","last_active":"2024-05-10T03:29:48.791287+0000","last_peered":"2024-05-10T03:29:48.791287+0000","last_clean":"2024-05-10T03:29:48.791287+0000","last_became_active":"2024-05-10T03:29:44.349396+0000","last_became_peered":"2024-05-10T03:29:44.349396+0000","last_unstale":"2024-05-10T03:29:48.791287+0000","last_undegraded":"2024-05-10T03:29:48.791287+0000","last_fullsized":"2024-05-10T03:29:48.791287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808295+0000","last_change":"2024-05-10T03:29:44.359086+0000","last_active":"2024-05-10T03:29:48.808295+0000","last_peered":"2024-05-10T03:29:48.808295+0000","last_clean":"2024-05-10T03:29:48.808295+0000","last_became_active":"2024-05-10T03:29:44.358776+0000","last_became_peered":"2024-05-10T03:29:44.358776+0000","last_unstale":"2024-05-10T03:29:48.808295+0000","last_undegraded":"2024-05-10T03:29:48.808295+0000","last_fullsized":"2024-05-10T03:29:48.808295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794937+0000","last_change":"2024-05-10T03:29:44.352293+0000","last_active":"2024-05-10T03:29:48.794937+0000","last_peered":"2024-05-10T03:29:48.794937+0000","last_clean":"2024-05-10T03:29:48.794937+0000","last_became_active":"2024-05-10T03:29:44.352017+0000","last_became_peered":"2024-05-10T03:29:44.352017+0000","last_unstale":"2024-05-10T03:29:48.794937+0000","last_undegraded":"2024-05-10T03:29:48.794937+0000","last_fullsized":"2024-05-10T03:29:48.794937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812361+0000","last_change":"2024-05-10T03:29:44.339931+0000","last_active":"2024-05-10T03:29:48.812361+0000","last_peered":"2024-05-10T03:29:48.812361+0000","last_clean":"2024-05-10T03:29:48.812361+0000","last_became_active":"2024-05-10T03:29:44.339638+0000","last_became_peered":"2024-05-10T03:29:44.339638+0000","last_unstale":"2024-05-10T03:29:48.812361+0000","last_undegraded":"2024-05-10T03:29:48.812361+0000","last_fullsized":"2024-05-10T03:29:48.812361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.815879+0000","last_change":"2024-05-10T03:29:44.343280+0000","last_active":"2024-05-10T03:29:48.815879+0000","last_peered":"2024-05-10T03:29:48.815879+0000","last_clean":"2024-05-10T03:29:48.815879+0000","last_became_active":"2024-05-10T03:29:44.342959+0000","last_became_peered":"2024-05-10T03:29:44.342959+0000","last_unstale":"2024-05-10T03:29:48.815879+0000","last_undegraded":"2024-05-10T03:29:48.815879+0000","last_fullsized":"2024-05-10T03:29:48.815879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786197+0000","last_change":"2024-05-10T03:29:44.351381+0000","last_active":"2024-05-10T03:29:48.786197+0000","last_peered":"2024-05-10T03:29:48.786197+0000","last_clean":"2024-05-10T03:29:48.786197+0000","last_became_active":"2024-05-10T03:29:44.351019+0000","last_became_peered":"2024-05-10T03:29:44.351019+0000","last_unstale":"2024-05-10T03:29:48.786197+0000","last_undegraded":"2024-05-10T03:29:48.786197+0000","last_fullsized":"2024-05-10T03:29:48.786197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801184+0000","last_change":"2024-05-10T03:29:44.344852+0000","last_active":"2024-05-10T03:29:48.801184+0000","last_peered":"2024-05-10T03:29:48.801184+0000","last_clean":"2024-05-10T03:29:48.801184+0000","last_became_active":"2024-05-10T03:29:44.344659+0000","last_became_peered":"2024-05-10T03:29:44.344659+0000","last_unstale":"2024-05-10T03:29:48.801184+0000","last_undegraded":"2024-05-10T03:29:48.801184+0000","last_fullsized":"2024-05-10T03:29:48.801184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792296+0000","last_change":"2024-05-10T03:29:44.335097+0000","last_active":"2024-05-10T03:29:48.792296+0000","last_peered":"2024-05-10T03:29:48.792296+0000","last_clean":"2024-05-10T03:29:48.792296+0000","last_became_active":"2024-05-10T03:29:44.332994+0000","last_became_peered":"2024-05-10T03:29:44.332994+0000","last_unstale":"2024-05-10T03:29:48.792296+0000","last_undegraded":"2024-05-10T03:29:48.792296+0000","last_fullsized":"2024-05-10T03:29:48.792296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.788203+0000","last_change":"2024-05-10T03:29:44.346157+0000","last_active":"2024-05-10T03:29:48.788203+0000","last_peered":"2024-05-10T03:29:48.788203+0000","last_clean":"2024-05-10T03:29:48.788203+0000","last_became_active":"2024-05-10T03:29:44.345819+0000","last_became_peered":"2024-05-10T03:29:44.345819+0000","last_unstale":"2024-05-10T03:29:48.788203+0000","last_undegraded":"2024-05-10T03:29:48.788203+0000","last_fullsized":"2024-05-10T03:29:48.788203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813266+0000","last_change":"2024-05-10T03:29:44.351136+0000","last_active":"2024-05-10T03:29:48.813266+0000","last_peered":"2024-05-10T03:29:48.813266+0000","last_clean":"2024-05-10T03:29:48.813266+0000","last_became_active":"2024-05-10T03:29:44.347556+0000","last_became_peered":"2024-05-10T03:29:44.347556+0000","last_unstale":"2024-05-10T03:29:48.813266+0000","last_undegraded":"2024-05-10T03:29:48.813266+0000","last_fullsized":"2024-05-10T03:29:48.813266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247953+0000","last_change":"2024-05-10T03:29:44.354749+0000","last_active":"2024-05-10T03:29:49.247953+0000","last_peered":"2024-05-10T03:29:49.247953+0000","last_clean":"2024-05-10T03:29:49.247953+0000","last_became_active":"2024-05-10T03:29:44.354474+0000","last_became_peered":"2024-05-10T03:29:44.354474+0000","last_unstale":"2024-05-10T03:29:49.247953+0000","last_undegraded":"2024-05-10T03:29:49.247953+0000","last_fullsized":"2024-05-10T03:29:49.247953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780909+0000","last_change":"2024-05-10T03:29:44.339173+0000","last_active":"2024-05-10T03:29:48.780909+0000","last_peered":"2024-05-10T03:29:48.780909+0000","last_clean":"2024-05-10T03:29:48.780909+0000","last_became_active":"2024-05-10T03:29:44.338815+0000","last_became_peered":"2024-05-10T03:29:44.338815+0000","last_unstale":"2024-05-10T03:29:48.780909+0000","last_undegraded":"2024-05-10T03:29:48.780909+0000","last_fullsized":"2024-05-10T03:29:48.780909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799655+0000","last_change":"2024-05-10T03:29:44.334824+0000","last_active":"2024-05-10T03:29:48.799655+0000","last_peered":"2024-05-10T03:29:48.799655+0000","last_clean":"2024-05-10T03:29:48.799655+0000","last_became_active":"2024-05-10T03:29:44.333616+0000","last_became_peered":"2024-05-10T03:29:44.333616+0000","last_unstale":"2024-05-10T03:29:48.799655+0000","last_undegraded":"2024-05-10T03:29:48.799655+0000","last_fullsized":"2024-05-10T03:29:48.799655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:38.441292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.785927+0000","last_change":"2024-05-10T03:29:44.339089+0000","last_active":"2024-05-10T03:29:48.785927+0000","last_peered":"2024-05-10T03:29:48.785927+0000","last_clean":"2024-05-10T03:29:48.785927+0000","last_became_active":"2024-05-10T03:29:44.336113+0000","last_became_peered":"2024-05-10T03:29:44.336113+0000","last_unstale":"2024-05-10T03:29:48.785927+0000","last_undegraded":"2024-05-10T03:29:48.785927+0000","last_fullsized":"2024-05-10T03:29:48.785927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794173+0000","last_change":"2024-05-10T03:29:44.340483+0000","last_active":"2024-05-10T03:29:48.794173+0000","last_peered":"2024-05-10T03:29:48.794173+0000","last_clean":"2024-05-10T03:29:48.794173+0000","last_became_active":"2024-05-10T03:29:44.340093+0000","last_became_peered":"2024-05-10T03:29:44.340093+0000","last_unstale":"2024-05-10T03:29:48.794173+0000","last_undegraded":"2024-05-10T03:29:48.794173+0000","last_fullsized":"2024-05-10T03:29:48.794173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790438+0000","last_change":"2024-05-10T03:29:44.345377+0000","last_active":"2024-05-10T03:29:48.790438+0000","last_peered":"2024-05-10T03:29:48.790438+0000","last_clean":"2024-05-10T03:29:48.790438+0000","last_became_active":"2024-05-10T03:29:44.345048+0000","last_became_peered":"2024-05-10T03:29:44.345048+0000","last_unstale":"2024-05-10T03:29:48.790438+0000","last_undegraded":"2024-05-10T03:29:48.790438+0000","last_fullsized":"2024-05-10T03:29:48.790438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:36.993303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794132+0000","last_change":"2024-05-10T03:29:44.349235+0000","last_active":"2024-05-10T03:29:48.794132+0000","last_peered":"2024-05-10T03:29:48.794132+0000","last_clean":"2024-05-10T03:29:48.794132+0000","last_became_active":"2024-05-10T03:29:44.348922+0000","last_became_peered":"2024-05-10T03:29:44.348922+0000","last_unstale":"2024-05-10T03:29:48.794132+0000","last_undegraded":"2024-05-10T03:29:48.794132+0000","last_fullsized":"2024-05-10T03:29:48.794132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791228+0000","last_change":"2024-05-10T03:29:44.342929+0000","last_active":"2024-05-10T03:29:48.791228+0000","last_peered":"2024-05-10T03:29:48.791228+0000","last_clean":"2024-05-10T03:29:48.791228+0000","last_became_active":"2024-05-10T03:29:44.342505+0000","last_became_peered":"2024-05-10T03:29:44.342505+0000","last_unstale":"2024-05-10T03:29:48.791228+0000","last_undegraded":"2024-05-10T03:29:48.791228+0000","last_fullsized":"2024-05-10T03:29:48.791228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811605+0000","last_change":"2024-05-10T03:29:44.338969+0000","last_active":"2024-05-10T03:29:48.811605+0000","last_peered":"2024-05-10T03:29:48.811605+0000","last_clean":"2024-05-10T03:29:48.811605+0000","last_became_active":"2024-05-10T03:29:44.337703+0000","last_became_peered":"2024-05-10T03:29:44.337703+0000","last_unstale":"2024-05-10T03:29:48.811605+0000","last_undegraded":"2024-05-10T03:29:48.811605+0000","last_fullsized":"2024-05-10T03:29:48.811605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812225+0000","last_change":"2024-05-10T03:29:44.348810+0000","last_active":"2024-05-10T03:29:48.812225+0000","last_peered":"2024-05-10T03:29:48.812225+0000","last_clean":"2024-05-10T03:29:48.812225+0000","last_became_active":"2024-05-10T03:29:44.348512+0000","last_became_peered":"2024-05-10T03:29:44.348512+0000","last_unstale":"2024-05-10T03:29:48.812225+0000","last_undegraded":"2024-05-10T03:29:48.812225+0000","last_fullsized":"2024-05-10T03:29:48.812225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249372+0000","last_change":"2024-05-10T03:29:44.344167+0000","last_active":"2024-05-10T03:29:49.249372+0000","last_peered":"2024-05-10T03:29:49.249372+0000","last_clean":"2024-05-10T03:29:49.249372+0000","last_became_active":"2024-05-10T03:29:44.343405+0000","last_became_peered":"2024-05-10T03:29:44.343405+0000","last_unstale":"2024-05-10T03:29:49.249372+0000","last_undegraded":"2024-05-10T03:29:49.249372+0000","last_fullsized":"2024-05-10T03:29:49.249372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808451+0000","last_change":"2024-05-10T03:29:44.337141+0000","last_active":"2024-05-10T03:29:48.808451+0000","last_peered":"2024-05-10T03:29:48.808451+0000","last_clean":"2024-05-10T03:29:48.808451+0000","last_became_active":"2024-05-10T03:29:44.336801+0000","last_became_peered":"2024-05-10T03:29:44.336801+0000","last_unstale":"2024-05-10T03:29:48.808451+0000","last_undegraded":"2024-05-10T03:29:48.808451+0000","last_fullsized":"2024-05-10T03:29:48.808451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242013+0000","last_change":"2024-05-10T03:29:44.339252+0000","last_active":"2024-05-10T03:29:49.242013+0000","last_peered":"2024-05-10T03:29:49.242013+0000","last_clean":"2024-05-10T03:29:49.242013+0000","last_became_active":"2024-05-10T03:29:44.338870+0000","last_became_peered":"2024-05-10T03:29:44.338870+0000","last_unstale":"2024-05-10T03:29:49.242013+0000","last_undegraded":"2024-05-10T03:29:49.242013+0000","last_fullsized":"2024-05-10T03:29:49.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792099+0000","last_change":"2024-05-10T03:29:44.341080+0000","last_active":"2024-05-10T03:29:48.792099+0000","last_peered":"2024-05-10T03:29:48.792099+0000","last_clean":"2024-05-10T03:29:48.792099+0000","last_became_active":"2024-05-10T03:29:44.340720+0000","last_became_peered":"2024-05-10T03:29:44.340720+0000","last_unstale":"2024-05-10T03:29:48.792099+0000","last_undegraded":"2024-05-10T03:29:48.792099+0000","last_fullsized":"2024-05-10T03:29:48.792099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241944+0000","last_change":"2024-05-10T03:29:44.345293+0000","last_active":"2024-05-10T03:29:49.241944+0000","last_peered":"2024-05-10T03:29:49.241944+0000","last_clean":"2024-05-10T03:29:49.241944+0000","last_became_active":"2024-05-10T03:29:44.344092+0000","last_became_peered":"2024-05-10T03:29:44.344092+0000","last_unstale":"2024-05-10T03:29:49.241944+0000","last_undegraded":"2024-05-10T03:29:49.241944+0000","last_fullsized":"2024-05-10T03:29:49.241944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797675+0000","last_change":"2024-05-10T03:29:44.359220+0000","last_active":"2024-05-10T03:29:48.797675+0000","last_peered":"2024-05-10T03:29:48.797675+0000","last_clean":"2024-05-10T03:29:48.797675+0000","last_became_active":"2024-05-10T03:29:44.358849+0000","last_became_peered":"2024-05-10T03:29:44.358849+0000","last_unstale":"2024-05-10T03:29:48.797675+0000","last_undegraded":"2024-05-10T03:29:48.797675+0000","last_fullsized":"2024-05-10T03:29:48.797675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:54.507386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795357+0000","last_change":"2024-05-10T03:29:44.349124+0000","last_active":"2024-05-10T03:29:48.795357+0000","last_peered":"2024-05-10T03:29:48.795357+0000","last_clean":"2024-05-10T03:29:48.795357+0000","last_became_active":"2024-05-10T03:29:44.348841+0000","last_became_peered":"2024-05-10T03:29:44.348841+0000","last_unstale":"2024-05-10T03:29:48.795357+0000","last_undegraded":"2024-05-10T03:29:48.795357+0000","last_fullsized":"2024-05-10T03:29:48.795357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795763+0000","last_change":"2024-05-10T03:29:44.342192+0000","last_active":"2024-05-10T03:29:48.795763+0000","last_peered":"2024-05-10T03:29:48.795763+0000","last_clean":"2024-05-10T03:29:48.795763+0000","last_became_active":"2024-05-10T03:29:44.341267+0000","last_became_peered":"2024-05-10T03:29:44.341267+0000","last_unstale":"2024-05-10T03:29:48.795763+0000","last_undegraded":"2024-05-10T03:29:48.795763+0000","last_fullsized":"2024-05-10T03:29:48.795763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249243+0000","last_change":"2024-05-10T03:29:44.344420+0000","last_active":"2024-05-10T03:29:49.249243+0000","last_peered":"2024-05-10T03:29:49.249243+0000","last_clean":"2024-05-10T03:29:49.249243+0000","last_became_active":"2024-05-10T03:29:44.343841+0000","last_became_peered":"2024-05-10T03:29:44.343841+0000","last_unstale":"2024-05-10T03:29:49.249243+0000","last_undegraded":"2024-05-10T03:29:49.249243+0000","last_fullsized":"2024-05-10T03:29:49.249243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.133}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.833}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.149}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:30:14.326 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:30:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.642+0000 7f7653e00640 1 Processor -- start 2024-05-10T03:30:14.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.642+0000 7f7653e00640 1 -- start start 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f764c06b6e0 0x7f764c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 0x7f764c165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c162380 0x7f764c162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0711c0 con 0x7f764c06b6e0 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c06baf0 con 0x7f764c1659c0 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.644+0000 7f7653e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0715e0 con 0x7f764c162380 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f764c06b6e0 0x7f764c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f764c06b6e0 0x7f764c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33378/0 (socket says 172.21.15.55:33378) 2024-05-10T03:30:14.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 -- 172.21.15.55:0/2612135753 learned_addr learned my addr 172.21.15.55:0/2612135753 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7650a00640 1 --2- 172.21.15.55:0/2612135753 >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 0x7f764c165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 -- 172.21.15.55:0/2612135753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c162380 msgr2=0x7f764c162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651e00640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c162380 0x7f764c162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c162380 0x7f764c162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 -- 172.21.15.55:0/2612135753 >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 msgr2=0x7f764c165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 --2- 172.21.15.55:0/2612135753 >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 0x7f764c165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7651400640 1 -- 172.21.15.55:0/2612135753 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c166310 con 0x7f764c06b6e0 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.645+0000 7f7650a00640 1 --2- 172.21.15.55:0/2612135753 >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 0x7f764c165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:14.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.646+0000 7f7651400640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c06b6e0 0x7f764c0ce3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f764c06d1d0 tx=0x7f764002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a866bdb740ee9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.647+0000 7f764be00640 1 -- 172.21.15.55:0/2612135753 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764002ae90 con 0x7f764c06b6e0 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.647+0000 7f764be00640 1 -- 172.21.15.55:0/2612135753 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7640032440 con 0x7f764c06b6e0 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.647+0000 7f764be00640 1 -- 172.21.15.55:0/2612135753 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400327d0 con 0x7f764c06b6e0 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.647+0000 7f7653e00640 1 -- 172.21.15.55:0/2612135753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c06b6e0 msgr2=0x7f764c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.647+0000 7f7653e00640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c06b6e0 0x7f764c0ce3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f764c06d1d0 tx=0x7f764002a140 comp rx=0 tx=0).stop 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 -- 172.21.15.55:0/2612135753 shutdown_connections 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c162380 0x7f764c162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 --2- 172.21.15.55:0/2612135753 >> v2:172.21.15.162:3300/0 conn(0x7f764c1659c0 0x7f764c165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 --2- 172.21.15.55:0/2612135753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c06b6e0 0x7f764c0ce3b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 -- 172.21.15.55:0/2612135753 >> 172.21.15.55:0/2612135753 conn(0x7f764c0cbb90 msgr2=0x7f764c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 -- 172.21.15.55:0/2612135753 shutdown_connections 2024-05-10T03:30:14.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.648+0000 7f7653e00640 1 -- 172.21.15.55:0/2612135753 wait complete. 2024-05-10T03:30:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.651+0000 7f7653e00640 1 Processor -- start 2024-05-10T03:30:14.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.651+0000 7f7653e00640 1 -- start start 2024-05-10T03:30:14.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f764c06b6e0 0x7f764c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c1659c0 0x7f764c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c148a50 con 0x7f764c162380 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1495e0 con 0x7f764c06b6e0 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7653e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c149830 con 0x7f764c1659c0 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7650a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7650a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33384/0 (socket says 172.21.15.55:33384) 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7650a00640 1 -- 172.21.15.55:0/1613877104 learned_addr learned my addr 172.21.15.55:0/1613877104 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7651400640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f764c06b6e0 0x7f764c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.653+0000 7f7651e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c1659c0 0x7f764c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 -- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c1659c0 msgr2=0x7f764c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c1659c0 0x7f764c15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 -- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f764c06b6e0 msgr2=0x7f764c146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f764c06b6e0 0x7f764c146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 -- 172.21.15.55:0/1613877104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400094d0 con 0x7f764c162380 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7650a00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f763800c7f0 tx=0x7f763800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638007820 con 0x7f764c162380 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638007980 con 0x7f764c162380 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638007cf0 con 0x7f764c162380 2024-05-10T03:30:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.654+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c149b10 con 0x7f764c162380 2024-05-10T03:30:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.655+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c14a0d0 con 0x7f764c162380 2024-05-10T03:30:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.655+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c147560 con 0x7f764c162380 2024-05-10T03:30:14.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.657+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f76380113f0 con 0x7f764c162380 2024-05-10T03:30:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.658+0000 7f7649600640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 0x7f762c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:14.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.658+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7638089a50 con 0x7f764c162380 2024-05-10T03:30:14.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.659+0000 7f7651400640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 0x7f762c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:14.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.660+0000 7f7651400640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 0x7f762c06d110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7640034000 tx=0x7f764002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:14.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.663+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f76380548a0 con 0x7f764c162380 2024-05-10T03:30:14.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.850+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f764c070260 con 0x7f762c06ac00 2024-05-10T03:30:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.877+0000 7f7649600640 1 -- 172.21.15.55:0/1613877104 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345044 (secure 0 0 0) 0x7f7640032b00 con 0x7f762c06ac00 2024-05-10T03:30:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:14.889 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 msgr2=0x7f762c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 0x7f762c06d110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7640034000 tx=0x7f764002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 msgr2=0x7f764c15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f763800c7f0 tx=0x7f763800ccb0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 shutdown_connections 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f764c1659c0 0x7f764c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f764c162380 0x7f764c15b0e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f762c06ac00 0x7f762c06d110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 --2- 172.21.15.55:0/1613877104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f764c06b6e0 0x7f764c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 >> 172.21.15.55:0/1613877104 conn(0x7f764c0cbb90 msgr2=0x7f764c0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 shutdown_connections 2024-05-10T03:30:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:14.885+0000 7f7653e00640 1 -- 172.21.15.55:0/1613877104 wait complete. 2024-05-10T03:30:14.917 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-05-10T03:30:12.954695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231796,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754742924,"statfs":{"total":773094113280,"available":772856754176,"internally_reserved":0,"allocated":3383296,"data_stored":2025532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005604"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.796272+0000","last_change":"2024-05-10T03:29:43.770873+0000","last_active":"2024-05-10T03:29:48.796272+0000","last_peered":"2024-05-10T03:29:48.796272+0000","last_clean":"2024-05-10T03:29:48.796272+0000","last_became_active":"2024-05-10T03:29:43.770443+0000","last_became_peered":"2024-05-10T03:29:43.770443+0000","last_unstale":"2024-05-10T03:29:48.796272+0000","last_undegraded":"2024-05-10T03:29:48.796272+0000","last_fullsized":"2024-05-10T03:29:48.796272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:55.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.814450+0000","last_change":"2024-05-10T03:29:44.352733+0000","last_active":"2024-05-10T03:29:48.814450+0000","last_peered":"2024-05-10T03:29:48.814450+0000","last_clean":"2024-05-10T03:29:48.814450+0000","last_became_active":"2024-05-10T03:29:44.351354+0000","last_became_peered":"2024-05-10T03:29:44.351354+0000","last_unstale":"2024-05-10T03:29:48.814450+0000","last_undegraded":"2024-05-10T03:29:48.814450+0000","last_fullsized":"2024-05-10T03:29:48.814450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367124+0000","last_change":"2024-05-10T03:29:49.255640+0000","last_active":"2024-05-10T03:29:49.367124+0000","last_peered":"2024-05-10T03:29:49.367124+0000","last_clean":"2024-05-10T03:29:49.367124+0000","last_became_active":"2024-05-10T03:29:49.254068+0000","last_became_peered":"2024-05-10T03:29:49.254068+0000","last_unstale":"2024-05-10T03:29:49.367124+0000","last_undegraded":"2024-05-10T03:29:49.367124+0000","last_fullsized":"2024-05-10T03:29:49.367124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:09.608357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813230+0000","last_change":"2024-05-10T03:29:43.768485+0000","last_active":"2024-05-10T03:29:48.813230+0000","last_peered":"2024-05-10T03:29:48.813230+0000","last_clean":"2024-05-10T03:29:48.813230+0000","last_became_active":"2024-05-10T03:29:43.767506+0000","last_became_peered":"2024-05-10T03:29:43.767506+0000","last_unstale":"2024-05-10T03:29:48.813230+0000","last_undegraded":"2024-05-10T03:29:48.813230+0000","last_fullsized":"2024-05-10T03:29:48.813230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795311+0000","last_change":"2024-05-10T03:29:44.339118+0000","last_active":"2024-05-10T03:29:48.795311+0000","last_peered":"2024-05-10T03:29:48.795311+0000","last_clean":"2024-05-10T03:29:48.795311+0000","last_became_active":"2024-05-10T03:29:44.338690+0000","last_became_peered":"2024-05-10T03:29:44.338690+0000","last_unstale":"2024-05-10T03:29:48.795311+0000","last_undegraded":"2024-05-10T03:29:48.795311+0000","last_fullsized":"2024-05-10T03:29:48.795311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:23.150055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909491+0000","last_change":"2024-05-10T03:29:48.819825+0000","last_active":"2024-05-10T03:29:49.909491+0000","last_peered":"2024-05-10T03:29:49.909491+0000","last_clean":"2024-05-10T03:29:49.909491+0000","last_became_active":"2024-05-10T03:29:48.818817+0000","last_became_peered":"2024-05-10T03:29:48.818817+0000","last_unstale":"2024-05-10T03:29:49.909491+0000","last_undegraded":"2024-05-10T03:29:49.909491+0000","last_fullsized":"2024-05-10T03:29:49.909491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:10.138375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247614+0000","last_change":"2024-05-10T03:29:43.777967+0000","last_active":"2024-05-10T03:29:49.247614+0000","last_peered":"2024-05-10T03:29:49.247614+0000","last_clean":"2024-05-10T03:29:49.247614+0000","last_became_active":"2024-05-10T03:29:43.777585+0000","last_became_peered":"2024-05-10T03:29:43.777585+0000","last_unstale":"2024-05-10T03:29:49.247614+0000","last_undegraded":"2024-05-10T03:29:49.247614+0000","last_fullsized":"2024-05-10T03:29:49.247614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797875+0000","last_change":"2024-05-10T03:29:44.343238+0000","last_active":"2024-05-10T03:29:48.797875+0000","last_peered":"2024-05-10T03:29:48.797875+0000","last_clean":"2024-05-10T03:29:48.797875+0000","last_became_active":"2024-05-10T03:29:44.339308+0000","last_became_peered":"2024-05-10T03:29:44.339308+0000","last_unstale":"2024-05-10T03:29:48.797875+0000","last_undegraded":"2024-05-10T03:29:48.797875+0000","last_fullsized":"2024-05-10T03:29:48.797875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910600+0000","last_change":"2024-05-10T03:29:48.821031+0000","last_active":"2024-05-10T03:29:49.910600+0000","last_peered":"2024-05-10T03:29:49.910600+0000","last_clean":"2024-05-10T03:29:49.910600+0000","last_became_active":"2024-05-10T03:29:48.818081+0000","last_became_peered":"2024-05-10T03:29:48.818081+0000","last_unstale":"2024-05-10T03:29:49.910600+0000","last_undegraded":"2024-05-10T03:29:49.910600+0000","last_fullsized":"2024-05-10T03:29:49.910600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243722+0000","last_change":"2024-05-10T03:29:43.776394+0000","last_active":"2024-05-10T03:29:49.243722+0000","last_peered":"2024-05-10T03:29:49.243722+0000","last_clean":"2024-05-10T03:29:49.243722+0000","last_became_active":"2024-05-10T03:29:43.774971+0000","last_became_peered":"2024-05-10T03:29:43.774971+0000","last_unstale":"2024-05-10T03:29:49.243722+0000","last_undegraded":"2024-05-10T03:29:49.243722+0000","last_fullsized":"2024-05-10T03:29:49.243722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798050+0000","last_change":"2024-05-10T03:29:44.336984+0000","last_active":"2024-05-10T03:29:48.798050+0000","last_peered":"2024-05-10T03:29:48.798050+0000","last_clean":"2024-05-10T03:29:48.798050+0000","last_became_active":"2024-05-10T03:29:44.335925+0000","last_became_peered":"2024-05-10T03:29:44.335925+0000","last_unstale":"2024-05-10T03:29:48.798050+0000","last_undegraded":"2024-05-10T03:29:48.798050+0000","last_fullsized":"2024-05-10T03:29:48.798050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:25.887533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909455+0000","last_change":"2024-05-10T03:29:48.818620+0000","last_active":"2024-05-10T03:29:49.909455+0000","last_peered":"2024-05-10T03:29:49.909455+0000","last_clean":"2024-05-10T03:29:49.909455+0000","last_became_active":"2024-05-10T03:29:48.817907+0000","last_became_peered":"2024-05-10T03:29:48.817907+0000","last_unstale":"2024-05-10T03:29:49.909455+0000","last_undegraded":"2024-05-10T03:29:49.909455+0000","last_fullsized":"2024-05-10T03:29:49.909455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808538+0000","last_change":"2024-05-10T03:29:44.347407+0000","last_active":"2024-05-10T03:29:48.808538+0000","last_peered":"2024-05-10T03:29:48.808538+0000","last_clean":"2024-05-10T03:29:48.808538+0000","last_became_active":"2024-05-10T03:29:44.347072+0000","last_became_peered":"2024-05-10T03:29:44.347072+0000","last_unstale":"2024-05-10T03:29:48.808538+0000","last_undegraded":"2024-05-10T03:29:48.808538+0000","last_fullsized":"2024-05-10T03:29:48.808538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243215+0000","last_change":"2024-05-10T03:29:43.775422+0000","last_active":"2024-05-10T03:29:49.243215+0000","last_peered":"2024-05-10T03:29:49.243215+0000","last_clean":"2024-05-10T03:29:49.243215+0000","last_became_active":"2024-05-10T03:29:43.775083+0000","last_became_peered":"2024-05-10T03:29:43.775083+0000","last_unstale":"2024-05-10T03:29:49.243215+0000","last_undegraded":"2024-05-10T03:29:49.243215+0000","last_fullsized":"2024-05-10T03:29:49.243215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910633+0000","last_change":"2024-05-10T03:29:48.812082+0000","last_active":"2024-05-10T03:29:49.910633+0000","last_peered":"2024-05-10T03:29:49.910633+0000","last_clean":"2024-05-10T03:29:49.910633+0000","last_became_active":"2024-05-10T03:29:48.809401+0000","last_became_peered":"2024-05-10T03:29:48.809401+0000","last_unstale":"2024-05-10T03:29:49.910633+0000","last_undegraded":"2024-05-10T03:29:49.910633+0000","last_fullsized":"2024-05-10T03:29:49.910633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794221+0000","last_change":"2024-05-10T03:29:44.355438+0000","last_active":"2024-05-10T03:29:48.794221+0000","last_peered":"2024-05-10T03:29:48.794221+0000","last_clean":"2024-05-10T03:29:48.794221+0000","last_became_active":"2024-05-10T03:29:44.354716+0000","last_became_peered":"2024-05-10T03:29:44.354716+0000","last_unstale":"2024-05-10T03:29:48.794221+0000","last_undegraded":"2024-05-10T03:29:48.794221+0000","last_fullsized":"2024-05-10T03:29:48.794221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797885+0000","last_change":"2024-05-10T03:29:43.761646+0000","last_active":"2024-05-10T03:29:48.797885+0000","last_peered":"2024-05-10T03:29:48.797885+0000","last_clean":"2024-05-10T03:29:48.797885+0000","last_became_active":"2024-05-10T03:29:43.759174+0000","last_became_peered":"2024-05-10T03:29:43.759174+0000","last_unstale":"2024-05-10T03:29:48.797885+0000","last_undegraded":"2024-05-10T03:29:48.797885+0000","last_fullsized":"2024-05-10T03:29:48.797885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910230+0000","last_change":"2024-05-10T03:29:48.822517+0000","last_active":"2024-05-10T03:29:49.910230+0000","last_peered":"2024-05-10T03:29:49.910230+0000","last_clean":"2024-05-10T03:29:49.910230+0000","last_became_active":"2024-05-10T03:29:48.821950+0000","last_became_peered":"2024-05-10T03:29:48.821950+0000","last_unstale":"2024-05-10T03:29:49.910230+0000","last_undegraded":"2024-05-10T03:29:49.910230+0000","last_fullsized":"2024-05-10T03:29:49.910230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:56:50.388163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808692+0000","last_change":"2024-05-10T03:29:44.324359+0000","last_active":"2024-05-10T03:29:48.808692+0000","last_peered":"2024-05-10T03:29:48.808692+0000","last_clean":"2024-05-10T03:29:48.808692+0000","last_became_active":"2024-05-10T03:29:44.323910+0000","last_became_peered":"2024-05-10T03:29:44.323910+0000","last_unstale":"2024-05-10T03:29:48.808692+0000","last_undegraded":"2024-05-10T03:29:48.808692+0000","last_fullsized":"2024-05-10T03:29:48.808692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794312+0000","last_change":"2024-05-10T03:29:43.771353+0000","last_active":"2024-05-10T03:29:48.794312+0000","last_peered":"2024-05-10T03:29:48.794312+0000","last_clean":"2024-05-10T03:29:48.794312+0000","last_became_active":"2024-05-10T03:29:43.769654+0000","last_became_peered":"2024-05-10T03:29:43.769654+0000","last_unstale":"2024-05-10T03:29:48.794312+0000","last_undegraded":"2024-05-10T03:29:48.794312+0000","last_fullsized":"2024-05-10T03:29:48.794312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494881+0000","last_change":"2024-05-10T03:29:49.254990+0000","last_active":"2024-05-10T03:29:49.494881+0000","last_peered":"2024-05-10T03:29:49.494881+0000","last_clean":"2024-05-10T03:29:49.494881+0000","last_became_active":"2024-05-10T03:29:49.253789+0000","last_became_peered":"2024-05-10T03:29:49.253789+0000","last_unstale":"2024-05-10T03:29:49.494881+0000","last_undegraded":"2024-05-10T03:29:49.494881+0000","last_fullsized":"2024-05-10T03:29:49.494881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790993+0000","last_change":"2024-05-10T03:29:44.347866+0000","last_active":"2024-05-10T03:29:48.790993+0000","last_peered":"2024-05-10T03:29:48.790993+0000","last_clean":"2024-05-10T03:29:48.790993+0000","last_became_active":"2024-05-10T03:29:44.346222+0000","last_became_peered":"2024-05-10T03:29:44.346222+0000","last_unstale":"2024-05-10T03:29:48.790993+0000","last_undegraded":"2024-05-10T03:29:48.790993+0000","last_fullsized":"2024-05-10T03:29:48.790993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241506+0000","last_change":"2024-05-10T03:29:43.775452+0000","last_active":"2024-05-10T03:29:49.241506+0000","last_peered":"2024-05-10T03:29:49.241506+0000","last_clean":"2024-05-10T03:29:49.241506+0000","last_became_active":"2024-05-10T03:29:43.774420+0000","last_became_peered":"2024-05-10T03:29:43.774420+0000","last_unstale":"2024-05-10T03:29:49.241506+0000","last_undegraded":"2024-05-10T03:29:49.241506+0000","last_fullsized":"2024-05-10T03:29:49.241506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910708+0000","last_change":"2024-05-10T03:29:48.815642+0000","last_active":"2024-05-10T03:29:49.910708+0000","last_peered":"2024-05-10T03:29:49.910708+0000","last_clean":"2024-05-10T03:29:49.910708+0000","last_became_active":"2024-05-10T03:29:48.815170+0000","last_became_peered":"2024-05-10T03:29:48.815170+0000","last_unstale":"2024-05-10T03:29:49.910708+0000","last_undegraded":"2024-05-10T03:29:49.910708+0000","last_fullsized":"2024-05-10T03:29:49.910708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.338211+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.337821+0000","last_became_peered":"2024-05-10T03:29:44.337821+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811440+0000","last_change":"2024-05-10T03:29:43.774006+0000","last_active":"2024-05-10T03:29:48.811440+0000","last_peered":"2024-05-10T03:29:48.811440+0000","last_clean":"2024-05-10T03:29:48.811440+0000","last_became_active":"2024-05-10T03:29:43.773614+0000","last_became_peered":"2024-05-10T03:29:43.773614+0000","last_unstale":"2024-05-10T03:29:48.811440+0000","last_undegraded":"2024-05-10T03:29:48.811440+0000","last_fullsized":"2024-05-10T03:29:48.811440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:33.437326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910293+0000","last_change":"2024-05-10T03:29:49.254777+0000","last_active":"2024-05-10T03:29:49.910293+0000","last_peered":"2024-05-10T03:29:49.910293+0000","last_clean":"2024-05-10T03:29:49.910293+0000","last_became_active":"2024-05-10T03:29:49.253585+0000","last_became_peered":"2024-05-10T03:29:49.253585+0000","last_unstale":"2024-05-10T03:29:49.910293+0000","last_undegraded":"2024-05-10T03:29:49.910293+0000","last_fullsized":"2024-05-10T03:29:49.910293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:58.544334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813189+0000","last_change":"2024-05-10T03:29:44.344906+0000","last_active":"2024-05-10T03:29:48.813189+0000","last_peered":"2024-05-10T03:29:48.813189+0000","last_clean":"2024-05-10T03:29:48.813189+0000","last_became_active":"2024-05-10T03:29:44.344600+0000","last_became_peered":"2024-05-10T03:29:44.344600+0000","last_unstale":"2024-05-10T03:29:48.813189+0000","last_undegraded":"2024-05-10T03:29:48.813189+0000","last_fullsized":"2024-05-10T03:29:48.813189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795841+0000","last_change":"2024-05-10T03:29:43.768246+0000","last_active":"2024-05-10T03:29:48.795841+0000","last_peered":"2024-05-10T03:29:48.795841+0000","last_clean":"2024-05-10T03:29:48.795841+0000","last_became_active":"2024-05-10T03:29:43.767828+0000","last_became_peered":"2024-05-10T03:29:43.767828+0000","last_unstale":"2024-05-10T03:29:48.795841+0000","last_undegraded":"2024-05-10T03:29:48.795841+0000","last_fullsized":"2024-05-10T03:29:48.795841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366780+0000","last_change":"2024-05-10T03:29:49.246734+0000","last_active":"2024-05-10T03:29:49.366780+0000","last_peered":"2024-05-10T03:29:49.366780+0000","last_clean":"2024-05-10T03:29:49.366780+0000","last_became_active":"2024-05-10T03:29:49.246242+0000","last_became_peered":"2024-05-10T03:29:49.246242+0000","last_unstale":"2024-05-10T03:29:49.366780+0000","last_undegraded":"2024-05-10T03:29:49.366780+0000","last_fullsized":"2024-05-10T03:29:49.366780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:58:50.675077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797404+0000","last_change":"2024-05-10T03:29:44.342815+0000","last_active":"2024-05-10T03:29:48.797404+0000","last_peered":"2024-05-10T03:29:48.797404+0000","last_clean":"2024-05-10T03:29:48.797404+0000","last_became_active":"2024-05-10T03:29:44.340796+0000","last_became_peered":"2024-05-10T03:29:44.340796+0000","last_unstale":"2024-05-10T03:29:48.797404+0000","last_undegraded":"2024-05-10T03:29:48.797404+0000","last_fullsized":"2024-05-10T03:29:48.797404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:01:08.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.248230+0000","last_change":"2024-05-10T03:29:43.775843+0000","last_active":"2024-05-10T03:29:49.248230+0000","last_peered":"2024-05-10T03:29:49.248230+0000","last_clean":"2024-05-10T03:29:49.248230+0000","last_became_active":"2024-05-10T03:29:43.775096+0000","last_became_peered":"2024-05-10T03:29:43.775096+0000","last_unstale":"2024-05-10T03:29:49.248230+0000","last_undegraded":"2024-05-10T03:29:49.248230+0000","last_fullsized":"2024-05-10T03:29:49.248230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:22.134113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909045+0000","last_change":"2024-05-10T03:29:49.250836+0000","last_active":"2024-05-10T03:29:49.909045+0000","last_peered":"2024-05-10T03:29:49.909045+0000","last_clean":"2024-05-10T03:29:49.909045+0000","last_became_active":"2024-05-10T03:29:49.249127+0000","last_became_peered":"2024-05-10T03:29:49.249127+0000","last_unstale":"2024-05-10T03:29:49.909045+0000","last_undegraded":"2024-05-10T03:29:49.909045+0000","last_fullsized":"2024-05-10T03:29:49.909045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798180+0000","last_change":"2024-05-10T03:29:44.349438+0000","last_active":"2024-05-10T03:29:48.798180+0000","last_peered":"2024-05-10T03:29:48.798180+0000","last_clean":"2024-05-10T03:29:48.798180+0000","last_became_active":"2024-05-10T03:29:44.348987+0000","last_became_peered":"2024-05-10T03:29:44.348987+0000","last_unstale":"2024-05-10T03:29:48.798180+0000","last_undegraded":"2024-05-10T03:29:48.798180+0000","last_fullsized":"2024-05-10T03:29:48.798180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797443+0000","last_change":"2024-05-10T03:29:43.765358+0000","last_active":"2024-05-10T03:29:48.797443+0000","last_peered":"2024-05-10T03:29:48.797443+0000","last_clean":"2024-05-10T03:29:48.797443+0000","last_became_active":"2024-05-10T03:29:43.764855+0000","last_became_peered":"2024-05-10T03:29:43.764855+0000","last_unstale":"2024-05-10T03:29:48.797443+0000","last_undegraded":"2024-05-10T03:29:48.797443+0000","last_fullsized":"2024-05-10T03:29:48.797443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494037+0000","last_change":"2024-05-10T03:29:48.807909+0000","last_active":"2024-05-10T03:29:49.494037+0000","last_peered":"2024-05-10T03:29:49.494037+0000","last_clean":"2024-05-10T03:29:49.494037+0000","last_became_active":"2024-05-10T03:29:48.807403+0000","last_became_peered":"2024-05-10T03:29:48.807403+0000","last_unstale":"2024-05-10T03:29:49.494037+0000","last_undegraded":"2024-05-10T03:29:49.494037+0000","last_fullsized":"2024-05-10T03:29:49.494037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:12.323596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791871+0000","last_change":"2024-05-10T03:29:44.352509+0000","last_active":"2024-05-10T03:29:48.791871+0000","last_peered":"2024-05-10T03:29:48.791871+0000","last_clean":"2024-05-10T03:29:48.791871+0000","last_became_active":"2024-05-10T03:29:44.349456+0000","last_became_peered":"2024-05-10T03:29:44.349456+0000","last_unstale":"2024-05-10T03:29:48.791871+0000","last_undegraded":"2024-05-10T03:29:48.791871+0000","last_fullsized":"2024-05-10T03:29:48.791871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:10.785937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792782+0000","last_change":"2024-05-10T03:29:43.754724+0000","last_active":"2024-05-10T03:29:48.792782+0000","last_peered":"2024-05-10T03:29:48.792782+0000","last_clean":"2024-05-10T03:29:48.792782+0000","last_became_active":"2024-05-10T03:29:43.754159+0000","last_became_peered":"2024-05-10T03:29:43.754159+0000","last_unstale":"2024-05-10T03:29:48.792782+0000","last_undegraded":"2024-05-10T03:29:48.792782+0000","last_fullsized":"2024-05-10T03:29:48.792782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494653+0000","last_change":"2024-05-10T03:29:48.822295+0000","last_active":"2024-05-10T03:29:49.494653+0000","last_peered":"2024-05-10T03:29:49.494653+0000","last_clean":"2024-05-10T03:29:49.494653+0000","last_became_active":"2024-05-10T03:29:48.821883+0000","last_became_peered":"2024-05-10T03:29:48.821883+0000","last_unstale":"2024-05-10T03:29:49.494653+0000","last_undegraded":"2024-05-10T03:29:49.494653+0000","last_fullsized":"2024-05-10T03:29:49.494653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799865+0000","last_change":"2024-05-10T03:29:44.330889+0000","last_active":"2024-05-10T03:29:48.799865+0000","last_peered":"2024-05-10T03:29:48.799865+0000","last_clean":"2024-05-10T03:29:48.799865+0000","last_became_active":"2024-05-10T03:29:44.327692+0000","last_became_peered":"2024-05-10T03:29:44.327692+0000","last_unstale":"2024-05-10T03:29:48.799865+0000","last_undegraded":"2024-05-10T03:29:48.799865+0000","last_fullsized":"2024-05-10T03:29:48.799865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780603+0000","last_change":"2024-05-10T03:29:43.760419+0000","last_active":"2024-05-10T03:29:48.780603+0000","last_peered":"2024-05-10T03:29:48.780603+0000","last_clean":"2024-05-10T03:29:48.780603+0000","last_became_active":"2024-05-10T03:29:43.760082+0000","last_became_peered":"2024-05-10T03:29:43.760082+0000","last_unstale":"2024-05-10T03:29:48.780603+0000","last_undegraded":"2024-05-10T03:29:48.780603+0000","last_fullsized":"2024-05-10T03:29:48.780603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494952+0000","last_change":"2024-05-10T03:29:48.819719+0000","last_active":"2024-05-10T03:29:49.494952+0000","last_peered":"2024-05-10T03:29:49.494952+0000","last_clean":"2024-05-10T03:29:49.494952+0000","last_became_active":"2024-05-10T03:29:48.818741+0000","last_became_peered":"2024-05-10T03:29:48.818741+0000","last_unstale":"2024-05-10T03:29:49.494952+0000","last_undegraded":"2024-05-10T03:29:49.494952+0000","last_fullsized":"2024-05-10T03:29:49.494952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:46:50.233880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.787035+0000","last_change":"2024-05-10T03:29:44.354949+0000","last_active":"2024-05-10T03:29:48.787035+0000","last_peered":"2024-05-10T03:29:48.787035+0000","last_clean":"2024-05-10T03:29:48.787035+0000","last_became_active":"2024-05-10T03:29:44.354451+0000","last_became_peered":"2024-05-10T03:29:44.354451+0000","last_unstale":"2024-05-10T03:29:48.787035+0000","last_undegraded":"2024-05-10T03:29:48.787035+0000","last_fullsized":"2024-05-10T03:29:48.787035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241517+0000","last_change":"2024-05-10T03:29:43.776046+0000","last_active":"2024-05-10T03:29:49.241517+0000","last_peered":"2024-05-10T03:29:49.241517+0000","last_clean":"2024-05-10T03:29:49.241517+0000","last_became_active":"2024-05-10T03:29:43.775739+0000","last_became_peered":"2024-05-10T03:29:43.775739+0000","last_unstale":"2024-05-10T03:29:49.241517+0000","last_undegraded":"2024-05-10T03:29:49.241517+0000","last_fullsized":"2024-05-10T03:29:49.241517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909517+0000","last_change":"2024-05-10T03:29:49.247763+0000","last_active":"2024-05-10T03:29:49.909517+0000","last_peered":"2024-05-10T03:29:49.909517+0000","last_clean":"2024-05-10T03:29:49.909517+0000","last_became_active":"2024-05-10T03:29:49.246481+0000","last_became_peered":"2024-05-10T03:29:49.246481+0000","last_unstale":"2024-05-10T03:29:49.909517+0000","last_undegraded":"2024-05-10T03:29:49.909517+0000","last_fullsized":"2024-05-10T03:29:49.909517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.793334+0000","last_change":"2024-05-10T03:29:44.337125+0000","last_active":"2024-05-10T03:29:48.793334+0000","last_peered":"2024-05-10T03:29:48.793334+0000","last_clean":"2024-05-10T03:29:48.793334+0000","last_became_active":"2024-05-10T03:29:44.336773+0000","last_became_peered":"2024-05-10T03:29:44.336773+0000","last_unstale":"2024-05-10T03:29:48.793334+0000","last_undegraded":"2024-05-10T03:29:48.793334+0000","last_fullsized":"2024-05-10T03:29:48.793334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797167+0000","last_change":"2024-05-10T03:29:43.759632+0000","last_active":"2024-05-10T03:29:48.797167+0000","last_peered":"2024-05-10T03:29:48.797167+0000","last_clean":"2024-05-10T03:29:48.797167+0000","last_became_active":"2024-05-10T03:29:43.758627+0000","last_became_peered":"2024-05-10T03:29:43.758627+0000","last_unstale":"2024-05-10T03:29:48.797167+0000","last_undegraded":"2024-05-10T03:29:48.797167+0000","last_fullsized":"2024-05-10T03:29:48.797167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241817+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:29:49.241817+0000","last_peered":"2024-05-10T03:29:49.241817+0000","last_clean":"2024-05-10T03:29:49.241817+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:29:49.241817+0000","last_undegraded":"2024-05-10T03:29:49.241817+0000","last_fullsized":"2024-05-10T03:29:49.241817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373998+0000","last_change":"2024-05-10T03:29:49.258186+0000","last_active":"2024-05-10T03:29:49.373998+0000","last_peered":"2024-05-10T03:29:49.373998+0000","last_clean":"2024-05-10T03:29:49.373998+0000","last_became_active":"2024-05-10T03:29:49.256940+0000","last_became_peered":"2024-05-10T03:29:49.256940+0000","last_unstale":"2024-05-10T03:29:49.373998+0000","last_undegraded":"2024-05-10T03:29:49.373998+0000","last_fullsized":"2024-05-10T03:29:49.373998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:56.951686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247185+0000","last_change":"2024-05-10T03:29:44.333742+0000","last_active":"2024-05-10T03:29:49.247185+0000","last_peered":"2024-05-10T03:29:49.247185+0000","last_clean":"2024-05-10T03:29:49.247185+0000","last_became_active":"2024-05-10T03:29:44.333369+0000","last_became_peered":"2024-05-10T03:29:44.333369+0000","last_unstale":"2024-05-10T03:29:49.247185+0000","last_undegraded":"2024-05-10T03:29:49.247185+0000","last_fullsized":"2024-05-10T03:29:49.247185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:02.931303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.782735+0000","last_change":"2024-05-10T03:29:43.759722+0000","last_active":"2024-05-10T03:29:48.782735+0000","last_peered":"2024-05-10T03:29:48.782735+0000","last_clean":"2024-05-10T03:29:48.782735+0000","last_became_active":"2024-05-10T03:29:43.759323+0000","last_became_peered":"2024-05-10T03:29:43.759323+0000","last_unstale":"2024-05-10T03:29:48.782735+0000","last_undegraded":"2024-05-10T03:29:48.782735+0000","last_fullsized":"2024-05-10T03:29:48.782735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:03:20.074508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.374329+0000","last_change":"2024-05-10T03:29:49.246585+0000","last_active":"2024-05-10T03:29:49.374329+0000","last_peered":"2024-05-10T03:29:49.374329+0000","last_clean":"2024-05-10T03:29:49.374329+0000","last_became_active":"2024-05-10T03:29:49.245297+0000","last_became_peered":"2024-05-10T03:29:49.245297+0000","last_unstale":"2024-05-10T03:29:49.374329+0000","last_undegraded":"2024-05-10T03:29:49.374329+0000","last_fullsized":"2024-05-10T03:29:49.374329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:50:46.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797859+0000","last_change":"2024-05-10T03:29:44.341567+0000","last_active":"2024-05-10T03:29:48.797859+0000","last_peered":"2024-05-10T03:29:48.797859+0000","last_clean":"2024-05-10T03:29:48.797859+0000","last_became_active":"2024-05-10T03:29:44.341235+0000","last_became_peered":"2024-05-10T03:29:44.341235+0000","last_unstale":"2024-05-10T03:29:48.797859+0000","last_undegraded":"2024-05-10T03:29:48.797859+0000","last_fullsized":"2024-05-10T03:29:48.797859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795601+0000","last_change":"2024-05-10T03:29:43.756510+0000","last_active":"2024-05-10T03:29:48.795601+0000","last_peered":"2024-05-10T03:29:48.795601+0000","last_clean":"2024-05-10T03:29:48.795601+0000","last_became_active":"2024-05-10T03:29:43.756205+0000","last_became_peered":"2024-05-10T03:29:43.756205+0000","last_unstale":"2024-05-10T03:29:48.795601+0000","last_undegraded":"2024-05-10T03:29:48.795601+0000","last_fullsized":"2024-05-10T03:29:48.795601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.908687+0000","last_change":"2024-05-10T03:29:48.806390+0000","last_active":"2024-05-10T03:29:49.908687+0000","last_peered":"2024-05-10T03:29:49.908687+0000","last_clean":"2024-05-10T03:29:49.908687+0000","last_became_active":"2024-05-10T03:29:48.805905+0000","last_became_peered":"2024-05-10T03:29:48.805905+0000","last_unstale":"2024-05-10T03:29:49.908687+0000","last_undegraded":"2024-05-10T03:29:49.908687+0000","last_fullsized":"2024-05-10T03:29:49.908687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249530+0000","last_change":"2024-05-10T03:29:44.348672+0000","last_active":"2024-05-10T03:29:49.249530+0000","last_peered":"2024-05-10T03:29:49.249530+0000","last_clean":"2024-05-10T03:29:49.249530+0000","last_became_active":"2024-05-10T03:29:44.348328+0000","last_became_peered":"2024-05-10T03:29:44.348328+0000","last_unstale":"2024-05-10T03:29:49.249530+0000","last_undegraded":"2024-05-10T03:29:49.249530+0000","last_fullsized":"2024-05-10T03:29:49.249530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781197+0000","last_change":"2024-05-10T03:29:43.760410+0000","last_active":"2024-05-10T03:29:48.781197+0000","last_peered":"2024-05-10T03:29:48.781197+0000","last_clean":"2024-05-10T03:29:48.781197+0000","last_became_active":"2024-05-10T03:29:43.760137+0000","last_became_peered":"2024-05-10T03:29:43.760137+0000","last_unstale":"2024-05-10T03:29:48.781197+0000","last_undegraded":"2024-05-10T03:29:48.781197+0000","last_fullsized":"2024-05-10T03:29:48.781197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.370957+0000","last_change":"2024-05-10T03:29:49.256630+0000","last_active":"2024-05-10T03:29:49.370957+0000","last_peered":"2024-05-10T03:29:49.370957+0000","last_clean":"2024-05-10T03:29:49.370957+0000","last_became_active":"2024-05-10T03:29:49.246648+0000","last_became_peered":"2024-05-10T03:29:49.246648+0000","last_unstale":"2024-05-10T03:29:49.370957+0000","last_undegraded":"2024-05-10T03:29:49.370957+0000","last_fullsized":"2024-05-10T03:29:49.370957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:28.169726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.244841+0000","last_change":"2024-05-10T03:29:44.347464+0000","last_active":"2024-05-10T03:29:49.244841+0000","last_peered":"2024-05-10T03:29:49.244841+0000","last_clean":"2024-05-10T03:29:49.244841+0000","last_became_active":"2024-05-10T03:29:44.347135+0000","last_became_peered":"2024-05-10T03:29:44.347135+0000","last_unstale":"2024-05-10T03:29:49.244841+0000","last_undegraded":"2024-05-10T03:29:49.244841+0000","last_fullsized":"2024-05-10T03:29:49.244841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:06:59.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792883+0000","last_change":"2024-05-10T03:29:43.758164+0000","last_active":"2024-05-10T03:29:48.792883+0000","last_peered":"2024-05-10T03:29:48.792883+0000","last_clean":"2024-05-10T03:29:48.792883+0000","last_became_active":"2024-05-10T03:29:43.757793+0000","last_became_peered":"2024-05-10T03:29:43.757793+0000","last_unstale":"2024-05-10T03:29:48.792883+0000","last_undegraded":"2024-05-10T03:29:48.792883+0000","last_fullsized":"2024-05-10T03:29:48.792883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:39.499394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909165+0000","last_change":"2024-05-10T03:29:48.821419+0000","last_active":"2024-05-10T03:29:49.909165+0000","last_peered":"2024-05-10T03:29:49.909165+0000","last_clean":"2024-05-10T03:29:49.909165+0000","last_became_active":"2024-05-10T03:29:48.820977+0000","last_became_peered":"2024-05-10T03:29:48.820977+0000","last_unstale":"2024-05-10T03:29:49.909165+0000","last_undegraded":"2024-05-10T03:29:49.909165+0000","last_fullsized":"2024-05-10T03:29:49.909165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:17.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241985+0000","last_change":"2024-05-10T03:29:44.349656+0000","last_active":"2024-05-10T03:29:49.241985+0000","last_peered":"2024-05-10T03:29:49.241985+0000","last_clean":"2024-05-10T03:29:49.241985+0000","last_became_active":"2024-05-10T03:29:44.349360+0000","last_became_peered":"2024-05-10T03:29:44.349360+0000","last_unstale":"2024-05-10T03:29:49.241985+0000","last_undegraded":"2024-05-10T03:29:49.241985+0000","last_fullsized":"2024-05-10T03:29:49.241985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:32.828301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.781272+0000","last_change":"2024-05-10T03:29:43.755515+0000","last_active":"2024-05-10T03:29:48.781272+0000","last_peered":"2024-05-10T03:29:48.781272+0000","last_clean":"2024-05-10T03:29:48.781272+0000","last_became_active":"2024-05-10T03:29:43.755202+0000","last_became_peered":"2024-05-10T03:29:43.755202+0000","last_unstale":"2024-05-10T03:29:48.781272+0000","last_undegraded":"2024-05-10T03:29:48.781272+0000","last_fullsized":"2024-05-10T03:29:48.781272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909234+0000","last_change":"2024-05-10T03:29:48.816001+0000","last_active":"2024-05-10T03:29:49.909234+0000","last_peered":"2024-05-10T03:29:49.909234+0000","last_clean":"2024-05-10T03:29:49.909234+0000","last_became_active":"2024-05-10T03:29:48.815291+0000","last_became_peered":"2024-05-10T03:29:48.815291+0000","last_unstale":"2024-05-10T03:29:49.909234+0000","last_undegraded":"2024-05-10T03:29:49.909234+0000","last_fullsized":"2024-05-10T03:29:49.909234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:25.601246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812139+0000","last_change":"2024-05-10T03:29:44.334673+0000","last_active":"2024-05-10T03:29:48.812139+0000","last_peered":"2024-05-10T03:29:48.812139+0000","last_clean":"2024-05-10T03:29:48.812139+0000","last_became_active":"2024-05-10T03:29:44.333855+0000","last_became_peered":"2024-05-10T03:29:44.333855+0000","last_unstale":"2024-05-10T03:29:48.812139+0000","last_undegraded":"2024-05-10T03:29:48.812139+0000","last_fullsized":"2024-05-10T03:29:48.812139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.807723+0000","last_change":"2024-05-10T03:29:43.768430+0000","last_active":"2024-05-10T03:29:48.807723+0000","last_peered":"2024-05-10T03:29:48.807723+0000","last_clean":"2024-05-10T03:29:48.807723+0000","last_became_active":"2024-05-10T03:29:43.767491+0000","last_became_peered":"2024-05-10T03:29:43.767491+0000","last_unstale":"2024-05-10T03:29:48.807723+0000","last_undegraded":"2024-05-10T03:29:48.807723+0000","last_fullsized":"2024-05-10T03:29:48.807723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:23:48.690627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.368290+0000","last_change":"2024-05-10T03:29:49.247514+0000","last_active":"2024-05-10T03:29:49.368290+0000","last_peered":"2024-05-10T03:29:49.368290+0000","last_clean":"2024-05-10T03:29:49.368290+0000","last_became_active":"2024-05-10T03:29:49.246239+0000","last_became_peered":"2024-05-10T03:29:49.246239+0000","last_unstale":"2024-05-10T03:29:49.368290+0000","last_undegraded":"2024-05-10T03:29:49.368290+0000","last_fullsized":"2024-05-10T03:29:49.368290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:20:20.616507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242291+0000","last_change":"2024-05-10T03:29:44.348653+0000","last_active":"2024-05-10T03:29:49.242291+0000","last_peered":"2024-05-10T03:29:49.242291+0000","last_clean":"2024-05-10T03:29:49.242291+0000","last_became_active":"2024-05-10T03:29:44.348301+0000","last_became_peered":"2024-05-10T03:29:44.348301+0000","last_unstale":"2024-05-10T03:29:49.242291+0000","last_undegraded":"2024-05-10T03:29:49.242291+0000","last_fullsized":"2024-05-10T03:29:49.242291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792608+0000","last_change":"2024-05-10T03:29:43.760877+0000","last_active":"2024-05-10T03:29:48.792608+0000","last_peered":"2024-05-10T03:29:48.792608+0000","last_clean":"2024-05-10T03:29:48.792608+0000","last_became_active":"2024-05-10T03:29:43.760572+0000","last_became_peered":"2024-05-10T03:29:43.760572+0000","last_unstale":"2024-05-10T03:29:48.792608+0000","last_undegraded":"2024-05-10T03:29:48.792608+0000","last_fullsized":"2024-05-10T03:29:48.792608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:12.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367784+0000","last_change":"2024-05-10T03:29:49.246283+0000","last_active":"2024-05-10T03:29:49.367784+0000","last_peered":"2024-05-10T03:29:49.367784+0000","last_clean":"2024-05-10T03:29:49.367784+0000","last_became_active":"2024-05-10T03:29:49.243969+0000","last_became_peered":"2024-05-10T03:29:49.243969+0000","last_unstale":"2024-05-10T03:29:49.367784+0000","last_undegraded":"2024-05-10T03:29:49.367784+0000","last_fullsized":"2024-05-10T03:29:49.367784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:50:30.586336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795670+0000","last_change":"2024-05-10T03:29:44.350944+0000","last_active":"2024-05-10T03:29:48.795670+0000","last_peered":"2024-05-10T03:29:48.795670+0000","last_clean":"2024-05-10T03:29:48.795670+0000","last_became_active":"2024-05-10T03:29:44.350411+0000","last_became_peered":"2024-05-10T03:29:44.350411+0000","last_unstale":"2024-05-10T03:29:48.795670+0000","last_undegraded":"2024-05-10T03:29:48.795670+0000","last_fullsized":"2024-05-10T03:29:48.795670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791490+0000","last_change":"2024-05-10T03:29:43.763410+0000","last_active":"2024-05-10T03:29:48.791490+0000","last_peered":"2024-05-10T03:29:48.791490+0000","last_clean":"2024-05-10T03:29:48.791490+0000","last_became_active":"2024-05-10T03:29:43.761799+0000","last_became_peered":"2024-05-10T03:29:43.761799+0000","last_unstale":"2024-05-10T03:29:48.791490+0000","last_undegraded":"2024-05-10T03:29:48.791490+0000","last_fullsized":"2024-05-10T03:29:48.791490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:27.283436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.373121+0000","last_change":"2024-05-10T03:29:49.254044+0000","last_active":"2024-05-10T03:29:49.373121+0000","last_peered":"2024-05-10T03:29:49.373121+0000","last_clean":"2024-05-10T03:29:49.373121+0000","last_became_active":"2024-05-10T03:29:49.252991+0000","last_became_peered":"2024-05-10T03:29:49.252991+0000","last_unstale":"2024-05-10T03:29:49.373121+0000","last_undegraded":"2024-05-10T03:29:49.373121+0000","last_fullsized":"2024-05-10T03:29:49.373121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:18:14.955082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.243191+0000","last_change":"2024-05-10T03:29:44.361247+0000","last_active":"2024-05-10T03:29:49.243191+0000","last_peered":"2024-05-10T03:29:49.243191+0000","last_clean":"2024-05-10T03:29:49.243191+0000","last_became_active":"2024-05-10T03:29:44.360953+0000","last_became_peered":"2024-05-10T03:29:44.360953+0000","last_unstale":"2024-05-10T03:29:49.243191+0000","last_undegraded":"2024-05-10T03:29:49.243191+0000","last_fullsized":"2024-05-10T03:29:49.243191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:06.101428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:30:04.976035+0000","last_change":"2024-05-10T03:29:43.773652+0000","last_active":"2024-05-10T03:30:04.976035+0000","last_peered":"2024-05-10T03:30:04.976035+0000","last_clean":"2024-05-10T03:30:04.976035+0000","last_became_active":"2024-05-10T03:29:43.773313+0000","last_became_peered":"2024-05-10T03:29:43.773313+0000","last_unstale":"2024-05-10T03:30:04.976035+0000","last_undegraded":"2024-05-10T03:30:04.976035+0000","last_fullsized":"2024-05-10T03:30:04.976035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:09.769995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.494687+0000","last_change":"2024-05-10T03:29:48.814023+0000","last_active":"2024-05-10T03:29:49.494687+0000","last_peered":"2024-05-10T03:29:49.494687+0000","last_clean":"2024-05-10T03:29:49.494687+0000","last_became_active":"2024-05-10T03:29:48.813592+0000","last_became_peered":"2024-05-10T03:29:48.813592+0000","last_unstale":"2024-05-10T03:29:49.494687+0000","last_undegraded":"2024-05-10T03:29:49.494687+0000","last_fullsized":"2024-05-10T03:29:49.494687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:23:57.005008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791481+0000","last_change":"2024-05-10T03:29:44.337189+0000","last_active":"2024-05-10T03:29:48.791481+0000","last_peered":"2024-05-10T03:29:48.791481+0000","last_clean":"2024-05-10T03:29:48.791481+0000","last_became_active":"2024-05-10T03:29:44.336858+0000","last_became_peered":"2024-05-10T03:29:44.336858+0000","last_unstale":"2024-05-10T03:29:48.791481+0000","last_undegraded":"2024-05-10T03:29:48.791481+0000","last_fullsized":"2024-05-10T03:29:48.791481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:34.453268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795391+0000","last_change":"2024-05-10T03:29:43.763074+0000","last_active":"2024-05-10T03:29:48.795391+0000","last_peered":"2024-05-10T03:29:48.795391+0000","last_clean":"2024-05-10T03:29:48.795391+0000","last_became_active":"2024-05-10T03:29:43.762746+0000","last_became_peered":"2024-05-10T03:29:43.762746+0000","last_unstale":"2024-05-10T03:29:48.795391+0000","last_undegraded":"2024-05-10T03:29:48.795391+0000","last_fullsized":"2024-05-10T03:29:48.795391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.910002+0000","last_change":"2024-05-10T03:29:48.811643+0000","last_active":"2024-05-10T03:29:49.910002+0000","last_peered":"2024-05-10T03:29:49.910002+0000","last_clean":"2024-05-10T03:29:49.910002+0000","last_became_active":"2024-05-10T03:29:48.811181+0000","last_became_peered":"2024-05-10T03:29:48.811181+0000","last_unstale":"2024-05-10T03:29:49.910002+0000","last_undegraded":"2024-05-10T03:29:49.910002+0000","last_fullsized":"2024-05-10T03:29:49.910002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795416+0000","last_change":"2024-05-10T03:29:44.339071+0000","last_active":"2024-05-10T03:29:48.795416+0000","last_peered":"2024-05-10T03:29:48.795416+0000","last_clean":"2024-05-10T03:29:48.795416+0000","last_became_active":"2024-05-10T03:29:44.338222+0000","last_became_peered":"2024-05-10T03:29:44.338222+0000","last_unstale":"2024-05-10T03:29:48.795416+0000","last_undegraded":"2024-05-10T03:29:48.795416+0000","last_fullsized":"2024-05-10T03:29:48.795416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798392+0000","last_change":"2024-05-10T03:29:43.769611+0000","last_active":"2024-05-10T03:29:48.798392+0000","last_peered":"2024-05-10T03:29:48.798392+0000","last_clean":"2024-05-10T03:29:48.798392+0000","last_became_active":"2024-05-10T03:29:43.769245+0000","last_became_peered":"2024-05-10T03:29:43.769245+0000","last_unstale":"2024-05-10T03:29:48.798392+0000","last_undegraded":"2024-05-10T03:29:48.798392+0000","last_fullsized":"2024-05-10T03:29:48.798392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:07:50.360382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367049+0000","last_change":"2024-05-10T03:29:48.823083+0000","last_active":"2024-05-10T03:29:49.367049+0000","last_peered":"2024-05-10T03:29:49.367049+0000","last_clean":"2024-05-10T03:29:49.367049+0000","last_became_active":"2024-05-10T03:29:48.817306+0000","last_became_peered":"2024-05-10T03:29:48.817306+0000","last_unstale":"2024-05-10T03:29:49.367049+0000","last_undegraded":"2024-05-10T03:29:49.367049+0000","last_fullsized":"2024-05-10T03:29:49.367049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:31:18.655235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.798233+0000","last_change":"2024-05-10T03:29:44.330232+0000","last_active":"2024-05-10T03:29:48.798233+0000","last_peered":"2024-05-10T03:29:48.798233+0000","last_clean":"2024-05-10T03:29:48.798233+0000","last_became_active":"2024-05-10T03:29:44.329857+0000","last_became_peered":"2024-05-10T03:29:44.329857+0000","last_unstale":"2024-05-10T03:29:48.798233+0000","last_undegraded":"2024-05-10T03:29:48.798233+0000","last_fullsized":"2024-05-10T03:29:48.798233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811307+0000","last_change":"2024-05-10T03:29:43.771585+0000","last_active":"2024-05-10T03:29:48.811307+0000","last_peered":"2024-05-10T03:29:48.811307+0000","last_clean":"2024-05-10T03:29:48.811307+0000","last_became_active":"2024-05-10T03:29:43.771253+0000","last_became_peered":"2024-05-10T03:29:43.771253+0000","last_unstale":"2024-05-10T03:29:48.811307+0000","last_undegraded":"2024-05-10T03:29:48.811307+0000","last_fullsized":"2024-05-10T03:29:48.811307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:58.932988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.366899+0000","last_change":"2024-05-10T03:29:49.245635+0000","last_active":"2024-05-10T03:29:49.366899+0000","last_peered":"2024-05-10T03:29:49.366899+0000","last_clean":"2024-05-10T03:29:49.366899+0000","last_became_active":"2024-05-10T03:29:49.245125+0000","last_became_peered":"2024-05-10T03:29:49.245125+0000","last_unstale":"2024-05-10T03:29:49.366899+0000","last_undegraded":"2024-05-10T03:29:49.366899+0000","last_fullsized":"2024-05-10T03:29:49.366899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:05:36.375913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.909603+0000","last_change":"2024-05-10T03:29:49.253042+0000","last_active":"2024-05-10T03:29:49.909603+0000","last_peered":"2024-05-10T03:29:49.909603+0000","last_clean":"2024-05-10T03:29:49.909603+0000","last_became_active":"2024-05-10T03:29:49.252051+0000","last_became_peered":"2024-05-10T03:29:49.252051+0000","last_unstale":"2024-05-10T03:29:49.909603+0000","last_undegraded":"2024-05-10T03:29:49.909603+0000","last_fullsized":"2024-05-10T03:29:49.909603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:57:00.757246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794396+0000","last_change":"2024-05-10T03:29:44.346922+0000","last_active":"2024-05-10T03:29:48.794396+0000","last_peered":"2024-05-10T03:29:48.794396+0000","last_clean":"2024-05-10T03:29:48.794396+0000","last_became_active":"2024-05-10T03:29:44.346260+0000","last_became_peered":"2024-05-10T03:29:44.346260+0000","last_unstale":"2024-05-10T03:29:48.794396+0000","last_undegraded":"2024-05-10T03:29:48.794396+0000","last_fullsized":"2024-05-10T03:29:48.794396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786593+0000","last_change":"2024-05-10T03:29:43.761072+0000","last_active":"2024-05-10T03:29:48.786593+0000","last_peered":"2024-05-10T03:29:48.786593+0000","last_clean":"2024-05-10T03:29:48.786593+0000","last_became_active":"2024-05-10T03:29:43.760425+0000","last_became_peered":"2024-05-10T03:29:43.760425+0000","last_unstale":"2024-05-10T03:29:48.786593+0000","last_undegraded":"2024-05-10T03:29:48.786593+0000","last_fullsized":"2024-05-10T03:29:48.786593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.375073+0000","last_change":"2024-05-10T03:29:48.818277+0000","last_active":"2024-05-10T03:29:49.375073+0000","last_peered":"2024-05-10T03:29:49.375073+0000","last_clean":"2024-05-10T03:29:49.375073+0000","last_became_active":"2024-05-10T03:29:48.817666+0000","last_became_peered":"2024-05-10T03:29:48.817666+0000","last_unstale":"2024-05-10T03:29:49.375073+0000","last_undegraded":"2024-05-10T03:29:49.375073+0000","last_fullsized":"2024-05-10T03:29:49.375073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:53:12.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808591+0000","last_change":"2024-05-10T03:29:44.356141+0000","last_active":"2024-05-10T03:29:48.808591+0000","last_peered":"2024-05-10T03:29:48.808591+0000","last_clean":"2024-05-10T03:29:48.808591+0000","last_became_active":"2024-05-10T03:29:44.355691+0000","last_became_peered":"2024-05-10T03:29:44.355691+0000","last_unstale":"2024-05-10T03:29:48.808591+0000","last_undegraded":"2024-05-10T03:29:48.808591+0000","last_fullsized":"2024-05-10T03:29:48.808591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801357+0000","last_change":"2024-05-10T03:29:43.767957+0000","last_active":"2024-05-10T03:29:48.801357+0000","last_peered":"2024-05-10T03:29:48.801357+0000","last_clean":"2024-05-10T03:29:48.801357+0000","last_became_active":"2024-05-10T03:29:43.767083+0000","last_became_peered":"2024-05-10T03:29:43.767083+0000","last_unstale":"2024-05-10T03:29:48.801357+0000","last_undegraded":"2024-05-10T03:29:48.801357+0000","last_fullsized":"2024-05-10T03:29:48.801357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:40:33.006557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.809728+0000","last_change":"2024-05-10T03:29:43.767890+0000","last_active":"2024-05-10T03:29:48.809728+0000","last_peered":"2024-05-10T03:29:48.809728+0000","last_clean":"2024-05-10T03:29:48.809728+0000","last_became_active":"2024-05-10T03:29:43.767550+0000","last_became_peered":"2024-05-10T03:29:43.767550+0000","last_unstale":"2024-05-10T03:29:48.809728+0000","last_undegraded":"2024-05-10T03:29:48.809728+0000","last_fullsized":"2024-05-10T03:29:48.809728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:15.035052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792435+0000","last_change":"2024-05-10T03:29:44.346806+0000","last_active":"2024-05-10T03:29:48.792435+0000","last_peered":"2024-05-10T03:29:48.792435+0000","last_clean":"2024-05-10T03:29:48.792435+0000","last_became_active":"2024-05-10T03:29:44.346484+0000","last_became_peered":"2024-05-10T03:29:44.346484+0000","last_unstale":"2024-05-10T03:29:48.792435+0000","last_undegraded":"2024-05-10T03:29:48.792435+0000","last_fullsized":"2024-05-10T03:29:48.792435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:50.377829+0000","last_change":"2024-05-10T03:29:48.818619+0000","last_active":"2024-05-10T03:29:50.377829+0000","last_peered":"2024-05-10T03:29:50.377829+0000","last_clean":"2024-05-10T03:29:50.377829+0000","last_became_active":"2024-05-10T03:29:48.818125+0000","last_became_peered":"2024-05-10T03:29:48.818125+0000","last_unstale":"2024-05-10T03:29:50.377829+0000","last_undegraded":"2024-05-10T03:29:50.377829+0000","last_fullsized":"2024-05-10T03:29:50.377829+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:58.075131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794286+0000","last_change":"2024-05-10T03:29:43.771030+0000","last_active":"2024-05-10T03:29:48.794286+0000","last_peered":"2024-05-10T03:29:48.794286+0000","last_clean":"2024-05-10T03:29:48.794286+0000","last_became_active":"2024-05-10T03:29:43.770630+0000","last_became_peered":"2024-05-10T03:29:43.770630+0000","last_unstale":"2024-05-10T03:29:48.794286+0000","last_undegraded":"2024-05-10T03:29:48.794286+0000","last_fullsized":"2024-05-10T03:29:48.794286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:42.695168+0000","last_clean_scrub_stamp":"2024-05-10T03:29:42.695168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:02.928099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791595+0000","last_change":"2024-05-10T03:29:44.330198+0000","last_active":"2024-05-10T03:29:48.791595+0000","last_peered":"2024-05-10T03:29:48.791595+0000","last_clean":"2024-05-10T03:29:48.791595+0000","last_became_active":"2024-05-10T03:29:44.329871+0000","last_became_peered":"2024-05-10T03:29:44.329871+0000","last_unstale":"2024-05-10T03:29:48.791595+0000","last_undegraded":"2024-05-10T03:29:48.791595+0000","last_fullsized":"2024-05-10T03:29:48.791595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.367463+0000","last_change":"2024-05-10T03:29:48.822151+0000","last_active":"2024-05-10T03:29:49.367463+0000","last_peered":"2024-05-10T03:29:49.367463+0000","last_clean":"2024-05-10T03:29:49.367463+0000","last_became_active":"2024-05-10T03:29:48.819369+0000","last_became_peered":"2024-05-10T03:29:48.819369+0000","last_unstale":"2024-05-10T03:29:49.367463+0000","last_undegraded":"2024-05-10T03:29:49.367463+0000","last_fullsized":"2024-05-10T03:29:49.367463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:47.748057+0000","last_clean_scrub_stamp":"2024-05-10T03:29:47.748057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:58.725548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808987+0000","last_change":"2024-05-10T03:29:44.334873+0000","last_active":"2024-05-10T03:29:48.808987+0000","last_peered":"2024-05-10T03:29:48.808987+0000","last_clean":"2024-05-10T03:29:48.808987+0000","last_became_active":"2024-05-10T03:29:44.334365+0000","last_became_peered":"2024-05-10T03:29:44.334365+0000","last_unstale":"2024-05-10T03:29:48.808987+0000","last_undegraded":"2024-05-10T03:29:48.808987+0000","last_fullsized":"2024-05-10T03:29:48.808987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780300+0000","last_change":"2024-05-10T03:29:44.333950+0000","last_active":"2024-05-10T03:29:48.780300+0000","last_peered":"2024-05-10T03:29:48.780300+0000","last_clean":"2024-05-10T03:29:48.780300+0000","last_became_active":"2024-05-10T03:29:44.331498+0000","last_became_peered":"2024-05-10T03:29:44.331498+0000","last_unstale":"2024-05-10T03:29:48.780300+0000","last_undegraded":"2024-05-10T03:29:48.780300+0000","last_fullsized":"2024-05-10T03:29:48.780300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:52:59.948930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792455+0000","last_change":"2024-05-10T03:29:44.341676+0000","last_active":"2024-05-10T03:29:48.792455+0000","last_peered":"2024-05-10T03:29:48.792455+0000","last_clean":"2024-05-10T03:29:48.792455+0000","last_became_active":"2024-05-10T03:29:44.341296+0000","last_became_peered":"2024-05-10T03:29:44.341296+0000","last_unstale":"2024-05-10T03:29:48.792455+0000","last_undegraded":"2024-05-10T03:29:48.792455+0000","last_fullsized":"2024-05-10T03:29:48.792455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791287+0000","last_change":"2024-05-10T03:29:44.349694+0000","last_active":"2024-05-10T03:29:48.791287+0000","last_peered":"2024-05-10T03:29:48.791287+0000","last_clean":"2024-05-10T03:29:48.791287+0000","last_became_active":"2024-05-10T03:29:44.349396+0000","last_became_peered":"2024-05-10T03:29:44.349396+0000","last_unstale":"2024-05-10T03:29:48.791287+0000","last_undegraded":"2024-05-10T03:29:48.791287+0000","last_fullsized":"2024-05-10T03:29:48.791287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808295+0000","last_change":"2024-05-10T03:29:44.359086+0000","last_active":"2024-05-10T03:29:48.808295+0000","last_peered":"2024-05-10T03:29:48.808295+0000","last_clean":"2024-05-10T03:29:48.808295+0000","last_became_active":"2024-05-10T03:29:44.358776+0000","last_became_peered":"2024-05-10T03:29:44.358776+0000","last_unstale":"2024-05-10T03:29:48.808295+0000","last_undegraded":"2024-05-10T03:29:48.808295+0000","last_fullsized":"2024-05-10T03:29:48.808295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:13:18.867110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794937+0000","last_change":"2024-05-10T03:29:44.352293+0000","last_active":"2024-05-10T03:29:48.794937+0000","last_peered":"2024-05-10T03:29:48.794937+0000","last_clean":"2024-05-10T03:29:48.794937+0000","last_became_active":"2024-05-10T03:29:44.352017+0000","last_became_peered":"2024-05-10T03:29:44.352017+0000","last_unstale":"2024-05-10T03:29:48.794937+0000","last_undegraded":"2024-05-10T03:29:48.794937+0000","last_fullsized":"2024-05-10T03:29:48.794937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812361+0000","last_change":"2024-05-10T03:29:44.339931+0000","last_active":"2024-05-10T03:29:48.812361+0000","last_peered":"2024-05-10T03:29:48.812361+0000","last_clean":"2024-05-10T03:29:48.812361+0000","last_became_active":"2024-05-10T03:29:44.339638+0000","last_became_peered":"2024-05-10T03:29:44.339638+0000","last_unstale":"2024-05-10T03:29:48.812361+0000","last_undegraded":"2024-05-10T03:29:48.812361+0000","last_fullsized":"2024-05-10T03:29:48.812361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:16.401578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.815879+0000","last_change":"2024-05-10T03:29:44.343280+0000","last_active":"2024-05-10T03:29:48.815879+0000","last_peered":"2024-05-10T03:29:48.815879+0000","last_clean":"2024-05-10T03:29:48.815879+0000","last_became_active":"2024-05-10T03:29:44.342959+0000","last_became_peered":"2024-05-10T03:29:44.342959+0000","last_unstale":"2024-05-10T03:29:48.815879+0000","last_undegraded":"2024-05-10T03:29:48.815879+0000","last_fullsized":"2024-05-10T03:29:48.815879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.786197+0000","last_change":"2024-05-10T03:29:44.351381+0000","last_active":"2024-05-10T03:29:48.786197+0000","last_peered":"2024-05-10T03:29:48.786197+0000","last_clean":"2024-05-10T03:29:48.786197+0000","last_became_active":"2024-05-10T03:29:44.351019+0000","last_became_peered":"2024-05-10T03:29:44.351019+0000","last_unstale":"2024-05-10T03:29:48.786197+0000","last_undegraded":"2024-05-10T03:29:48.786197+0000","last_fullsized":"2024-05-10T03:29:48.786197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:13:23.913088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.801184+0000","last_change":"2024-05-10T03:29:44.344852+0000","last_active":"2024-05-10T03:29:48.801184+0000","last_peered":"2024-05-10T03:29:48.801184+0000","last_clean":"2024-05-10T03:29:48.801184+0000","last_became_active":"2024-05-10T03:29:44.344659+0000","last_became_peered":"2024-05-10T03:29:44.344659+0000","last_unstale":"2024-05-10T03:29:48.801184+0000","last_undegraded":"2024-05-10T03:29:48.801184+0000","last_fullsized":"2024-05-10T03:29:48.801184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792296+0000","last_change":"2024-05-10T03:29:44.335097+0000","last_active":"2024-05-10T03:29:48.792296+0000","last_peered":"2024-05-10T03:29:48.792296+0000","last_clean":"2024-05-10T03:29:48.792296+0000","last_became_active":"2024-05-10T03:29:44.332994+0000","last_became_peered":"2024-05-10T03:29:44.332994+0000","last_unstale":"2024-05-10T03:29:48.792296+0000","last_undegraded":"2024-05-10T03:29:48.792296+0000","last_fullsized":"2024-05-10T03:29:48.792296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:40:45.868497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.788203+0000","last_change":"2024-05-10T03:29:44.346157+0000","last_active":"2024-05-10T03:29:48.788203+0000","last_peered":"2024-05-10T03:29:48.788203+0000","last_clean":"2024-05-10T03:29:48.788203+0000","last_became_active":"2024-05-10T03:29:44.345819+0000","last_became_peered":"2024-05-10T03:29:44.345819+0000","last_unstale":"2024-05-10T03:29:48.788203+0000","last_undegraded":"2024-05-10T03:29:48.788203+0000","last_fullsized":"2024-05-10T03:29:48.788203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.813266+0000","last_change":"2024-05-10T03:29:44.351136+0000","last_active":"2024-05-10T03:29:48.813266+0000","last_peered":"2024-05-10T03:29:48.813266+0000","last_clean":"2024-05-10T03:29:48.813266+0000","last_became_active":"2024-05-10T03:29:44.347556+0000","last_became_peered":"2024-05-10T03:29:44.347556+0000","last_unstale":"2024-05-10T03:29:48.813266+0000","last_undegraded":"2024-05-10T03:29:48.813266+0000","last_fullsized":"2024-05-10T03:29:48.813266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:17:46.453723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.247953+0000","last_change":"2024-05-10T03:29:44.354749+0000","last_active":"2024-05-10T03:29:49.247953+0000","last_peered":"2024-05-10T03:29:49.247953+0000","last_clean":"2024-05-10T03:29:49.247953+0000","last_became_active":"2024-05-10T03:29:44.354474+0000","last_became_peered":"2024-05-10T03:29:44.354474+0000","last_unstale":"2024-05-10T03:29:49.247953+0000","last_undegraded":"2024-05-10T03:29:49.247953+0000","last_fullsized":"2024-05-10T03:29:49.247953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:27:48.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.780909+0000","last_change":"2024-05-10T03:29:44.339173+0000","last_active":"2024-05-10T03:29:48.780909+0000","last_peered":"2024-05-10T03:29:48.780909+0000","last_clean":"2024-05-10T03:29:48.780909+0000","last_became_active":"2024-05-10T03:29:44.338815+0000","last_became_peered":"2024-05-10T03:29:44.338815+0000","last_unstale":"2024-05-10T03:29:48.780909+0000","last_undegraded":"2024-05-10T03:29:48.780909+0000","last_fullsized":"2024-05-10T03:29:48.780909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.799655+0000","last_change":"2024-05-10T03:29:44.334824+0000","last_active":"2024-05-10T03:29:48.799655+0000","last_peered":"2024-05-10T03:29:48.799655+0000","last_clean":"2024-05-10T03:29:48.799655+0000","last_became_active":"2024-05-10T03:29:44.333616+0000","last_became_peered":"2024-05-10T03:29:44.333616+0000","last_unstale":"2024-05-10T03:29:48.799655+0000","last_undegraded":"2024-05-10T03:29:48.799655+0000","last_fullsized":"2024-05-10T03:29:48.799655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:38.441292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.785927+0000","last_change":"2024-05-10T03:29:44.339089+0000","last_active":"2024-05-10T03:29:48.785927+0000","last_peered":"2024-05-10T03:29:48.785927+0000","last_clean":"2024-05-10T03:29:48.785927+0000","last_became_active":"2024-05-10T03:29:44.336113+0000","last_became_peered":"2024-05-10T03:29:44.336113+0000","last_unstale":"2024-05-10T03:29:48.785927+0000","last_undegraded":"2024-05-10T03:29:48.785927+0000","last_fullsized":"2024-05-10T03:29:48.785927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794173+0000","last_change":"2024-05-10T03:29:44.340483+0000","last_active":"2024-05-10T03:29:48.794173+0000","last_peered":"2024-05-10T03:29:48.794173+0000","last_clean":"2024-05-10T03:29:48.794173+0000","last_became_active":"2024-05-10T03:29:44.340093+0000","last_became_peered":"2024-05-10T03:29:44.340093+0000","last_unstale":"2024-05-10T03:29:48.794173+0000","last_undegraded":"2024-05-10T03:29:48.794173+0000","last_fullsized":"2024-05-10T03:29:48.794173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:04:23.301866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.790438+0000","last_change":"2024-05-10T03:29:44.345377+0000","last_active":"2024-05-10T03:29:48.790438+0000","last_peered":"2024-05-10T03:29:48.790438+0000","last_clean":"2024-05-10T03:29:48.790438+0000","last_became_active":"2024-05-10T03:29:44.345048+0000","last_became_peered":"2024-05-10T03:29:44.345048+0000","last_unstale":"2024-05-10T03:29:48.790438+0000","last_undegraded":"2024-05-10T03:29:48.790438+0000","last_fullsized":"2024-05-10T03:29:48.790438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:36.993303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.794132+0000","last_change":"2024-05-10T03:29:44.349235+0000","last_active":"2024-05-10T03:29:48.794132+0000","last_peered":"2024-05-10T03:29:48.794132+0000","last_clean":"2024-05-10T03:29:48.794132+0000","last_became_active":"2024-05-10T03:29:44.348922+0000","last_became_peered":"2024-05-10T03:29:44.348922+0000","last_unstale":"2024-05-10T03:29:48.794132+0000","last_undegraded":"2024-05-10T03:29:48.794132+0000","last_fullsized":"2024-05-10T03:29:48.794132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.791228+0000","last_change":"2024-05-10T03:29:44.342929+0000","last_active":"2024-05-10T03:29:48.791228+0000","last_peered":"2024-05-10T03:29:48.791228+0000","last_clean":"2024-05-10T03:29:48.791228+0000","last_became_active":"2024-05-10T03:29:44.342505+0000","last_became_peered":"2024-05-10T03:29:44.342505+0000","last_unstale":"2024-05-10T03:29:48.791228+0000","last_undegraded":"2024-05-10T03:29:48.791228+0000","last_fullsized":"2024-05-10T03:29:48.791228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:13.234710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.811605+0000","last_change":"2024-05-10T03:29:44.338969+0000","last_active":"2024-05-10T03:29:48.811605+0000","last_peered":"2024-05-10T03:29:48.811605+0000","last_clean":"2024-05-10T03:29:48.811605+0000","last_became_active":"2024-05-10T03:29:44.337703+0000","last_became_peered":"2024-05-10T03:29:44.337703+0000","last_unstale":"2024-05-10T03:29:48.811605+0000","last_undegraded":"2024-05-10T03:29:48.811605+0000","last_fullsized":"2024-05-10T03:29:48.811605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:13.652539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.812225+0000","last_change":"2024-05-10T03:29:44.348810+0000","last_active":"2024-05-10T03:29:48.812225+0000","last_peered":"2024-05-10T03:29:48.812225+0000","last_clean":"2024-05-10T03:29:48.812225+0000","last_became_active":"2024-05-10T03:29:44.348512+0000","last_became_peered":"2024-05-10T03:29:44.348512+0000","last_unstale":"2024-05-10T03:29:48.812225+0000","last_undegraded":"2024-05-10T03:29:48.812225+0000","last_fullsized":"2024-05-10T03:29:48.812225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:30:57.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249372+0000","last_change":"2024-05-10T03:29:44.344167+0000","last_active":"2024-05-10T03:29:49.249372+0000","last_peered":"2024-05-10T03:29:49.249372+0000","last_clean":"2024-05-10T03:29:49.249372+0000","last_became_active":"2024-05-10T03:29:44.343405+0000","last_became_peered":"2024-05-10T03:29:44.343405+0000","last_unstale":"2024-05-10T03:29:49.249372+0000","last_undegraded":"2024-05-10T03:29:49.249372+0000","last_fullsized":"2024-05-10T03:29:49.249372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.808451+0000","last_change":"2024-05-10T03:29:44.337141+0000","last_active":"2024-05-10T03:29:48.808451+0000","last_peered":"2024-05-10T03:29:48.808451+0000","last_clean":"2024-05-10T03:29:48.808451+0000","last_became_active":"2024-05-10T03:29:44.336801+0000","last_became_peered":"2024-05-10T03:29:44.336801+0000","last_unstale":"2024-05-10T03:29:48.808451+0000","last_undegraded":"2024-05-10T03:29:48.808451+0000","last_fullsized":"2024-05-10T03:29:48.808451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:51.917316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.242013+0000","last_change":"2024-05-10T03:29:44.339252+0000","last_active":"2024-05-10T03:29:49.242013+0000","last_peered":"2024-05-10T03:29:49.242013+0000","last_clean":"2024-05-10T03:29:49.242013+0000","last_became_active":"2024-05-10T03:29:44.338870+0000","last_became_peered":"2024-05-10T03:29:44.338870+0000","last_unstale":"2024-05-10T03:29:49.242013+0000","last_undegraded":"2024-05-10T03:29:49.242013+0000","last_fullsized":"2024-05-10T03:29:49.242013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.792099+0000","last_change":"2024-05-10T03:29:44.341080+0000","last_active":"2024-05-10T03:29:48.792099+0000","last_peered":"2024-05-10T03:29:48.792099+0000","last_clean":"2024-05-10T03:29:48.792099+0000","last_became_active":"2024-05-10T03:29:44.340720+0000","last_became_peered":"2024-05-10T03:29:44.340720+0000","last_unstale":"2024-05-10T03:29:48.792099+0000","last_undegraded":"2024-05-10T03:29:48.792099+0000","last_fullsized":"2024-05-10T03:29:48.792099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:11.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.241944+0000","last_change":"2024-05-10T03:29:44.345293+0000","last_active":"2024-05-10T03:29:49.241944+0000","last_peered":"2024-05-10T03:29:49.241944+0000","last_clean":"2024-05-10T03:29:49.241944+0000","last_became_active":"2024-05-10T03:29:44.344092+0000","last_became_peered":"2024-05-10T03:29:44.344092+0000","last_unstale":"2024-05-10T03:29:49.241944+0000","last_undegraded":"2024-05-10T03:29:49.241944+0000","last_fullsized":"2024-05-10T03:29:49.241944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:33:58.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.797675+0000","last_change":"2024-05-10T03:29:44.359220+0000","last_active":"2024-05-10T03:29:48.797675+0000","last_peered":"2024-05-10T03:29:48.797675+0000","last_clean":"2024-05-10T03:29:48.797675+0000","last_became_active":"2024-05-10T03:29:44.358849+0000","last_became_peered":"2024-05-10T03:29:44.358849+0000","last_unstale":"2024-05-10T03:29:48.797675+0000","last_undegraded":"2024-05-10T03:29:48.797675+0000","last_fullsized":"2024-05-10T03:29:48.797675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:54.507386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795357+0000","last_change":"2024-05-10T03:29:44.349124+0000","last_active":"2024-05-10T03:29:48.795357+0000","last_peered":"2024-05-10T03:29:48.795357+0000","last_clean":"2024-05-10T03:29:48.795357+0000","last_became_active":"2024-05-10T03:29:44.348841+0000","last_became_peered":"2024-05-10T03:29:44.348841+0000","last_unstale":"2024-05-10T03:29:48.795357+0000","last_undegraded":"2024-05-10T03:29:48.795357+0000","last_fullsized":"2024-05-10T03:29:48.795357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:25:59.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:48.795763+0000","last_change":"2024-05-10T03:29:44.342192+0000","last_active":"2024-05-10T03:29:48.795763+0000","last_peered":"2024-05-10T03:29:48.795763+0000","last_clean":"2024-05-10T03:29:48.795763+0000","last_became_active":"2024-05-10T03:29:44.341267+0000","last_became_peered":"2024-05-10T03:29:44.341267+0000","last_unstale":"2024-05-10T03:29:48.795763+0000","last_undegraded":"2024-05-10T03:29:48.795763+0000","last_fullsized":"2024-05-10T03:29:48.795763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:54.038184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-10T03:29:49.249243+0000","last_change":"2024-05-10T03:29:44.344420+0000","last_active":"2024-05-10T03:29:49.249243+0000","last_peered":"2024-05-10T03:29:49.249243+0000","last_clean":"2024-05-10T03:29:49.249243+0000","last_became_active":"2024-05-10T03:29:44.343841+0000","last_became_peered":"2024-05-10T03:29:44.343841+0000","last_unstale":"2024-05-10T03:29:49.249243+0000","last_undegraded":"2024-05-10T03:29:49.249243+0000","last_fullsized":"2024-05-10T03:29:49.249243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:43.711110+0000","last_clean_scrub_stamp":"2024-05-10T03:29:43.711110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:23:08.501356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.133}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":108247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.833}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.53}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9730000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.575}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.518}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3650000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.149}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:30:14.923 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T03:30:14.923 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T03:30:14.923 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-10T03:30:14.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-10T03:30:15.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.227+0000 7fce3a200640 1 Processor -- start 2024-05-10T03:30:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.227+0000 7fce3a200640 1 -- start start 2024-05-10T03:30:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.229+0000 7fce3a200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce34071db0 0x7fce34158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce3a200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fce340b3860 0x7fce340b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce3a200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce340708b0 con 0x7fce3406a4b0 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce3a200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce340b2490 con 0x7fce340b3860 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce3a200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce34070050 con 0x7fce34071db0 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fce340b3860 0x7fce340b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce32a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fce3406a4b0 0x7fce340ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38044/0 (socket says 172.21.15.55:38044) 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce32a00640 1 -- 172.21.15.55:0/2680831374 learned_addr learned my addr 172.21.15.55:0/2680831374 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33400640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce34071db0 0x7fce34158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33e00640 1 -- 172.21.15.55:0/2680831374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce34071db0 msgr2=0x7fce34158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33e00640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce34071db0 0x7fce34158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33e00640 1 -- 172.21.15.55:0/2680831374 >> v1:172.21.15.55:6789/0 conn(0x7fce3406a4b0 legacy=0x7fce340ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.230+0000 7fce33e00640 1 -- 172.21.15.55:0/2680831374 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3415b940 con 0x7fce340b3860 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.232+0000 7fce33e00640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce340b3c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009d90 tx=0x7fce2c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae14c430f27b82f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.232+0000 7fce32000640 1 -- 172.21.15.55:0/2680831374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c031600 con 0x7fce340b3860 2024-05-10T03:30:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.232+0000 7fce32000640 1 -- 172.21.15.55:0/2680831374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c02d050 con 0x7fce340b3860 2024-05-10T03:30:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.232+0000 7fce32000640 1 -- 172.21.15.55:0/2680831374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c031980 con 0x7fce340b3860 2024-05-10T03:30:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 -- 172.21.15.55:0/2680831374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 msgr2=0x7fce340b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce340b3c70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009d90 tx=0x7fce2c02a5d0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 -- 172.21.15.55:0/2680831374 shutdown_connections 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce340b3c70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 --2- 172.21.15.55:0/2680831374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce34071db0 0x7fce34158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 -- 172.21.15.55:0/2680831374 >> 172.21.15.55:0/2680831374 conn(0x7fce340cbb90 msgr2=0x7fce340cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 -- 172.21.15.55:0/2680831374 shutdown_connections 2024-05-10T03:30:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.233+0000 7fce3a200640 1 -- 172.21.15.55:0/2680831374 wait complete. 2024-05-10T03:30:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.236+0000 7fce3a200640 1 Processor -- start 2024-05-10T03:30:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.236+0000 7fce3a200640 1 -- start start 2024-05-10T03:30:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.238+0000 7fce3a200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce3406a4b0 0x7fce340a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.238+0000 7fce3a200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 0x7fce34156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.238+0000 7fce3a200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce3a200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34131f90 con 0x7fce340b3860 2024-05-10T03:30:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce3a200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3413f560 con 0x7fce34071db0 2024-05-10T03:30:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce3a200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3413f7b0 con 0x7fce3406a4b0 2024-05-10T03:30:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33396/0 (socket says 172.21.15.55:33396) 2024-05-10T03:30:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 -- 172.21.15.55:0/74216521 learned_addr learned my addr 172.21.15.55:0/74216521 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33400640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce3406a4b0 0x7fce340a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce32a00640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 0x7fce34156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 -- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce3406a4b0 msgr2=0x7fce340a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce3406a4b0 0x7fce340a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 -- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 msgr2=0x7fce34156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 0x7fce34156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce33e00640 1 -- 172.21.15.55:0/74216521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c009a70 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.239+0000 7fce32a00640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 0x7fce34156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce33e00640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009eb0 tx=0x7fce2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c03b820 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c03b980 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3413fa90 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c03bcf0 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.240+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34140050 con 0x7fce340b3860 2024-05-10T03:30:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.242+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34066f60 con 0x7fce340b3860 2024-05-10T03:30:15.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.243+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fce2c04d3f0 con 0x7fce340b3860 2024-05-10T03:30:15.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.245+0000 7fce2b400640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 0x7fce1006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.245+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce2c0aae20 con 0x7fce340b3860 2024-05-10T03:30:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.246+0000 7fce33400640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 0x7fce1006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.247+0000 7fce33400640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 0x7fce1006d140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce24006fd0 tx=0x7fce24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.251+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fce2c075c70 con 0x7fce340b3860 2024-05-10T03:30:15.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.488+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fce34067220 con 0x7fce340b3860 2024-05-10T03:30:15.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.489+0000 7fce2b400640 1 -- 172.21.15.55:0/74216521 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fce2c079290 con 0x7fce340b3860 2024-05-10T03:30:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:15.494 INFO:teuthology.orchestra.run.smithi055.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 msgr2=0x7fce1006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 0x7fce1006d140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce24006fd0 tx=0x7fce24008040 comp rx=0 tx=0).stop 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 msgr2=0x7fce34141150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009eb0 tx=0x7fce2c004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 shutdown_connections 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce340b3860 0x7fce34141150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce1006ac30 0x7fce1006d140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce34071db0 0x7fce34156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 --2- 172.21.15.55:0/74216521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce3406a4b0 0x7fce340a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.491+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 >> 172.21.15.55:0/74216521 conn(0x7fce340cbb90 msgr2=0x7fce34159950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:15.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.492+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 shutdown_connections 2024-05-10T03:30:15.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.492+0000 7fce3a200640 1 -- 172.21.15.55:0/74216521 wait complete. 2024-05-10T03:30:15.524 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-10T03:30:15.524 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.801+0000 7f90f5200640 1 Processor -- start 2024-05-10T03:30:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.801+0000 7f90f5200640 1 -- start start 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90f5200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f90f0144710 0x7f90f0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90f5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 0x7f90f00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90f5200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f006d350 con 0x7f90f00b2fc0 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90f5200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f006c740 con 0x7f90f0144710 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90f5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f006ca20 con 0x7f90f006d7e0 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90efe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f90f00b2fc0 0x7f90f015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38054/0 (socket says 172.21.15.55:38054) 2024-05-10T03:30:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.803+0000 7f90efe00640 1 -- 172.21.15.55:0/3786883858 learned_addr learned my addr 172.21.15.55:0/3786883858 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:15.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90ef400640 1 --2- 172.21.15.55:0/3786883858 >> v2:172.21.15.162:3300/0 conn(0x7f90f0144710 0x7f90f0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90e7e00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 0x7f90f00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013467617 0 0) 0x7f90f006d350 con 0x7f90f00b2fc0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d4003520 con 0x7f90f00b2fc0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1688881120 0 0) 0x7f90d4003520 con 0x7f90f00b2fc0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 msgr2=0x7f90f00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 0x7f90f00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 >> v2:172.21.15.162:3300/0 conn(0x7f90f0144710 msgr2=0x7f90f0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 --2- 172.21.15.55:0/3786883858 >> v2:172.21.15.162:3300/0 conn(0x7f90f0144710 0x7f90f0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f006cd00 con 0x7f90f00b2fc0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f90e80019f0 con 0x7f90f00b2fc0 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 >> v1:172.21.15.55:6789/0 conn(0x7f90f00b2fc0 legacy=0x7f90f015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.804+0000 7f90eea00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90d4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90eea00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90d40047c0 0x7f90d4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90eea00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 0x7f90d4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f006c740 con 0x7f90d40039e0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f006ca20 con 0x7f90d40047c0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d400ba80 con 0x7f90f006d7e0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90d4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 -- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 msgr2=0x7f90d4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f006d7e0 0x7f90d4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90e7e00640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90d40047c0 0x7f90d4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 -- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90d40047c0 msgr2=0x7f90d4004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90d40047c0 0x7f90d4004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.805+0000 7f90ef400640 1 -- 172.21.15.55:0/3786883858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4003520 con 0x7f90d40039e0 2024-05-10T03:30:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.806+0000 7f90ef400640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90d4003e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009a90 tx=0x7f90d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615b4747d9dddd46 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0031400 con 0x7f90d40039e0 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e002d050 con 0x7f90d40039e0 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90eea00640 1 -- 172.21.15.55:0/3786883858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00317a0 con 0x7f90d40039e0 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 -- 172.21.15.55:0/3786883858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 msgr2=0x7f90d4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 --2- 172.21.15.55:0/3786883858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90d4003e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009a90 tx=0x7f90d40057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90efe00640 1 -- 172.21.15.55:0/3786883858 reap_dead start 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 -- 172.21.15.55:0/3786883858 shutdown_connections 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 -- 172.21.15.55:0/3786883858 >> 172.21.15.55:0/3786883858 conn(0x7f90f006a110 msgr2=0x7f90f006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 -- 172.21.15.55:0/3786883858 shutdown_connections 2024-05-10T03:30:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.807+0000 7f90f5200640 1 -- 172.21.15.55:0/3786883858 wait complete. 2024-05-10T03:30:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.810+0000 7f90f5200640 1 Processor -- start 2024-05-10T03:30:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.810+0000 7f90f5200640 1 -- start start 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90f0140050 0x7f90f0140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f01409a0 0x7f90f015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015c620 con 0x7f90d40039e0 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f013cb50 con 0x7f90f0140050 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90f5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015dd80 con 0x7f90f01409a0 2024-05-10T03:30:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.811+0000 7f90ef400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33424/0 (socket says 172.21.15.55:33424) 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 -- 172.21.15.55:0/655370407 learned_addr learned my addr 172.21.15.55:0/655370407 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90efe00640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f01409a0 0x7f90f015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 -- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f01409a0 msgr2=0x7f90f015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f01409a0 0x7f90f015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 -- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90f0140050 msgr2=0x7f90f0140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90f0140050 0x7f90f0140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 -- 172.21.15.55:0/655370407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f015e820 con 0x7f90d40039e0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.812+0000 7f90ef400640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009a90 tx=0x7f90e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.813+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e003a810 con 0x7f90d40039e0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.813+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e003a970 con 0x7f90d40039e0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.813+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e003ad20 con 0x7f90d40039e0 2024-05-10T03:30:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.813+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f015bc80 con 0x7f90d40039e0 2024-05-10T03:30:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.813+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f015c240 con 0x7f90d40039e0 2024-05-10T03:30:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.814+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f015c890 con 0x7f90d40039e0 2024-05-10T03:30:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.815+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f90e0038040 con 0x7f90d40039e0 2024-05-10T03:30:15.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.816+0000 7f90e7400640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 0x7f90cc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:15.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.816+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f90e00a99b0 con 0x7f90d40039e0 2024-05-10T03:30:15.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.816+0000 7f90e7e00640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 0x7f90cc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:15.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.818+0000 7f90e7e00640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 0x7f90cc06d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90f01f2a60 tx=0x7f90d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:15.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:15.821+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f90e00748b0 con 0x7f90d40039e0 2024-05-10T03:30:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.016+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90f0066ce0 con 0x7f90d40039e0 2024-05-10T03:30:16.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.019+0000 7f90e7400640 1 -- 172.21.15.55:0/655370407 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f90e004a090 con 0x7f90d40039e0 2024-05-10T03:30:16.024 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:16.025 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 msgr2=0x7f90cc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 0x7f90cc06d110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f90f01f2a60 tx=0x7f90d8008040 comp rx=0 tx=0).stop 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 msgr2=0x7f90f013c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009a90 tx=0x7f90e0004730 comp rx=0 tx=0).stop 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 shutdown_connections 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90f01409a0 0x7f90f015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f90cc06ac00 0x7f90cc06d110 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90f0140050 0x7f90f0140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 --2- 172.21.15.55:0/655370407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90d40039e0 0x7f90f013c610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 >> 172.21.15.55:0/655370407 conn(0x7f90f006a110 msgr2=0x7f90f00b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:16.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 shutdown_connections 2024-05-10T03:30:16.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.022+0000 7f90f5200640 1 -- 172.21.15.55:0/655370407 wait complete. 2024-05-10T03:30:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:16.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:16.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.328+0000 7fbf5b200640 1 Processor -- start 2024-05-10T03:30:16.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.328+0000 7fbf5b200640 1 -- start start 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.330+0000 7fbf5b200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbf5c0ce7c0 0x7fbf5c0709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.330+0000 7fbf5b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 0x7fbf5c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.330+0000 7fbf5b200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c0cbc10 con 0x7fbf5c0cb800 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.330+0000 7fbf5b200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c15e640 con 0x7fbf5c0ce7c0 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.330+0000 7fbf5b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b3e20 con 0x7fbf5c06b120 2024-05-10T03:30:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf59e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbf5c0cb800 0x7fbf5c15df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38062/0 (socket says 172.21.15.55:38062) 2024-05-10T03:30:16.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf59e00640 1 -- 172.21.15.55:0/2969471700 learned_addr learned my addr 172.21.15.55:0/2969471700 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:16.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf59400640 1 --2- 172.21.15.55:0/2969471700 >> v2:172.21.15.162:3300/0 conn(0x7fbf5c0ce7c0 0x7fbf5c0709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf5a800640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 0x7fbf5c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268143591 0 0) 0x7fbf4c001110 con 0x7fbf5c0cb800 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.331+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf40005730 con 0x7fbf5c0cb800 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 533065367 0 0) 0x7fbf40005730 con 0x7fbf5c0cb800 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 msgr2=0x7fbf5c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 0x7fbf5c06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 >> v2:172.21.15.162:3300/0 conn(0x7fbf5c0ce7c0 msgr2=0x7fbf5c0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 --2- 172.21.15.55:0/2969471700 >> v2:172.21.15.162:3300/0 conn(0x7fbf5c0ce7c0 0x7fbf5c0709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c06ba70 con 0x7fbf5c0cb800 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbf4c001800 con 0x7fbf5c0cb800 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 >> v1:172.21.15.55:6789/0 conn(0x7fbf5c0cb800 legacy=0x7fbf5c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 0x7fbf40005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf40006170 0x7fbf400069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf40006f10 0x7fbf40007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c15e640 con 0x7fbf40006f10 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b3e20 con 0x7fbf40006170 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000dc70 con 0x7fbf5c06b120 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.332+0000 7fbf5a800640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf40006f10 0x7fbf40007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf59400640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf40006170 0x7fbf400069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf5a800640 1 -- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 msgr2=0x7fbf40005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf5a800640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c06b120 0x7fbf40005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf5a800640 1 -- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf40006170 msgr2=0x7fbf400069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf5a800640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf40006170 0x7fbf400069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.333+0000 7fbf5a800640 1 -- 172.21.15.55:0/2969471700 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40005730 con 0x7fbf40006f10 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.334+0000 7fbf5a800640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf40006f10 0x7fbf40007320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fc0 tx=0x7fbf40007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1390939ca3541675 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.334+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54031670 con 0x7fbf40006f10 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.334+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400f040 con 0x7fbf40006f10 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.334+0000 7fbf58a00640 1 -- 172.21.15.55:0/2969471700 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54031a10 con 0x7fbf40006f10 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 -- 172.21.15.55:0/2969471700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf40006f10 msgr2=0x7fbf40007320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 --2- 172.21.15.55:0/2969471700 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf40006f10 0x7fbf40007320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009fc0 tx=0x7fbf40007a00 comp rx=0 tx=0).stop 2024-05-10T03:30:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5a800640 1 -- 172.21.15.55:0/2969471700 reap_dead start 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 -- 172.21.15.55:0/2969471700 shutdown_connections 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 -- 172.21.15.55:0/2969471700 >> 172.21.15.55:0/2969471700 conn(0x7fbf5c06c750 msgr2=0x7fbf5c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 -- 172.21.15.55:0/2969471700 shutdown_connections 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.335+0000 7fbf5b200640 1 -- 172.21.15.55:0/2969471700 wait complete. 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.338+0000 7fbf5b200640 1 Processor -- start 2024-05-10T03:30:16.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.338+0000 7fbf5b200640 1 -- start start 2024-05-10T03:30:16.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 0x7fbf5c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c0ae0c0 0x7fbf5c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c157a80 con 0x7fbf5c15c440 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c15b9f0 con 0x7fbf5c0adcb0 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf5b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c15bfb0 con 0x7fbf5c0ae0c0 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf59e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf59e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33448/0 (socket says 172.21.15.55:33448) 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.340+0000 7fbf59e00640 1 -- 172.21.15.55:0/1254809109 learned_addr learned my addr 172.21.15.55:0/1254809109 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59400640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 0x7fbf5c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 -- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c0ae0c0 msgr2=0x7fbf5c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c0ae0c0 0x7fbf5c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 -- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 msgr2=0x7fbf5c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 0x7fbf5c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 -- 172.21.15.55:0/1254809109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c1575a0 con 0x7fbf5c15c440 2024-05-10T03:30:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59400640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 0x7fbf5c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf59e00640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ca70 tx=0x7fbf4c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.341+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c004cb0 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.342+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c004e10 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.342+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c007310 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.342+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c155180 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.342+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c15bc40 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.343+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c04e290 con 0x7fbf5c15c440 2024-05-10T03:30:16.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.344+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbf4c007470 con 0x7fbf5c15c440 2024-05-10T03:30:16.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.346+0000 7fbf52000640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 0x7fbf3c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.346+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf4c088840 con 0x7fbf5c15c440 2024-05-10T03:30:16.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.346+0000 7fbf59400640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 0x7fbf3c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.348+0000 7fbf59400640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 0x7fbf3c06d110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf48006e80 tx=0x7fbf48006dd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.351+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbf4c053720 con 0x7fbf5c15c440 2024-05-10T03:30:16.544 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:30:16.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:30:16.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.558+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf5c0cbc10 con 0x7fbf5c15c440 2024-05-10T03:30:16.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.558+0000 7fbf52000640 1 -- 172.21.15.55:0/1254809109 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf4c018020 con 0x7fbf5c15c440 2024-05-10T03:30:16.563 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 msgr2=0x7fbf3c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 0x7fbf3c06d110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf48006e80 tx=0x7fbf48006dd0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 msgr2=0x7fbf5c1502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ca70 tx=0x7fbf4c00cf30 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 shutdown_connections 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf5c0ae0c0 0x7fbf5c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbf3c06ac00 0x7fbf3c06d110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf5c0adcb0 0x7fbf5c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 --2- 172.21.15.55:0/1254809109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf5c15c440 0x7fbf5c1502f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 >> 172.21.15.55:0/1254809109 conn(0x7fbf5c06c750 msgr2=0x7fbf5c0b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 shutdown_connections 2024-05-10T03:30:16.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.561+0000 7fbf5b200640 1 -- 172.21.15.55:0/1254809109 wait complete. 2024-05-10T03:30:16.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:16.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.891+0000 7f11ab200640 1 Processor -- start 2024-05-10T03:30:16.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.892+0000 7f11ab200640 1 -- start start 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.894+0000 7f11ab200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.894+0000 7f11ab200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11a4070b50 0x7f11a40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.894+0000 7f11ab200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a4070f60 con 0x7f11a4161dd0 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.894+0000 7f11ab200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a406a110 con 0x7f11a4070b50 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11ab200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a415dd20 con 0x7f11a406daf0 2024-05-10T03:30:16.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11a9e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11a4161dd0 0x7f11a415d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38078/0 (socket says 172.21.15.55:38078) 2024-05-10T03:30:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11a9e00640 1 -- 172.21.15.55:0/3946185592 learned_addr learned my addr 172.21.15.55:0/3946185592 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11aa800640 1 --2- 172.21.15.55:0/3946185592 >> v2:172.21.15.55:3300/0 conn(0x7f11a4070b50 0x7f11a40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11a9400640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11a8a00640 1 -- 172.21.15.55:0/3946185592 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703894670 0 0) 0x7f11a4070f60 con 0x7f11a4161dd0 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.895+0000 7f11a8a00640 1 -- 172.21.15.55:0/3946185592 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f118c003520 con 0x7f11a4161dd0 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.896+0000 7f11aa800640 1 -- 172.21.15.55:0/3946185592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 msgr2=0x7f11a4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.896+0000 7f11aa800640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a4070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.896+0000 7f11aa800640 1 -- 172.21.15.55:0/3946185592 >> v1:172.21.15.55:6789/0 conn(0x7f11a4161dd0 legacy=0x7f11a415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.896+0000 7f11aa800640 1 -- 172.21.15.55:0/3946185592 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a415e000 con 0x7f11a4070b50 2024-05-10T03:30:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.897+0000 7f11aa800640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4070b50 0x7f11a40ce410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009d90 tx=0x7f11a002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf152e5b92db4bca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.897+0000 7f11a8a00640 1 -- 172.21.15.55:0/3946185592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0031600 con 0x7f11a4070b50 2024-05-10T03:30:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.897+0000 7f11a8a00640 1 -- 172.21.15.55:0/3946185592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a002d050 con 0x7f11a4070b50 2024-05-10T03:30:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.897+0000 7f11a8a00640 1 -- 172.21.15.55:0/3946185592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0031980 con 0x7f11a4070b50 2024-05-10T03:30:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 -- 172.21.15.55:0/3946185592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4070b50 msgr2=0x7f11a40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4070b50 0x7f11a40ce410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009d90 tx=0x7f11a002a5d0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 -- 172.21.15.55:0/3946185592 shutdown_connections 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a4070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 --2- 172.21.15.55:0/3946185592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4070b50 0x7f11a40ce410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 -- 172.21.15.55:0/3946185592 >> 172.21.15.55:0/3946185592 conn(0x7f11a40cc060 msgr2=0x7f11a40cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 -- 172.21.15.55:0/3946185592 shutdown_connections 2024-05-10T03:30:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.898+0000 7f11ab200640 1 -- 172.21.15.55:0/3946185592 wait complete. 2024-05-10T03:30:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.901+0000 7f11ab200640 1 Processor -- start 2024-05-10T03:30:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.901+0000 7f11ab200640 1 -- start start 2024-05-10T03:30:16.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.902+0000 7f11ab200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11ab200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11a4070b50 0x7f11a40ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11ab200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11ab200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40aefc0 con 0x7f11a4161dd0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11ab200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40c5280 con 0x7f11a4070b50 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11ab200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40c46b0 con 0x7f11a406daf0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11aa800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11a9400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11a4070b50 0x7f11a40ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11aa800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33452/0 (socket says 172.21.15.55:33452) 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a414cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:49000/0 (socket says 172.21.15.55:49000) 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.903+0000 7f11aa800640 1 -- 172.21.15.55:0/1277479569 learned_addr learned my addr 172.21.15.55:0/1277479569 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 -- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 msgr2=0x7f11a414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 -- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11a4070b50 msgr2=0x7f11a40ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11a4070b50 0x7f11a40ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 -- 172.21.15.55:0/1277479569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0009a70 con 0x7f11a4161dd0 2024-05-10T03:30:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11aa800640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009eb0 tx=0x7f11a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a003b820 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a003b980 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a40c4950 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.904+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a003bcf0 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.905+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a40c4f10 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.906+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a40c5410 con 0x7f11a4161dd0 2024-05-10T03:30:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.907+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f11a004d3f0 con 0x7f11a4161dd0 2024-05-10T03:30:16.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.908+0000 7f119e000640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 0x7f118806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:16.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.908+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11a00aac10 con 0x7f11a4161dd0 2024-05-10T03:30:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.908+0000 7f11a9e00640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 0x7f118806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:16.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.910+0000 7f11a9e00640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 0x7f118806d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11980011e0 tx=0x7f1198007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:16.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:16.913+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f11a00795a0 con 0x7f11a4161dd0 2024-05-10T03:30:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.115+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11a404e290 con 0x7f11a4161dd0 2024-05-10T03:30:17.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.116+0000 7f119e000640 1 -- 172.21.15.55:0/1277479569 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11a0079130 con 0x7f11a4161dd0 2024-05-10T03:30:17.121 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 msgr2=0x7f118806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 0x7f118806d110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11980011e0 tx=0x7f1198007d30 comp rx=0 tx=0).stop 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 msgr2=0x7f11a415bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009eb0 tx=0x7f11a0004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 shutdown_connections 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a4161dd0 0x7f11a415bd50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f118806ac00 0x7f118806d110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11a4070b50 0x7f11a40ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 --2- 172.21.15.55:0/1277479569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11a406daf0 0x7f11a414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 >> 172.21.15.55:0/1277479569 conn(0x7f11a40cc060 msgr2=0x7f11a40aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:17.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 shutdown_connections 2024-05-10T03:30:17.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.119+0000 7f11ab200640 1 -- 172.21.15.55:0/1277479569 wait complete. 2024-05-10T03:30:17.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:17.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.403+0000 7feb44e00640 1 Processor -- start 2024-05-10T03:30:17.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.403+0000 7feb44e00640 1 -- start start 2024-05-10T03:30:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb44e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feb4015d700 0x7feb4015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb44e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400b8ff0 con 0x7feb400cdf70 2024-05-10T03:30:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb44e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400b6d00 con 0x7feb400d1ba0 2024-05-10T03:30:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb44e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4015e050 con 0x7feb4015d700 2024-05-10T03:30:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb3fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feb400cdf70 0x7feb400ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38092/0 (socket says 172.21.15.55:38092) 2024-05-10T03:30:17.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.406+0000 7feb3fe00640 1 -- 172.21.15.55:0/1160601699 learned_addr learned my addr 172.21.15.55:0/1160601699 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb37e00640 1 --2- 172.21.15.55:0/1160601699 >> v2:172.21.15.162:3300/0 conn(0x7feb4015d700 0x7feb4015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509295631 0 0) 0x7feb400b8ff0 con 0x7feb400cdf70 2024-05-10T03:30:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb20003520 con 0x7feb400cdf70 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175222504 0 0) 0x7feb20003520 con 0x7feb400cdf70 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 >> v2:172.21.15.162:3300/0 conn(0x7feb4015d700 msgr2=0x7feb4015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 --2- 172.21.15.55:0/1160601699 >> v2:172.21.15.162:3300/0 conn(0x7feb4015d700 0x7feb4015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 >> v1:172.21.15.162:6789/0 conn(0x7feb400d1ba0 legacy=0x7feb400b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400b7b50 con 0x7feb400cdf70 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7feb380019f0 con 0x7feb400cdf70 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 >> v1:172.21.15.55:6789/0 conn(0x7feb400cdf70 legacy=0x7feb400ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb20003970 0x7feb20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 0x7feb20004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb20005230 0x7feb2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2000c190 con 0x7feb200047c0 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2000c300 con 0x7feb20005230 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.407+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2000c4c0 con 0x7feb20003970 2024-05-10T03:30:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 0x7feb20004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb3f400640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb20003970 0x7feb20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 -- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb20003970 msgr2=0x7feb20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb20003970 0x7feb20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 -- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb20005230 msgr2=0x7feb2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb20005230 0x7feb2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb37e00640 1 -- 172.21.15.55:0/1160601699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20003520 con 0x7feb200047c0 2024-05-10T03:30:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.408+0000 7feb3fe00640 1 -- 172.21.15.55:0/1160601699 reap_dead start 2024-05-10T03:30:17.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.410+0000 7feb37e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 0x7feb20004bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb2800ba30 tx=0x7feb200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8394e5273a89bb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.410+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2802ae10 con 0x7feb200047c0 2024-05-10T03:30:17.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.410+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2802e070 con 0x7feb200047c0 2024-05-10T03:30:17.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.410+0000 7feb3ea00640 1 -- 172.21.15.55:0/1160601699 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28034620 con 0x7feb200047c0 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 -- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 msgr2=0x7feb20004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 0x7feb20004bd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb2800ba30 tx=0x7feb200057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 -- 172.21.15.55:0/1160601699 shutdown_connections 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 --2- 172.21.15.55:0/1160601699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb200047c0 0x7feb20004bd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 -- 172.21.15.55:0/1160601699 >> 172.21.15.55:0/1160601699 conn(0x7feb400d0620 msgr2=0x7feb400d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 -- 172.21.15.55:0/1160601699 shutdown_connections 2024-05-10T03:30:17.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.411+0000 7feb44e00640 1 -- 172.21.15.55:0/1160601699 wait complete. 2024-05-10T03:30:17.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.414+0000 7feb44e00640 1 Processor -- start 2024-05-10T03:30:17.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.414+0000 7feb44e00640 1 -- start start 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb200047c0 0x7feb400ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb401f57c0 0x7feb400c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c8210 con 0x7feb400c7410 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401f5d70 con 0x7feb200047c0 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb44e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40157a70 con 0x7feb401f57c0 2024-05-10T03:30:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33466/0 (socket says 172.21.15.55:33466) 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb3fe00640 1 -- 172.21.15.55:0/1959686369 learned_addr learned my addr 172.21.15.55:0/1959686369 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.416+0000 7feb37e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb401f57c0 0x7feb400c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3f400640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb200047c0 0x7feb400ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 -- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb401f57c0 msgr2=0x7feb400c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb401f57c0 0x7feb400c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 -- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb200047c0 msgr2=0x7feb400ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb200047c0 0x7feb400ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 -- 172.21.15.55:0/1959686369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb401f6b40 con 0x7feb400c7410 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb3fe00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb38001560 tx=0x7feb3800ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38007c20 con 0x7feb400c7410 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800a610 con 0x7feb400c7410 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800a9c0 con 0x7feb400c7410 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.417+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400c7d60 con 0x7feb400c7410 2024-05-10T03:30:17.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.418+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401f62a0 con 0x7feb400c7410 2024-05-10T03:30:17.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.419+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4004e290 con 0x7feb400c7410 2024-05-10T03:30:17.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.420+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb3800ab20 con 0x7feb400c7410 2024-05-10T03:30:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.421+0000 7feb37400640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 0x7feb1c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.421+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feb38088f50 con 0x7feb400c7410 2024-05-10T03:30:17.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.421+0000 7feb3f400640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 0x7feb1c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.423+0000 7feb3f400640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 0x7feb1c06d140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb30007fd0 tx=0x7feb30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.425+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feb38057830 con 0x7feb400c7410 2024-05-10T03:30:17.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.619+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb400d14a0 con 0x7feb400c7410 2024-05-10T03:30:17.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.620+0000 7feb37400640 1 -- 172.21.15.55:0/1959686369 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb3801a020 con 0x7feb400c7410 2024-05-10T03:30:17.625 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 msgr2=0x7feb1c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 0x7feb1c06d140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb30007fd0 tx=0x7feb30009040 comp rx=0 tx=0).stop 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 msgr2=0x7feb400c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb38001560 tx=0x7feb3800ce80 comp rx=0 tx=0).stop 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 shutdown_connections 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb400c7410 0x7feb400c7820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb401f57c0 0x7feb400c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feb1c06ac30 0x7feb1c06d140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 --2- 172.21.15.55:0/1959686369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb200047c0 0x7feb400ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.623+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 >> 172.21.15.55:0/1959686369 conn(0x7feb400d0620 msgr2=0x7feb4006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.624+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 shutdown_connections 2024-05-10T03:30:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.624+0000 7feb44e00640 1 -- 172.21.15.55:0/1959686369 wait complete. 2024-05-10T03:30:17.653 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.917+0000 7f0757e00640 1 Processor -- start 2024-05-10T03:30:17.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.917+0000 7f0757e00640 1 -- start start 2024-05-10T03:30:17.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0757e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07580b2110 0x7f075815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0757e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580b1bc0 0x7f07580cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0757e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07580ceb00 con 0x7f07580ce090 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0757e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075815dcd0 con 0x7f07580b2110 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0757e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07580d2150 con 0x7f07580b1bc0 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.919+0000 7f0756a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07580b2110 0x7f075815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07580b2110 0x7f075815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33476/0 (socket says 172.21.15.55:33476) 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 -- 172.21.15.55:0/1691078772 learned_addr learned my addr 172.21.15.55:0/1691078772 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756000640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580b1bc0 0x7f07580cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 -- 172.21.15.55:0/1691078772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580b1bc0 msgr2=0x7f07580cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:17.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580b1bc0 0x7f07580cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 -- 172.21.15.55:0/1691078772 >> v1:172.21.15.55:6789/0 conn(0x7f07580ce090 legacy=0x7f0758161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.920+0000 7f0756a00640 1 -- 172.21.15.55:0/1691078772 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580b2910 con 0x7f07580b2110 2024-05-10T03:30:17.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.921+0000 7f0756a00640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f075815d790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0748009e50 tx=0x7f074802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36078e567c062339 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.921+0000 7f0755600640 1 -- 172.21.15.55:0/1691078772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074802aea0 con 0x7f07580b2110 2024-05-10T03:30:17.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.921+0000 7f0755600640 1 -- 172.21.15.55:0/1691078772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074802d050 con 0x7f07580b2110 2024-05-10T03:30:17.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0755600640 1 -- 172.21.15.55:0/1691078772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748031820 con 0x7f07580b2110 2024-05-10T03:30:17.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 -- 172.21.15.55:0/1691078772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 msgr2=0x7f075815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f075815d790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0748009e50 tx=0x7f074802a680 comp rx=0 tx=0).stop 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 -- 172.21.15.55:0/1691078772 shutdown_connections 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580b1bc0 0x7f07580cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 --2- 172.21.15.55:0/1691078772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f075815d790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 -- 172.21.15.55:0/1691078772 >> 172.21.15.55:0/1691078772 conn(0x7f07580d30d0 msgr2=0x7f07580d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 -- 172.21.15.55:0/1691078772 shutdown_connections 2024-05-10T03:30:17.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.922+0000 7f0757e00640 1 -- 172.21.15.55:0/1691078772 wait complete. 2024-05-10T03:30:17.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.925+0000 7f0757e00640 1 Processor -- start 2024-05-10T03:30:17.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.925+0000 7f0757e00640 1 -- start start 2024-05-10T03:30:17.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07580b1bc0 0x7f0758154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580ce090 0x7f0758151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580cbbc0 con 0x7f07580b2110 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0758152d40 con 0x7f07580b1bc0 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.926+0000 7f0757e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0758152f90 con 0x7f07580ce090 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33482/0 (socket says 172.21.15.55:33482) 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 -- 172.21.15.55:0/1708895814 learned_addr learned my addr 172.21.15.55:0/1708895814 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0757400640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580ce090 0x7f0758151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756a00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07580b1bc0 0x7f0758154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 -- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580ce090 msgr2=0x7f0758151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580ce090 0x7f0758151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 -- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07580b1bc0 msgr2=0x7f0758154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:17.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07580b1bc0 0x7f0758154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 -- 172.21.15.55:0/1708895814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0748009b30 con 0x7f07580b2110 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.927+0000 7f0756000640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f074400d5c0 tx=0x7f074400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.928+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07440125f0 con 0x7f07580b2110 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.928+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744012750 con 0x7f07580b2110 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.928+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744012b00 con 0x7f07580b2110 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.928+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0758153270 con 0x7f07580b2110 2024-05-10T03:30:17.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.928+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0758153830 con 0x7f07580b2110 2024-05-10T03:30:17.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.929+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075804e290 con 0x7f07580b2110 2024-05-10T03:30:17.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.930+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0744012c60 con 0x7f07580b2110 2024-05-10T03:30:17.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.931+0000 7f074ea00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 0x7f073806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:17.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.931+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0744089ba0 con 0x7f07580b2110 2024-05-10T03:30:17.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.932+0000 7f0756a00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 0x7f073806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.933+0000 7f0756a00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 0x7f073806d110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0748033000 tx=0x7f0748009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:17.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:17.941+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f07440549f0 con 0x7f07580b2110 2024-05-10T03:30:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.146+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f07580cd490 con 0x7f07580b2110 2024-05-10T03:30:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.151+0000 7f074ea00640 1 -- 172.21.15.55:0/1708895814 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0744058010 con 0x7f07580b2110 2024-05-10T03:30:18.155 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 msgr2=0x7f073806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 0x7f073806d110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0748033000 tx=0x7f0748009000 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 msgr2=0x7f0758160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f074400d5c0 tx=0x7f074400da80 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 shutdown_connections 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07580ce090 0x7f0758151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07580b2110 0x7f0758160830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f073806ac00 0x7f073806d110 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 --2- 172.21.15.55:0/1708895814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07580b1bc0 0x7f0758154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 >> 172.21.15.55:0/1708895814 conn(0x7f07580d30d0 msgr2=0x7f075815be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 shutdown_connections 2024-05-10T03:30:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.153+0000 7f0757e00640 1 -- 172.21.15.55:0/1708895814 wait complete. 2024-05-10T03:30:18.182 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:30:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.446+0000 7f8547c00640 1 Processor -- start 2024-05-10T03:30:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.446+0000 7f8547c00640 1 -- start start 2024-05-10T03:30:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f854006b6e0 0x7f85400ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f85401659c0 0x7f8540165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f8540162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400711c0 con 0x7f85401659c0 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854006baf0 con 0x7f854006b6e0 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.448+0000 7f8547c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85400715e0 con 0x7f8540162380 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f85401659c0 0x7f8540165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f85401659c0 0x7f8540165dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33494/0 (socket says 172.21.15.55:33494) 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 -- 172.21.15.55:0/867697419 learned_addr learned my addr 172.21.15.55:0/867697419 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f8545200640 1 --2- 172.21.15.55:0/867697419 >> v2:172.21.15.162:3300/0 conn(0x7f854006b6e0 0x7f85400ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f8545c00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f8540162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 -- 172.21.15.55:0/867697419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 msgr2=0x7f8540162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f8540162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 -- 172.21.15.55:0/867697419 >> v2:172.21.15.162:3300/0 conn(0x7f854006b6e0 msgr2=0x7f85400ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.449+0000 7f853fe00640 1 --2- 172.21.15.55:0/867697419 >> v2:172.21.15.162:3300/0 conn(0x7f854006b6e0 0x7f85400ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.450+0000 7f853fe00640 1 -- 172.21.15.55:0/867697419 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540166310 con 0x7f85401659c0 2024-05-10T03:30:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.451+0000 7f853fe00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f8540165dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f854006bcb0 tx=0x7f852c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0bb5f8f98466c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.451+0000 7f853f400640 1 -- 172.21.15.55:0/867697419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c031600 con 0x7f85401659c0 2024-05-10T03:30:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.451+0000 7f853f400640 1 -- 172.21.15.55:0/867697419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c02d050 con 0x7f85401659c0 2024-05-10T03:30:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.451+0000 7f853f400640 1 -- 172.21.15.55:0/867697419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c031980 con 0x7f85401659c0 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 -- 172.21.15.55:0/867697419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 msgr2=0x7f8540165dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f8540165dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f854006bcb0 tx=0x7f852c02a610 comp rx=0 tx=0).stop 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 -- 172.21.15.55:0/867697419 shutdown_connections 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f8540162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 --2- 172.21.15.55:0/867697419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f8540165dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 --2- 172.21.15.55:0/867697419 >> v2:172.21.15.162:3300/0 conn(0x7f854006b6e0 0x7f85400ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 -- 172.21.15.55:0/867697419 >> 172.21.15.55:0/867697419 conn(0x7f85400cbb90 msgr2=0x7f85400cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 -- 172.21.15.55:0/867697419 shutdown_connections 2024-05-10T03:30:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.452+0000 7f8547c00640 1 -- 172.21.15.55:0/867697419 wait complete. 2024-05-10T03:30:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.454+0000 7f8547c00640 1 Processor -- start 2024-05-10T03:30:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.454+0000 7f8547c00640 1 -- start start 2024-05-10T03:30:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 0x7f85401481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f85401f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540149940 con 0x7f85401659c0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85401fa070 con 0x7f854006b6e0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8547c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8540148720 con 0x7f8540162380 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8545c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8545c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33496/0 (socket says 172.21.15.55:33496) 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.456+0000 7f8545c00640 1 -- 172.21.15.55:0/2304823625 learned_addr learned my addr 172.21.15.55:0/2304823625 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f853fe00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f85401f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545200640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 0x7f85401481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 -- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 msgr2=0x7f85401f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f85401f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 -- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 msgr2=0x7f85401481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 0x7f85401481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 -- 172.21.15.55:0/2304823625 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009b30 con 0x7f85401659c0 2024-05-10T03:30:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545200640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 0x7f85401481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:18.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f8545c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f853800e7f0 tx=0x7f853800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:18.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.457+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85380094b0 con 0x7f85401659c0 2024-05-10T03:30:18.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.458+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853800c730 con 0x7f85401659c0 2024-05-10T03:30:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.458+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853800cae0 con 0x7f85401659c0 2024-05-10T03:30:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.458+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85401faed0 con 0x7f85401659c0 2024-05-10T03:30:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.458+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85401fa5a0 con 0x7f85401659c0 2024-05-10T03:30:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.459+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85401fb060 con 0x7f85401659c0 2024-05-10T03:30:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.460+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f853800cc40 con 0x7f85401659c0 2024-05-10T03:30:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.461+0000 7f853cc00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 0x7f852006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.461+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8538089b90 con 0x7f85401659c0 2024-05-10T03:30:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.462+0000 7f8545200640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 0x7f852006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.463+0000 7f8545200640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 0x7f852006d110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f854006d310 tx=0x7f8530008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:18.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.466+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8538054950 con 0x7f85401659c0 2024-05-10T03:30:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.710+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8540066f60 con 0x7f85401659c0 2024-05-10T03:30:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.711+0000 7f853cc00640 1 -- 172.21.15.55:0/2304823625 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f8538021090 con 0x7f85401659c0 2024-05-10T03:30:18.716 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:18.716 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:30:18.716 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 msgr2=0x7f852006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 0x7f852006d110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f854006d310 tx=0x7f8530008040 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 msgr2=0x7f85401f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f853800e7f0 tx=0x7f853800ecb0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 shutdown_connections 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85401659c0 0x7f85401f9ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8540162380 0x7f85401f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f852006ac00 0x7f852006d110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 --2- 172.21.15.55:0/2304823625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854006b6e0 0x7f85401481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 >> 172.21.15.55:0/2304823625 conn(0x7f85400cbb90 msgr2=0x7f85400b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 shutdown_connections 2024-05-10T03:30:18.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:18.714+0000 7f8547c00640 1 -- 172.21.15.55:0/2304823625 wait complete. 2024-05-10T03:30:18.747 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:19.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.020+0000 7f7fce200640 1 Processor -- start 2024-05-10T03:30:19.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.021+0000 7f7fce200640 1 -- start start 2024-05-10T03:30:19.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fce200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7fc8144710 0x7f7fc8144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fce200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fce200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc806d350 con 0x7f7fc806d7e0 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fce200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc806c740 con 0x7f7fc8144710 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fce200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806ca20 con 0x7f7fc80b3190 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fcce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7fc8144710 0x7f7fc8144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fc7e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7fc806d7e0 0x7f7fc80b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38112/0 (socket says 172.21.15.55:38112) 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fcce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7fc8144710 0x7f7fc8144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33510/0 (socket says 172.21.15.55:33510) 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.023+0000 7f7fc7e00640 1 -- 172.21.15.55:0/2720138820 learned_addr learned my addr 172.21.15.55:0/2720138820 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcd800640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcce00640 1 -- 172.21.15.55:0/2720138820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 msgr2=0x7f7fc80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcce00640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcce00640 1 -- 172.21.15.55:0/2720138820 >> v1:172.21.15.55:6789/0 conn(0x7f7fc806d7e0 legacy=0x7f7fc80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcce00640 1 -- 172.21.15.55:0/2720138820 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc806cd00 con 0x7f7fc8144710 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fc7400640 1 -- 172.21.15.55:0/2720138820 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277506616 0 0) 0x7f7fc806d350 con 0x7f7fc806d7e0 2024-05-10T03:30:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.024+0000 7f7fcd800640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:19.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.025+0000 7f7fcce00640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc8144120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0097a0 tx=0x7f7fbc02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598058e88c9a0f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.025+0000 7f7fc7400640 1 -- 172.21.15.55:0/2720138820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc02adb0 con 0x7f7fc8144710 2024-05-10T03:30:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.025+0000 7f7fc7400640 1 -- 172.21.15.55:0/2720138820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc032440 con 0x7f7fc8144710 2024-05-10T03:30:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.025+0000 7f7fc7400640 1 -- 172.21.15.55:0/2720138820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0327d0 con 0x7f7fc8144710 2024-05-10T03:30:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 -- 172.21.15.55:0/2720138820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 msgr2=0x7f7fc8144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc8144120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0097a0 tx=0x7f7fbc02a060 comp rx=0 tx=0).stop 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 -- 172.21.15.55:0/2720138820 shutdown_connections 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 --2- 172.21.15.55:0/2720138820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc8144120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 -- 172.21.15.55:0/2720138820 >> 172.21.15.55:0/2720138820 conn(0x7f7fc806a110 msgr2=0x7f7fc806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 -- 172.21.15.55:0/2720138820 shutdown_connections 2024-05-10T03:30:19.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.026+0000 7f7fce200640 1 -- 172.21.15.55:0/2720138820 wait complete. 2024-05-10T03:30:19.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.028+0000 7f7fce200640 1 Processor -- start 2024-05-10T03:30:19.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.029+0000 7f7fce200640 1 -- start start 2024-05-10T03:30:19.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc806d7e0 0x7f7fc815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80ab0d0 con 0x7f7fc8144710 2024-05-10T03:30:19.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80ae310 con 0x7f7fc806d7e0 2024-05-10T03:30:19.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fce200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80a8d30 con 0x7f7fc80b3190 2024-05-10T03:30:19.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fcd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fcd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33522/0 (socket says 172.21.15.55:33522) 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.030+0000 7f7fcd800640 1 -- 172.21.15.55:0/2602922503 learned_addr learned my addr 172.21.15.55:0/2602922503 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fc7e00640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcce00640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc806d7e0 0x7f7fc815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 -- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 msgr2=0x7f7fc80ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 -- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc806d7e0 msgr2=0x7f7fc815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc806d7e0 0x7f7fc815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 -- 172.21.15.55:0/2602922503 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc009400 con 0x7f7fc8144710 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.031+0000 7f7fcd800640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000cd80 tx=0x7f7fc000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.032+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000f680 con 0x7f7fc8144710 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.032+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000f800 con 0x7f7fc8144710 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.032+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc80a8fd0 con 0x7f7fc8144710 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.032+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000fbb0 con 0x7f7fc8144710 2024-05-10T03:30:19.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.032+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc80a9590 con 0x7f7fc8144710 2024-05-10T03:30:19.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.033+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8066ce0 con 0x7f7fc8144710 2024-05-10T03:30:19.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.034+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7fc000b040 con 0x7f7fc8144710 2024-05-10T03:30:19.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.035+0000 7f7fc4c00640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 0x7f7fa806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.035+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7fc008a0a0 con 0x7f7fc8144710 2024-05-10T03:30:19.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.036+0000 7f7fcce00640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 0x7f7fa806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.037+0000 7f7fcce00640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 0x7f7fa806d110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc034000 tx=0x7f7fbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.040+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7fc0054f00 con 0x7f7fc8144710 2024-05-10T03:30:19.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.242+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fc80cecb0 con 0x7f7fc8144710 2024-05-10T03:30:19.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.245+0000 7f7fc4c00640 1 -- 172.21.15.55:0/2602922503 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f7fc001d090 con 0x7f7fc8144710 2024-05-10T03:30:19.250 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:19.250 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 msgr2=0x7f7fa806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 0x7f7fa806d110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc034000 tx=0x7f7fbc02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 msgr2=0x7f7fc80adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000cd80 tx=0x7f7fc000a650 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 shutdown_connections 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fc8144710 0x7f7fc80adcd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80b3190 0x7f7fc80ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7fa806ac00 0x7f7fa806d110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 --2- 172.21.15.55:0/2602922503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc806d7e0 0x7f7fc815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 >> 172.21.15.55:0/2602922503 conn(0x7f7fc806a110 msgr2=0x7f7fc8066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 shutdown_connections 2024-05-10T03:30:19.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.248+0000 7f7fce200640 1 -- 172.21.15.55:0/2602922503 wait complete. 2024-05-10T03:30:19.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:19.276 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:19.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.552+0000 7ff167e00640 1 Processor -- start 2024-05-10T03:30:19.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.552+0000 7ff167e00640 1 -- start start 2024-05-10T03:30:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.554+0000 7ff167e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff16815e920 0x7ff1680cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.554+0000 7ff167e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff16806b120 0x7ff16806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff167e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680b2a30 con 0x7ff1680710f0 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff167e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16815a900 con 0x7ff16815e920 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff167e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16815eec0 con 0x7ff16806b120 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff16815e920 0x7ff1680cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff16815e920 0x7ff1680cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33534/0 (socket says 172.21.15.55:33534) 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 -- 172.21.15.55:0/3539275158 learned_addr learned my addr 172.21.15.55:0/3539275158 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 -- 172.21.15.55:0/3539275158 >> v2:172.21.15.162:3300/0 conn(0x7ff16806b120 msgr2=0x7ff16806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 --2- 172.21.15.55:0/3539275158 >> v2:172.21.15.162:3300/0 conn(0x7ff16806b120 0x7ff16806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 -- 172.21.15.55:0/3539275158 >> v1:172.21.15.162:6789/0 conn(0x7ff1680710f0 legacy=0x7ff168071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.555+0000 7ff166a00640 1 -- 172.21.15.55:0/3539275158 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16806ba70 con 0x7ff16815e920 2024-05-10T03:30:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.556+0000 7ff166a00640 1 --2- 172.21.15.55:0/3539275158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16815e920 0x7ff1680cc240 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff1680ceda0 tx=0x7ff15802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c0943273816b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.557+0000 7ff165600640 1 -- 172.21.15.55:0/3539275158 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15802f070 con 0x7ff16815e920 2024-05-10T03:30:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.557+0000 7ff165600640 1 -- 172.21.15.55:0/3539275158 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1580325a0 con 0x7ff16815e920 2024-05-10T03:30:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.557+0000 7ff165600640 1 -- 172.21.15.55:0/3539275158 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158032930 con 0x7ff16815e920 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.557+0000 7ff167e00640 1 -- 172.21.15.55:0/3539275158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16815e920 msgr2=0x7ff1680cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.557+0000 7ff167e00640 1 --2- 172.21.15.55:0/3539275158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16815e920 0x7ff1680cc240 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff1680ceda0 tx=0x7ff15802c8e0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 -- 172.21.15.55:0/3539275158 shutdown_connections 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 --2- 172.21.15.55:0/3539275158 >> v2:172.21.15.162:3300/0 conn(0x7ff16806b120 0x7ff16806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 --2- 172.21.15.55:0/3539275158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16815e920 0x7ff1680cc240 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 -- 172.21.15.55:0/3539275158 >> 172.21.15.55:0/3539275158 conn(0x7ff16806c750 msgr2=0x7ff1680b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:19.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 -- 172.21.15.55:0/3539275158 shutdown_connections 2024-05-10T03:30:19.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.558+0000 7ff167e00640 1 -- 172.21.15.55:0/3539275158 wait complete. 2024-05-10T03:30:19.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.560+0000 7ff167e00640 1 Processor -- start 2024-05-10T03:30:19.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.560+0000 7ff167e00640 1 -- start start 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.561+0000 7ff167e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.561+0000 7ff167e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 0x7ff1680cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff167e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff16815e920 0x7ff1680ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff167e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681f8160 con 0x7ff16806b120 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff167e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff168155ed0 con 0x7ff1680710f0 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff167e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681f7f10 con 0x7ff16815e920 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33540/0 (socket says 172.21.15.55:33540) 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 -- 172.21.15.55:0/64861021 learned_addr learned my addr 172.21.15.55:0/64861021 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:19.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 -- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff16815e920 msgr2=0x7ff1680ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff16815e920 0x7ff1680ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 -- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 msgr2=0x7ff1680cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166000640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 0x7ff1680cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 0x7ff1680cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166a00640 1 -- 172.21.15.55:0/64861021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15800b770 con 0x7ff16806b120 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.562+0000 7ff166000640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 0x7ff1680cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff166a00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff158000c00 tx=0x7ff158004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158033d30 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158033e90 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15803c660 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1681541e0 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.563+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff168152490 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.564+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16804e290 con 0x7ff16806b120 2024-05-10T03:30:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.565+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff15803c7c0 con 0x7ff16806b120 2024-05-10T03:30:19.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.566+0000 7ff15ea00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 0x7ff14806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:19.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.566+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff1580aa760 con 0x7ff16806b120 2024-05-10T03:30:19.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.567+0000 7ff166000640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 0x7ff14806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:19.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.568+0000 7ff166000640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 0x7ff14806d110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1680cbe40 tx=0x7ff154008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:19.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.571+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff1580756d0 con 0x7ff16806b120 2024-05-10T03:30:19.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.759+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1680b2300 con 0x7ff16806b120 2024-05-10T03:30:19.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.760+0000 7ff15ea00640 1 -- 172.21.15.55:0/64861021 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff158078cf0 con 0x7ff16806b120 2024-05-10T03:30:19.765 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:19.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 msgr2=0x7ff14806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 0x7ff14806d110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff1680cbe40 tx=0x7ff154008040 comp rx=0 tx=0).stop 2024-05-10T03:30:19.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 msgr2=0x7ff1681426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:19.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff158000c00 tx=0x7ff158004730 comp rx=0 tx=0).stop 2024-05-10T03:30:19.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 shutdown_connections 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff16815e920 0x7ff1680ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff14806ac00 0x7ff14806d110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1680710f0 0x7ff1680cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 --2- 172.21.15.55:0/64861021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff16806b120 0x7ff1681426f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 >> 172.21.15.55:0/64861021 conn(0x7ff16806c750 msgr2=0x7ff1681431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 shutdown_connections 2024-05-10T03:30:19.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:19.762+0000 7ff167e00640 1 -- 172.21.15.55:0/64861021 wait complete. 2024-05-10T03:30:19.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:20.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.069+0000 7fbb72400640 1 Processor -- start 2024-05-10T03:30:20.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.070+0000 7fbb72400640 1 -- start start 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb72400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb6c0719f0 0x7fbb6c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb72400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb6c06b840 0x7fbb6c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb72400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c15fdf0 con 0x7fbb6c071e00 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb72400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c15f1e0 con 0x7fbb6c06b840 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb72400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c15f4c0 con 0x7fbb6c0719f0 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb6be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb6c06b840 0x7fbb6c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.072+0000 7fbb6be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb6c06b840 0x7fbb6c0700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33554/0 (socket says 172.21.15.55:33554) 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb6be00640 1 -- 172.21.15.55:0/2913451543 learned_addr learned my addr 172.21.15.55:0/2913451543 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:20.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb71000640 1 --2- 172.21.15.55:0/2913451543 >> v2:172.21.15.162:3300/0 conn(0x7fbb6c0719f0 0x7fbb6c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb6be00640 1 -- 172.21.15.55:0/2913451543 >> v2:172.21.15.162:3300/0 conn(0x7fbb6c0719f0 msgr2=0x7fbb6c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:20.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb6be00640 1 --2- 172.21.15.55:0/2913451543 >> v2:172.21.15.162:3300/0 conn(0x7fbb6c0719f0 0x7fbb6c06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb6be00640 1 -- 172.21.15.55:0/2913451543 >> v1:172.21.15.162:6789/0 conn(0x7fbb6c071e00 legacy=0x7fbb6c070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:20.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.073+0000 7fbb6be00640 1 -- 172.21.15.55:0/2913451543 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c15f7a0 con 0x7fbb6c06b840 2024-05-10T03:30:20.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.074+0000 7fbb6be00640 1 --2- 172.21.15.55:0/2913451543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0700e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0b41f0 tx=0x7fbb5802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d358f9739f615d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.074+0000 7fbb6b400640 1 -- 172.21.15.55:0/2913451543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58032440 con 0x7fbb6c06b840 2024-05-10T03:30:20.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb6b400640 1 -- 172.21.15.55:0/2913451543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580325a0 con 0x7fbb6c06b840 2024-05-10T03:30:20.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb6b400640 1 -- 172.21.15.55:0/2913451543 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58032930 con 0x7fbb6c06b840 2024-05-10T03:30:20.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 -- 172.21.15.55:0/2913451543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 msgr2=0x7fbb6c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 --2- 172.21.15.55:0/2913451543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0700e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0b41f0 tx=0x7fbb5802a1a0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 -- 172.21.15.55:0/2913451543 shutdown_connections 2024-05-10T03:30:20.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 --2- 172.21.15.55:0/2913451543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0700e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 --2- 172.21.15.55:0/2913451543 >> v2:172.21.15.162:3300/0 conn(0x7fbb6c0719f0 0x7fbb6c06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.075+0000 7fbb72400640 1 -- 172.21.15.55:0/2913451543 >> 172.21.15.55:0/2913451543 conn(0x7fbb6c06c750 msgr2=0x7fbb6c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:20.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.076+0000 7fbb72400640 1 -- 172.21.15.55:0/2913451543 shutdown_connections 2024-05-10T03:30:20.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.076+0000 7fbb72400640 1 -- 172.21.15.55:0/2913451543 wait complete. 2024-05-10T03:30:20.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.078+0000 7fbb72400640 1 Processor -- start 2024-05-10T03:30:20.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.078+0000 7fbb72400640 1 -- start start 2024-05-10T03:30:20.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 0x7fbb6c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb6c071e00 0x7fbb6c0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c154830 con 0x7fbb6c06b840 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c43f0 con 0x7fbb6c0719f0 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb72400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c56e0 con 0x7fbb6c071e00 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33560/0 (socket says 172.21.15.55:33560) 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71000640 1 -- 172.21.15.55:0/1103917994 learned_addr learned my addr 172.21.15.55:0/1103917994 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb6be00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 0x7fbb6c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71000640 1 -- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb6c071e00 msgr2=0x7fbb6c0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71a00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb6c071e00 0x7fbb6c0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.080+0000 7fbb71000640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb6c071e00 0x7fbb6c0b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb71000640 1 -- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 msgr2=0x7fbb6c156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb71000640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 0x7fbb6c156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb71000640 1 -- 172.21.15.55:0/1103917994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb580094d0 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb6be00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 0x7fbb6c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb71000640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d5f0 tx=0x7fbb6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600125f0 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60012750 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60012b00 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0c4640 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.081+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c0c4c00 con 0x7fbb6c06b840 2024-05-10T03:30:20.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.083+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c04e290 con 0x7fbb6c06b840 2024-05-10T03:30:20.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.084+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb60012c60 con 0x7fbb6c06b840 2024-05-10T03:30:20.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.085+0000 7fbb68c00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 0x7fbb4c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.085+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbb60089bc0 con 0x7fbb6c06b840 2024-05-10T03:30:20.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.086+0000 7fbb6be00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 0x7fbb4c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.087+0000 7fbb6be00640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 0x7fbb4c06d110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb58034000 tx=0x7fbb5802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.090+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb60054aa0 con 0x7fbb6c06b840 2024-05-10T03:30:20.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.290+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb6c143f70 con 0x7fbb6c06b840 2024-05-10T03:30:20.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.291+0000 7fbb68c00640 1 -- 172.21.15.55:0/1103917994 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb600580c0 con 0x7fbb6c06b840 2024-05-10T03:30:20.296 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:20.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 msgr2=0x7fbb4c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 0x7fbb4c06d110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb58034000 tx=0x7fbb5802e000 comp rx=0 tx=0).stop 2024-05-10T03:30:20.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 msgr2=0x7fbb6c0c3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d5f0 tx=0x7fbb6000dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 shutdown_connections 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb6c071e00 0x7fbb6c0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb4c06ac00 0x7fbb4c06d110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb6c0719f0 0x7fbb6c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 --2- 172.21.15.55:0/1103917994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb6c06b840 0x7fbb6c0c3df0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.294+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 >> 172.21.15.55:0/1103917994 conn(0x7fbb6c06c750 msgr2=0x7fbb6c15e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.295+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 shutdown_connections 2024-05-10T03:30:20.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.295+0000 7fbb72400640 1 -- 172.21.15.55:0/1103917994 wait complete. 2024-05-10T03:30:20.327 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:20.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.617+0000 7f3741e00640 1 Processor -- start 2024-05-10T03:30:20.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.617+0000 7f3741e00640 1 -- start start 2024-05-10T03:30:20.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f3741e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f373c06fec0 0x7f373c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f3741e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0b4360 con 0x7f373c071970 2024-05-10T03:30:20.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f3741e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0b34f0 con 0x7f373c06b830 2024-05-10T03:30:20.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f3741e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c15ec50 con 0x7f373c06fec0 2024-05-10T03:30:20.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f373be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f373c071970 0x7f373c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38124/0 (socket says 172.21.15.55:38124) 2024-05-10T03:30:20.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.619+0000 7f373be00640 1 -- 172.21.15.55:0/1476846297 learned_addr learned my addr 172.21.15.55:0/1476846297 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f3740a00640 1 --2- 172.21.15.55:0/1476846297 >> v2:172.21.15.162:3300/0 conn(0x7f373c06fec0 0x7f373c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185433610 0 0) 0x7f373c0b4360 con 0x7f373c071970 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3724003520 con 0x7f373c071970 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990595493 0 0) 0x7f3724003520 con 0x7f373c071970 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 >> v2:172.21.15.162:3300/0 conn(0x7f373c06fec0 msgr2=0x7f373c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 --2- 172.21.15.55:0/1476846297 >> v2:172.21.15.162:3300/0 conn(0x7f373c06fec0 0x7f373c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 >> v1:172.21.15.162:6789/0 conn(0x7f373c06b830 legacy=0x7f373c0ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c15b6a0 con 0x7f373c071970 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f37280017f0 con 0x7f373c071970 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.620+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 >> v1:172.21.15.55:6789/0 conn(0x7f373c071970 legacy=0x7f373c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 0x7f3724003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240047c0 0x7f3724004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724005210 0x7f372400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c15ec50 con 0x7f3724003970 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372400c1a0 con 0x7f37240047c0 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372400c360 con 0x7f3724005210 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 0x7f3724003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3741400640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724005210 0x7f372400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 -- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724005210 msgr2=0x7f372400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:20.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724005210 0x7f372400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 -- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240047c0 msgr2=0x7f3724004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:20.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240047c0 0x7f3724004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3740a00640 1 -- 172.21.15.55:0/1476846297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724003520 con 0x7f3724003970 2024-05-10T03:30:20.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3741400640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724005210 0x7f372400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:20.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.621+0000 7f3741400640 1 -- 172.21.15.55:0/1476846297 reap_dead start 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.623+0000 7f3740a00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 0x7f3724003de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f373c06f0d0 tx=0x7f37240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78df852b51f0a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.623+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730031400 con 0x7f3724003970 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.623+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373002d050 con 0x7f3724003970 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.623+0000 7f373b400640 1 -- 172.21.15.55:0/1476846297 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300317a0 con 0x7f3724003970 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 -- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 msgr2=0x7f3724003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 0x7f3724003de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f373c06f0d0 tx=0x7f37240057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 -- 172.21.15.55:0/1476846297 shutdown_connections 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 --2- 172.21.15.55:0/1476846297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3724003970 0x7f3724003de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 -- 172.21.15.55:0/1476846297 >> 172.21.15.55:0/1476846297 conn(0x7f373c06c730 msgr2=0x7f373c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:20.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 -- 172.21.15.55:0/1476846297 shutdown_connections 2024-05-10T03:30:20.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.624+0000 7f3741e00640 1 -- 172.21.15.55:0/1476846297 wait complete. 2024-05-10T03:30:20.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.626+0000 7f3741e00640 1 Processor -- start 2024-05-10T03:30:20.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.626+0000 7f3741e00640 1 -- start start 2024-05-10T03:30:20.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724003970 0x7f373c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373c15cf30 0x7f373c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c13ad50 con 0x7f373c139fe0 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c1388a0 con 0x7f373c15cf30 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c13ab00 con 0x7f3724003970 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33578/0 (socket says 172.21.15.55:33578) 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.628+0000 7f3741400640 1 -- 172.21.15.55:0/2693712318 learned_addr learned my addr 172.21.15.55:0/2693712318 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3740a00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724003970 0x7f373c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f373be00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373c15cf30 0x7f373c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 -- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724003970 msgr2=0x7f373c0cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724003970 0x7f373c0cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 -- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373c15cf30 msgr2=0x7f373c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373c15cf30 0x7f373c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 -- 172.21.15.55:0/2693712318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0cad40 con 0x7f373c139fe0 2024-05-10T03:30:20.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.629+0000 7f3741400640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f373c15dd20 tx=0x7f373400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.630+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734003b70 con 0x7f373c139fe0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.630+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734003cf0 con 0x7f373c139fe0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.630+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c1f3880 con 0x7f373c139fe0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.630+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340052b0 con 0x7f373c139fe0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.630+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0ca6f0 con 0x7f373c139fe0 2024-05-10T03:30:20.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.631+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c0caed0 con 0x7f373c139fe0 2024-05-10T03:30:20.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.632+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3734005410 con 0x7f373c139fe0 2024-05-10T03:30:20.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.633+0000 7f3738c00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 0x7f371c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:20.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.633+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3734089020 con 0x7f373c139fe0 2024-05-10T03:30:20.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.634+0000 7f3740a00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 0x7f371c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:20.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.635+0000 7f3740a00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 0x7f371c06d110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3730033000 tx=0x7f3730008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:20.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.640+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3734053de0 con 0x7f373c139fe0 2024-05-10T03:30:20.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.846+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f373c0ca170 con 0x7f373c139fe0 2024-05-10T03:30:20.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.847+0000 7f3738c00640 1 -- 172.21.15.55:0/2693712318 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3734018030 con 0x7f373c139fe0 2024-05-10T03:30:20.851 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 msgr2=0x7f371c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 0x7f371c06d110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3730033000 tx=0x7f3730008030 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 msgr2=0x7f373c13a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f373c15dd20 tx=0x7f373400dc00 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 shutdown_connections 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f373c139fe0 0x7f373c13a3f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f371c06ac00 0x7f371c06d110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373c15cf30 0x7f373c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 --2- 172.21.15.55:0/2693712318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724003970 0x7f373c0cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:20.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 >> 172.21.15.55:0/2693712318 conn(0x7f373c06c730 msgr2=0x7f373c15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:20.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 shutdown_connections 2024-05-10T03:30:20.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:20.850+0000 7f3741e00640 1 -- 172.21.15.55:0/2693712318 wait complete. 2024-05-10T03:30:20.881 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.158+0000 7f4355200640 1 Processor -- start 2024-05-10T03:30:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.159+0000 7f4355200640 1 -- start start 2024-05-10T03:30:21.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f4355200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43500719f0 0x7f435006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f4355200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435015fdf0 con 0x7f435006b840 2024-05-10T03:30:21.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f4355200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435015f1e0 con 0x7f4350071e00 2024-05-10T03:30:21.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f4355200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4350141d10 con 0x7f43500719f0 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43500719f0 0x7f435006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43500719f0 0x7f435006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33594/0 (socket says 172.21.15.55:33594) 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f4347e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f435006b840 0x7f43500700e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38136/0 (socket says 172.21.15.55:38136) 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 -- 172.21.15.55:0/4097727442 learned_addr learned my addr 172.21.15.55:0/4097727442 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 -- 172.21.15.55:0/4097727442 >> v1:172.21.15.162:6789/0 conn(0x7f4350071e00 legacy=0x7f4350141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434ea00640 1 -- 172.21.15.55:0/4097727442 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986566752 0 0) 0x7f435015fdf0 con 0x7f435006b840 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 -- 172.21.15.55:0/4097727442 >> v1:172.21.15.55:6789/0 conn(0x7f435006b840 legacy=0x7f43500700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.162+0000 7f434f400640 1 -- 172.21.15.55:0/4097727442 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435015f4c0 con 0x7f43500719f0 2024-05-10T03:30:21.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.163+0000 7f434f400640 1 --2- 172.21.15.55:0/4097727442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f435006b300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4340009870 tx=0x7f434002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8028c031e86329 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.164+0000 7f434ea00640 1 -- 172.21.15.55:0/4097727442 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340032440 con 0x7f43500719f0 2024-05-10T03:30:21.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.164+0000 7f434ea00640 1 -- 172.21.15.55:0/4097727442 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43400325a0 con 0x7f43500719f0 2024-05-10T03:30:21.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.164+0000 7f434ea00640 1 -- 172.21.15.55:0/4097727442 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340032930 con 0x7f43500719f0 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.164+0000 7f4355200640 1 -- 172.21.15.55:0/4097727442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 msgr2=0x7f435006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.164+0000 7f4355200640 1 --2- 172.21.15.55:0/4097727442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f435006b300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4340009870 tx=0x7f434002a270 comp rx=0 tx=0).stop 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.165+0000 7f4355200640 1 -- 172.21.15.55:0/4097727442 shutdown_connections 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.165+0000 7f4355200640 1 --2- 172.21.15.55:0/4097727442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f435006b300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.165+0000 7f4355200640 1 -- 172.21.15.55:0/4097727442 >> 172.21.15.55:0/4097727442 conn(0x7f435006c750 msgr2=0x7f43500b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.165+0000 7f4355200640 1 -- 172.21.15.55:0/4097727442 shutdown_connections 2024-05-10T03:30:21.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.165+0000 7f4355200640 1 -- 172.21.15.55:0/4097727442 wait complete. 2024-05-10T03:30:21.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.168+0000 7f4355200640 1 Processor -- start 2024-05-10T03:30:21.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.168+0000 7f4355200640 1 -- start start 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f435006b840 0x7f4350137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4350071e00 0x7f43500adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350137b60 con 0x7f43500719f0 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350157570 con 0x7f4350071e00 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500b2850 con 0x7f435006b840 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33604/0 (socket says 172.21.15.55:33604) 2024-05-10T03:30:21.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 -- 172.21.15.55:0/3748989977 learned_addr learned my addr 172.21.15.55:0/3748989977 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 -- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f435006b840 msgr2=0x7f4350137620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f435006b840 0x7f4350137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 -- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4350071e00 msgr2=0x7f43500adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4350071e00 0x7f43500adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 -- 172.21.15.55:0/3748989977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400094d0 con 0x7f43500719f0 2024-05-10T03:30:21.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347e00640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f433800c3f0 tx=0x7f433800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380074e0 con 0x7f43500719f0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4338007660 con 0x7f43500719f0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338004a20 con 0x7f43500719f0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.170+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500b36b0 con 0x7f43500719f0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.171+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43500b2d80 con 0x7f43500719f0 2024-05-10T03:30:21.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.172+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435006e2e0 con 0x7f43500719f0 2024-05-10T03:30:21.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.174+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4338004b80 con 0x7f43500719f0 2024-05-10T03:30:21.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.174+0000 7f4347400640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 0x7f432c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.175+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43380894b0 con 0x7f43500719f0 2024-05-10T03:30:21.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.175+0000 7f434f400640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 0x7f432c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.176+0000 7f434f400640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 0x7f432c06d140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4340034000 tx=0x7f434002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.179+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4338054300 con 0x7f43500719f0 2024-05-10T03:30:21.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.386+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f435004e290 con 0x7f43500719f0 2024-05-10T03:30:21.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.387+0000 7f4347400640 1 -- 172.21.15.55:0/3748989977 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4338057920 con 0x7f43500719f0 2024-05-10T03:30:21.392 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:21.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 msgr2=0x7f432c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 0x7f432c06d140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4340034000 tx=0x7f434002c000 comp rx=0 tx=0).stop 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 msgr2=0x7f43500ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f433800c3f0 tx=0x7f433800c8b0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 shutdown_connections 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f432c06ac30 0x7f432c06d140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4350071e00 0x7f43500adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43500719f0 0x7f43500ad980 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 --2- 172.21.15.55:0/3748989977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f435006b840 0x7f4350137620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.390+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 >> 172.21.15.55:0/3748989977 conn(0x7f435006c750 msgr2=0x7f4350143c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:21.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.391+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 shutdown_connections 2024-05-10T03:30:21.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.391+0000 7f4355200640 1 -- 172.21.15.55:0/3748989977 wait complete. 2024-05-10T03:30:21.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:30:21.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.700+0000 7f25b1000640 1 Processor -- start 2024-05-10T03:30:21.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.700+0000 7f25b1000640 1 -- start start 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25b1000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25ac0b2170 0x7f25ac15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25b1000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0d25a0 con 0x7f25ac1618b0 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25b1000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0b28d0 con 0x7f25ac0b6680 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25b1000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac0d2b20 con 0x7f25ac0b2170 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25abe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f25ac1618b0 0x7f25ac06aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38150/0 (socket says 172.21.15.55:38150) 2024-05-10T03:30:21.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25abe00640 1 -- 172.21.15.55:0/448287218 learned_addr learned my addr 172.21.15.55:0/448287218 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.702+0000 7f25aaa00640 1 --2- 172.21.15.55:0/448287218 >> v2:172.21.15.55:3300/0 conn(0x7f25ac0b2170 0x7f25ac15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aa000640 1 -- 172.21.15.55:0/448287218 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431713955 0 0) 0x7f25ac0d25a0 con 0x7f25ac1618b0 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aa000640 1 -- 172.21.15.55:0/448287218 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2590003520 con 0x7f25ac1618b0 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aaa00640 1 -- 172.21.15.55:0/448287218 >> v1:172.21.15.162:6789/0 conn(0x7f25ac0b6680 legacy=0x7f25ac1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aaa00640 1 -- 172.21.15.55:0/448287218 >> v1:172.21.15.55:6789/0 conn(0x7f25ac1618b0 legacy=0x7f25ac06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aaa00640 1 -- 172.21.15.55:0/448287218 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0d2840 con 0x7f25ac0b2170 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aaa00640 1 --2- 172.21.15.55:0/448287218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac0b2170 0x7f25ac15d5e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25980097a0 tx=0x7f259802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70252c45f0685983 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aa000640 1 -- 172.21.15.55:0/448287218 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598032440 con 0x7f25ac0b2170 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aa000640 1 -- 172.21.15.55:0/448287218 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25980325a0 con 0x7f25ac0b2170 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.704+0000 7f25aa000640 1 -- 172.21.15.55:0/448287218 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598032910 con 0x7f25ac0b2170 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 -- 172.21.15.55:0/448287218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac0b2170 msgr2=0x7f25ac15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 --2- 172.21.15.55:0/448287218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac0b2170 0x7f25ac15d5e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f25980097a0 tx=0x7f259802a230 comp rx=0 tx=0).stop 2024-05-10T03:30:21.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 -- 172.21.15.55:0/448287218 shutdown_connections 2024-05-10T03:30:21.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 --2- 172.21.15.55:0/448287218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac0b2170 0x7f25ac15d5e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 -- 172.21.15.55:0/448287218 >> 172.21.15.55:0/448287218 conn(0x7f25ac0b5730 msgr2=0x7f25ac0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:21.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 -- 172.21.15.55:0/448287218 shutdown_connections 2024-05-10T03:30:21.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.705+0000 7f25b1000640 1 -- 172.21.15.55:0/448287218 wait complete. 2024-05-10T03:30:21.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.709+0000 7f25b1000640 1 Processor -- start 2024-05-10T03:30:21.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.709+0000 7f25b1000640 1 -- start start 2024-05-10T03:30:21.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25ac0b2170 0x7f25ac151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 0x7f25ac157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac15db20 con 0x7f25ac1618b0 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b1720 con 0x7f25ac0b6680 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25b1000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b1970 con 0x7f25ac0b2170 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33614/0 (socket says 172.21.15.55:33614) 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 -- 172.21.15.55:0/2131070864 learned_addr learned my addr 172.21.15.55:0/2131070864 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25aaa00640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 0x7f25ac157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25ab400640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25ac0b2170 0x7f25ac151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 -- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25ac0b2170 msgr2=0x7f25ac151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25ac0b2170 0x7f25ac151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 -- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 msgr2=0x7f25ac157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 0x7f25ac157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25abe00640 1 -- 172.21.15.55:0/2131070864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598009400 con 0x7f25ac1618b0 2024-05-10T03:30:21.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.710+0000 7f25aaa00640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 0x7f25ac157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25abe00640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25a400cad0 tx=0x7f25a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400ac30 con 0x7f25ac1618b0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400ad90 con 0x7f25ac1618b0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4012620 con 0x7f25ac1618b0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac153620 con 0x7f25ac1618b0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.711+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac153be0 con 0x7f25ac1618b0 2024-05-10T03:30:21.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.714+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac067740 con 0x7f25ac1618b0 2024-05-10T03:30:21.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.715+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f25a4007070 con 0x7f25ac1618b0 2024-05-10T03:30:21.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.715+0000 7f25a3400640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 0x7f258c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:21.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.715+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f25a40895c0 con 0x7f25ac1618b0 2024-05-10T03:30:21.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.715+0000 7f25ab400640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 0x7f258c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:21.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.716+0000 7f25ab400640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 0x7f258c06d110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f259c000f60 tx=0x7f259c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:21.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.719+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f25a40544b0 con 0x7f25ac1618b0 2024-05-10T03:30:21.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.959+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25ac0d1c20 con 0x7f25ac1618b0 2024-05-10T03:30:21.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.960+0000 7f25a3400640 1 -- 172.21.15.55:0/2131070864 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f25a4057ad0 con 0x7f25ac1618b0 2024-05-10T03:30:21.964 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:21.964 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:30:21.965 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:30:21.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 msgr2=0x7f258c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 0x7f258c06d110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f259c000f60 tx=0x7f259c009040 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 msgr2=0x7f25ac1582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25a400cad0 tx=0x7f25a400cf90 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 shutdown_connections 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25ac1618b0 0x7f25ac1582a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f258c06ac00 0x7f258c06d110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25ac0b6680 0x7f25ac157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 --2- 172.21.15.55:0/2131070864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25ac0b2170 0x7f25ac151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:21.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.963+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 >> 172.21.15.55:0/2131070864 conn(0x7f25ac0b5730 msgr2=0x7f25ac0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:21.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.964+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 shutdown_connections 2024-05-10T03:30:21.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:21.964+0000 7f25b1000640 1 -- 172.21.15.55:0/2131070864 wait complete. 2024-05-10T03:30:21.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.281+0000 7f0112200640 1 Processor -- start 2024-05-10T03:30:22.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.281+0000 7f0112200640 1 -- start start 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f010c15a5d0 0x7f010c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010c0b46b0 0x7f010c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c0b4ac0 con 0x7f010c0b46b0 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c15ad30 con 0x7f010c15a5d0 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0112200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c15b010 con 0x7f010c0cb730 2024-05-10T03:30:22.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0111800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010c0b46b0 0x7f010c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0111800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f010c0b46b0 0x7f010c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33620/0 (socket says 172.21.15.55:33620) 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f0111800640 1 -- 172.21.15.55:0/3336384862 learned_addr learned my addr 172.21.15.55:0/3336384862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.284+0000 7f010be00640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0110e00640 1 --2- 172.21.15.55:0/3336384862 >> v2:172.21.15.162:3300/0 conn(0x7f010c15a5d0 0x7f010c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0111800640 1 -- 172.21.15.55:0/3336384862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 msgr2=0x7f010c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0111800640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0111800640 1 -- 172.21.15.55:0/3336384862 >> v2:172.21.15.162:3300/0 conn(0x7f010c15a5d0 msgr2=0x7f010c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0111800640 1 --2- 172.21.15.55:0/3336384862 >> v2:172.21.15.162:3300/0 conn(0x7f010c15a5d0 0x7f010c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.285+0000 7f0111800640 1 -- 172.21.15.55:0/3336384862 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c15b2f0 con 0x7f010c0b46b0 2024-05-10T03:30:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.286+0000 7f0111800640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c15df40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f010c06deb0 tx=0x7f010402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb2e69557b586ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.286+0000 7f010b400640 1 -- 172.21.15.55:0/3336384862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104031600 con 0x7f010c0b46b0 2024-05-10T03:30:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.286+0000 7f010b400640 1 -- 172.21.15.55:0/3336384862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010402d050 con 0x7f010c0b46b0 2024-05-10T03:30:22.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.286+0000 7f010b400640 1 -- 172.21.15.55:0/3336384862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104031980 con 0x7f010c0b46b0 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 -- 172.21.15.55:0/3336384862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 msgr2=0x7f010c15df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c15df40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f010c06deb0 tx=0x7f010402a6a0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 -- 172.21.15.55:0/3336384862 shutdown_connections 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c15df40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 --2- 172.21.15.55:0/3336384862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 --2- 172.21.15.55:0/3336384862 >> v2:172.21.15.162:3300/0 conn(0x7f010c15a5d0 0x7f010c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 -- 172.21.15.55:0/3336384862 >> 172.21.15.55:0/3336384862 conn(0x7f010c06a110 msgr2=0x7f010c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 -- 172.21.15.55:0/3336384862 shutdown_connections 2024-05-10T03:30:22.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.287+0000 7f0112200640 1 -- 172.21.15.55:0/3336384862 wait complete. 2024-05-10T03:30:22.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.290+0000 7f0112200640 1 Processor -- start 2024-05-10T03:30:22.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.290+0000 7f0112200640 1 -- start start 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f010c15a5d0 0x7f010c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c154a80 con 0x7f010c0b46b0 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1567c0 con 0x7f010c15a5d0 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0112200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1f7440 con 0x7f010c0cb730 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33632/0 (socket says 172.21.15.55:33632) 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 -- 172.21.15.55:0/658167078 learned_addr learned my addr 172.21.15.55:0/658167078 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f010be00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 -- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 msgr2=0x7f010c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 -- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f010c15a5d0 msgr2=0x7f010c152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f010c15a5d0 0x7f010c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f0110e00640 1 -- 172.21.15.55:0/658167078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009b30 con 0x7f010c0b46b0 2024-05-10T03:30:22.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.292+0000 7f010be00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0110e00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f010000d5d0 tx=0x7f010000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100004240 con 0x7f010c0b46b0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01000043a0 con 0x7f010c0b46b0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100003830 con 0x7f010c0b46b0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c1f7720 con 0x7f010c0b46b0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.293+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c1f7ce0 con 0x7f010c0b46b0 2024-05-10T03:30:22.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.295+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c1f7ea0 con 0x7f010c0b46b0 2024-05-10T03:30:22.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.296+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0100003990 con 0x7f010c0b46b0 2024-05-10T03:30:22.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.297+0000 7f0108c00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 0x7f00ec06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.297+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01000897d0 con 0x7f010c0b46b0 2024-05-10T03:30:22.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.298+0000 7f010be00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 0x7f00ec06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.299+0000 7f010be00640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 0x7f00ec06d160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f00f8006fd0 tx=0x7f00f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.302+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0100054710 con 0x7f010c0b46b0 2024-05-10T03:30:22.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.497+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f010c066ce0 con 0x7f010c0b46b0 2024-05-10T03:30:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.499+0000 7f0108c00640 1 -- 172.21.15.55:0/658167078 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17878 (secure 0 0 0) 0x7f0100057d30 con 0x7f010c0b46b0 2024-05-10T03:30:22.504 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:22.504 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:29:49.356868+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 msgr2=0x7f00ec06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 0x7f00ec06d160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f00f8006fd0 tx=0x7f00f8008040 comp rx=0 tx=0).stop 2024-05-10T03:30:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 msgr2=0x7f010c156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f010000d5d0 tx=0x7f010000da90 comp rx=0 tx=0).stop 2024-05-10T03:30:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 shutdown_connections 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00ec06ac50 0x7f00ec06d160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f010c15a5d0 0x7f010c152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f010c0cb730 0x7f010c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 --2- 172.21.15.55:0/658167078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f010c0b46b0 0x7f010c156170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 >> 172.21.15.55:0/658167078 conn(0x7f010c06a110 msgr2=0x7f010c0706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 shutdown_connections 2024-05-10T03:30:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.502+0000 7f0112200640 1 -- 172.21.15.55:0/658167078 wait complete. 2024-05-10T03:30:22.530 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:30:22.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.811+0000 7fcc47e00640 1 Processor -- start 2024-05-10T03:30:22.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.811+0000 7fcc47e00640 1 -- start start 2024-05-10T03:30:22.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc4815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc481618b0 0x7fcc4806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc480d25a0 con 0x7fcc480b6680 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc480d2b20 con 0x7fcc481618b0 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4815db20 con 0x7fcc480b2170 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc481618b0 0x7fcc4806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc481618b0 0x7fcc4806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33636/0 (socket says 172.21.15.55:33636) 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc47400640 1 -- 172.21.15.55:0/797432635 learned_addr learned my addr 172.21.15.55:0/797432635 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:22.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.814+0000 7fcc46000640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc4815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.815+0000 7fcc47400640 1 -- 172.21.15.55:0/797432635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 msgr2=0x7fcc4815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:22.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.815+0000 7fcc47400640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc4815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.815+0000 7fcc47400640 1 -- 172.21.15.55:0/797432635 >> v1:172.21.15.162:6789/0 conn(0x7fcc480b6680 legacy=0x7fcc481611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:22.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.815+0000 7fcc47400640 1 -- 172.21.15.55:0/797432635 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480b28d0 con 0x7fcc481618b0 2024-05-10T03:30:22.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.816+0000 7fcc47400640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc4806aab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc4002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cf0b8174c057d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.816+0000 7fcc45600640 1 -- 172.21.15.55:0/797432635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40032440 con 0x7fcc481618b0 2024-05-10T03:30:22.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.816+0000 7fcc45600640 1 -- 172.21.15.55:0/797432635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc400325a0 con 0x7fcc481618b0 2024-05-10T03:30:22.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.816+0000 7fcc45600640 1 -- 172.21.15.55:0/797432635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40032930 con 0x7fcc481618b0 2024-05-10T03:30:22.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 -- 172.21.15.55:0/797432635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 msgr2=0x7fcc4806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc4806aab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc4002a210 comp rx=0 tx=0).stop 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 -- 172.21.15.55:0/797432635 shutdown_connections 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc4806aab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 --2- 172.21.15.55:0/797432635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc4815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 -- 172.21.15.55:0/797432635 >> 172.21.15.55:0/797432635 conn(0x7fcc480b5730 msgr2=0x7fcc480d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 -- 172.21.15.55:0/797432635 shutdown_connections 2024-05-10T03:30:22.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.817+0000 7fcc47e00640 1 -- 172.21.15.55:0/797432635 wait complete. 2024-05-10T03:30:22.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.820+0000 7fcc47e00640 1 Processor -- start 2024-05-10T03:30:22.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.820+0000 7fcc47e00640 1 -- start start 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.821+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc481e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.821+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc480b6680 0x7fcc480cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.821+0000 7fcc47e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.821+0000 7fcc47e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4814a420 con 0x7fcc481618b0 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc481e81e0 con 0x7fcc480b6680 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc481e8430 con 0x7fcc480b2170 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33640/0 (socket says 172.21.15.55:33640) 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 -- 172.21.15.55:0/3359474881 learned_addr learned my addr 172.21.15.55:0/3359474881 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc46000640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc480b6680 0x7fcc480cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc46a00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc481e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 -- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 msgr2=0x7fcc481e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc481e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 -- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc480b6680 msgr2=0x7fcc480cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:22.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc480b6680 0x7fcc480cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:22.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.822+0000 7fcc47400640 1 -- 172.21.15.55:0/3359474881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400094d0 con 0x7fcc481618b0 2024-05-10T03:30:22.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc47400640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc4002a250 tx=0x7fcc400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40032440 con 0x7fcc481618b0 2024-05-10T03:30:22.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40030b80 con 0x7fcc481618b0 2024-05-10T03:30:22.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc481e8710 con 0x7fcc481618b0 2024-05-10T03:30:22.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4003d420 con 0x7fcc481618b0 2024-05-10T03:30:22.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.823+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc481e8cd0 con 0x7fcc481618b0 2024-05-10T03:30:22.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.825+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4804e290 con 0x7fcc481618b0 2024-05-10T03:30:22.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.826+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcc40039040 con 0x7fcc481618b0 2024-05-10T03:30:22.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.826+0000 7fcc3ea00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 0x7fcc2806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:22.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.827+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcc400aa200 con 0x7fcc481618b0 2024-05-10T03:30:22.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.827+0000 7fcc46a00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 0x7fcc2806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:22.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.828+0000 7fcc46a00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 0x7fcc2806d140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009f60 tx=0x7fcc380094f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:22.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:22.831+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc40075050 con 0x7fcc481618b0 2024-05-10T03:30:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.071+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc480b3300 con 0x7fcc481618b0 2024-05-10T03:30:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.073+0000 7fcc3ea00640 1 -- 172.21.15.55:0/3359474881 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7fcc40078670 con 0x7fcc481618b0 2024-05-10T03:30:23.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:23.077 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:30:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.075+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 msgr2=0x7fcc2806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.075+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 0x7fcc2806d140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009f60 tx=0x7fcc380094f0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.075+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 msgr2=0x7fcc48148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.075+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc4002a250 tx=0x7fcc400046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 shutdown_connections 2024-05-10T03:30:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc481618b0 0x7fcc48148db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcc2806ac30 0x7fcc2806d140 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc480b6680 0x7fcc480cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 --2- 172.21.15.55:0/3359474881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc480b2170 0x7fcc481e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 >> 172.21.15.55:0/3359474881 conn(0x7fcc480b5730 msgr2=0x7fcc480b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 shutdown_connections 2024-05-10T03:30:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.076+0000 7fcc47e00640 1 -- 172.21.15.55:0/3359474881 wait complete. 2024-05-10T03:30:23.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:29:44.296174+0000', 'modified': '2024-05-10T03:30:08.098152+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/2047857054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 2047857054}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 2047857054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:30:23.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:30:23.512 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-10T03:30:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.808+0000 7f88c6000640 1 Processor -- start 2024-05-10T03:30:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.809+0000 7f88c6000640 1 -- start start 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c0154630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c6000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88c00cdef0 0x7f88c00ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c6000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c00b5d90 con 0x7f88c006bb80 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c6000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c00cf400 con 0x7f88c00cdef0 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c6000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00b4730 con 0x7f88c00b4320 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c4c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f88c006bb80 0x7f88c015e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48424/0 (socket says 172.21.15.55:48424) 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c4c00640 1 -- 172.21.15.55:0/1907494863 learned_addr learned my addr 172.21.15.55:0/1907494863 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.811+0000 7f88c5600640 1 --2- 172.21.15.55:0/1907494863 >> v2:172.21.15.55:3300/0 conn(0x7f88c00cdef0 0x7f88c00ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.812+0000 7f88bfe00640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c0154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.812+0000 7f88c5600640 1 -- 172.21.15.55:0/1907494863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 msgr2=0x7f88c0154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.812+0000 7f88c5600640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c0154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.812+0000 7f88c5600640 1 -- 172.21.15.55:0/1907494863 >> v1:172.21.15.55:6789/0 conn(0x7f88c006bb80 legacy=0x7f88c015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.812+0000 7f88c5600640 1 -- 172.21.15.55:0/1907494863 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00ce840 con 0x7f88c00cdef0 2024-05-10T03:30:23.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.813+0000 7f88c5600640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c00ce300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009dc0 tx=0x7f88b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d317d7d23ae527 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.813+0000 7f88bf400640 1 -- 172.21.15.55:0/1907494863 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8007740 con 0x7f88c00cdef0 2024-05-10T03:30:23.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.813+0000 7f88bf400640 1 -- 172.21.15.55:0/1907494863 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b802d050 con 0x7f88c00cdef0 2024-05-10T03:30:23.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.813+0000 7f88bf400640 1 -- 172.21.15.55:0/1907494863 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8007ac0 con 0x7f88c00cdef0 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 -- 172.21.15.55:0/1907494863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 msgr2=0x7f88c00ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c00ce300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009dc0 tx=0x7f88b802a710 comp rx=0 tx=0).stop 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 -- 172.21.15.55:0/1907494863 shutdown_connections 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c0154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 --2- 172.21.15.55:0/1907494863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c00ce300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 -- 172.21.15.55:0/1907494863 >> 172.21.15.55:0/1907494863 conn(0x7f88c006a110 msgr2=0x7f88c006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.814+0000 7f88c6000640 1 -- 172.21.15.55:0/1907494863 shutdown_connections 2024-05-10T03:30:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.815+0000 7f88c6000640 1 -- 172.21.15.55:0/1907494863 wait complete. 2024-05-10T03:30:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.817+0000 7f88c6000640 1 Processor -- start 2024-05-10T03:30:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.817+0000 7f88c6000640 1 -- start start 2024-05-10T03:30:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88c006bb80 0x7f88c00a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0146cf0 con 0x7f88c00cdef0 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00ccc20 con 0x7f88c006bb80 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c6000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c00ca5b0 con 0x7f88c00b4320 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58638/0 (socket says 172.21.15.55:58638) 2024-05-10T03:30:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88c5600640 1 -- 172.21.15.55:0/2595031981 learned_addr learned my addr 172.21.15.55:0/2595031981 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.819+0000 7f88bfe00640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c4c00640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88c006bb80 0x7f88c00a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 -- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 msgr2=0x7f88c00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c00a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 -- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88c006bb80 msgr2=0x7f88c00a9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88c006bb80 0x7f88c00a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 -- 172.21.15.55:0/2595031981 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8009a70 con 0x7f88c00cdef0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c5600640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88b802ad00 tx=0x7f88b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88bcc00640 1 -- 172.21.15.55:0/2595031981 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b803b820 con 0x7f88c00cdef0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88bcc00640 1 -- 172.21.15.55:0/2595031981 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b803b980 con 0x7f88c00cdef0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88bcc00640 1 -- 172.21.15.55:0/2595031981 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b803bcf0 con 0x7f88c00cdef0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.820+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c00ca300 con 0x7f88c00cdef0 2024-05-10T03:30:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.821+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c00caae0 con 0x7f88c00cdef0 2024-05-10T03:30:23.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.823+0000 7f88bcc00640 1 -- 172.21.15.55:0/2595031981 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f88b8031400 con 0x7f88c00cdef0 2024-05-10T03:30:23.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.824+0000 7f88bcc00640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 0x7f88a406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.824+0000 7f88bcc00640 1 -- 172.21.15.55:0/2595031981 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f88b80ab020 con 0x7f88c00cdef0 2024-05-10T03:30:23.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.825+0000 7f88c4c00640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 0x7f88a406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.825+0000 7f88ce46b740 1 Processor -- start 2024-05-10T03:30:23.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.825+0000 7f88ce46b740 1 -- start start 2024-05-10T03:30:23.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.826+0000 7f88c4c00640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 0x7f88a406d110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005880 tx=0x7f88b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a69a3b0 0x56515aaa7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aaa7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515a9cba70 con 0x56515a6c3270 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515a9c9200 con 0x56515a6a2590 2024-05-10T03:30:23.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515a9c4af0 con 0x56515a69a3b0 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88c4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88c4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58642/0 (socket says 172.21.15.55:58642) 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.827+0000 7f88c4c00640 1 -- 172.21.15.55:0/4225351716 learned_addr learned my addr 172.21.15.55:0/4225351716 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88bfe00640 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a69a3b0 0x56515aaa7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c5600640 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aaa7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 -- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a69a3b0 msgr2=0x56515aaa7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a69a3b0 0x56515aaa7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 -- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 msgr2=0x56515aaa7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aaa7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 -- 172.21.15.55:0/4225351716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515a98fb90 con 0x56515a6c3270 2024-05-10T03:30:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88c4c00640 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005b20 tx=0x7f88b4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128052300326a681 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88ad200640 1 -- 172.21.15.55:0/4225351716 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400d700 con 0x56515a6c3270 2024-05-10T03:30:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88ad200640 1 -- 172.21.15.55:0/4225351716 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400d860 con 0x56515a6c3270 2024-05-10T03:30:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.828+0000 7f88ad200640 1 -- 172.21.15.55:0/4225351716 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400dbd0 con 0x56515a6c3270 2024-05-10T03:30:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 -- 172.21.15.55:0/4225351716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 msgr2=0x56515aa8ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005b20 tx=0x7f88b4010000 comp rx=0 tx=0).stop 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 -- 172.21.15.55:0/4225351716 shutdown_connections 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aaa7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a69a3b0 0x56515aaa7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 --2- 172.21.15.55:0/4225351716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6c3270 0x56515aa8ccd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 -- 172.21.15.55:0/4225351716 >> 172.21.15.55:0/4225351716 conn(0x56515a6b73c0 msgr2=0x56515aa62330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.829+0000 7f88ce46b740 1 -- 172.21.15.55:0/4225351716 shutdown_connections 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.830+0000 7f88ce46b740 1 -- 172.21.15.55:0/4225351716 wait complete. 2024-05-10T03:30:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.831+0000 7f88ce46b740 1 -- start start 2024-05-10T03:30:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.831+0000 7f88ce46b740 1 Processor -- start 2024-05-10T03:30:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa60f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515a966220 con 0x56515a6a2590 2024-05-10T03:30:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515a965ec0 con 0x56515a69a3b0 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515a72bf20 con 0x56515a6c3270 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88ce46b740 10 client.0 _ll_register_callbacks cb 0x56515a9be160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58644/0 (socket says 172.21.15.55:58644) 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 -- 172.21.15.55:0/3196960128 learned_addr learned my addr 172.21.15.55:0/3196960128 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88c4c00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa60f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 -- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 msgr2=0x56515aa9ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 -- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 msgr2=0x56515aa60f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa60f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.833+0000 7f88bfe00640 1 -- 172.21.15.55:0/3196960128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c00b5d90 con 0x56515a6a2590 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88bfe00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88b000bed0 tx=0x7f88b0011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5b762e8becb0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88a1600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0011e60 con 0x56515a6a2590 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f88b0015650 con 0x56515a6a2590 2024-05-10T03:30:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56515a9eec90 con 0x56515a6a2590 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b0015810 con 0x56515a6a2590 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88ce46b740 10 client.4782 fetch_fsmap learned FSMap version 9 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.834+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56515a9c3170 con 0x56515a6a2590 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56515a9eec90 con 0x56515a6a2590 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f88b0015970 con 0x56515a6a2590 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 10 client.4782 fetch_fsmap finished waiting for FSMap version 9 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 10 client.4782 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 20 client.4782 populate_metadata read hostname 'smithi055' 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515a681e40 0x56515aab9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 4 client.4782 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.835+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 --> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56515a6b83c0 con 0x56515a681e40 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.836+0000 7f88c4c00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515a681e40 0x56515aab9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.837+0000 7f88a1600640 10 client.4782 ms_handle_connect on v2:172.21.15.162:6835/2047857054 2024-05-10T03:30:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.837+0000 7f88c4c00640 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515a681e40 0x56515aab9fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70e436a56a2174f server_cookie=53f37df5ddd8a7d2 in_seq=0 out_seq=0 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.838+0000 7f88a1600640 1 -- 172.21.15.55:0/3196960128 <== mds.0 v2:172.21.15.162:6835/2047857054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56515a6b83c0 con 0x56515a681e40 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88a1600640 10 client.4782 handle_command_reply: tid=0 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 client.4782 shutdown 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 msgr2=0x56515aa61470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88b000bed0 tx=0x7f88b0011630 comp rx=0 tx=0).stop 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 shutdown_connections 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515a681e40 0x56515aab9fd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa61470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 --2- 172.21.15.55:0/3196960128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa60f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88a1600640 0 client.4782 ms_handle_reset on v2:172.21.15.162:6835/2047857054 2024-05-10T03:30:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 >> 172.21.15.55:0/3196960128 conn(0x56515a6b73c0 msgr2=0x56515aa62330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 shutdown_connections 2024-05-10T03:30:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.839+0000 7f88ce46b740 1 -- 172.21.15.55:0/3196960128 wait complete. 2024-05-10T03:30:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.840+0000 7f88ce46b740 20 client.4782 trim_cache size 0 max 16384 2024-05-10T03:30:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.857+0000 7f88ce46b740 1 Processor -- start 2024-05-10T03:30:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.857+0000 7f88ce46b740 1 -- start start 2024-05-10T03:30:23.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515aa5dad0 con 0x56515a6a2590 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515aaa4b20 con 0x56515a69a3b0 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515aa897b0 con 0x56515a6c3270 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58658/0 (socket says 172.21.15.55:58658) 2024-05-10T03:30:23.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88bfe00640 1 -- 172.21.15.55:0/2131392012 learned_addr learned my addr 172.21.15.55:0/2131392012 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88c5600640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88c4c00640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.859+0000 7f88bfe00640 1 -- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 msgr2=0x56515aa9ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88bfe00640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88bfe00640 1 -- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 msgr2=0x56515aa90bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88bfe00640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa90bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88bfe00640 1 -- 172.21.15.55:0/2131392012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515a9f8130 con 0x56515a6a2590 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88c5600640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88c4c00640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88bfe00640 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88b000bea0 tx=0x7f88b0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9779c5d319f293 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88adc00640 1 -- 172.21.15.55:0/2131392012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0011e60 con 0x56515a6a2590 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88adc00640 1 -- 172.21.15.55:0/2131392012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0015650 con 0x56515a6a2590 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.860+0000 7f88adc00640 1 -- 172.21.15.55:0/2131392012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00159e0 con 0x56515a6a2590 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 -- 172.21.15.55:0/2131392012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 msgr2=0x56515aa9fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88b000bea0 tx=0x7f88b0009000 comp rx=0 tx=0).stop 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 -- 172.21.15.55:0/2131392012 shutdown_connections 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aa9ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a6a2590 0x56515aa9fa10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 --2- 172.21.15.55:0/2131392012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a69a3b0 0x56515aa90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 -- 172.21.15.55:0/2131392012 >> 172.21.15.55:0/2131392012 conn(0x56515a6b73c0 msgr2=0x56515aa85d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 -- 172.21.15.55:0/2131392012 shutdown_connections 2024-05-10T03:30:23.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.861+0000 7f88ce46b740 1 -- 172.21.15.55:0/2131392012 wait complete. 2024-05-10T03:30:23.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.863+0000 7f88ce46b740 1 -- start start 2024-05-10T03:30:23.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.863+0000 7f88ce46b740 1 Processor -- start 2024-05-10T03:30:23.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aab8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aab9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515aab9850 con 0x56515a69a3b0 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515aa58a60 con 0x56515a6a2590 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515aaa2f30 con 0x56515a6c3270 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88ce46b740 10 client.0 _ll_register_callbacks cb 0x7f88b000b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88c4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88c4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58666/0 (socket says 172.21.15.55:58666) 2024-05-10T03:30:23.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.865+0000 7f88c4c00640 1 -- 172.21.15.55:0/1373362155 learned_addr learned my addr 172.21.15.55:0/1373362155 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 -- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 msgr2=0x56515aab9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aab9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 -- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 msgr2=0x56515aab8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88bfe00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aab8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aab8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 -- 172.21.15.55:0/1373362155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56515a9c3170 con 0x56515a69a3b0 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88c4c00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f88b4013000 tx=0x7f88b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc0fc5c70ed9d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88a1600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:30:23.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4027040 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f88b400dde0 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.866+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88b400d860 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b400d700 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 10 client.4788 fetch_fsmap learned FSMap version 9 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56515a965430 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400d860 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f88b402e3f0 con 0x56515a69a3b0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 10 client.4788 fetch_fsmap finished waiting for FSMap version 9 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 10 client.4788 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 20 client.4788 populate_metadata read hostname 'smithi055' 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515ab107e0 0x56515aaa90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 4 client.4788 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.867+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 --> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56515a6c96c0 con 0x56515ab107e0 2024-05-10T03:30:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.868+0000 7f88bfe00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515ab107e0 0x56515aaa90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:23.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.869+0000 7f88bfe00640 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515ab107e0 0x56515aaa90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4bd0a603b21158 server_cookie=dbf7d2d7668c768 in_seq=0 out_seq=0 2024-05-10T03:30:23.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.869+0000 7f88a1600640 10 client.4788 ms_handle_connect on v2:172.21.15.162:6835/2047857054 2024-05-10T03:30:23.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88a1600640 1 -- 172.21.15.55:0/1373362155 <== mds.0 v2:172.21.15.162:6835/2047857054 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56515a6c96c0 con 0x56515ab107e0 2024-05-10T03:30:23.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88a1600640 10 client.4788 handle_command_reply: tid=0 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 client.4788 shutdown 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 msgr2=0x56515aac0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f88b4013000 tx=0x7f88b4000ed0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 shutdown_connections 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:6835/2047857054,v1:172.21.15.162:6837/2047857054] conn(0x56515ab107e0 0x56515aaa90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56515a6c3270 0x56515aab9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56515a6a2590 0x56515aab8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 --2- 172.21.15.55:0/1373362155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56515a69a3b0 0x56515aac0b10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88a1600640 0 client.4788 ms_handle_reset on v2:172.21.15.162:6835/2047857054 2024-05-10T03:30:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 >> 172.21.15.55:0/1373362155 conn(0x56515a6b73c0 msgr2=0x56515aa60010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 shutdown_connections 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 1 -- 172.21.15.55:0/1373362155 wait complete. 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.870+0000 7f88ce46b740 20 client.4788 trim_cache size 0 max 16384 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4294, 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:30:23.876 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 39.566054565999998, 2024-05-10T03:30:23.877 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 9, 2024-05-10T03:30:23.877 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 26, 2024-05-10T03:30:23.877 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 25, 2024-05-10T03:30:23.877 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 45.479253618000001 2024-05-10T03:30:23.877 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 msgr2=0x7f88a406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 0x7f88a406d110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005880 tx=0x7f88b4009040 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 msgr2=0x7f88c006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88b802ad00 tx=0x7f88b8004a40 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 shutdown_connections 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cdef0 0x7f88c006e6c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f88c00b4320 0x7f88c00a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f88a406ac00 0x7f88a406d110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 --2- 172.21.15.55:0/2595031981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88c006bb80 0x7f88c00a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 >> 172.21.15.55:0/2595031981 conn(0x7f88c006a110 msgr2=0x7f88c01509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 shutdown_connections 2024-05-10T03:30:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:23.871+0000 7f88c6000640 1 -- 172.21.15.55:0/2595031981 wait complete. 2024-05-10T03:30:23.901 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T03:30:23.909 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T03:30:23.909 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T03:30:23.910 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T03:30:23.910 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T03:30:23.924 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T03:30:23.924 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-10T03:30:23.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:23.925 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:30:23.983 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T03:30:23.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:23.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T03:30:24.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.077 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-10T03:30:24.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.177 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:30:24.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.223 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-10T03:30:24.342 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2024-05-10T03:30:24.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:30:24.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.346 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:30:24.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.394 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-05-10T03:30:24.525 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2024-05-10T03:30:24.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:30:24.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.529 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:30:24.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:24.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-05-10T03:30:24.655 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2024-05-10T03:30:24.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:30:24.658 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-10T03:30:24.658 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T03:30:24.658 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:30:24.658 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:30:24.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.947+0000 7f1b8c800640 1 Processor -- start 2024-05-10T03:30:24.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.947+0000 7f1b8c800640 1 -- start start 2024-05-10T03:30:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.950+0000 7f1b8c800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1b8415e940 0x7f1b8415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.950+0000 7f1b8c800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840d2d60 con 0x7f1b840ce150 2024-05-10T03:30:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.950+0000 7f1b8c800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840d2150 con 0x7f1b84165590 2024-05-10T03:30:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.950+0000 7f1b8c800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b841612d0 con 0x7f1b8415e940 2024-05-10T03:30:24.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.951+0000 7f1b89400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1b840ce150 0x7f1b840ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48436/0 (socket says 172.21.15.55:48436) 2024-05-10T03:30:24.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.951+0000 7f1b89400640 1 -- 172.21.15.55:0/3191622053 learned_addr learned my addr 172.21.15.55:0/3191622053 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.951+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3191622053 >> v2:172.21.15.162:3300/0 conn(0x7f1b8415e940 0x7f1b8415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.951+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64194533 0 0) 0x7f1b840d2d60 con 0x7f1b840ce150 2024-05-10T03:30:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.951+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b70003520 con 0x7f1b840ce150 2024-05-10T03:30:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835426891 0 0) 0x7f1b70003520 con 0x7f1b840ce150 2024-05-10T03:30:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 >> v2:172.21.15.162:3300/0 conn(0x7f1b8415e940 msgr2=0x7f1b8415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 --2- 172.21.15.55:0/3191622053 >> v2:172.21.15.162:3300/0 conn(0x7f1b8415e940 0x7f1b8415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 >> v1:172.21.15.162:6789/0 conn(0x7f1b84165590 legacy=0x7f1b841679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840d2430 con 0x7f1b840ce150 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f1b74001670 con 0x7f1b840ce150 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 >> v1:172.21.15.55:6789/0 conn(0x7f1b840ce150 legacy=0x7f1b840ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70003970 0x7f1b70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b700047c0 0x7f1b70004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 0x7f1b7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841612d0 con 0x7f1b70005180 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7000c120 con 0x7f1b700047c0 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7000c2e0 con 0x7f1b70003970 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3191622053 >> v2:172.21.15.162:3300/0 conn(0x7f1b8415e940 0x7f1b8415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 0x7f1b7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.952+0000 7f1b89400640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b700047c0 0x7f1b70004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70003970 0x7f1b70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b8a800640 1 -- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70003970 msgr2=0x7f1b70003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70003970 0x7f1b70003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b8a800640 1 -- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b700047c0 msgr2=0x7f1b70004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b700047c0 0x7f1b70004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b8a800640 1 -- 172.21.15.55:0/3191622053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b70003520 con 0x7f1b70005180 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.953+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70003970 0x7f1b70003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.954+0000 7f1b8a800640 1 -- 172.21.15.55:0/3191622053 reap_dead start 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.954+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 0x7f1b7000b980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b80001720 tx=0x7f1b700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1a4efe36186cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.954+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800052d0 con 0x7f1b70005180 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.954+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80005430 con 0x7f1b70005180 2024-05-10T03:30:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.954+0000 7f1b88a00640 1 -- 172.21.15.55:0/3191622053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8002aae0 con 0x7f1b70005180 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 -- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 msgr2=0x7f1b7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 0x7f1b7000b980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b80001720 tx=0x7f1b700057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 -- 172.21.15.55:0/3191622053 shutdown_connections 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3191622053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b70005180 0x7f1b7000b980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 -- 172.21.15.55:0/3191622053 >> 172.21.15.55:0/3191622053 conn(0x7f1b840d33a0 msgr2=0x7f1b840d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 -- 172.21.15.55:0/3191622053 shutdown_connections 2024-05-10T03:30:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.955+0000 7f1b8c800640 1 -- 172.21.15.55:0/3191622053 wait complete. 2024-05-10T03:30:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.958+0000 7f1b8c800640 1 Processor -- start 2024-05-10T03:30:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.958+0000 7f1b8c800640 1 -- start start 2024-05-10T03:30:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70005180 0x7f1b84162910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b84154670 0x7f1b840cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84156f00 con 0x7f1b84154a80 2024-05-10T03:30:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84158890 con 0x7f1b84154670 2024-05-10T03:30:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.959+0000 7f1b8c800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841596f0 con 0x7f1b70005180 2024-05-10T03:30:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58678/0 (socket says 172.21.15.55:58678) 2024-05-10T03:30:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 -- 172.21.15.55:0/3467272444 learned_addr learned my addr 172.21.15.55:0/3467272444 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70005180 0x7f1b84162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b89400640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b84154670 0x7f1b840cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 -- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70005180 msgr2=0x7f1b84162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70005180 0x7f1b84162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 -- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b84154670 msgr2=0x7f1b840cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b84154670 0x7f1b840cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 -- 172.21.15.55:0/3467272444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84158ae0 con 0x7f1b84154a80 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.960+0000 7f1b8a800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b8002a4b0 tx=0x7f1b80001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.961+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80038070 con 0x7f1b84154a80 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.961+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80000ed0 con 0x7f1b84154a80 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.961+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80003a00 con 0x7f1b84154a80 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.961+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84158dc0 con 0x7f1b84154a80 2024-05-10T03:30:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.961+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84159380 con 0x7f1b84154a80 2024-05-10T03:30:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.963+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84159830 con 0x7f1b84154a80 2024-05-10T03:30:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.963+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1b80003b60 con 0x7f1b84154a80 2024-05-10T03:30:24.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.964+0000 7f1b7a000640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 0x7f1b6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:24.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.964+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1b800a8850 con 0x7f1b84154a80 2024-05-10T03:30:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.965+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 0x7f1b6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.966+0000 7f1b89e00640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 0x7f1b6806d140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c003d80 tx=0x7f1b7c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:24.969+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1b84159830 con 0x7f1b84154a80 2024-05-10T03:30:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.211+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b84159830 con 0x7f1b84154a80 2024-05-10T03:30:25.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.212+0000 7f1b7a000640 1 -- 172.21.15.55:0/3467272444 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1b84159830 con 0x7f1b84154a80 2024-05-10T03:30:25.215 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T03:30:25.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 msgr2=0x7f1b6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:25.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 0x7f1b6806d140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c003d80 tx=0x7f1b7c00b040 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 msgr2=0x7f1b840cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b8002a4b0 tx=0x7f1b80001120 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 shutdown_connections 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b84154a80 0x7f1b840cd850 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b6806ac30 0x7f1b6806d140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b84154670 0x7f1b840cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 --2- 172.21.15.55:0/3467272444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b70005180 0x7f1b84162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 >> 172.21.15.55:0/3467272444 conn(0x7f1b840d33a0 msgr2=0x7f1b8415f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 shutdown_connections 2024-05-10T03:30:25.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:25.215+0000 7f1b8c800640 1 -- 172.21.15.55:0/3467272444 wait complete. 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T03:30:25.247 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T03:30:25.248 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:30:25.248 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:30:25.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.248 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:30:25.302 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:25.303 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:30:25.304 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T03:30:25.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T03:30:25.305 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:25.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.491 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:30:25.557 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:30:25.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.561 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:30:25.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:30:25.606 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:25.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.805 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:30:25.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.858 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-10T03:30:25.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:25.906 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:25.906 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:25.906 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:30:25.906 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T03:30:25.906 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:26.025 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T03:30:26.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:26.025 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T03:30:26.026 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:26.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:26.240 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:26.240 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:26.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:30:26.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T03:30:26.241 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:26.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:26.382 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:30:26.431 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:30:26.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:26.431 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:30:26.478 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:30:26.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:26.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:30:26.689 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:30:26.690 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:30:26.690 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:27.420 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:30:27.420 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:30:27.420 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:27.420 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:27.421 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:27.483 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:30:27.484 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:30:27.484 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:30:27.488 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:30:27.488 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:30:27.488 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:30:27.488 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:30:27.488 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:30:27.493 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:30:27.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:27.494 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:30:27.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:27.579 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:27.638 INFO:teuthology.orchestra.run.smithi055.stdout:client4657 2024-05-10T03:30:27.638 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:27.638 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:27.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:27.692 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:27.753 INFO:teuthology.orchestra.run.smithi055.stdout:client4657 2024-05-10T03:30:27.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:27.754 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:27.814 INFO:teuthology.orchestra.run.smithi055.stdout:client4657 2024-05-10T03:30:27.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:27.815 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/340d5548-dd78-4f40-a3a8-44ee99f8e0d8.client4657/status') 2024-05-10T03:30:27.901 INFO:teuthology.orchestra.run.smithi055.stdout:instance: client.4657 (3)172.21.15.55:0/1546425852 2024-05-10T03:30:27.901 INFO:teuthology.orchestra.run.smithi055.stdout:blocklisted: false 2024-05-10T03:30:27.901 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2024-05-10T03:30:27.901 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2024-05-10T03:30:27.902 INFO:teuthology.orchestra.run.smithi055.stderr:70 bytes copied, 0.000108307 s, 646 kB/s 2024-05-10T03:30:27.906 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-10T03:30:27.906 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T03:30:27.906 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-10T03:30:27.906 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:30:28.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.187+0000 7f8edb200640 1 Processor -- start 2024-05-10T03:30:28.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.187+0000 7f8edb200640 1 -- start start 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8edc0b6680 0x7f8edc15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8edc1616e0 0x7f8edc161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc0d2b20 con 0x7f8edc1616e0 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc15db20 con 0x7f8edc0b6680 2024-05-10T03:30:28.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.189+0000 7f8edb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0d23b0 con 0x7f8edc0b2170 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.190+0000 7f8eda800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8edc1616e0 0x7f8edc161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.190+0000 7f8eda800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8edc1616e0 0x7f8edc161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58690/0 (socket says 172.21.15.55:58690) 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.190+0000 7f8eda800640 1 -- 172.21.15.55:0/1151748314 learned_addr learned my addr 172.21.15.55:0/1151748314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.190+0000 7f8ed9400640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.190+0000 7f8ed9e00640 1 --2- 172.21.15.55:0/1151748314 >> v2:172.21.15.162:3300/0 conn(0x7f8edc0b6680 0x7f8edc15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.191+0000 7f8eda800640 1 -- 172.21.15.55:0/1151748314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 msgr2=0x7f8edc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:28.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.191+0000 7f8eda800640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.191+0000 7f8eda800640 1 -- 172.21.15.55:0/1151748314 >> v2:172.21.15.162:3300/0 conn(0x7f8edc0b6680 msgr2=0x7f8edc15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:28.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.191+0000 7f8eda800640 1 --2- 172.21.15.55:0/1151748314 >> v2:172.21.15.162:3300/0 conn(0x7f8edc0b6680 0x7f8edc15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.191+0000 7f8eda800640 1 -- 172.21.15.55:0/1151748314 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0d25a0 con 0x7f8edc1616e0 2024-05-10T03:30:28.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.192+0000 7f8eda800640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc1616e0 0x7f8edc161af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8edc08c070 tx=0x7f8ed402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c621cfa7130bd6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:28.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.192+0000 7f8ed8a00640 1 -- 172.21.15.55:0/1151748314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4031600 con 0x7f8edc1616e0 2024-05-10T03:30:28.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.192+0000 7f8ed8a00640 1 -- 172.21.15.55:0/1151748314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed402d050 con 0x7f8edc1616e0 2024-05-10T03:30:28.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.192+0000 7f8ed8a00640 1 -- 172.21.15.55:0/1151748314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4031980 con 0x7f8edc1616e0 2024-05-10T03:30:28.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 -- 172.21.15.55:0/1151748314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc1616e0 msgr2=0x7f8edc161af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc1616e0 0x7f8edc161af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8edc08c070 tx=0x7f8ed402a690 comp rx=0 tx=0).stop 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 -- 172.21.15.55:0/1151748314 shutdown_connections 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc1616e0 0x7f8edc161af0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 --2- 172.21.15.55:0/1151748314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 --2- 172.21.15.55:0/1151748314 >> v2:172.21.15.162:3300/0 conn(0x7f8edc0b6680 0x7f8edc15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 -- 172.21.15.55:0/1151748314 >> 172.21.15.55:0/1151748314 conn(0x7f8edc0b5730 msgr2=0x7f8edc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 -- 172.21.15.55:0/1151748314 shutdown_connections 2024-05-10T03:30:28.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.193+0000 7f8edb200640 1 -- 172.21.15.55:0/1151748314 wait complete. 2024-05-10T03:30:28.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.196+0000 7f8edb200640 1 Processor -- start 2024-05-10T03:30:28.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.196+0000 7f8edb200640 1 -- start start 2024-05-10T03:30:28.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.197+0000 7f8edb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.197+0000 7f8edb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8edb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8edc1616e0 0x7f8edc1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8edb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15fe00 con 0x7f8edc0b6680 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8edb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1537a0 con 0x7f8edc1616e0 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8edb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1539e0 con 0x7f8edc0b2170 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58696/0 (socket says 172.21.15.55:58696) 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 -- 172.21.15.55:0/3432364023 learned_addr learned my addr 172.21.15.55:0/3432364023 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9e00640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8eda800640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8edc1616e0 0x7f8edc1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 -- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 msgr2=0x7f8edc148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 -- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8edc1616e0 msgr2=0x7f8edc1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8edc1616e0 0x7f8edc1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.198+0000 7f8ed9400640 1 -- 172.21.15.55:0/3432364023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4009b30 con 0x7f8edc0b6680 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8ed9400640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d5f0 tx=0x7f8ec800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80125f0 con 0x7f8edc0b6680 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8012750 con 0x7f8edc0b6680 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8012b00 con 0x7f8edc0b6680 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc156670 con 0x7f8edc0b6680 2024-05-10T03:30:28.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.199+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc153c30 con 0x7f8edc0b6680 2024-05-10T03:30:28.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.200+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc156820 con 0x7f8edc0b6680 2024-05-10T03:30:28.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.201+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8ec8012c60 con 0x7f8edc0b6680 2024-05-10T03:30:28.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.203+0000 7f8ed2000640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 0x7f8ebc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:28.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.203+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8ec8089d60 con 0x7f8edc0b6680 2024-05-10T03:30:28.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.204+0000 7f8ed9e00640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 0x7f8ebc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.205+0000 7f8ed9e00640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 0x7f8ebc06d110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006fd0 tx=0x7f8ecc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.208+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8edc156820 con 0x7f8edc0b6680 2024-05-10T03:30:28.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.445+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8edc156820 con 0x7f8edc0b6680 2024-05-10T03:30:28.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.446+0000 7f8ed2000640 1 -- 172.21.15.55:0/3432364023 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8edc156820 con 0x7f8edc0b6680 2024-05-10T03:30:28.449 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.448+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 msgr2=0x7f8ebc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.448+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 0x7f8ebc06d110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006fd0 tx=0x7f8ecc008040 comp rx=0 tx=0).stop 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.448+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 msgr2=0x7f8edc148ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.448+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d5f0 tx=0x7f8ec800dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 shutdown_connections 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ebc06ac00 0x7f8ebc06d110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8edc1616e0 0x7f8edc1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8edc0b6680 0x7f8edc148ba0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 --2- 172.21.15.55:0/3432364023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8edc0b2170 0x7f8edc148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:28.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 >> 172.21.15.55:0/3432364023 conn(0x7f8edc0b5730 msgr2=0x7f8edc0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:28.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 shutdown_connections 2024-05-10T03:30:28.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:28.449+0000 7f8edb200640 1 -- 172.21.15.55:0/3432364023 wait complete. 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:30:28.482 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-10T03:30:28.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.483 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:30:28.533 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42424sec preferred_lft 42424sec 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:30:28.534 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:30:28.535 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:28.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.536 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:30:28.619 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:30:28.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.623 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:30:28.674 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:30:28.674 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:30:28.674 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:30:28.674 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:30:28.674 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:30:28.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:30:28.675 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:28.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.834 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:30:28.884 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:30:28.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.885 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-05-10T03:30:28.938 INFO:teuthology.orchestra.run.smithi055.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-10T03:30:28.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:28.939 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:28.940 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:28.940 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:30:28.940 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-10T03:30:28.940 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:29.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:29.068 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:29.155 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 72:a8:76:1f:cc:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::70a8:76ff:fe1f:ccaa/64 scope link 2024-05-10T03:30:29.156 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:29.160 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-10T03:30:29.249 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 72:a8:76:1f:cc:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::70a8:76ff:fe1f:ccaa/64 scope link 2024-05-10T03:30:29.250 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:29.256 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-10T03:30:29.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:29.256 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:29.256 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-10T03:30:29.257 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:29.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:29.476 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:30:29.476 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:30:29.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.1 up 2024-05-10T03:30:29.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.1 master ceph-brx 2024-05-10T03:30:29.476 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:30:29.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:29.624 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-10T03:30:29.675 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-10T03:30:29.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:29.676 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-10T03:30:29.728 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:30:29.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:29.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T03:30:29.944 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T03:30:29.944 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T03:30:29.944 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:29.974 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T03:30:29.974 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T03:30:29.974 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:29.974 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:29.974 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:30.009 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-10T03:30:30.009 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-10T03:30:30.009 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:30:30.009 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:30.010 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:30.010 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:30:30.010 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:30:30.010 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:30:30.011 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-10T03:30:30.011 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:30:30.011 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:30:30.011 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:30:30.012 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:30:30.017 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:30:30.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:30.018 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-10T03:30:30.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:30.098 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:30.158 INFO:teuthology.orchestra.run.smithi055.stdout:client4803 2024-05-10T03:30:30.159 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:30.159 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:30.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:30.205 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:30.311 INFO:teuthology.orchestra.run.smithi055.stdout:client4803 2024-05-10T03:30:30.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:30.312 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:30.376 INFO:teuthology.orchestra.run.smithi055.stdout:client4803 2024-05-10T03:30:30.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:30.377 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/340d5548-dd78-4f40-a3a8-44ee99f8e0d8.client4803/status') 2024-05-10T03:30:30.463 INFO:teuthology.orchestra.run.smithi055.stdout:instance: client.4803 (3)192.168.0.2:0/694727918 2024-05-10T03:30:30.463 INFO:teuthology.orchestra.run.smithi055.stdout:blocklisted: false 2024-05-10T03:30:30.463 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2024-05-10T03:30:30.463 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2024-05-10T03:30:30.463 INFO:teuthology.orchestra.run.smithi055.stderr:68 bytes copied, 9.0742e-05 s, 749 kB/s 2024-05-10T03:30:30.467 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-10T03:30:30.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T03:30:30.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:30:30.467 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:30:30.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.746+0000 7ff1d4a00640 1 Processor -- start 2024-05-10T03:30:30.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.746+0000 7ff1d4a00640 1 -- start start 2024-05-10T03:30:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.748+0000 7ff1d4a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1d015a5d0 0x7ff1d00cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.748+0000 7ff1d4a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1d00b4880 0x7ff1d00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.748+0000 7ff1d4a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d015ad30 con 0x7ff1d00cb730 2024-05-10T03:30:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.748+0000 7ff1d4a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d015b010 con 0x7ff1d00b4880 2024-05-10T03:30:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.748+0000 7ff1d4a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d015b2f0 con 0x7ff1d015a5d0 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1d00b4880 0x7ff1d00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1d00b4880 0x7ff1d00ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58698/0 (socket says 172.21.15.55:58698) 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 -- 172.21.15.55:0/1487561907 learned_addr learned my addr 172.21.15.55:0/1487561907 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cf400640 1 --2- 172.21.15.55:0/1487561907 >> v2:172.21.15.162:3300/0 conn(0x7ff1d015a5d0 0x7ff1d00cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 -- 172.21.15.55:0/1487561907 >> v2:172.21.15.162:3300/0 conn(0x7ff1d015a5d0 msgr2=0x7ff1d00cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 --2- 172.21.15.55:0/1487561907 >> v2:172.21.15.162:3300/0 conn(0x7ff1d015a5d0 0x7ff1d00cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 -- 172.21.15.55:0/1487561907 >> v1:172.21.15.162:6789/0 conn(0x7ff1d00cb730 legacy=0x7ff1d00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.749+0000 7ff1cfe00640 1 -- 172.21.15.55:0/1487561907 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d015b5d0 con 0x7ff1d00b4880 2024-05-10T03:30:30.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.751+0000 7ff1cfe00640 1 --2- 172.21.15.55:0/1487561907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00b4880 0x7ff1d00ccc80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d006deb0 tx=0x7ff1c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e041148ba0570afd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:30.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.751+0000 7ff1ce000640 1 -- 172.21.15.55:0/1487561907 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8031600 con 0x7ff1d00b4880 2024-05-10T03:30:30.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.751+0000 7ff1ce000640 1 -- 172.21.15.55:0/1487561907 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c802d050 con 0x7ff1d00b4880 2024-05-10T03:30:30.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.751+0000 7ff1ce000640 1 -- 172.21.15.55:0/1487561907 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8031980 con 0x7ff1d00b4880 2024-05-10T03:30:30.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 -- 172.21.15.55:0/1487561907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00b4880 msgr2=0x7ff1d00ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/1487561907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00b4880 0x7ff1d00ccc80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff1d006deb0 tx=0x7ff1c802a6f0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 -- 172.21.15.55:0/1487561907 shutdown_connections 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/1487561907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00b4880 0x7ff1d00ccc80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/1487561907 >> v2:172.21.15.162:3300/0 conn(0x7ff1d015a5d0 0x7ff1d00cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 -- 172.21.15.55:0/1487561907 >> 172.21.15.55:0/1487561907 conn(0x7ff1d006a110 msgr2=0x7ff1d006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.752+0000 7ff1d4a00640 1 -- 172.21.15.55:0/1487561907 shutdown_connections 2024-05-10T03:30:30.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.753+0000 7ff1d4a00640 1 -- 172.21.15.55:0/1487561907 wait complete. 2024-05-10T03:30:30.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.755+0000 7ff1d4a00640 1 Processor -- start 2024-05-10T03:30:30.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.755+0000 7ff1d4a00640 1 -- start start 2024-05-10T03:30:30.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1d015a5d0 0x7ff1d0151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 0x7ff1d0158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d014a170 con 0x7ff1d00cb730 2024-05-10T03:30:30.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d013eb60 con 0x7ff1d015a5d0 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1d4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d0151570 con 0x7ff1d013ef80 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58708/0 (socket says 172.21.15.55:58708) 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.757+0000 7ff1cf400640 1 -- 172.21.15.55:0/3151947297 learned_addr learned my addr 172.21.15.55:0/3151947297 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cea00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1d015a5d0 0x7ff1d0151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cfe00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 0x7ff1d0158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 -- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 msgr2=0x7ff1d0158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:30.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 0x7ff1d0158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 -- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1d015a5d0 msgr2=0x7ff1d0151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:30.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1d015a5d0 0x7ff1d0151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:30.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 -- 172.21.15.55:0/3151947297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8009b30 con 0x7ff1d00cb730 2024-05-10T03:30:30.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cfe00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 0x7ff1d0158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:30.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.758+0000 7ff1cf400640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000d5d0 tx=0x7ff1c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:30.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.759+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0004240 con 0x7ff1d00cb730 2024-05-10T03:30:30.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.759+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c00043a0 con 0x7ff1d00cb730 2024-05-10T03:30:30.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.759+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0003830 con 0x7ff1d00cb730 2024-05-10T03:30:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.759+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0149c40 con 0x7ff1d00cb730 2024-05-10T03:30:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.759+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d014a670 con 0x7ff1d00cb730 2024-05-10T03:30:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.760+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d014ccc0 con 0x7ff1d00cb730 2024-05-10T03:30:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.761+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff1c0003990 con 0x7ff1d00cb730 2024-05-10T03:30:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.762+0000 7ff1c7400640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 0x7ff1b006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:30.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.762+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff1c0089890 con 0x7ff1d00cb730 2024-05-10T03:30:30.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.763+0000 7ff1cea00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 0x7ff1b006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:30.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.764+0000 7ff1cea00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 0x7ff1b006d140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006fd0 tx=0x7ff1bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:30.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:30.767+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff1c0054540 con 0x7ff1d00cb730 2024-05-10T03:30:31.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.004+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1d014a670 con 0x7ff1d00cb730 2024-05-10T03:30:31.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.006+0000 7ff1c7400640 1 -- 172.21.15.55:0/3151947297 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff1d014a670 con 0x7ff1d00cb730 2024-05-10T03:30:31.009 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.008+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 msgr2=0x7ff1b006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.008+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 0x7ff1b006d140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006fd0 tx=0x7ff1bc008040 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.008+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 msgr2=0x7ff1d014c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.008+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000d5d0 tx=0x7ff1c000da90 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 shutdown_connections 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1d013ef80 0x7ff1d0158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff1b006ac30 0x7ff1b006d140 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1d015a5d0 0x7ff1d0151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 --2- 172.21.15.55:0/3151947297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1d00cb730 0x7ff1d014c780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 >> 172.21.15.55:0/3151947297 conn(0x7ff1d006a110 msgr2=0x7ff1d0152870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 shutdown_connections 2024-05-10T03:30:31.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:31.009+0000 7ff1d4a00640 1 -- 172.21.15.55:0/3151947297 wait complete. 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T03:30:31.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T03:30:31.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:30:31.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:30:31.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:31.040 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:31.097 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:30:31.098 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:31.099 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T03:30:31.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:31.099 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add name ceph-brx type bridge 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr flush dev ceph-brx 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set ceph-brx up 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T03:30:31.100 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:31.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:31.533 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:30:31.609 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:30:31.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:31.613 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:30:31.675 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:30:31.675 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:30:31.675 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:30:31.675 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:30:31.675 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:30:31.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:30:31.676 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:32.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:32.334 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:30:32.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:32.388 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2024-05-10T03:30:32.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:32.435 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:32.435 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:32.436 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:30:32.436 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-10T03:30:32.436 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:32.595 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-10T03:30:32.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-10T03:30:32.596 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:32.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:32.821 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:32.821 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:32.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:30:32.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T03:30:32.821 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:32.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:32.955 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:30:33.000 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:30:33.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:33.000 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:30:33.093 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:30:33.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:33.094 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:30:33.314 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:30:33.315 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:30:33.315 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:34.088 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:30:34.089 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:30:34.089 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:34.089 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:34.089 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:30:34.139 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:30:34.143 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:30:34.143 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:30:34.143 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:30:34.143 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:30:34.143 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:30:34.148 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:30:34.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:34.256 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:30:34.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:34.349 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:34.407 INFO:teuthology.orchestra.run.smithi162.stdout:client4669 2024-05-10T03:30:34.408 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:30:34.408 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:34.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:34.458 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:34.560 INFO:teuthology.orchestra.run.smithi162.stdout:client4669 2024-05-10T03:30:34.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:34.561 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:34.619 INFO:teuthology.orchestra.run.smithi162.stdout:client4669 2024-05-10T03:30:34.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:34.620 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/340d5548-dd78-4f40-a3a8-44ee99f8e0d8.client4669/status') 2024-05-10T03:30:34.700 INFO:teuthology.orchestra.run.smithi162.stdout:instance: client.4669 (3)192.168.0.1:0/2862672099 2024-05-10T03:30:34.700 INFO:teuthology.orchestra.run.smithi162.stdout:blocklisted: false 2024-05-10T03:30:34.700 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2024-05-10T03:30:34.701 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2024-05-10T03:30:34.701 INFO:teuthology.orchestra.run.smithi162.stderr:69 bytes copied, 8.1252e-05 s, 849 kB/s 2024-05-10T03:30:34.704 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-10T03:30:34.704 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T03:30:34.704 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-10T03:30:34.704 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:30:35.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.003+0000 7f503ec00640 1 Processor -- start 2024-05-10T03:30:35.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.003+0000 7f503ec00640 1 -- start start 2024-05-10T03:30:35.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.005+0000 7f503ec00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f50380b6680 0x7f503815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:35.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.005+0000 7f503ec00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380d25a0 con 0x7f50380b2170 2024-05-10T03:30:35.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.005+0000 7f503ec00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380b28d0 con 0x7f50381618b0 2024-05-10T03:30:35.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.005+0000 7f503ec00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380d2b20 con 0x7f50380b6680 2024-05-10T03:30:35.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503d800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f50380b6680 0x7f503815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:35.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503ce00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f50380b2170 0x7f50381611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54440/0 (socket says 172.21.15.55:54440) 2024-05-10T03:30:35.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503ce00640 1 -- 172.21.15.55:0/4176540677 learned_addr learned my addr 172.21.15.55:0/4176540677 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:35.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503d800640 1 -- 172.21.15.55:0/4176540677 >> v1:172.21.15.162:6789/0 conn(0x7f50381618b0 legacy=0x7f503806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:35.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503d800640 1 -- 172.21.15.55:0/4176540677 >> v1:172.21.15.55:6789/0 conn(0x7f50380b2170 legacy=0x7f50381611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f503d800640 1 -- 172.21.15.55:0/4176540677 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380d2840 con 0x7f50380b6680 2024-05-10T03:30:35.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.006+0000 7f5037e00640 1 -- 172.21.15.55:0/4176540677 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195953763 0 0) 0x7f50380d25a0 con 0x7f50380b2170 2024-05-10T03:30:35.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.007+0000 7f503d800640 1 --2- 172.21.15.55:0/4176540677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f503815d5e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a12f28e3bc05fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.007+0000 7f5037e00640 1 -- 172.21.15.55:0/4176540677 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c007740 con 0x7f50380b6680 2024-05-10T03:30:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.008+0000 7f5037e00640 1 -- 172.21.15.55:0/4176540677 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c02d050 con 0x7f50380b6680 2024-05-10T03:30:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.008+0000 7f5037e00640 1 -- 172.21.15.55:0/4176540677 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c007ac0 con 0x7f50380b6680 2024-05-10T03:30:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.008+0000 7f503ec00640 1 -- 172.21.15.55:0/4176540677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 msgr2=0x7f503815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.008+0000 7f503ec00640 1 --2- 172.21.15.55:0/4176540677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f503815d5e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c02a6a0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.009+0000 7f503ec00640 1 -- 172.21.15.55:0/4176540677 shutdown_connections 2024-05-10T03:30:35.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.009+0000 7f503ec00640 1 --2- 172.21.15.55:0/4176540677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f503815d5e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.009+0000 7f503ec00640 1 -- 172.21.15.55:0/4176540677 >> 172.21.15.55:0/4176540677 conn(0x7f50380b5730 msgr2=0x7f50380d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:35.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.009+0000 7f503ec00640 1 -- 172.21.15.55:0/4176540677 shutdown_connections 2024-05-10T03:30:35.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.009+0000 7f503ec00640 1 -- 172.21.15.55:0/4176540677 wait complete. 2024-05-10T03:30:35.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.011+0000 7f503ec00640 1 Processor -- start 2024-05-10T03:30:35.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.011+0000 7f503ec00640 1 -- start start 2024-05-10T03:30:35.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b2170 0x7f5038160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:35.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:35.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50381618b0 0x7f50381e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:35.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503815db20 con 0x7f50380b6680 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038144450 con 0x7f50381618b0 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038159e10 con 0x7f50380b2170 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.013+0000 7f503ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50976/0 (socket says 172.21.15.55:50976) 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 -- 172.21.15.55:0/3947039165 learned_addr learned my addr 172.21.15.55:0/3947039165 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503d800640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b2170 0x7f5038160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503e200640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50381618b0 0x7f50381e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 -- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b2170 msgr2=0x7f5038160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b2170 0x7f5038160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 -- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50381618b0 msgr2=0x7f50381e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50381618b0 0x7f50381e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 -- 172.21.15.55:0/3947039165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c009a70 con 0x7f50380b6680 2024-05-10T03:30:35.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.014+0000 7f503ce00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f502400c460 tx=0x7f502400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:35.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.015+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240074e0 con 0x7f50380b6680 2024-05-10T03:30:35.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.015+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024007660 con 0x7f50380b6680 2024-05-10T03:30:35.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.015+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024004a20 con 0x7f50380b6680 2024-05-10T03:30:35.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.015+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5038156b50 con 0x7f50380b6680 2024-05-10T03:30:35.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.015+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503815a340 con 0x7f50380b6680 2024-05-10T03:30:35.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.016+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5038067740 con 0x7f50380b6680 2024-05-10T03:30:35.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.017+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5024004b80 con 0x7f50380b6680 2024-05-10T03:30:35.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.019+0000 7f5035600640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 0x7f501806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:35.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.019+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5024088e20 con 0x7f50380b6680 2024-05-10T03:30:35.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.020+0000 7f503d800640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 0x7f501806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:35.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.021+0000 7f503d800640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 0x7f501806d160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f502c034000 tx=0x7f502c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.024+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5024053b50 con 0x7f50380b6680 2024-05-10T03:30:35.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.265+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f503804e290 con 0x7f50380b6680 2024-05-10T03:30:35.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.266+0000 7f5035600640 1 -- 172.21.15.55:0/3947039165 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5024018030 con 0x7f50380b6680 2024-05-10T03:30:35.270 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T03:30:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 msgr2=0x7f501806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 0x7f501806d160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f502c034000 tx=0x7f502c02f000 comp rx=0 tx=0).stop 2024-05-10T03:30:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 msgr2=0x7f5038157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f502400c460 tx=0x7f502400c920 comp rx=0 tx=0).stop 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 shutdown_connections 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac50 0x7f501806d160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50381618b0 0x7f50381e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50380b6680 0x7f5038157000 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 --2- 172.21.15.55:0/3947039165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b2170 0x7f5038160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 >> 172.21.15.55:0/3947039165 conn(0x7f50380b5730 msgr2=0x7f50380cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 shutdown_connections 2024-05-10T03:30:35.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:35.269+0000 7f503ec00640 1 -- 172.21.15.55:0/3947039165 wait complete. 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:30:35.299 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-10T03:30:35.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.299 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:30:35.358 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:35.358 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:35.358 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:30:35.359 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42441sec preferred_lft 42441sec 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:30:35.360 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.361 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:30:35.436 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:30:35.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.440 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:30:35.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:30:35.494 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:35.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.643 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:30:35.694 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:30:35.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.695 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2024-05-10T03:30:35.746 INFO:teuthology.orchestra.run.smithi162.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-10T03:30:35.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.747 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:35.747 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:35.747 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:30:35.747 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-10T03:30:35.747 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:35.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.867 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-10T03:30:35.946 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:35.946 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:35.946 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 56:a7:63:75:07:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::54a7:63ff:fe75:7e6/64 scope link 2024-05-10T03:30:35.947 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:35.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:35.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-10T03:30:36.031 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:30:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 56:a7:63:75:07:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::54a7:63ff:fe75:7e6/64 scope link 2024-05-10T03:30:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:30:36.036 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-10T03:30:36.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-10T03:30:36.036 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:36.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.247 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:30:36.247 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:30:36.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.1 up 2024-05-10T03:30:36.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.1 master ceph-brx 2024-05-10T03:30:36.247 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:30:36.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:36.397 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-10T03:30:36.444 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-10T03:30:36.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:30:36.444 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-10T03:30:36.537 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:30:36.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T03:30:36.748 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T03:30:36.749 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T03:30:36.749 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:36.771 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T03:30:36.771 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T03:30:36.771 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:36.771 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:36.772 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:30:36.805 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:30:36.806 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:30:36.807 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-10T03:30:36.807 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:30:36.807 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:30:36.807 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:30:36.807 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:30:36.812 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:30:36.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.812 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-10T03:30:36.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.888 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:36.941 INFO:teuthology.orchestra.run.smithi162.stdout:client4520 2024-05-10T03:30:36.941 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:30:36.941 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:36.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:36.990 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:37.091 INFO:teuthology.orchestra.run.smithi162.stdout:client4520 2024-05-10T03:30:37.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:37.091 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:30:37.145 INFO:teuthology.orchestra.run.smithi162.stdout:client4520 2024-05-10T03:30:37.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:37.145 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/340d5548-dd78-4f40-a3a8-44ee99f8e0d8.client4520/status') 2024-05-10T03:30:37.226 INFO:teuthology.orchestra.run.smithi162.stdout:instance: client.4520 (3)192.168.0.2:0/2180144219 2024-05-10T03:30:37.227 INFO:teuthology.orchestra.run.smithi162.stdout:blocklisted: false 2024-05-10T03:30:37.227 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records in 2024-05-10T03:30:37.227 INFO:teuthology.orchestra.run.smithi162.stderr:0+1 records out 2024-05-10T03:30:37.227 INFO:teuthology.orchestra.run.smithi162.stderr:69 bytes copied, 7.4334e-05 s, 928 kB/s 2024-05-10T03:30:37.231 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-10T03:30:37.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:37.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.540+0000 7f5cdd000640 1 Processor -- start 2024-05-10T03:30:37.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.540+0000 7f5cdd000640 1 -- start start 2024-05-10T03:30:37.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.542+0000 7f5cdd000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5cd8144b60 0x7f5cd8162840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.542+0000 7f5cdd000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd80b1f60 con 0x7f5cd80b2d00 2024-05-10T03:30:37.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.542+0000 7f5cdd000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd8162ed0 con 0x7f5cd8147900 2024-05-10T03:30:37.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.542+0000 7f5cdd000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80b2220 con 0x7f5cd8144b60 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.543+0000 7f5cd7400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5cd80b2d00 0x7f5cd81471f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54456/0 (socket says 172.21.15.55:54456) 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.543+0000 7f5cd7400640 1 -- 172.21.15.55:0/1461859164 learned_addr learned my addr 172.21.15.55:0/1461859164 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.543+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/1461859164 >> v2:172.21.15.162:3300/0 conn(0x7f5cd8144b60 0x7f5cd8162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223408821 0 0) 0x7f5cd80b1f60 con 0x7f5cd80b2d00 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cbc003520 con 0x7f5cd80b2d00 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171937776 0 0) 0x7f5cbc003520 con 0x7f5cd80b2d00 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 >> v2:172.21.15.162:3300/0 conn(0x7f5cd8144b60 msgr2=0x7f5cd8162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 --2- 172.21.15.55:0/1461859164 >> v2:172.21.15.162:3300/0 conn(0x7f5cd8144b60 0x7f5cd8162840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 >> v1:172.21.15.162:6789/0 conn(0x7f5cd8147900 legacy=0x7f5cd8162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80b2500 con 0x7f5cd80b2d00 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f5cc8001670 con 0x7f5cd80b2d00 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 >> v1:172.21.15.55:6789/0 conn(0x7f5cd80b2d00 legacy=0x7f5cd81471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc0039e0 0x7f5cbc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc0047c0 0x7f5cbc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 0x7f5cbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80b2220 con 0x7f5cbc005310 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc00c210 con 0x7f5cbc0039e0 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.544+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc00c3d0 con 0x7f5cbc0047c0 2024-05-10T03:30:37.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 0x7f5cbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7400640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc0039e0 0x7f5cbc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd6a00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc0047c0 0x7f5cbc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 -- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc0047c0 msgr2=0x7f5cbc004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc0047c0 0x7f5cbc004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 -- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc0039e0 msgr2=0x7f5cbc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc0039e0 0x7f5cbc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7e00640 1 -- 172.21.15.55:0/1461859164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc003520 con 0x7f5cbc005310 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd7400640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cbc0039e0 0x7f5cbc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.545+0000 7f5cd6a00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc0047c0 0x7f5cbc004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.546+0000 7f5cd7e00640 1 -- 172.21.15.55:0/1461859164 reap_dead start 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.546+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 0x7f5cbc00b980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009fc0 tx=0x7f5cbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8442cde5eaae480c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003b90 con 0x7f5cbc005310 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd000f040 con 0x7f5cbc005310 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cd6000640 1 -- 172.21.15.55:0/1461859164 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004020 con 0x7f5cbc005310 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 -- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 msgr2=0x7f5cbc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 0x7f5cbc00b980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009fc0 tx=0x7f5cbc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 -- 172.21.15.55:0/1461859164 shutdown_connections 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 --2- 172.21.15.55:0/1461859164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cbc005310 0x7f5cbc00b980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 -- 172.21.15.55:0/1461859164 >> 172.21.15.55:0/1461859164 conn(0x7f5cd806a110 msgr2=0x7f5cd806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 -- 172.21.15.55:0/1461859164 shutdown_connections 2024-05-10T03:30:37.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.547+0000 7f5cdd000640 1 -- 172.21.15.55:0/1461859164 wait complete. 2024-05-10T03:30:37.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.550+0000 7f5cdd000640 1 Processor -- start 2024-05-10T03:30:37.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.550+0000 7f5cdd000640 1 -- start start 2024-05-10T03:30:37.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 0x7f5cd81496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 0x7f5cd8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8148b90 con 0x7f5cd8149c20 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd815af80 con 0x7f5cd81f7f80 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cdd000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8147cc0 con 0x7f5cbc005310 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50996/0 (socket says 172.21.15.55:50996) 2024-05-10T03:30:37.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.552+0000 7f5cd6a00640 1 -- 172.21.15.55:0/2757158735 learned_addr learned my addr 172.21.15.55:0/2757158735 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:37.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd7400640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 0x7f5cd81496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 0x7f5cd8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 -- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 msgr2=0x7f5cd81496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 0x7f5cd81496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 -- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 msgr2=0x7f5cd8159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 0x7f5cd8159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 -- 172.21.15.55:0/2757158735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8148fb0 con 0x7f5cd8149c20 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd7400640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 0x7f5cd81496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd7e00640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 0x7f5cd8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cd6a00640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d560 tx=0x7f5cc400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4005480 con 0x7f5cd8149c20 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4003610 con 0x7f5cd8149c20 2024-05-10T03:30:37.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd814a030 con 0x7f5cd8149c20 2024-05-10T03:30:37.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4003980 con 0x7f5cd8149c20 2024-05-10T03:30:37.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.553+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd81481f0 con 0x7f5cd8149c20 2024-05-10T03:30:37.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.555+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd8066ce0 con 0x7f5cd8149c20 2024-05-10T03:30:37.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.556+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5cc4003b00 con 0x7f5cd8149c20 2024-05-10T03:30:37.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.557+0000 7f5ccf400640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 0x7f5cb806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:37.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.557+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5cc40896c0 con 0x7f5cd8149c20 2024-05-10T03:30:37.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.557+0000 7f5cd7400640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 0x7f5cb806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:37.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.559+0000 7f5cd7400640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 0x7f5cb806d140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8001200 tx=0x7f5cc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:37.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.562+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5cc4054400 con 0x7f5cd8149c20 2024-05-10T03:30:37.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.763+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cd8070e60 con 0x7f5cd8149c20 2024-05-10T03:30:37.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.766+0000 7f5ccf400640 1 -- 172.21.15.55:0/2757158735 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f5cc4001230 con 0x7f5cd8149c20 2024-05-10T03:30:37.769 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:37.770 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 msgr2=0x7f5cb806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 0x7f5cb806d140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8001200 tx=0x7f5cc8009040 comp rx=0 tx=0).stop 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 msgr2=0x7f5cd81f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d560 tx=0x7f5cc400da20 comp rx=0 tx=0).stop 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 shutdown_connections 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5cb806ac30 0x7f5cb806d140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cd81f7f80 0x7f5cd8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cd8149c20 0x7f5cd81f7a40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 --2- 172.21.15.55:0/2757158735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cbc005310 0x7f5cd81496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.768+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 >> 172.21.15.55:0/2757158735 conn(0x7f5cd806a110 msgr2=0x7f5cd81631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.769+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 shutdown_connections 2024-05-10T03:30:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:37.769+0000 7f5cdd000640 1 -- 172.21.15.55:0/2757158735 wait complete. 2024-05-10T03:30:37.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:37.802 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:38.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.092+0000 7f7155a00640 1 Processor -- start 2024-05-10T03:30:38.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.092+0000 7f7155a00640 1 -- start start 2024-05-10T03:30:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f7155a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71500710f0 0x7f71501405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f7155a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7150071e00 0x7f7150140b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f7155a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715015fdf0 con 0x7f71500719f0 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f7155a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715015f1e0 con 0x7f71500710f0 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f7155a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715015f4c0 con 0x7f7150071e00 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f714fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f71500719f0 0x7f715006b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54458/0 (socket says 172.21.15.55:54458) 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f714fe00640 1 -- 172.21.15.55:0/400407585 learned_addr learned my addr 172.21.15.55:0/400407585 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.095+0000 7f714f400640 1 --2- 172.21.15.55:0/400407585 >> v2:172.21.15.55:3300/0 conn(0x7f71500710f0 0x7f71501405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f7155000640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7150071e00 0x7f7150140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714ea00640 1 -- 172.21.15.55:0/400407585 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232492232 0 0) 0x7f715015fdf0 con 0x7f71500719f0 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714ea00640 1 -- 172.21.15.55:0/400407585 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7134003520 con 0x7f71500719f0 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714f400640 1 -- 172.21.15.55:0/400407585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7150071e00 msgr2=0x7f7150140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714f400640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7150071e00 0x7f7150140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714f400640 1 -- 172.21.15.55:0/400407585 >> v1:172.21.15.55:6789/0 conn(0x7f71500719f0 legacy=0x7f715006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.096+0000 7f714f400640 1 -- 172.21.15.55:0/400407585 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715015f7a0 con 0x7f71500710f0 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.097+0000 7f714f400640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71500710f0 0x7f71501405f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f713c0097a0 tx=0x7f713c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c873b3f3269603e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.097+0000 7f714ea00640 1 -- 172.21.15.55:0/400407585 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c02ae50 con 0x7f71500710f0 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.097+0000 7f714ea00640 1 -- 172.21.15.55:0/400407585 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c032440 con 0x7f71500710f0 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f714ea00640 1 -- 172.21.15.55:0/400407585 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c0327b0 con 0x7f71500710f0 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 -- 172.21.15.55:0/400407585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71500710f0 msgr2=0x7f71501405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71500710f0 0x7f71501405f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f713c0097a0 tx=0x7f713c02a100 comp rx=0 tx=0).stop 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 -- 172.21.15.55:0/400407585 shutdown_connections 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7150071e00 0x7f7150140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 --2- 172.21.15.55:0/400407585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71500710f0 0x7f71501405f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 -- 172.21.15.55:0/400407585 >> 172.21.15.55:0/400407585 conn(0x7f715006c750 msgr2=0x7f71500b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 -- 172.21.15.55:0/400407585 shutdown_connections 2024-05-10T03:30:38.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.098+0000 7f7155a00640 1 -- 172.21.15.55:0/400407585 wait complete. 2024-05-10T03:30:38.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.101+0000 7f7155a00640 1 Processor -- start 2024-05-10T03:30:38.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.101+0000 7f7155a00640 1 -- start start 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71500710f0 0x7f7150156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71500719f0 0x7f715015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71500b2640 con 0x7f7150071e00 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715013f9a0 con 0x7f71500719f0 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71501f42f0 con 0x7f71500710f0 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51008/0 (socket says 172.21.15.55:51008) 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.103+0000 7f7155000640 1 -- 172.21.15.55:0/3210989584 learned_addr learned my addr 172.21.15.55:0/3210989584 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f714fe00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71500710f0 0x7f7150156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f714f400640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71500719f0 0x7f715015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 -- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71500710f0 msgr2=0x7f7150156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71500710f0 0x7f7150156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 -- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71500719f0 msgr2=0x7f715015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71500719f0 0x7f715015cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 -- 172.21.15.55:0/3210989584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c009400 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155000640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f714800efc0 tx=0x7f714800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148009ba0 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714800f040 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715013f6f0 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.104+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480146c0 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.105+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71501f3ca0 con 0x7f7150071e00 2024-05-10T03:30:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.106+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715004e290 con 0x7f7150071e00 2024-05-10T03:30:38.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.107+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7148014820 con 0x7f7150071e00 2024-05-10T03:30:38.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.108+0000 7f7147e00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 0x7f713006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.108+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f714808a620 con 0x7f7150071e00 2024-05-10T03:30:38.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.108+0000 7f714fe00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 0x7f713006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.110+0000 7f714fe00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 0x7f713006d110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7140005880 tx=0x7f7140009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.112+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7148055360 con 0x7f7150071e00 2024-05-10T03:30:38.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.313+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71500703d0 con 0x7f7150071e00 2024-05-10T03:30:38.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.314+0000 7f7147e00640 1 -- 172.21.15.55:0/3210989584 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7148058980 con 0x7f7150071e00 2024-05-10T03:30:38.317 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:38.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.316+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 msgr2=0x7f713006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.316+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 0x7f713006d110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7140005880 tx=0x7f7140009040 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.316+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 msgr2=0x7f715015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.316+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f714800efc0 tx=0x7f714800c490 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 shutdown_connections 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7150071e00 0x7f715015d510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f713006ac00 0x7f713006d110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71500719f0 0x7f715015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 --2- 172.21.15.55:0/3210989584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71500710f0 0x7f7150156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 >> 172.21.15.55:0/3210989584 conn(0x7f715006c750 msgr2=0x7f715006b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 shutdown_connections 2024-05-10T03:30:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.317+0000 7f7155a00640 1 -- 172.21.15.55:0/3210989584 wait complete. 2024-05-10T03:30:38.348 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.620+0000 7f2520e00640 1 Processor -- start 2024-05-10T03:30:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.621+0000 7f2520e00640 1 -- start start 2024-05-10T03:30:38.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2520e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f251c0cd750 0x7f251c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2520e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c071ce0 con 0x7f251c141a30 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2520e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c070460 con 0x7f251c070050 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2520e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c070f40 con 0x7f251c0cd750 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2513e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f251c141a30 0x7f251c15bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54472/0 (socket says 172.21.15.55:54472) 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f2513e00640 1 -- 172.21.15.55:0/3247689094 learned_addr learned my addr 172.21.15.55:0/3247689094 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.623+0000 7f251a800640 1 --2- 172.21.15.55:0/3247689094 >> v2:172.21.15.55:3300/0 conn(0x7f251c0cd750 0x7f251c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.624+0000 7f2519400640 1 -- 172.21.15.55:0/3247689094 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356499646 0 0) 0x7f2504001110 con 0x7f251c141a30 2024-05-10T03:30:38.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.624+0000 7f2519400640 1 -- 172.21.15.55:0/3247689094 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2500003520 con 0x7f251c141a30 2024-05-10T03:30:38.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.624+0000 7f251a800640 1 -- 172.21.15.55:0/3247689094 >> v1:172.21.15.162:6789/0 conn(0x7f251c070050 legacy=0x7f251c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.624+0000 7f251a800640 1 -- 172.21.15.55:0/3247689094 >> v1:172.21.15.55:6789/0 conn(0x7f251c141a30 legacy=0x7f251c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.624+0000 7f251a800640 1 -- 172.21.15.55:0/3247689094 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c141e40 con 0x7f251c0cd750 2024-05-10T03:30:38.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.625+0000 7f251a800640 1 --2- 172.21.15.55:0/3247689094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c0cd750 0x7f251c06b120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2514009d90 tx=0x7f251402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e4185d746d04e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.625+0000 7f2519400640 1 -- 172.21.15.55:0/3247689094 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514007740 con 0x7f251c0cd750 2024-05-10T03:30:38.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.626+0000 7f2519400640 1 -- 172.21.15.55:0/3247689094 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251402d050 con 0x7f251c0cd750 2024-05-10T03:30:38.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.626+0000 7f2519400640 1 -- 172.21.15.55:0/3247689094 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514007ac0 con 0x7f251c0cd750 2024-05-10T03:30:38.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.626+0000 7f2520e00640 1 -- 172.21.15.55:0/3247689094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c0cd750 msgr2=0x7f251c06b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.626+0000 7f2520e00640 1 --2- 172.21.15.55:0/3247689094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c0cd750 0x7f251c06b120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2514009d90 tx=0x7f251402a6f0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.627+0000 7f2520e00640 1 -- 172.21.15.55:0/3247689094 shutdown_connections 2024-05-10T03:30:38.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.627+0000 7f2520e00640 1 --2- 172.21.15.55:0/3247689094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c0cd750 0x7f251c06b120 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.627+0000 7f2520e00640 1 -- 172.21.15.55:0/3247689094 >> 172.21.15.55:0/3247689094 conn(0x7f251c06c120 msgr2=0x7f251c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:38.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.627+0000 7f2520e00640 1 -- 172.21.15.55:0/3247689094 shutdown_connections 2024-05-10T03:30:38.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.627+0000 7f2520e00640 1 -- 172.21.15.55:0/3247689094 wait complete. 2024-05-10T03:30:38.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.629+0000 7f2520e00640 1 Processor -- start 2024-05-10T03:30:38.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.629+0000 7f2520e00640 1 -- start start 2024-05-10T03:30:38.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 0x7f251c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f251c141a30 0x7f251c13be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1e6280 con 0x7f251c070050 2024-05-10T03:30:38.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0b1f50 con 0x7f251c0cd750 2024-05-10T03:30:38.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2520e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c13b9a0 con 0x7f251c141a30 2024-05-10T03:30:38.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2519e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2519e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51026/0 (socket says 172.21.15.55:51026) 2024-05-10T03:30:38.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2519e00640 1 -- 172.21.15.55:0/4282833127 learned_addr learned my addr 172.21.15.55:0/4282833127 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.631+0000 7f2513e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 0x7f251c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f251a800640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f251c141a30 0x7f251c13be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 -- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f251c141a30 msgr2=0x7f251c13be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f251c141a30 0x7f251c13be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 -- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 msgr2=0x7f251c156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 0x7f251c156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 -- 172.21.15.55:0/4282833127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009a70 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2513e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 0x7f251c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2519e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c3c0 tx=0x7f250c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c0052d0 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c005450 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c003830 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c13c370 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.632+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c13b070 con 0x7f251c070050 2024-05-10T03:30:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.634+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c066f60 con 0x7f251c070050 2024-05-10T03:30:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.635+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f250c003990 con 0x7f251c070050 2024-05-10T03:30:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.636+0000 7f2512000640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 0x7f24f806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.636+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f250c088a40 con 0x7f251c070050 2024-05-10T03:30:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.637+0000 7f2513e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 0x7f24f806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.638+0000 7f2513e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 0x7f24f806d110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f251c15cfd0 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:38.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.641+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f250c08ae60 con 0x7f251c070050 2024-05-10T03:30:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.844+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f251c070460 con 0x7f251c070050 2024-05-10T03:30:38.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.845+0000 7f2512000640 1 -- 172.21.15.55:0/4282833127 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f251c070460 con 0x7f251c070050 2024-05-10T03:30:38.848 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.847+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 msgr2=0x7f24f806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.847+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 0x7f24f806d110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f251c15cfd0 tx=0x7f2504009040 comp rx=0 tx=0).stop 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.847+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 msgr2=0x7f251c0ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.847+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c3c0 tx=0x7f250c00c880 comp rx=0 tx=0).stop 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 shutdown_connections 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f251c141a30 0x7f251c13be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24f806ac00 0x7f24f806d110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f251c0cd750 0x7f251c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 --2- 172.21.15.55:0/4282833127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f251c070050 0x7f251c0ae1b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:38.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 >> 172.21.15.55:0/4282833127 conn(0x7f251c06c120 msgr2=0x7f251c15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:38.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 shutdown_connections 2024-05-10T03:30:38.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:38.848+0000 7f2520e00640 1 -- 172.21.15.55:0/4282833127 wait complete. 2024-05-10T03:30:38.881 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:39.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.164+0000 7fd8ef800640 1 Processor -- start 2024-05-10T03:30:39.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.165+0000 7fd8ef800640 1 -- start start 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ef800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8e815fe50 0x7fd8e8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ef800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ef800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e806ad20 con 0x7fd8e80b2140 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ef800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e8141f50 con 0x7fd8e815fe50 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ef800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e806a110 con 0x7fd8e80b2550 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.167+0000 7fd8ece00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8e815fe50 0x7fd8e8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8e815fe50 0x7fd8e8141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51030/0 (socket says 172.21.15.55:51030) 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 -- 172.21.15.55:0/1357160105 learned_addr learned my addr 172.21.15.55:0/1357160105 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ed800640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 -- 172.21.15.55:0/1357160105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 msgr2=0x7fd8e806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 -- 172.21.15.55:0/1357160105 >> v1:172.21.15.162:6789/0 conn(0x7fd8e80b2140 legacy=0x7fd8e8142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.168+0000 7fd8ece00640 1 -- 172.21.15.55:0/1357160105 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e806a3f0 con 0x7fd8e815fe50 2024-05-10T03:30:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.169+0000 7fd8ece00640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e815fe50 0x7fd8e8141a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009870 tx=0x7fd8dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ce4c6a7cbd53da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.170+0000 7fd8e7400640 1 -- 172.21.15.55:0/1357160105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032640 con 0x7fd8e815fe50 2024-05-10T03:30:39.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.170+0000 7fd8e7400640 1 -- 172.21.15.55:0/1357160105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0327a0 con 0x7fd8e815fe50 2024-05-10T03:30:39.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.170+0000 7fd8e7400640 1 -- 172.21.15.55:0/1357160105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc032b30 con 0x7fd8e815fe50 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.170+0000 7fd8ef800640 1 -- 172.21.15.55:0/1357160105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e815fe50 msgr2=0x7fd8e8141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.170+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e815fe50 0x7fd8e8141a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009870 tx=0x7fd8dc02a300 comp rx=0 tx=0).stop 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 -- 172.21.15.55:0/1357160105 shutdown_connections 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1357160105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e815fe50 0x7fd8e8141a10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 -- 172.21.15.55:0/1357160105 >> 172.21.15.55:0/1357160105 conn(0x7fd8e80cbb90 msgr2=0x7fd8e80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:39.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 -- 172.21.15.55:0/1357160105 shutdown_connections 2024-05-10T03:30:39.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.171+0000 7fd8ef800640 1 -- 172.21.15.55:0/1357160105 wait complete. 2024-05-10T03:30:39.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.173+0000 7fd8ef800640 1 Processor -- start 2024-05-10T03:30:39.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.173+0000 7fd8ef800640 1 -- start start 2024-05-10T03:30:39.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 0x7fd8e813b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e81f4eb0 con 0x7fd8e80b2140 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80701d0 con 0x7fd8e815fe50 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ef800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80704b0 con 0x7fd8e80b2550 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ece00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ece00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51040/0 (socket says 172.21.15.55:51040) 2024-05-10T03:30:39.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.175+0000 7fd8ece00640 1 -- 172.21.15.55:0/1927494317 learned_addr learned my addr 172.21.15.55:0/1927494317 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:39.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ed800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 0x7fd8e813b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8e7e00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 -- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 msgr2=0x7fd8e815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 -- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 msgr2=0x7fd8e813b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 0x7fd8e813b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 -- 172.21.15.55:0/1927494317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0094d0 con 0x7fd8e80b2140 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ed800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 0x7fd8e813b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8e7e00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8ece00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.176+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc030850 con 0x7fd8e80b2140 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.177+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0309b0 con 0x7fd8e80b2140 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.177+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc030d60 con 0x7fd8e80b2140 2024-05-10T03:30:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.177+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e8070790 con 0x7fd8e80b2140 2024-05-10T03:30:39.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.177+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e81f4bf0 con 0x7fd8e80b2140 2024-05-10T03:30:39.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.178+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e8066f60 con 0x7fd8e80b2140 2024-05-10T03:30:39.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.179+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd8dc039040 con 0x7fd8e80b2140 2024-05-10T03:30:39.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.180+0000 7fd8e4c00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 0x7fd8c806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.180+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd8dc0aa320 con 0x7fd8e80b2140 2024-05-10T03:30:39.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.180+0000 7fd8e7e00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 0x7fd8c806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.182+0000 7fd8e7e00640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 0x7fd8c806d110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8e815e0a0 tx=0x7fd8d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.185+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd8dc075060 con 0x7fd8e80b2140 2024-05-10T03:30:39.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.381+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8e806b120 con 0x7fd8e80b2140 2024-05-10T03:30:39.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.382+0000 7fd8e4c00640 1 -- 172.21.15.55:0/1927494317 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8dc078680 con 0x7fd8e80b2140 2024-05-10T03:30:39.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.384+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 msgr2=0x7fd8c806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.384+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 0x7fd8c806d110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8e815e0a0 tx=0x7fd8d4009040 comp rx=0 tx=0).stop 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.384+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 msgr2=0x7fd8e815ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.384+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 shutdown_connections 2024-05-10T03:30:39.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8c806ac00 0x7fd8c806d110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e815fe50 0x7fd8e813b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e80b2550 0x7fd8e815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 --2- 172.21.15.55:0/1927494317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8e80b2140 0x7fd8e815ae90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 >> 172.21.15.55:0/1927494317 conn(0x7fd8e80cbb90 msgr2=0x7fd8e80aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 shutdown_connections 2024-05-10T03:30:39.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.385+0000 7fd8ef800640 1 -- 172.21.15.55:0/1927494317 wait complete. 2024-05-10T03:30:39.416 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.704+0000 7f2296a00640 1 Processor -- start 2024-05-10T03:30:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.704+0000 7f2296a00640 1 -- start start 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.706+0000 7f2296a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b23f0 0x7f2290161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f2296a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22900ce740 0x7f229015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f2296a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900d29f0 con 0x7f22900b8d70 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f2296a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22901623d0 con 0x7f22900b23f0 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f2296a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22900d1de0 con 0x7f22900ce740 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b23f0 0x7f2290161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b23f0 0x7f2290161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51042/0 (socket says 172.21.15.55:51042) 2024-05-10T03:30:39.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 -- 172.21.15.55:0/1018083249 learned_addr learned my addr 172.21.15.55:0/1018083249 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:39.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 -- 172.21.15.55:0/1018083249 >> v2:172.21.15.162:3300/0 conn(0x7f22900ce740 msgr2=0x7f229015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:39.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 --2- 172.21.15.55:0/1018083249 >> v2:172.21.15.162:3300/0 conn(0x7f22900ce740 0x7f229015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 -- 172.21.15.55:0/1018083249 >> v1:172.21.15.162:6789/0 conn(0x7f22900b8d70 legacy=0x7f22900ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:39.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.707+0000 7f228fe00640 1 -- 172.21.15.55:0/1018083249 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900d20c0 con 0x7f22900b23f0 2024-05-10T03:30:39.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.709+0000 7f228fe00640 1 --2- 172.21.15.55:0/1018083249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f2290161e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b35bf1c1e8e81c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.709+0000 7f228f400640 1 -- 172.21.15.55:0/1018083249 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280032440 con 0x7f22900b23f0 2024-05-10T03:30:39.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.709+0000 7f228f400640 1 -- 172.21.15.55:0/1018083249 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22800325a0 con 0x7f22900b23f0 2024-05-10T03:30:39.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.709+0000 7f228f400640 1 -- 172.21.15.55:0/1018083249 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280032930 con 0x7f22900b23f0 2024-05-10T03:30:39.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 -- 172.21.15.55:0/1018083249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 msgr2=0x7f2290161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 --2- 172.21.15.55:0/1018083249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f2290161e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228002a250 comp rx=0 tx=0).stop 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 -- 172.21.15.55:0/1018083249 shutdown_connections 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 --2- 172.21.15.55:0/1018083249 >> v2:172.21.15.162:3300/0 conn(0x7f22900ce740 0x7f229015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 --2- 172.21.15.55:0/1018083249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f2290161e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 -- 172.21.15.55:0/1018083249 >> 172.21.15.55:0/1018083249 conn(0x7f22900d0850 msgr2=0x7f22900d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 -- 172.21.15.55:0/1018083249 shutdown_connections 2024-05-10T03:30:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.710+0000 7f2296a00640 1 -- 172.21.15.55:0/1018083249 wait complete. 2024-05-10T03:30:39.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.712+0000 7f2296a00640 1 Processor -- start 2024-05-10T03:30:39.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.713+0000 7f2296a00640 1 -- start start 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 0x7f2290146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 0x7f2290146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229015a420 con 0x7f22900b23f0 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290145530 con 0x7f22900b8d70 2024-05-10T03:30:39.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f2296a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22901472a0 con 0x7f2290159d20 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.714+0000 7f228fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51050/0 (socket says 172.21.15.55:51050) 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 -- 172.21.15.55:0/3348232068 learned_addr learned my addr 172.21.15.55:0/3348232068 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 -- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 msgr2=0x7f2290146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f2294a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 0x7f2290146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f2287e00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 0x7f2290146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 0x7f2290146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 -- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 msgr2=0x7f2290146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 0x7f2290146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 -- 172.21.15.55:0/3348232068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22800094d0 con 0x7f22900b23f0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f2294a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 0x7f2290146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f2287e00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 0x7f2290146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.715+0000 7f228fe00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f22800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.716+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280004230 con 0x7f22900b23f0 2024-05-10T03:30:39.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.716+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900b9300 con 0x7f22900b23f0 2024-05-10T03:30:39.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.716+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2280004390 con 0x7f22900b23f0 2024-05-10T03:30:39.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.716+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280030b40 con 0x7f22900b23f0 2024-05-10T03:30:39.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.716+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290145a60 con 0x7f22900b23f0 2024-05-10T03:30:39.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.717+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290067220 con 0x7f22900b23f0 2024-05-10T03:30:39.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.718+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2280039040 con 0x7f22900b23f0 2024-05-10T03:30:39.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.719+0000 7f228cc00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 0x7f226c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:39.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.719+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f22800a9f90 con 0x7f22900b23f0 2024-05-10T03:30:39.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.720+0000 7f2287e00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 0x7f226c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:39.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.721+0000 7f2287e00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 0x7f226c06d140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f229014bbb0 tx=0x7f2278008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:39.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.724+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f22800dd900 con 0x7f22900b23f0 2024-05-10T03:30:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.931+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2290067220 con 0x7f22900b23f0 2024-05-10T03:30:39.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.931+0000 7f228cc00640 1 -- 172.21.15.55:0/3348232068 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f228003f0a0 con 0x7f22900b23f0 2024-05-10T03:30:39.935 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:39.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.934+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 msgr2=0x7f226c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.934+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 0x7f226c06d140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f229014bbb0 tx=0x7f2278008040 comp rx=0 tx=0).stop 2024-05-10T03:30:39.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.934+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 msgr2=0x7f22900c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:39.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.934+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f22800046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 shutdown_connections 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2290159d20 0x7f2290146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f226c06ac30 0x7f226c06d140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22900b8d70 0x7f2290146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 --2- 172.21.15.55:0/3348232068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b23f0 0x7f22900c6560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 >> 172.21.15.55:0/3348232068 conn(0x7f22900d0850 msgr2=0x7f22900b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 shutdown_connections 2024-05-10T03:30:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:39.935+0000 7f2296a00640 1 -- 172.21.15.55:0/3348232068 wait complete. 2024-05-10T03:30:39.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:40.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.242+0000 7f474be00640 1 Processor -- start 2024-05-10T03:30:40.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.243+0000 7f474be00640 1 -- start start 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.245+0000 7f474be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474c15a5d0 0x7f474c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.245+0000 7f474be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f474c0b4880 0x7f474c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.245+0000 7f474be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c15ad30 con 0x7f474c0cb730 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.245+0000 7f474be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c15b010 con 0x7f474c15a5d0 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.245+0000 7f474be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c15b2f0 con 0x7f474c0b4880 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474c15a5d0 0x7f474c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474c15a5d0 0x7f474c0cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51058/0 (socket says 172.21.15.55:51058) 2024-05-10T03:30:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 -- 172.21.15.55:0/2921577573 learned_addr learned my addr 172.21.15.55:0/2921577573 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474b400640 1 --2- 172.21.15.55:0/2921577573 >> v2:172.21.15.162:3300/0 conn(0x7f474c0b4880 0x7f474c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 -- 172.21.15.55:0/2921577573 >> v2:172.21.15.162:3300/0 conn(0x7f474c0b4880 msgr2=0x7f474c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 --2- 172.21.15.55:0/2921577573 >> v2:172.21.15.162:3300/0 conn(0x7f474c0b4880 0x7f474c0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 -- 172.21.15.55:0/2921577573 >> v1:172.21.15.162:6789/0 conn(0x7f474c0cb730 legacy=0x7f474c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.246+0000 7f474aa00640 1 -- 172.21.15.55:0/2921577573 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c15b5d0 con 0x7f474c15a5d0 2024-05-10T03:30:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.247+0000 7f474aa00640 1 --2- 172.21.15.55:0/2921577573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c15a5d0 0x7f474c0cb1f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f474c0caff0 tx=0x7f473c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a55a49eb149fe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.248+0000 7f474a000640 1 -- 172.21.15.55:0/2921577573 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c02f070 con 0x7f474c15a5d0 2024-05-10T03:30:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.248+0000 7f474a000640 1 -- 172.21.15.55:0/2921577573 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c0325a0 con 0x7f474c15a5d0 2024-05-10T03:30:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.248+0000 7f474a000640 1 -- 172.21.15.55:0/2921577573 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c032930 con 0x7f474c15a5d0 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.248+0000 7f474be00640 1 -- 172.21.15.55:0/2921577573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c15a5d0 msgr2=0x7f474c0cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.248+0000 7f474be00640 1 --2- 172.21.15.55:0/2921577573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c15a5d0 0x7f474c0cb1f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f474c0caff0 tx=0x7f473c02c8e0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 -- 172.21.15.55:0/2921577573 shutdown_connections 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 --2- 172.21.15.55:0/2921577573 >> v2:172.21.15.162:3300/0 conn(0x7f474c0b4880 0x7f474c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 --2- 172.21.15.55:0/2921577573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c15a5d0 0x7f474c0cb1f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 -- 172.21.15.55:0/2921577573 >> 172.21.15.55:0/2921577573 conn(0x7f474c06a110 msgr2=0x7f474c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 -- 172.21.15.55:0/2921577573 shutdown_connections 2024-05-10T03:30:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.249+0000 7f474be00640 1 -- 172.21.15.55:0/2921577573 wait complete. 2024-05-10T03:30:40.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.251+0000 7f474be00640 1 Processor -- start 2024-05-10T03:30:40.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.251+0000 7f474be00640 1 -- start start 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474c0cb730 0x7f474c1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474c15a5d0 0x7f474c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1557a0 con 0x7f474c1592d0 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c157fe0 con 0x7f474c15a5d0 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c158e40 con 0x7f474c0cb730 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51072/0 (socket says 172.21.15.55:51072) 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.253+0000 7f474b400640 1 -- 172.21.15.55:0/2212699239 learned_addr learned my addr 172.21.15.55:0/2212699239 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474aa00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474c0cb730 0x7f474c1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f4743e00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474c15a5d0 0x7f474c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 -- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474c0cb730 msgr2=0x7f474c1476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474c0cb730 0x7f474c1476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 -- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474c15a5d0 msgr2=0x7f474c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474c15a5d0 0x7f474c06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 -- 172.21.15.55:0/2212699239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c00b770 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.254+0000 7f474b400640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f474400cc60 tx=0x7f474400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.255+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400ce70 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.255+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744003990 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.255+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744003d20 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.255+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c158230 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.255+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c1587f0 con 0x7f474c1592d0 2024-05-10T03:30:40.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.256+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474c066ce0 con 0x7f474c1592d0 2024-05-10T03:30:40.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.258+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4744003e80 con 0x7f474c1592d0 2024-05-10T03:30:40.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.258+0000 7f4742a00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 0x7f472806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.259+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4744089c70 con 0x7f474c1592d0 2024-05-10T03:30:40.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.259+0000 7f474aa00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 0x7f472806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.261+0000 7f474aa00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 0x7f472806d140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f473c037000 tx=0x7f473c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.263+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4744054920 con 0x7f474c1592d0 2024-05-10T03:30:40.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.481+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f474c04e290 con 0x7f474c1592d0 2024-05-10T03:30:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.483+0000 7f4742a00640 1 -- 172.21.15.55:0/2212699239 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f474401d030 con 0x7f474c1592d0 2024-05-10T03:30:40.487 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:40.487 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 msgr2=0x7f472806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 0x7f472806d140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f473c037000 tx=0x7f473c032000 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 msgr2=0x7f474c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f474400cc60 tx=0x7f474400a5c0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 shutdown_connections 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474c1592d0 0x7f474c06d110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac30 0x7f472806d140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474c15a5d0 0x7f474c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 --2- 172.21.15.55:0/2212699239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474c0cb730 0x7f474c1476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 >> 172.21.15.55:0/2212699239 conn(0x7f474c06a110 msgr2=0x7f474c15e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 shutdown_connections 2024-05-10T03:30:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.486+0000 7f474be00640 1 -- 172.21.15.55:0/2212699239 wait complete. 2024-05-10T03:30:40.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:40.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.806+0000 7f073a600640 1 Processor -- start 2024-05-10T03:30:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.806+0000 7f073a600640 1 -- start start 2024-05-10T03:30:40.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.808+0000 7f073a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.808+0000 7f073a600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 0x7f07340700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.808+0000 7f073a600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073415fdf0 con 0x7f0734071e00 2024-05-10T03:30:40.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.808+0000 7f073a600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073415f1e0 con 0x7f073406b840 2024-05-10T03:30:40.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.808+0000 7f073a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073415f4c0 con 0x7f07340719f0 2024-05-10T03:30:40.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.809+0000 7f0739200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.809+0000 7f0739200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48792/0 (socket says 172.21.15.55:48792) 2024-05-10T03:30:40.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.809+0000 7f0739200640 1 -- 172.21.15.55:0/469871750 learned_addr learned my addr 172.21.15.55:0/469871750 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:40.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.809+0000 7f0733e00640 1 --2- 172.21.15.55:0/469871750 >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 0x7f07340700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0733400640 1 -- 172.21.15.55:0/469871750 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526369122 0 0) 0x7f073415fdf0 con 0x7f0734071e00 2024-05-10T03:30:40.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0733400640 1 -- 172.21.15.55:0/469871750 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c003520 con 0x7f0734071e00 2024-05-10T03:30:40.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0739200640 1 -- 172.21.15.55:0/469871750 >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 msgr2=0x7f07340700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:40.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0739200640 1 --2- 172.21.15.55:0/469871750 >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 0x7f07340700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0739200640 1 -- 172.21.15.55:0/469871750 >> v1:172.21.15.162:6789/0 conn(0x7f0734071e00 legacy=0x7f0734070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0739200640 1 -- 172.21.15.55:0/469871750 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073415f7a0 con 0x7f07340719f0 2024-05-10T03:30:40.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.810+0000 7f0733e00640 1 --2- 172.21.15.55:0/469871750 >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 0x7f07340700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:40.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.811+0000 7f0739200640 1 --2- 172.21.15.55:0/469871750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0728009870 tx=0x7f072802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954908f786b5a2b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.812+0000 7f0733400640 1 -- 172.21.15.55:0/469871750 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728032440 con 0x7f07340719f0 2024-05-10T03:30:40.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.812+0000 7f0733400640 1 -- 172.21.15.55:0/469871750 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07280325a0 con 0x7f07340719f0 2024-05-10T03:30:40.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.812+0000 7f0733400640 1 -- 172.21.15.55:0/469871750 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728032930 con 0x7f07340719f0 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 -- 172.21.15.55:0/469871750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 msgr2=0x7f073406b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 --2- 172.21.15.55:0/469871750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0728009870 tx=0x7f072802a1d0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 -- 172.21.15.55:0/469871750 shutdown_connections 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 --2- 172.21.15.55:0/469871750 >> v2:172.21.15.162:3300/0 conn(0x7f073406b840 0x7f07340700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 --2- 172.21.15.55:0/469871750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07340719f0 0x7f073406b300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 -- 172.21.15.55:0/469871750 >> 172.21.15.55:0/469871750 conn(0x7f073406c750 msgr2=0x7f07340b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.813+0000 7f073a600640 1 -- 172.21.15.55:0/469871750 shutdown_connections 2024-05-10T03:30:40.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.814+0000 7f073a600640 1 -- 172.21.15.55:0/469871750 wait complete. 2024-05-10T03:30:40.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.816+0000 7f073a600640 1 Processor -- start 2024-05-10T03:30:40.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.816+0000 7f073a600640 1 -- start start 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073406b840 0x7f073413b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0734071e00 0x7f07340b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c8170 con 0x7f07340719f0 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073413e4f0 con 0x7f073406b840 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f073a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c69a0 con 0x7f0734071e00 2024-05-10T03:30:40.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f0733e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f0733e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51076/0 (socket says 172.21.15.55:51076) 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.818+0000 7f0733e00640 1 -- 172.21.15.55:0/3267794175 learned_addr learned my addr 172.21.15.55:0/3267794175 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0739200640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073406b840 0x7f073413b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0739c00640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0734071e00 0x7f07340b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 -- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0734071e00 msgr2=0x7f07340b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0734071e00 0x7f07340b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 -- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073406b840 msgr2=0x7f073413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073406b840 0x7f073413b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 -- 172.21.15.55:0/3267794175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720009400 con 0x7f07340719f0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0733e00640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f072000e900 tx=0x7f072000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.819+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720009660 con 0x7f07340719f0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.820+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000ca00 con 0x7f07340719f0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.820+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07280094d0 con 0x7f07340719f0 2024-05-10T03:30:40.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.820+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072000cdb0 con 0x7f07340719f0 2024-05-10T03:30:40.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.820+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073413ea20 con 0x7f07340719f0 2024-05-10T03:30:40.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.821+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340680c0 con 0x7f07340719f0 2024-05-10T03:30:40.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.822+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0720013470 con 0x7f07340719f0 2024-05-10T03:30:40.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.824+0000 7f0730c00640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 0x7f071406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:40.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.824+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f0720089b70 con 0x7f07340719f0 2024-05-10T03:30:40.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.824+0000 7f0739200640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 0x7f071406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:40.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.826+0000 7f0739200640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 0x7f071406d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0728034000 tx=0x7f072802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:40.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:40.828+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f07200547e0 con 0x7f07340719f0 2024-05-10T03:30:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.028+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f073404e290 con 0x7f07340719f0 2024-05-10T03:30:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.029+0000 7f0730c00640 1 -- 172.21.15.55:0/3267794175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0720057e00 con 0x7f07340719f0 2024-05-10T03:30:41.032 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 msgr2=0x7f071406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 0x7f071406d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0728034000 tx=0x7f072802e000 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 msgr2=0x7f073413d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f072000e900 tx=0x7f072000edc0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 shutdown_connections 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0734071e00 0x7f07340b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07340719f0 0x7f073413d9d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071406ac00 0x7f071406d110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 --2- 172.21.15.55:0/3267794175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073406b840 0x7f073413b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 >> 172.21.15.55:0/3267794175 conn(0x7f073406c750 msgr2=0x7f073415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.032+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 shutdown_connections 2024-05-10T03:30:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.033+0000 7f073a600640 1 -- 172.21.15.55:0/3267794175 wait complete. 2024-05-10T03:30:41.065 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:41.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.348+0000 7f8d2be00640 1 Processor -- start 2024-05-10T03:30:41.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.348+0000 7f8d2be00640 1 -- start start 2024-05-10T03:30:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.350+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.350+0000 7f8d2be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8d2c0cdb70 0x7f8d2c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.350+0000 7f8d2be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b2a30 con 0x7f8d2c15e920 2024-05-10T03:30:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.350+0000 7f8d2be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c15a900 con 0x7f8d2c0cdb70 2024-05-10T03:30:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.350+0000 7f8d2be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15eec0 con 0x7f8d2c06b830 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.351+0000 7f8d2a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.351+0000 7f8d2a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48804/0 (socket says 172.21.15.55:48804) 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.351+0000 7f8d2a000640 1 -- 172.21.15.55:0/205741485 learned_addr learned my addr 172.21.15.55:0/205741485 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.351+0000 7f8d2aa00640 1 --1- 172.21.15.55:0/205741485 >> v1:172.21.15.162:6789/0 conn(0x7f8d2c15e920 0x7f8d2c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:56236/0 (socket says 172.21.15.55:56236) 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.351+0000 7f8d2b400640 1 --2- 172.21.15.55:0/205741485 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c0cdb70 0x7f8d2c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.352+0000 7f8d2a000640 1 -- 172.21.15.55:0/205741485 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c0cdb70 msgr2=0x7f8d2c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.352+0000 7f8d2a000640 1 --2- 172.21.15.55:0/205741485 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c0cdb70 0x7f8d2c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.352+0000 7f8d2a000640 1 -- 172.21.15.55:0/205741485 >> v1:172.21.15.162:6789/0 conn(0x7f8d2c15e920 legacy=0x7f8d2c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.352+0000 7f8d2a000640 1 -- 172.21.15.55:0/205741485 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c15b8b0 con 0x7f8d2c06b830 2024-05-10T03:30:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.353+0000 7f8d2a000640 1 --2- 172.21.15.55:0/205741485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800bb10 tx=0x7f8d1802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcd92893b6a1d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.354+0000 7f8d29600640 1 -- 172.21.15.55:0/205741485 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18032820 con 0x7f8d2c06b830 2024-05-10T03:30:41.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.354+0000 7f8d29600640 1 -- 172.21.15.55:0/205741485 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802e040 con 0x7f8d2c06b830 2024-05-10T03:30:41.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.354+0000 7f8d29600640 1 -- 172.21.15.55:0/205741485 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18032ba0 con 0x7f8d2c06b830 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 -- 172.21.15.55:0/205741485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 msgr2=0x7f8d2c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 --2- 172.21.15.55:0/205741485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800bb10 tx=0x7f8d1802c920 comp rx=0 tx=0).stop 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 -- 172.21.15.55:0/205741485 shutdown_connections 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 --2- 172.21.15.55:0/205741485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b830 0x7f8d2c0cc240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 --2- 172.21.15.55:0/205741485 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c0cdb70 0x7f8d2c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 -- 172.21.15.55:0/205741485 >> 172.21.15.55:0/205741485 conn(0x7f8d2c06c750 msgr2=0x7f8d2c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 -- 172.21.15.55:0/205741485 shutdown_connections 2024-05-10T03:30:41.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.355+0000 7f8d2be00640 1 -- 172.21.15.55:0/205741485 wait complete. 2024-05-10T03:30:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.358+0000 7f8d2be00640 1 Processor -- start 2024-05-10T03:30:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.359+0000 7f8d2be00640 1 -- start start 2024-05-10T03:30:41.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 0x7f8d2c13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 0x7f8d2c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c141d90 con 0x7f8d2c13c740 2024-05-10T03:30:41.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c3df0 con 0x7f8d2c0cdb70 2024-05-10T03:30:41.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c4030 con 0x7f8d2c15e920 2024-05-10T03:30:41.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51084/0 (socket says 172.21.15.55:51084) 2024-05-10T03:30:41.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2b400640 1 -- 172.21.15.55:0/1087177294 learned_addr learned my addr 172.21.15.55:0/1087177294 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2a000640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 0x7f8d2c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2aa00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 0x7f8d2c13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.361+0000 7f8d2b400640 1 -- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 msgr2=0x7f8d2c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2b400640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 0x7f8d2c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2b400640 1 -- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 msgr2=0x7f8d2c13bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2b400640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 0x7f8d2c13bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2b400640 1 -- 172.21.15.55:0/1087177294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1800b770 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2aa00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 0x7f8d2c13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2a000640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 0x7f8d2c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2b400640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400efc0 tx=0x7f8d2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24009b20 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400f040 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c158950 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240146c0 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.362+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c158020 con 0x7f8d2c13c740 2024-05-10T03:30:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.364+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c158ae0 con 0x7f8d2c13c740 2024-05-10T03:30:41.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.365+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d24014820 con 0x7f8d2c13c740 2024-05-10T03:30:41.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.366+0000 7f8d23400640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 0x7f8d0c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.366+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8d2408a760 con 0x7f8d2c13c740 2024-05-10T03:30:41.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.366+0000 7f8d2aa00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 0x7f8d0c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.368+0000 7f8d2aa00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 0x7f8d0c06d110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000f60 tx=0x7f8d1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.370+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8d24055420 con 0x7f8d2c13c740 2024-05-10T03:30:41.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.573+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d2c04e290 con 0x7f8d2c13c740 2024-05-10T03:30:41.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.574+0000 7f8d23400640 1 -- 172.21.15.55:0/1087177294 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d24009e60 con 0x7f8d2c13c740 2024-05-10T03:30:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 msgr2=0x7f8d0c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 0x7f8d0c06d110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c000f60 tx=0x7f8d1c009040 comp rx=0 tx=0).stop 2024-05-10T03:30:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 msgr2=0x7f8d2c0cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400efc0 tx=0x7f8d2400c490 comp rx=0 tx=0).stop 2024-05-10T03:30:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 shutdown_connections 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c13c740 0x7f8d2c0cb6f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c15e920 0x7f8d2c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.577+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0c06ac00 0x7f8d0c06d110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.578+0000 7f8d2be00640 1 --2- 172.21.15.55:0/1087177294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c0cdb70 0x7f8d2c13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.578+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 >> 172.21.15.55:0/1087177294 conn(0x7f8d2c06c750 msgr2=0x7f8d2c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.578+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 shutdown_connections 2024-05-10T03:30:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.578+0000 7f8d2be00640 1 -- 172.21.15.55:0/1087177294 wait complete. 2024-05-10T03:30:41.612 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:41.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.893+0000 7f5ac6e00640 1 Processor -- start 2024-05-10T03:30:41.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.894+0000 7f5ac6e00640 1 -- start start 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac0143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac01435d0 0x7f5ac00ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5ac006b120 0x7f5ac006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac00cdbe0 con 0x7f5ac01435d0 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac015e8b0 con 0x7f5ac006b120 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.895+0000 7f5ac6e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac00ce140 con 0x7f5ac00b2040 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac01435d0 0x7f5ac00ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ac01435d0 0x7f5ac00ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51098/0 (socket says 172.21.15.55:51098) 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5000640 1 -- 172.21.15.55:0/3459748419 learned_addr learned my addr 172.21.15.55:0/3459748419 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5a00640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac6400640 1 --2- 172.21.15.55:0/3459748419 >> v2:172.21.15.162:3300/0 conn(0x7f5ac006b120 0x7f5ac006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5000640 1 -- 172.21.15.55:0/3459748419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 msgr2=0x7f5ac0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.896+0000 7f5ac5000640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.897+0000 7f5ac5000640 1 -- 172.21.15.55:0/3459748419 >> v2:172.21.15.162:3300/0 conn(0x7f5ac006b120 msgr2=0x7f5ac006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.897+0000 7f5ac5000640 1 --2- 172.21.15.55:0/3459748419 >> v2:172.21.15.162:3300/0 conn(0x7f5ac006b120 0x7f5ac006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.897+0000 7f5ac5000640 1 -- 172.21.15.55:0/3459748419 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac006ba70 con 0x7f5ac01435d0 2024-05-10T03:30:41.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.898+0000 7f5ac5000640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac01435d0 0x7f5ac00ce5d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009880 tx=0x7f5aac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2a91a618da2060 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.898+0000 7f5abfe00640 1 -- 172.21.15.55:0/3459748419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac02ae90 con 0x7f5ac01435d0 2024-05-10T03:30:41.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.898+0000 7f5abfe00640 1 -- 172.21.15.55:0/3459748419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac004030 con 0x7f5ac01435d0 2024-05-10T03:30:41.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.898+0000 7f5abfe00640 1 -- 172.21.15.55:0/3459748419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0043e0 con 0x7f5ac01435d0 2024-05-10T03:30:41.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 -- 172.21.15.55:0/3459748419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac01435d0 msgr2=0x7f5ac00ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac01435d0 0x7f5ac00ce5d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009880 tx=0x7f5aac02a180 comp rx=0 tx=0).stop 2024-05-10T03:30:41.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 -- 172.21.15.55:0/3459748419 shutdown_connections 2024-05-10T03:30:41.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/3459748419 >> v2:172.21.15.162:3300/0 conn(0x7f5ac006b120 0x7f5ac006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac01435d0 0x7f5ac00ce5d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/3459748419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 -- 172.21.15.55:0/3459748419 >> 172.21.15.55:0/3459748419 conn(0x7f5ac006c730 msgr2=0x7f5ac00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:41.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 -- 172.21.15.55:0/3459748419 shutdown_connections 2024-05-10T03:30:41.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.899+0000 7f5ac6e00640 1 -- 172.21.15.55:0/3459748419 wait complete. 2024-05-10T03:30:41.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.901+0000 7f5ac6e00640 1 Processor -- start 2024-05-10T03:30:41.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.902+0000 7f5ac6e00640 1 -- start start 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac00ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 0x7f5ac0141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f6d60 con 0x7f5ac006b120 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f40f0 con 0x7f5ac01435d0 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac6e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac00aa050 con 0x7f5ac00b2040 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac5a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac5a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51108/0 (socket says 172.21.15.55:51108) 2024-05-10T03:30:41.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.903+0000 7f5ac5a00640 1 -- 172.21.15.55:0/730663170 learned_addr learned my addr 172.21.15.55:0/730663170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:41.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5000640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac00ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac6400640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 0x7f5ac0141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 -- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 msgr2=0x7f5ac00ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac00ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 -- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 msgr2=0x7f5ac0141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 0x7f5ac0141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 -- 172.21.15.55:0/730663170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac009460 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac6400640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 0x7f5ac0141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5ac5a00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400d5f0 tx=0x7f5ab400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.904+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40125f0 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.905+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4012750 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.905+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4012b00 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.905+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac00aaeb0 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.905+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac00aa580 con 0x7f5ac006b120 2024-05-10T03:30:41.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.906+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac00b1680 con 0x7f5ac006b120 2024-05-10T03:30:41.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.907+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5ab4012c60 con 0x7f5ac006b120 2024-05-10T03:30:41.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.908+0000 7f5abd600640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 0x7f5aa006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:41.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.908+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5ab4089d70 con 0x7f5ac006b120 2024-05-10T03:30:41.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.909+0000 7f5ac5000640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 0x7f5aa006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:41.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.910+0000 7f5ac5000640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 0x7f5aa006d110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:41.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:41.912+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5ab4054a20 con 0x7f5ac006b120 2024-05-10T03:30:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.112+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ac00cd1c0 con 0x7f5ac006b120 2024-05-10T03:30:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.113+0000 7f5abd600640 1 -- 172.21.15.55:0/730663170 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ab4058040 con 0x7f5ac006b120 2024-05-10T03:30:42.116 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 msgr2=0x7f5aa006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 0x7f5aa006d110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5aac033000 tx=0x7f5aac02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 msgr2=0x7f5ac00acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400d5f0 tx=0x7f5ab400dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 shutdown_connections 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5aa006ac00 0x7f5aa006d110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ac01435d0 0x7f5ac0141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ac00b2040 0x7f5ac00ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 --2- 172.21.15.55:0/730663170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ac006b120 0x7f5ac00acc70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 >> 172.21.15.55:0/730663170 conn(0x7f5ac006c730 msgr2=0x7f5ac00aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 shutdown_connections 2024-05-10T03:30:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.116+0000 7f5ac6e00640 1 -- 172.21.15.55:0/730663170 wait complete. 2024-05-10T03:30:42.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:42.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.431+0000 7f3352e00640 1 Processor -- start 2024-05-10T03:30:42.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.432+0000 7f3352e00640 1 -- start start 2024-05-10T03:30:42.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.434+0000 7f3352e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f334c141a30 0x7f334c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.434+0000 7f3352e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c071ce0 con 0x7f334c15bce0 2024-05-10T03:30:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.434+0000 7f3352e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c070460 con 0x7f334c070050 2024-05-10T03:30:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.434+0000 7f3352e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c070f40 con 0x7f334c141a30 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.435+0000 7f3350e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f334c15bce0 0x7f334c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54486/0 (socket says 172.21.15.55:54486) 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.435+0000 7f3350e00640 1 -- 172.21.15.55:0/3189717100 learned_addr learned my addr 172.21.15.55:0/3189717100 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.435+0000 7f334b400640 1 --2- 172.21.15.55:0/3189717100 >> v2:172.21.15.162:3300/0 conn(0x7f334c141a30 0x7f334c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.435+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680349934 0 0) 0x7f3344001490 con 0x7f334c15bce0 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.435+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3330003520 con 0x7f334c15bce0 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793710915 0 0) 0x7f3330003520 con 0x7f334c15bce0 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 >> v2:172.21.15.162:3300/0 conn(0x7f334c141a30 msgr2=0x7f334c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 --2- 172.21.15.55:0/3189717100 >> v2:172.21.15.162:3300/0 conn(0x7f334c141a30 0x7f334c0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 >> v1:172.21.15.162:6789/0 conn(0x7f334c070050 legacy=0x7f334c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c141e40 con 0x7f334c15bce0 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f3344001b80 con 0x7f334c15bce0 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 >> v1:172.21.15.55:6789/0 conn(0x7f334c15bce0 legacy=0x7f334c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 0x7f3330003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33300047c0 0x7f3330004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3330005180 0x7f333000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c070f40 con 0x7f3330003970 2024-05-10T03:30:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333000c120 con 0x7f3330005180 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333000c2e0 con 0x7f33300047c0 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f334be00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 0x7f3330003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.436+0000 7f3350e00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3330005180 0x7f333000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334be00640 1 -- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33300047c0 msgr2=0x7f3330004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334b400640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33300047c0 0x7f3330004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334be00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33300047c0 0x7f3330004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334be00640 1 -- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3330005180 msgr2=0x7f333000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334be00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3330005180 0x7f333000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f334be00640 1 -- 172.21.15.55:0/3189717100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330003520 con 0x7f3330003970 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f3350e00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3330005180 0x7f333000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:42.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.437+0000 7f3350e00640 1 -- 172.21.15.55:0/3189717100 reap_dead start 2024-05-10T03:30:42.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.438+0000 7f334be00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 0x7f3330003de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f333c001580 tx=0x7f33300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b3b32746c0394 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.438+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c0052d0 con 0x7f3330003970 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.438+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c005430 con 0x7f3330003970 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.438+0000 7f334aa00640 1 -- 172.21.15.55:0/3189717100 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c02a940 con 0x7f3330003970 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 -- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 msgr2=0x7f3330003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 0x7f3330003de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f333c001580 tx=0x7f33300057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 -- 172.21.15.55:0/3189717100 shutdown_connections 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 --2- 172.21.15.55:0/3189717100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3330003970 0x7f3330003de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 -- 172.21.15.55:0/3189717100 >> 172.21.15.55:0/3189717100 conn(0x7f334c06c120 msgr2=0x7f334c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 -- 172.21.15.55:0/3189717100 shutdown_connections 2024-05-10T03:30:42.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.439+0000 7f3352e00640 1 -- 172.21.15.55:0/3189717100 wait complete. 2024-05-10T03:30:42.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.441+0000 7f3352e00640 1 Processor -- start 2024-05-10T03:30:42.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.442+0000 7f3352e00640 1 -- start start 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 0x7f334c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334c1f6fc0 0x7f334c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1f4340 con 0x7f334c1f6bb0 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1448d0 con 0x7f334c1f6fc0 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f3352e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1f3eb0 con 0x7f3330003970 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f334b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f334b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51112/0 (socket says 172.21.15.55:51112) 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.443+0000 7f334b400640 1 -- 172.21.15.55:0/1647699274 learned_addr learned my addr 172.21.15.55:0/1647699274 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334be00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 0x7f334c1440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f3350e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334c1f6fc0 0x7f334c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 -- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 msgr2=0x7f334c1440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 0x7f334c1440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 -- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334c1f6fc0 msgr2=0x7f334c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334c1f6fc0 0x7f334c0b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 -- 172.21.15.55:0/1647699274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c13f0b0 con 0x7f334c1f6bb0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334be00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 0x7f334c1440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f334b400640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3338010910 tx=0x7f3338010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.444+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338009690 con 0x7f334c1f6bb0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.445+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800ec20 con 0x7f334c1f6bb0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.445+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c144610 con 0x7f334c1f6bb0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.445+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333801c400 con 0x7f334c1f6bb0 2024-05-10T03:30:42.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.445+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c144e00 con 0x7f334c1f6bb0 2024-05-10T03:30:42.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.446+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c066f60 con 0x7f334c1f6bb0 2024-05-10T03:30:42.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.447+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f333801c560 con 0x7f334c1f6bb0 2024-05-10T03:30:42.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.448+0000 7f3343e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 0x7f332c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.448+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f3338089d60 con 0x7f334c1f6bb0 2024-05-10T03:30:42.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.449+0000 7f334be00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 0x7f332c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.450+0000 7f334be00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 0x7f332c06d160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f333c032000 tx=0x7f333c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:42.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.453+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3338054bd0 con 0x7f334c1f6bb0 2024-05-10T03:30:42.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.659+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f334c067220 con 0x7f334c1f6bb0 2024-05-10T03:30:42.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.660+0000 7f3343e00640 1 -- 172.21.15.55:0/1647699274 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33380581f0 con 0x7f334c1f6bb0 2024-05-10T03:30:42.664 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 msgr2=0x7f332c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 0x7f332c06d160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f333c032000 tx=0x7f333c02e000 comp rx=0 tx=0).stop 2024-05-10T03:30:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 msgr2=0x7f334c0b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3338010910 tx=0x7f3338010dd0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 shutdown_connections 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f332c06ac50 0x7f332c06d160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f334c1f6fc0 0x7f334c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f334c1f6bb0 0x7f334c0b36a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 --2- 172.21.15.55:0/1647699274 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3330003970 0x7f334c1440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.663+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 >> 172.21.15.55:0/1647699274 conn(0x7f334c06c120 msgr2=0x7f334c15c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.664+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 shutdown_connections 2024-05-10T03:30:42.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.664+0000 7f3352e00640 1 -- 172.21.15.55:0/1647699274 wait complete. 2024-05-10T03:30:42.694 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:42.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.979+0000 7f6930a00640 1 Processor -- start 2024-05-10T03:30:42.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.979+0000 7f6930a00640 1 -- start start 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.981+0000 7f6930a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f692c144710 0x7f692c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.981+0000 7f6930a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c0b3190 0x7f692c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f6930a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c06d350 con 0x7f692c06d7e0 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f6930a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c06c740 con 0x7f692c144710 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f6930a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c06ca20 con 0x7f692c0b3190 2024-05-10T03:30:42.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f692aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f692c06d7e0 0x7f692c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44012/0 (socket says 172.21.15.55:44012) 2024-05-10T03:30:42.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f692aa00640 1 -- 172.21.15.55:0/4252516832 learned_addr learned my addr 172.21.15.55:0/4252516832 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:42.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f692b400640 1 --2- 172.21.15.55:0/4252516832 >> v2:172.21.15.55:3300/0 conn(0x7f692c144710 0x7f692c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.982+0000 7f692be00640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c0b3190 0x7f692c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692a000640 1 -- 172.21.15.55:0/4252516832 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336403916 0 0) 0x7f692c06d350 con 0x7f692c06d7e0 2024-05-10T03:30:42.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692a000640 1 -- 172.21.15.55:0/4252516832 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6910003520 con 0x7f692c06d7e0 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692b400640 1 -- 172.21.15.55:0/4252516832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c0b3190 msgr2=0x7f692c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692b400640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c0b3190 0x7f692c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692b400640 1 -- 172.21.15.55:0/4252516832 >> v1:172.21.15.55:6789/0 conn(0x7f692c06d7e0 legacy=0x7f692c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.983+0000 7f692b400640 1 -- 172.21.15.55:0/4252516832 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c06cd00 con 0x7f692c144710 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.984+0000 7f692b400640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c144710 0x7f692c144120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f691c0097a0 tx=0x7f691c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f633144d4e4fbfb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:42.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.984+0000 7f692a000640 1 -- 172.21.15.55:0/4252516832 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c02adb0 con 0x7f692c144710 2024-05-10T03:30:42.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.984+0000 7f692a000640 1 -- 172.21.15.55:0/4252516832 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c032440 con 0x7f692c144710 2024-05-10T03:30:42.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.984+0000 7f692a000640 1 -- 172.21.15.55:0/4252516832 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0327b0 con 0x7f692c144710 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 -- 172.21.15.55:0/4252516832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c144710 msgr2=0x7f692c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c144710 0x7f692c144120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f691c0097a0 tx=0x7f691c02a060 comp rx=0 tx=0).stop 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 -- 172.21.15.55:0/4252516832 shutdown_connections 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c0b3190 0x7f692c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 --2- 172.21.15.55:0/4252516832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c144710 0x7f692c144120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 -- 172.21.15.55:0/4252516832 >> 172.21.15.55:0/4252516832 conn(0x7f692c06a110 msgr2=0x7f692c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 -- 172.21.15.55:0/4252516832 shutdown_connections 2024-05-10T03:30:42.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.985+0000 7f6930a00640 1 -- 172.21.15.55:0/4252516832 wait complete. 2024-05-10T03:30:42.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.988+0000 7f6930a00640 1 Processor -- start 2024-05-10T03:30:42.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.988+0000 7f6930a00640 1 -- start start 2024-05-10T03:30:42.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 0x7f692c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c144710 0x7f692c1341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c135200 con 0x7f692c06d7e0 2024-05-10T03:30:42.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c1416c0 con 0x7f692c0b3190 2024-05-10T03:30:42.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f6930a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c141910 con 0x7f692c144710 2024-05-10T03:30:42.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f692b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f692b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53452/0 (socket says 172.21.15.55:53452) 2024-05-10T03:30:42.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.990+0000 7f692b400640 1 -- 172.21.15.55:0/2063174938 learned_addr learned my addr 172.21.15.55:0/2063174938 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692aa00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 0x7f692c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692be00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c144710 0x7f692c1341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 -- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c144710 msgr2=0x7f692c1341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c144710 0x7f692c1341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 -- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 msgr2=0x7f692c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 0x7f692c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 -- 172.21.15.55:0/2063174938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009400 con 0x7f692c06d7e0 2024-05-10T03:30:42.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692aa00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 0x7f692c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f692b400640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098c0 tx=0x7f691c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c03c8d0 con 0x7f692c06d7e0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c03ca30 con 0x7f692c06d7e0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c03cde0 con 0x7f692c06d7e0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.991+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c141bf0 con 0x7f692c06d7e0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.992+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c1421b0 con 0x7f692c06d7e0 2024-05-10T03:30:42.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.993+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c066ce0 con 0x7f692c06d7e0 2024-05-10T03:30:42.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.994+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f691c03b3a0 con 0x7f692c06d7e0 2024-05-10T03:30:42.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.995+0000 7f6923400640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 0x7f690c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:42.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.995+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f691c0aa370 con 0x7f692c06d7e0 2024-05-10T03:30:42.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.996+0000 7f692aa00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 0x7f690c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:42.997+0000 7f692aa00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 0x7f690c06d140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f692c0afaa0 tx=0x7f6918009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:43.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.000+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f691c078be0 con 0x7f692c06d7e0 2024-05-10T03:30:43.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.201+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f692c04e290 con 0x7f692c06d7e0 2024-05-10T03:30:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.205+0000 7f6923400640 1 -- 172.21.15.55:0/2063174938 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f691c078770 con 0x7f692c06d7e0 2024-05-10T03:30:43.209 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:43.209 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.207+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 msgr2=0x7f690c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.207+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 0x7f690c06d140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f692c0afaa0 tx=0x7f6918009040 comp rx=0 tx=0).stop 2024-05-10T03:30:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.207+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 msgr2=0x7f692c070170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.207+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098c0 tx=0x7f691c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 shutdown_connections 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f692c144710 0x7f692c1341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f690c06ac30 0x7f690c06d140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f692c0b3190 0x7f692c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 --2- 172.21.15.55:0/2063174938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f692c06d7e0 0x7f692c070170 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 >> 172.21.15.55:0/2063174938 conn(0x7f692c06a110 msgr2=0x7f692c0662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 shutdown_connections 2024-05-10T03:30:43.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.208+0000 7f6930a00640 1 -- 172.21.15.55:0/2063174938 wait complete. 2024-05-10T03:30:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:43.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:43.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.526+0000 7f7d27c00640 1 Processor -- start 2024-05-10T03:30:43.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.526+0000 7f7d27c00640 1 -- start start 2024-05-10T03:30:43.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d27c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d2015e200 0x7f7d2006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d27c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2006ff50 0x7f7d200b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d27c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2015ef50 con 0x7f7d200b2190 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d27c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200b0a00 con 0x7f7d2006ff50 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d27c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200b61a0 con 0x7f7d2015e200 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d25c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2006ff50 0x7f7d200b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d25200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7d200b2190 0x7f7d2015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44028/0 (socket says 172.21.15.55:44028) 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d25200640 1 -- 172.21.15.55:0/2757408334 learned_addr learned my addr 172.21.15.55:0/2757408334 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.529+0000 7f7d1fe00640 1 --2- 172.21.15.55:0/2757408334 >> v2:172.21.15.162:3300/0 conn(0x7f7d2015e200 0x7f7d2006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.530+0000 7f7d25c00640 1 -- 172.21.15.55:0/2757408334 >> v2:172.21.15.162:3300/0 conn(0x7f7d2015e200 msgr2=0x7f7d2006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.530+0000 7f7d25c00640 1 --2- 172.21.15.55:0/2757408334 >> v2:172.21.15.162:3300/0 conn(0x7f7d2015e200 0x7f7d2006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.530+0000 7f7d25c00640 1 -- 172.21.15.55:0/2757408334 >> v1:172.21.15.55:6789/0 conn(0x7f7d200b2190 legacy=0x7f7d2015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.530+0000 7f7d25c00640 1 -- 172.21.15.55:0/2757408334 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20070360 con 0x7f7d2006ff50 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.531+0000 7f7d25c00640 1 --2- 172.21.15.55:0/2757408334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2006ff50 0x7f7d200b0e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d200d2140 tx=0x7f7d1802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e942a14305cba7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.531+0000 7f7d1f400640 1 -- 172.21.15.55:0/2757408334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1802aea0 con 0x7f7d2006ff50 2024-05-10T03:30:43.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.531+0000 7f7d1f400640 1 -- 172.21.15.55:0/2757408334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1802d050 con 0x7f7d2006ff50 2024-05-10T03:30:43.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.531+0000 7f7d1f400640 1 -- 172.21.15.55:0/2757408334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18007960 con 0x7f7d2006ff50 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 -- 172.21.15.55:0/2757408334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2006ff50 msgr2=0x7f7d200b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 --2- 172.21.15.55:0/2757408334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2006ff50 0x7f7d200b0e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d200d2140 tx=0x7f7d1802a680 comp rx=0 tx=0).stop 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 -- 172.21.15.55:0/2757408334 shutdown_connections 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 --2- 172.21.15.55:0/2757408334 >> v2:172.21.15.162:3300/0 conn(0x7f7d2015e200 0x7f7d2006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 --2- 172.21.15.55:0/2757408334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2006ff50 0x7f7d200b0e20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 -- 172.21.15.55:0/2757408334 >> 172.21.15.55:0/2757408334 conn(0x7f7d200b36e0 msgr2=0x7f7d200d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 -- 172.21.15.55:0/2757408334 shutdown_connections 2024-05-10T03:30:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.532+0000 7f7d27c00640 1 -- 172.21.15.55:0/2757408334 wait complete. 2024-05-10T03:30:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.535+0000 7f7d27c00640 1 Processor -- start 2024-05-10T03:30:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.535+0000 7f7d27c00640 1 -- start start 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2006ff50 0x7f7d20148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2015e200 0x7f7d2015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200cd130 con 0x7f7d200b2190 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20157e30 con 0x7f7d2015e200 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.536+0000 7f7d27c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200ca400 con 0x7f7d2006ff50 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53462/0 (socket says 172.21.15.55:53462) 2024-05-10T03:30:43.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 -- 172.21.15.55:0/1984640598 learned_addr learned my addr 172.21.15.55:0/1984640598 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:43.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d25c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2015e200 0x7f7d2015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d25200640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2006ff50 0x7f7d20148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 -- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2006ff50 msgr2=0x7f7d20148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2006ff50 0x7f7d20148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 -- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2015e200 msgr2=0x7f7d2015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2015e200 0x7f7d2015c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.537+0000 7f7d1fe00640 1 -- 172.21.15.55:0/1984640598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009a70 con 0x7f7d200b2190 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d1fe00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00d5f0 tx=0x7f7d0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0125f0 con 0x7f7d200b2190 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c012750 con 0x7f7d200b2190 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c012b00 con 0x7f7d200b2190 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d200ca930 con 0x7f7d200b2190 2024-05-10T03:30:43.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.538+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d20148d50 con 0x7f7d200b2190 2024-05-10T03:30:43.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.539+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d200d1980 con 0x7f7d200b2190 2024-05-10T03:30:43.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.540+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d0c012c60 con 0x7f7d200b2190 2024-05-10T03:30:43.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.542+0000 7f7d1cc00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 0x7f7d0006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:43.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.542+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f7d0c089c00 con 0x7f7d200b2190 2024-05-10T03:30:43.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.542+0000 7f7d25200640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 0x7f7d0006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:43.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.544+0000 7f7d25200640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 0x7f7d0006d140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7d14005880 tx=0x7f7d14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:43.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.547+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d0c0549c0 con 0x7f7d200b2190 2024-05-10T03:30:43.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.747+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d200b3fa0 con 0x7f7d200b2190 2024-05-10T03:30:43.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.747+0000 7f7d1cc00640 1 -- 172.21.15.55:0/1984640598 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d0c057fe0 con 0x7f7d200b2190 2024-05-10T03:30:43.751 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:43.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 msgr2=0x7f7d0006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 0x7f7d0006d140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7d14005880 tx=0x7f7d14009040 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 msgr2=0x7f7d20160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00d5f0 tx=0x7f7d0c00dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 shutdown_connections 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0006ac30 0x7f7d0006d140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2015e200 0x7f7d2015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d200b2190 0x7f7d20160af0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 --2- 172.21.15.55:0/1984640598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2006ff50 0x7f7d20148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 >> 172.21.15.55:0/1984640598 conn(0x7f7d200b36e0 msgr2=0x7f7d20159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 shutdown_connections 2024-05-10T03:30:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:43.750+0000 7f7d27c00640 1 -- 172.21.15.55:0/1984640598 wait complete. 2024-05-10T03:30:43.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.066+0000 7f9505c00640 1 Processor -- start 2024-05-10T03:30:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.066+0000 7f9505c00640 1 -- start start 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.068+0000 7f9505c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f9500143330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.068+0000 7f9505c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9500162340 con 0x7f950006c440 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.068+0000 7f9505c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95000cddb0 con 0x7f9500161f30 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.068+0000 7f9505c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9500143870 con 0x7f9500142f20 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.069+0000 7f94ff400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f950006c440 0x7f950006c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44042/0 (socket says 172.21.15.55:44042) 2024-05-10T03:30:44.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.069+0000 7f94ff400640 1 -- 172.21.15.55:0/365349046 learned_addr learned my addr 172.21.15.55:0/365349046 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.069+0000 7f94fe000640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f9500143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.069+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098471990 0 0) 0x7f94f8001110 con 0x7f950006c440 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.069+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e4003520 con 0x7f950006c440 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 251553328 0 0) 0x7f94e4003520 con 0x7f950006c440 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 msgr2=0x7f9500143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f9500143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 >> v1:172.21.15.162:6789/0 conn(0x7f9500161f30 legacy=0x7f9500140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500143a90 con 0x7f950006c440 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f94f8001800 con 0x7f950006c440 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 >> v1:172.21.15.55:6789/0 conn(0x7f950006c440 legacy=0x7f950006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e40039a0 0x7f94e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40047c0 0x7f94e4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f94e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9500143870 con 0x7f94e40039a0 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e400bab0 con 0x7f94e40047c0 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e400bc70 con 0x7f9500142f20 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fea00640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e40039a0 0x7f94e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.070+0000 7f94fe000640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40047c0 0x7f94e4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94ff400640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f94e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94fea00640 1 -- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 msgr2=0x7f94e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94fea00640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9500142f20 0x7f94e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94fea00640 1 -- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40047c0 msgr2=0x7f94e4004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94fea00640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40047c0 0x7f94e4004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.071+0000 7f94fea00640 1 -- 172.21.15.55:0/365349046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4003520 con 0x7f94e40039a0 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.072+0000 7f94fea00640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e40039a0 0x7f94e4003e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001720 tx=0x7f94e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479cd7b6f5ba4ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.072+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80052d0 con 0x7f94e40039a0 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.072+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8005430 con 0x7f94e40039a0 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.072+0000 7f94fd600640 1 -- 172.21.15.55:0/365349046 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e802abc0 con 0x7f94e40039a0 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 -- 172.21.15.55:0/365349046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e40039a0 msgr2=0x7f94e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 --2- 172.21.15.55:0/365349046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e40039a0 0x7f94e4003e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001720 tx=0x7f94e40057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f94ff400640 1 -- 172.21.15.55:0/365349046 reap_dead start 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 -- 172.21.15.55:0/365349046 shutdown_connections 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 -- 172.21.15.55:0/365349046 >> 172.21.15.55:0/365349046 conn(0x7f95000b2ed0 msgr2=0x7f95000b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 -- 172.21.15.55:0/365349046 shutdown_connections 2024-05-10T03:30:44.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.073+0000 7f9505c00640 1 -- 172.21.15.55:0/365349046 wait complete. 2024-05-10T03:30:44.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.075+0000 7f9505c00640 1 Processor -- start 2024-05-10T03:30:44.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.076+0000 7f9505c00640 1 -- start start 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40039a0 0x7f950013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 0x7f95001e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950015d360 con 0x7f950015bc60 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9500140090 con 0x7f94e40039a0 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f9505c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95001e8420 con 0x7f95000cc7e0 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f94fe000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.077+0000 7f94fe000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53468/0 (socket says 172.21.15.55:53468) 2024-05-10T03:30:44.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 -- 172.21.15.55:0/2325394265 learned_addr learned my addr 172.21.15.55:0/2325394265 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fea00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40039a0 0x7f950013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94ff400640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 0x7f95001e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 -- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 msgr2=0x7f95001e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 0x7f95001e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 -- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40039a0 msgr2=0x7f950013fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40039a0 0x7f950013fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 -- 172.21.15.55:0/2325394265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950015c490 con 0x7f950015bc60 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94ff400640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 0x7f95001e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.078+0000 7f94fe000640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009fd0 tx=0x7f94f00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.079+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f000e030 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.079+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f0003990 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.079+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0003b20 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.079+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95000ccbf0 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.079+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950015c9c0 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.080+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9500160fe0 con 0x7f950015bc60 2024-05-10T03:30:44.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.081+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f94f0003c80 con 0x7f950015bc60 2024-05-10T03:30:44.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.082+0000 7f94f6a00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 0x7f94dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.083+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f94f000f070 con 0x7f950015bc60 2024-05-10T03:30:44.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.083+0000 7f94fea00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 0x7f94dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.085+0000 7f94fea00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 0x7f94dc06d140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94e8032000 tx=0x7f94e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.088+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f94f0055010 con 0x7f950015bc60 2024-05-10T03:30:44.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.293+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9500067220 con 0x7f950015bc60 2024-05-10T03:30:44.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.294+0000 7f94f6a00640 1 -- 172.21.15.55:0/2325394265 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94f0058630 con 0x7f950015bc60 2024-05-10T03:30:44.297 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:44.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 msgr2=0x7f94dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 0x7f94dc06d140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94e8032000 tx=0x7f94e802e000 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 msgr2=0x7f95000cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009fd0 tx=0x7f94f00115a0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 shutdown_connections 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95000cc7e0 0x7f95001e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f950015bc60 0x7f95000cc2a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94dc06ac30 0x7f94dc06d140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 --2- 172.21.15.55:0/2325394265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e40039a0 0x7f950013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 >> 172.21.15.55:0/2325394265 conn(0x7f95000b2ed0 msgr2=0x7f95000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 shutdown_connections 2024-05-10T03:30:44.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.297+0000 7f9505c00640 1 -- 172.21.15.55:0/2325394265 wait complete. 2024-05-10T03:30:44.327 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:44.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.601+0000 7f9d23400640 1 Processor -- start 2024-05-10T03:30:44.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.601+0000 7f9d23400640 1 -- start start 2024-05-10T03:30:44.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.603+0000 7f9d23400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d1c06a4b0 0x7f9d1c158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.604+0000 7f9d23400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.604+0000 7f9d23400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c0708b0 con 0x7f9d1c071db0 2024-05-10T03:30:44.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.604+0000 7f9d23400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0b2490 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.604+0000 7f9d23400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c070050 con 0x7f9d1c0b3860 2024-05-10T03:30:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d21400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d21400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47942/0 (socket says 172.21.15.55:47942) 2024-05-10T03:30:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d21400640 1 -- 172.21.15.55:0/62471218 learned_addr learned my addr 172.21.15.55:0/62471218 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d20a00640 1 --1- 172.21.15.55:0/62471218 >> v1:172.21.15.162:6789/0 conn(0x7f9d1c071db0 0x7f9d1c0ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:37534/0 (socket says 172.21.15.55:37534) 2024-05-10T03:30:44.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d1be00640 1 --2- 172.21.15.55:0/62471218 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c06a4b0 0x7f9d1c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d21400640 1 -- 172.21.15.55:0/62471218 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c06a4b0 msgr2=0x7f9d1c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.605+0000 7f9d21400640 1 --2- 172.21.15.55:0/62471218 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c06a4b0 0x7f9d1c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.606+0000 7f9d21400640 1 -- 172.21.15.55:0/62471218 >> v1:172.21.15.162:6789/0 conn(0x7f9d1c071db0 legacy=0x7f9d1c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.606+0000 7f9d21400640 1 -- 172.21.15.55:0/62471218 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c15b940 con 0x7f9d1c0b3860 2024-05-10T03:30:44.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.607+0000 7f9d21400640 1 --2- 172.21.15.55:0/62471218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009870 tx=0x7f9d1402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ce4f4160351af server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.607+0000 7f9d1b400640 1 -- 172.21.15.55:0/62471218 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14032440 con 0x7f9d1c0b3860 2024-05-10T03:30:44.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.607+0000 7f9d1b400640 1 -- 172.21.15.55:0/62471218 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d140325a0 con 0x7f9d1c0b3860 2024-05-10T03:30:44.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.607+0000 7f9d1b400640 1 -- 172.21.15.55:0/62471218 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14032930 con 0x7f9d1c0b3860 2024-05-10T03:30:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.608+0000 7f9d23400640 1 -- 172.21.15.55:0/62471218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 msgr2=0x7f9d1c0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.608+0000 7f9d23400640 1 --2- 172.21.15.55:0/62471218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009870 tx=0x7f9d1402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 -- 172.21.15.55:0/62471218 shutdown_connections 2024-05-10T03:30:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 --2- 172.21.15.55:0/62471218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c0b3860 0x7f9d1c0b3c70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 --2- 172.21.15.55:0/62471218 >> v2:172.21.15.162:3300/0 conn(0x7f9d1c06a4b0 0x7f9d1c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 -- 172.21.15.55:0/62471218 >> 172.21.15.55:0/62471218 conn(0x7f9d1c0cbb90 msgr2=0x7f9d1c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 -- 172.21.15.55:0/62471218 shutdown_connections 2024-05-10T03:30:44.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.609+0000 7f9d23400640 1 -- 172.21.15.55:0/62471218 wait complete. 2024-05-10T03:30:44.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.611+0000 7f9d23400640 1 Processor -- start 2024-05-10T03:30:44.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.612+0000 7f9d23400640 1 -- start start 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c071db0 0x7f9d1c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c0b3860 0x7f9d1c139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c159700 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c138b40 con 0x7f9d1c0b3860 2024-05-10T03:30:44.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.613+0000 7f9d23400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c154a90 con 0x7f9d1c071db0 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53474/0 (socket says 172.21.15.55:53474) 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 -- 172.21.15.55:0/2681676523 learned_addr learned my addr 172.21.15.55:0/2681676523 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d1be00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c071db0 0x7f9d1c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d21400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c0b3860 0x7f9d1c139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 -- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c071db0 msgr2=0x7f9d1c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c071db0 0x7f9d1c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 -- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c0b3860 msgr2=0x7f9d1c139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c0b3860 0x7f9d1c139df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.614+0000 7f9d20a00640 1 -- 172.21.15.55:0/2681676523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140094d0 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d20a00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c710 tx=0x7f9d0c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00ce20 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c00a5e0 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c138d50 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00a990 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.615+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c139310 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.617+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c066f60 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.617+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9d0c00aaf0 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.618+0000 7f9d18c00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 0x7f9cfc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:44.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.618+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9d0c088ec0 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.619+0000 7f9d1be00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 0x7f9cfc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.621+0000 7f9d1be00640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 0x7f9cfc06d140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c15d9b0 tx=0x7f9d08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.623+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9d0c053c40 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.831+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d1c15b360 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.832+0000 7f9d18c00640 1 -- 172.21.15.55:0/2681676523 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d0c0170a0 con 0x7f9d1c06a4b0 2024-05-10T03:30:44.836 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:44.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.835+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 msgr2=0x7f9cfc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.835+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 0x7f9cfc06d140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c15d9b0 tx=0x7f9d08008040 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.835+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 msgr2=0x7f9d1c158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.835+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00c710 tx=0x7f9d0c00cbd0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 shutdown_connections 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cfc06ac30 0x7f9cfc06d140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d1c0b3860 0x7f9d1c139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d1c071db0 0x7f9d1c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 --2- 172.21.15.55:0/2681676523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1c06a4b0 0x7f9d1c158f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 >> 172.21.15.55:0/2681676523 conn(0x7f9d1c0cbb90 msgr2=0x7f9d1c13e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 shutdown_connections 2024-05-10T03:30:44.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:44.836+0000 7f9d23400640 1 -- 172.21.15.55:0/2681676523 wait complete. 2024-05-10T03:30:44.866 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:45.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.138+0000 7fb245e00640 1 Processor -- start 2024-05-10T03:30:45.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.138+0000 7fb245e00640 1 -- start start 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.141+0000 7fb245e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb240141a30 0x7fb2400cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.141+0000 7fb245e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb240071ce0 con 0x7fb24015bce0 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.141+0000 7fb245e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb240070460 con 0x7fb240070050 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.141+0000 7fb245e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb240070f40 con 0x7fb240141a30 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.141+0000 7fb236000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb240141a30 0x7fb2400cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb23f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb24015bce0 0x7fb24006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44046/0 (socket says 172.21.15.55:44046) 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb23f400640 1 -- 172.21.15.55:0/2480780947 learned_addr learned my addr 172.21.15.55:0/2480780947 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:45.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb23ea00640 1 --1- 172.21.15.55:0/2480780947 >> v1:172.21.15.162:6789/0 conn(0x7fb240070050 0x7fb24015f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:37544/0 (socket says 172.21.15.55:37544) 2024-05-10T03:30:45.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb236000640 1 -- 172.21.15.55:0/2480780947 >> v1:172.21.15.162:6789/0 conn(0x7fb240070050 legacy=0x7fb24015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb236000640 1 -- 172.21.15.55:0/2480780947 >> v1:172.21.15.55:6789/0 conn(0x7fb24015bce0 legacy=0x7fb24006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb23e000640 1 -- 172.21.15.55:0/2480780947 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696791896 0 0) 0x7fb238001110 con 0x7fb24015bce0 2024-05-10T03:30:45.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.142+0000 7fb236000640 1 -- 172.21.15.55:0/2480780947 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb240141e40 con 0x7fb240141a30 2024-05-10T03:30:45.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.143+0000 7fb236000640 1 --2- 172.21.15.55:0/2480780947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb240141a30 0x7fb2400cd750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb2280097a0 tx=0x7fb22802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8875fdf1d31053d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.144+0000 7fb23e000640 1 -- 172.21.15.55:0/2480780947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228032440 con 0x7fb240141a30 2024-05-10T03:30:45.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.144+0000 7fb23e000640 1 -- 172.21.15.55:0/2480780947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2280325a0 con 0x7fb240141a30 2024-05-10T03:30:45.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.144+0000 7fb23e000640 1 -- 172.21.15.55:0/2480780947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228032930 con 0x7fb240141a30 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.144+0000 7fb245e00640 1 -- 172.21.15.55:0/2480780947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb240141a30 msgr2=0x7fb2400cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.144+0000 7fb245e00640 1 --2- 172.21.15.55:0/2480780947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb240141a30 0x7fb2400cd750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb2280097a0 tx=0x7fb22802a230 comp rx=0 tx=0).stop 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.145+0000 7fb245e00640 1 -- 172.21.15.55:0/2480780947 shutdown_connections 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.145+0000 7fb245e00640 1 --2- 172.21.15.55:0/2480780947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb240141a30 0x7fb2400cd750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.145+0000 7fb245e00640 1 -- 172.21.15.55:0/2480780947 >> 172.21.15.55:0/2480780947 conn(0x7fb24006c120 msgr2=0x7fb24006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.145+0000 7fb245e00640 1 -- 172.21.15.55:0/2480780947 shutdown_connections 2024-05-10T03:30:45.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.145+0000 7fb245e00640 1 -- 172.21.15.55:0/2480780947 wait complete. 2024-05-10T03:30:45.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.147+0000 7fb245e00640 1 Processor -- start 2024-05-10T03:30:45.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.147+0000 7fb245e00640 1 -- start start 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 0x7fb240158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb240141a30 0x7fb2401f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401f86a0 con 0x7fb24015bce0 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401e56d0 con 0x7fb240070050 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb245e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401f8450 con 0x7fb240141a30 2024-05-10T03:30:45.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb23f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb23f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53486/0 (socket says 172.21.15.55:53486) 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.149+0000 7fb23f400640 1 -- 172.21.15.55:0/3793371878 learned_addr learned my addr 172.21.15.55:0/3793371878 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb236000640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb240141a30 0x7fb2401f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23ea00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 0x7fb240158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 -- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb240141a30 msgr2=0x7fb2401f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb240141a30 0x7fb2401f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 -- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 msgr2=0x7fb240158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 0x7fb240158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 -- 172.21.15.55:0/3793371878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009400 con 0x7fb24015bce0 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23ea00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 0x7fb240158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:45.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb23f400640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb238001eb0 tx=0x7fb23800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2380074e0 con 0x7fb24015bce0 2024-05-10T03:30:45.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb238007640 con 0x7fb24015bce0 2024-05-10T03:30:45.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238004a00 con 0x7fb24015bce0 2024-05-10T03:30:45.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.150+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2400b2d80 con 0x7fb24015bce0 2024-05-10T03:30:45.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.151+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2400b3340 con 0x7fb24015bce0 2024-05-10T03:30:45.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.152+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb240066f60 con 0x7fb24015bce0 2024-05-10T03:30:45.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.153+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb238004b60 con 0x7fb24015bce0 2024-05-10T03:30:45.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.154+0000 7fb237e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 0x7fb21c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.154+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb238089560 con 0x7fb24015bce0 2024-05-10T03:30:45.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.154+0000 7fb23ea00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 0x7fb21c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.155+0000 7fb23ea00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 0x7fb21c06d110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb230007fd0 tx=0x7fb230009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.158+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb238054210 con 0x7fb24015bce0 2024-05-10T03:30:45.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.359+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb240067220 con 0x7fb24015bce0 2024-05-10T03:30:45.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.360+0000 7fb237e00640 1 -- 172.21.15.55:0/3793371878 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb238057830 con 0x7fb24015bce0 2024-05-10T03:30:45.364 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 msgr2=0x7fb21c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 0x7fb21c06d110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb230007fd0 tx=0x7fb230009040 comp rx=0 tx=0).stop 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 msgr2=0x7fb24015aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb238001eb0 tx=0x7fb23800c870 comp rx=0 tx=0).stop 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 shutdown_connections 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb24015bce0 0x7fb24015aff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb240141a30 0x7fb2401f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb21c06ac00 0x7fb21c06d110 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 --2- 172.21.15.55:0/3793371878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb240070050 0x7fb240158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.363+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 >> 172.21.15.55:0/3793371878 conn(0x7fb24006c120 msgr2=0x7fb24013e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:45.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.364+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 shutdown_connections 2024-05-10T03:30:45.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.364+0000 7fb245e00640 1 -- 172.21.15.55:0/3793371878 wait complete. 2024-05-10T03:30:45.393 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.672+0000 7fec9ca00640 1 Processor -- start 2024-05-10T03:30:45.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.672+0000 7fec9ca00640 1 -- start start 2024-05-10T03:30:45.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec9ca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec980b6680 0x7fec9815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec9ca00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980d25a0 con 0x7fec981618b0 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec9ca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980b28d0 con 0x7fec980b2170 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec9ca00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec980d2b20 con 0x7fec980b6680 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec96800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec980b6680 0x7fec9815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec97200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fec981618b0 0x7fec9806aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44058/0 (socket says 172.21.15.55:44058) 2024-05-10T03:30:45.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.674+0000 7fec97200640 1 -- 172.21.15.55:0/163925781 learned_addr learned my addr 172.21.15.55:0/163925781 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:45.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.675+0000 7fec96800640 1 -- 172.21.15.55:0/163925781 >> v1:172.21.15.162:6789/0 conn(0x7fec980b2170 legacy=0x7fec981611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:45.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.675+0000 7fec96800640 1 -- 172.21.15.55:0/163925781 >> v1:172.21.15.55:6789/0 conn(0x7fec981618b0 legacy=0x7fec9806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.675+0000 7fec96800640 1 -- 172.21.15.55:0/163925781 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980d2840 con 0x7fec980b6680 2024-05-10T03:30:45.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.675+0000 7fec95400640 1 -- 172.21.15.55:0/163925781 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395948209 0 0) 0x7fec980d25a0 con 0x7fec981618b0 2024-05-10T03:30:45.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.676+0000 7fec96800640 1 --2- 172.21.15.55:0/163925781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b6680 0x7fec9815d5e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec800097a0 tx=0x7fec8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109c4a82474ea3ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.676+0000 7fec95400640 1 -- 172.21.15.55:0/163925781 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80032440 con 0x7fec980b6680 2024-05-10T03:30:45.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.676+0000 7fec95400640 1 -- 172.21.15.55:0/163925781 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec800325a0 con 0x7fec980b6680 2024-05-10T03:30:45.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.676+0000 7fec95400640 1 -- 172.21.15.55:0/163925781 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80032930 con 0x7fec980b6680 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 -- 172.21.15.55:0/163925781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b6680 msgr2=0x7fec9815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 --2- 172.21.15.55:0/163925781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b6680 0x7fec9815d5e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec800097a0 tx=0x7fec8002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 -- 172.21.15.55:0/163925781 shutdown_connections 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 --2- 172.21.15.55:0/163925781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b6680 0x7fec9815d5e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 -- 172.21.15.55:0/163925781 >> 172.21.15.55:0/163925781 conn(0x7fec980b5730 msgr2=0x7fec980d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 -- 172.21.15.55:0/163925781 shutdown_connections 2024-05-10T03:30:45.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.677+0000 7fec9ca00640 1 -- 172.21.15.55:0/163925781 wait complete. 2024-05-10T03:30:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.680+0000 7fec9ca00640 1 Processor -- start 2024-05-10T03:30:45.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.680+0000 7fec9ca00640 1 -- start start 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.681+0000 7fec9ca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.681+0000 7fec9ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec980b6680 0x7fec98155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.681+0000 7fec9ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec981618b0 0x7fec981562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.681+0000 7fec9ca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9815db20 con 0x7fec980b2170 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec9ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec981433b0 con 0x7fec981618b0 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec9ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec98159dd0 con 0x7fec980b6680 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53508/0 (socket says 172.21.15.55:53508) 2024-05-10T03:30:45.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 -- 172.21.15.55:0/3893328017 learned_addr learned my addr 172.21.15.55:0/3893328017 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 -- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec980b6680 msgr2=0x7fec98155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec980b6680 0x7fec98155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 -- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec981618b0 msgr2=0x7fec981562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec981618b0 0x7fec981562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.682+0000 7fec96800640 1 -- 172.21.15.55:0/3893328017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80009400 con 0x7fec980b2170 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec96800640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec800097a0 tx=0x7fec800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80030b80 con 0x7fec980b2170 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec80030ce0 con 0x7fec980b2170 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8003b640 con 0x7fec980b2170 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9815a020 con 0x7fec980b2170 2024-05-10T03:30:45.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.683+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec981552b0 con 0x7fec980b2170 2024-05-10T03:30:45.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.684+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec98067740 con 0x7fec980b2170 2024-05-10T03:30:45.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.686+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fec80039040 con 0x7fec980b2170 2024-05-10T03:30:45.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.687+0000 7fec8ea00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 0x7fec7406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:45.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.687+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fec800aa210 con 0x7fec980b2170 2024-05-10T03:30:45.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.687+0000 7fec95e00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 0x7fec7406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:45.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.689+0000 7fec95e00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 0x7fec7406d140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec88006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:45.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.691+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fec80074ec0 con 0x7fec980b2170 2024-05-10T03:30:45.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.894+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9804e290 con 0x7fec980b2170 2024-05-10T03:30:45.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.898+0000 7fec8ea00640 1 -- 172.21.15.55:0/3893328017 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7fec80006e10 con 0x7fec980b2170 2024-05-10T03:30:45.901 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:45.902 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:45.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 msgr2=0x7fec7406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 0x7fec7406d140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec88006bc0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 msgr2=0x7fec98160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fec800097a0 tx=0x7fec800046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 shutdown_connections 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fec7406ac30 0x7fec7406d140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec981618b0 0x7fec981562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec980b6680 0x7fec98155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 --2- 172.21.15.55:0/3893328017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec980b2170 0x7fec98160300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 >> 172.21.15.55:0/3893328017 conn(0x7fec980b5730 msgr2=0x7fec980cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 shutdown_connections 2024-05-10T03:30:45.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:45.901+0000 7fec9ca00640 1 -- 172.21.15.55:0/3893328017 wait complete. 2024-05-10T03:30:45.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:45.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:46.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.221+0000 7f4a6b200640 1 Processor -- start 2024-05-10T03:30:46.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.221+0000 7f4a6b200640 1 -- start start 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.223+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 0x7f4a6c140060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.223+0000 7f4a6b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a6c0cd190 0x7f4a6c06a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.223+0000 7f4a6b200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4a6c06ae80 0x7f4a6c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a6b200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c1405a0 con 0x7f4a6c0cd190 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a6b200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6c06a1b0 con 0x7f4a6c06ae80 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a6b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c15e500 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a6c0cd190 0x7f4a6c06a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a6c0cd190 0x7f4a6c06a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53524/0 (socket says 172.21.15.55:53524) 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 -- 172.21.15.55:0/2167987454 learned_addr learned my addr 172.21.15.55:0/2167987454 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a6a800640 1 --2- 172.21.15.55:0/2167987454 >> v2:172.21.15.162:3300/0 conn(0x7f4a6c06ae80 0x7f4a6c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69e00640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 0x7f4a6c140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 -- 172.21.15.55:0/2167987454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 msgr2=0x7f4a6c140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 0x7f4a6c140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 -- 172.21.15.55:0/2167987454 >> v2:172.21.15.162:3300/0 conn(0x7f4a6c06ae80 msgr2=0x7f4a6c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.224+0000 7f4a69400640 1 --2- 172.21.15.55:0/2167987454 >> v2:172.21.15.162:3300/0 conn(0x7f4a6c06ae80 0x7f4a6c0b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.225+0000 7f4a69400640 1 -- 172.21.15.55:0/2167987454 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0cf030 con 0x7f4a6c0cd190 2024-05-10T03:30:46.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.225+0000 7f4a69e00640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 0x7f4a6c140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:46.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.226+0000 7f4a69400640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c06a940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0705a0 tx=0x7f4a5802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c03876e4758207 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.226+0000 7f4a68a00640 1 -- 172.21.15.55:0/2167987454 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58030050 con 0x7f4a6c0cd190 2024-05-10T03:30:46.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a68a00640 1 -- 172.21.15.55:0/2167987454 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a58032700 con 0x7f4a6c0cd190 2024-05-10T03:30:46.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a68a00640 1 -- 172.21.15.55:0/2167987454 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58032a90 con 0x7f4a6c0cd190 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 -- 172.21.15.55:0/2167987454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c0cd190 msgr2=0x7f4a6c06a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c06a940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0705a0 tx=0x7f4a5802c800 comp rx=0 tx=0).stop 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 -- 172.21.15.55:0/2167987454 shutdown_connections 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 --2- 172.21.15.55:0/2167987454 >> v2:172.21.15.162:3300/0 conn(0x7f4a6c06ae80 0x7f4a6c0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c06a940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 --2- 172.21.15.55:0/2167987454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c15e0f0 0x7f4a6c140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 -- 172.21.15.55:0/2167987454 >> 172.21.15.55:0/2167987454 conn(0x7f4a6c0cc060 msgr2=0x7f4a6c0cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:46.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.227+0000 7f4a6b200640 1 -- 172.21.15.55:0/2167987454 shutdown_connections 2024-05-10T03:30:46.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.228+0000 7f4a6b200640 1 -- 172.21.15.55:0/2167987454 wait complete. 2024-05-10T03:30:46.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.230+0000 7f4a6b200640 1 Processor -- start 2024-05-10T03:30:46.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.230+0000 7f4a6b200640 1 -- start start 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c06ae80 0x7f4a6c1f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c1f19c0 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0a8820 con 0x7f4a6c0cd190 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c1f35b0 con 0x7f4a6c06ae80 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53526/0 (socket says 172.21.15.55:53526) 2024-05-10T03:30:46.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a6a800640 1 -- 172.21.15.55:0/870208742 learned_addr learned my addr 172.21.15.55:0/870208742 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a69400640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.232+0000 7f4a69e00640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c06ae80 0x7f4a6c1f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 -- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c06ae80 msgr2=0x7f4a6c1f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c06ae80 0x7f4a6c1f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 -- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a6c0cd190 msgr2=0x7f4a6c0706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c0706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 -- 172.21.15.55:0/870208742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5800b770 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6a800640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a64007e60 tx=0x7f4a6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64014420 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a640145a0 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64014950 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.233+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c0cd720 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.234+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c0a8d50 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.235+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c04e290 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.236+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a64014ab0 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.237+0000 7f4a62000640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 0x7f4a4c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.237+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4a6408a190 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.238+0000 7f4a69e00640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 0x7f4a4c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.239+0000 7f4a69e00640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 0x7f4a4c06d110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00a8d0 tx=0x7f4a5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.242+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a64054e40 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.442+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a6c1f02a0 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.443+0000 7f4a62000640 1 -- 172.21.15.55:0/870208742 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a64058460 con 0x7f4a6c15e0f0 2024-05-10T03:30:46.446 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:46.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 msgr2=0x7f4a4c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 0x7f4a4c06d110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00a8d0 tx=0x7f4a5c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:46.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 msgr2=0x7f4a6c0abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a64007e60 tx=0x7f4a6400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 shutdown_connections 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a6c15e0f0 0x7f4a6c0abb00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a4c06ac00 0x7f4a4c06d110 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a6c0cd190 0x7f4a6c0706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 --2- 172.21.15.55:0/870208742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a6c06ae80 0x7f4a6c1f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 >> 172.21.15.55:0/870208742 conn(0x7f4a6c0cc060 msgr2=0x7f4a6c1590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:46.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.446+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 shutdown_connections 2024-05-10T03:30:46.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.447+0000 7f4a6b200640 1 -- 172.21.15.55:0/870208742 wait complete. 2024-05-10T03:30:46.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:46.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:30:46.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:30:46.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.741+0000 7f8b33000640 1 Processor -- start 2024-05-10T03:30:46.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.742+0000 7f8b33000640 1 -- start start 2024-05-10T03:30:46.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.743+0000 7f8b33000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8b2c141a30 0x7f8b2c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.743+0000 7f8b33000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b33000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c071ce0 con 0x7f8b2c070050 2024-05-10T03:30:46.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b33000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c070f40 con 0x7f8b2c15bce0 2024-05-10T03:30:46.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b33000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c15f280 con 0x7f8b2c141a30 2024-05-10T03:30:46.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b31000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b31000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53540/0 (socket says 172.21.15.55:53540) 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b31000640 1 -- 172.21.15.55:0/183212621 learned_addr learned my addr 172.21.15.55:0/183212621 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b2b400640 1 --2- 172.21.15.55:0/183212621 >> v2:172.21.15.162:3300/0 conn(0x7f8b2c141a30 0x7f8b2c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.744+0000 7f8b31000640 1 -- 172.21.15.55:0/183212621 >> v2:172.21.15.162:3300/0 conn(0x7f8b2c141a30 msgr2=0x7f8b2c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.745+0000 7f8b31000640 1 --2- 172.21.15.55:0/183212621 >> v2:172.21.15.162:3300/0 conn(0x7f8b2c141a30 0x7f8b2c0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.745+0000 7f8b31000640 1 -- 172.21.15.55:0/183212621 >> v1:172.21.15.162:6789/0 conn(0x7f8b2c070050 legacy=0x7f8b2c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.745+0000 7f8b31000640 1 -- 172.21.15.55:0/183212621 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c070460 con 0x7f8b2c15bce0 2024-05-10T03:30:46.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.746+0000 7f8b31000640 1 --2- 172.21.15.55:0/183212621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c066c20 tx=0x7f8b2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91da07bfbaffdea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.746+0000 7f8b2aa00640 1 -- 172.21.15.55:0/183212621 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24032440 con 0x7f8b2c15bce0 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.746+0000 7f8b2aa00640 1 -- 172.21.15.55:0/183212621 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240325a0 con 0x7f8b2c15bce0 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.746+0000 7f8b2aa00640 1 -- 172.21.15.55:0/183212621 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24032930 con 0x7f8b2c15bce0 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 -- 172.21.15.55:0/183212621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c15bce0 msgr2=0x7f8b2c15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 --2- 172.21.15.55:0/183212621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c066c20 tx=0x7f8b2402a230 comp rx=0 tx=0).stop 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 -- 172.21.15.55:0/183212621 shutdown_connections 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 --2- 172.21.15.55:0/183212621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c15c0f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 --2- 172.21.15.55:0/183212621 >> v2:172.21.15.162:3300/0 conn(0x7f8b2c141a30 0x7f8b2c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 -- 172.21.15.55:0/183212621 >> 172.21.15.55:0/183212621 conn(0x7f8b2c06c120 msgr2=0x7f8b2c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 -- 172.21.15.55:0/183212621 shutdown_connections 2024-05-10T03:30:46.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.747+0000 7f8b33000640 1 -- 172.21.15.55:0/183212621 wait complete. 2024-05-10T03:30:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.749+0000 7f8b33000640 1 Processor -- start 2024-05-10T03:30:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.750+0000 7f8b33000640 1 -- start start 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b2c070050 0x7f8b2c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c1f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c140ad0 con 0x7f8b2c141a30 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c1f13c0 con 0x7f8b2c15bce0 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.751+0000 7f8b33000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c13fec0 con 0x7f8b2c070050 2024-05-10T03:30:46.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53554/0 (socket says 172.21.15.55:53554) 2024-05-10T03:30:46.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 -- 172.21.15.55:0/4171330233 learned_addr learned my addr 172.21.15.55:0/4171330233 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:46.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b31000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c1f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2be00640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b2c070050 0x7f8b2c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 -- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b2c070050 msgr2=0x7f8b2c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b2c070050 0x7f8b2c0adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 -- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b2c15bce0 msgr2=0x7f8b2c1f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c1f0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.752+0000 7f8b2b400640 1 -- 172.21.15.55:0/4171330233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b240094d0 con 0x7f8b2c141a30 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b2b400640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400d5f0 tx=0x7f8b1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b140125f0 con 0x7f8b2c141a30 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14012750 con 0x7f8b2c141a30 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14012b00 con 0x7f8b2c141a30 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c1401a0 con 0x7f8b2c141a30 2024-05-10T03:30:46.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.753+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c140760 con 0x7f8b2c141a30 2024-05-10T03:30:46.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.755+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c066f60 con 0x7f8b2c141a30 2024-05-10T03:30:46.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.756+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8b14012c60 con 0x7f8b2c141a30 2024-05-10T03:30:46.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.757+0000 7f8b23e00640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 0x7f8b0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:46.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.757+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8b14089b90 con 0x7f8b2c141a30 2024-05-10T03:30:46.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.757+0000 7f8b2be00640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 0x7f8b0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:46.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.759+0000 7f8b2be00640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 0x7f8b0806d140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c15ce50 tx=0x7f8b1c0094d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:46.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.765+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8b140548d0 con 0x7f8b2c141a30 2024-05-10T03:30:46.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.968+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b2c067220 con 0x7f8b2c141a30 2024-05-10T03:30:46.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.968+0000 7f8b23e00640 1 -- 172.21.15.55:0/4171330233 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b14057ef0 con 0x7f8b2c141a30 2024-05-10T03:30:46.972 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 msgr2=0x7f8b0806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 0x7f8b0806d140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c15ce50 tx=0x7f8b1c0094d0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 msgr2=0x7f8b2c1f0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400d5f0 tx=0x7f8b1400dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 shutdown_connections 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8b0806ac30 0x7f8b0806d140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b2c15bce0 0x7f8b2c1f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b2c141a30 0x7f8b2c1f0940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 --2- 172.21.15.55:0/4171330233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b2c070050 0x7f8b2c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:46.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 >> 172.21.15.55:0/4171330233 conn(0x7f8b2c06c120 msgr2=0x7f8b2c13e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:46.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 shutdown_connections 2024-05-10T03:30:46.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:46.971+0000 7f8b33000640 1 -- 172.21.15.55:0/4171330233 wait complete. 2024-05-10T03:30:47.001 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.310+0000 7f844e800640 1 Processor -- start 2024-05-10T03:30:47.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.310+0000 7f844e800640 1 -- start start 2024-05-10T03:30:47.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f844815fe50 0x7f8448141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84480b2140 0x7f8448142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 0x7f8448142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844806ad20 con 0x7f84480b2140 2024-05-10T03:30:47.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8448141f50 con 0x7f844815fe50 2024-05-10T03:30:47.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.312+0000 7f844e800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844806a110 con 0x7f84480b2550 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84480b2140 0x7f8448142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84480b2140 0x7f8448142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53568/0 (socket says 172.21.15.55:53568) 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 -- 172.21.15.55:0/827132720 learned_addr learned my addr 172.21.15.55:0/827132720 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f8447e00640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 0x7f8448142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f8447400640 1 --2- 172.21.15.55:0/827132720 >> v2:172.21.15.162:3300/0 conn(0x7f844815fe50 0x7f8448141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 -- 172.21.15.55:0/827132720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 msgr2=0x7f8448142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 0x7f8448142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 -- 172.21.15.55:0/827132720 >> v2:172.21.15.162:3300/0 conn(0x7f844815fe50 msgr2=0x7f8448141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 --2- 172.21.15.55:0/827132720 >> v2:172.21.15.162:3300/0 conn(0x7f844815fe50 0x7f8448141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f843fe00640 1 -- 172.21.15.55:0/827132720 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844806a3f0 con 0x7f84480b2140 2024-05-10T03:30:47.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.313+0000 7f8447e00640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 0x7f8448142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:30:47.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.315+0000 7f843fe00640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f8448142440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f844806e520 tx=0x7f843002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87898dfe9198ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.315+0000 7f8446a00640 1 -- 172.21.15.55:0/827132720 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430032440 con 0x7f84480b2140 2024-05-10T03:30:47.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.315+0000 7f8446a00640 1 -- 172.21.15.55:0/827132720 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84300325a0 con 0x7f84480b2140 2024-05-10T03:30:47.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.315+0000 7f8446a00640 1 -- 172.21.15.55:0/827132720 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430032930 con 0x7f84480b2140 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.316+0000 7f844e800640 1 -- 172.21.15.55:0/827132720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 msgr2=0x7f8448142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.316+0000 7f844e800640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f8448142440 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f844806e520 tx=0x7f843002a270 comp rx=0 tx=0).stop 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.316+0000 7f844e800640 1 -- 172.21.15.55:0/827132720 shutdown_connections 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.316+0000 7f844e800640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84480b2550 0x7f8448142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.317+0000 7f844e800640 1 --2- 172.21.15.55:0/827132720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f8448142440 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.317+0000 7f844e800640 1 --2- 172.21.15.55:0/827132720 >> v2:172.21.15.162:3300/0 conn(0x7f844815fe50 0x7f8448141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.317+0000 7f844e800640 1 -- 172.21.15.55:0/827132720 >> 172.21.15.55:0/827132720 conn(0x7f84480cbb90 msgr2=0x7f84480cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.317+0000 7f844e800640 1 -- 172.21.15.55:0/827132720 shutdown_connections 2024-05-10T03:30:47.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.317+0000 7f844e800640 1 -- 172.21.15.55:0/827132720 wait complete. 2024-05-10T03:30:47.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.319+0000 7f844e800640 1 Processor -- start 2024-05-10T03:30:47.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.320+0000 7f844e800640 1 -- start start 2024-05-10T03:30:47.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84480b2550 0x7f8448142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 0x7f8448142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481e50a0 con 0x7f84480b2140 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84481f5230 con 0x7f84480b2550 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f844e800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8448142c00 con 0x7f844815fe50 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f8447400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f8447400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53572/0 (socket says 172.21.15.55:53572) 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.321+0000 7f8447400640 1 -- 172.21.15.55:0/1798734422 learned_addr learned my addr 172.21.15.55:0/1798734422 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f843fe00640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84480b2550 0x7f8448142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447e00640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 0x7f8448142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 -- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 msgr2=0x7f8448142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 0x7f8448142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 -- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84480b2550 msgr2=0x7f8448142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84480b2550 0x7f8448142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 -- 172.21.15.55:0/1798734422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300094d0 con 0x7f84480b2140 2024-05-10T03:30:47.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447e00640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 0x7f8448142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:47.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.322+0000 7f8447400640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f843800d650 tx=0x7f843800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.323+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800b6c0 con 0x7f84480b2140 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.323+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843800b820 con 0x7f84480b2140 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.323+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800bbd0 con 0x7f84480b2140 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.323+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8448143a60 con 0x7f84480b2140 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.323+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844813cf00 con 0x7f84480b2140 2024-05-10T03:30:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.324+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8448066f60 con 0x7f84480b2140 2024-05-10T03:30:47.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.325+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f843800bd30 con 0x7f84480b2140 2024-05-10T03:30:47.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.326+0000 7f843f400640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 0x7f842406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.326+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8438089c50 con 0x7f84480b2140 2024-05-10T03:30:47.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.327+0000 7f843fe00640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 0x7f842406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.329+0000 7f843fe00640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 0x7f842406d140 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8430034000 tx=0x7f843002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.331+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8438054900 con 0x7f84480b2140 2024-05-10T03:30:47.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.533+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8448067220 con 0x7f84480b2140 2024-05-10T03:30:47.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.534+0000 7f843f400640 1 -- 172.21.15.55:0/1798734422 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8438057f20 con 0x7f84480b2140 2024-05-10T03:30:47.537 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 msgr2=0x7f842406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 0x7f842406d140 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8430034000 tx=0x7f843002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 msgr2=0x7f844813c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f843800d650 tx=0x7f843800db10 comp rx=0 tx=0).stop 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 shutdown_connections 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f844815fe50 0x7f8448142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f842406ac30 0x7f842406d140 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84480b2550 0x7f8448142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 --2- 172.21.15.55:0/1798734422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84480b2140 0x7f844813c9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 >> 172.21.15.55:0/1798734422 conn(0x7f84480cbb90 msgr2=0x7f844815bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 shutdown_connections 2024-05-10T03:30:47.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.537+0000 7f844e800640 1 -- 172.21.15.55:0/1798734422 wait complete. 2024-05-10T03:30:47.567 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.839+0000 7fbbc3e00640 1 Processor -- start 2024-05-10T03:30:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.839+0000 7fbbc3e00640 1 -- start start 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.841+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbbc406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc40b2e20 con 0x7fbbc406f2e0 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc406cb80 con 0x7fbbc406c770 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc415a8c0 con 0x7fbbc4154b00 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc3400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbbc406f2e0 0x7fbbc406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44074/0 (socket says 172.21.15.55:44074) 2024-05-10T03:30:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc3400640 1 -- 172.21.15.55:0/3925467731 learned_addr learned my addr 172.21.15.55:0/3925467731 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc2a00640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbbc406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701560061 0 0) 0x7fbbc40b2e20 con 0x7fbbc406f2e0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.842+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba8003520 con 0x7fbbc406f2e0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974043467 0 0) 0x7fbba8003520 con 0x7fbbc406f2e0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 msgr2=0x7fbbc406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbbc406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 >> v1:172.21.15.162:6789/0 conn(0x7fbbc406c770 legacy=0x7fbbc406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc415ab80 con 0x7fbbc406f2e0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbbbc001a10 con 0x7fbbc406f2e0 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 >> v1:172.21.15.55:6789/0 conn(0x7fbbc406f2e0 legacy=0x7fbbc406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8003a10 0x7fbba8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbba80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbba8004d00 0x7fbba8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc415a8c0 con 0x7fbba8004d00 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800bb40 con 0x7fbba8003a10 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800bd00 con 0x7fbbc4154b00 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.843+0000 7fbbc3400640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbba8004d00 0x7fbba8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc2000640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbba80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc2a00640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8003a10 0x7fbba8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc3400640 1 -- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 msgr2=0x7fbba80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc3400640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbba80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc3400640 1 -- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8003a10 msgr2=0x7fbba8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc3400640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8003a10 0x7fbba8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc3400640 1 -- 172.21.15.55:0/3925467731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8003520 con 0x7fbba8004d00 2024-05-10T03:30:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.844+0000 7fbbc2000640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc4154b00 0x7fbba80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.845+0000 7fbbc3400640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbba8004d00 0x7fbba8005190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40b5450 tx=0x7fbba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22d4b7caec1316 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.845+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc02b9a0 con 0x7fbba8004d00 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.845+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc02bb00 con 0x7fbba8004d00 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.845+0000 7fbbc1600640 1 -- 172.21.15.55:0/3925467731 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc034470 con 0x7fbba8004d00 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.846+0000 7fbbc3e00640 1 -- 172.21.15.55:0/3925467731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbba8004d00 msgr2=0x7fbba8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.846+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/3925467731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbba8004d00 0x7fbba8005190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40b5450 tx=0x7fbba80057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.846+0000 7fbbc3400640 1 -- 172.21.15.55:0/3925467731 reap_dead start 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.847+0000 7fbbc3e00640 1 -- 172.21.15.55:0/3925467731 shutdown_connections 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.847+0000 7fbbc3e00640 1 -- 172.21.15.55:0/3925467731 >> 172.21.15.55:0/3925467731 conn(0x7fbbc4070eb0 msgr2=0x7fbbc40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.847+0000 7fbbc3e00640 1 -- 172.21.15.55:0/3925467731 shutdown_connections 2024-05-10T03:30:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.847+0000 7fbbc3e00640 1 -- 172.21.15.55:0/3925467731 wait complete. 2024-05-10T03:30:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.849+0000 7fbbc3e00640 1 Processor -- start 2024-05-10T03:30:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.849+0000 7fbbc3e00640 1 -- start start 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8004d00 0x7fbbc40cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbc40ada40 0x7fbbc40ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40ac8e0 con 0x7fbbc41f6a50 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc415d140 con 0x7fbbc40ada40 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc41f73a0 con 0x7fbba8004d00 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53590/0 (socket says 172.21.15.55:53590) 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 -- 172.21.15.55:0/2822612120 learned_addr learned my addr 172.21.15.55:0/2822612120 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 -- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8004d00 msgr2=0x7fbbc40cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc3400640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbc40ada40 0x7fbbc40ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8004d00 0x7fbbc40cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 -- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbc40ada40 msgr2=0x7fbbc40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.851+0000 7fbbc2000640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbc40ada40 0x7fbbc40ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbc2000640 1 -- 172.21.15.55:0/2822612120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc40ae390 con 0x7fbbc41f6a50 2024-05-10T03:30:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbc2000640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d7d0 tx=0x7fbbb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0017070 con 0x7fbbc41f6a50 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0005070 con 0x7fbbc41f6a50 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0005440 con 0x7fbbc41f6a50 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc415d390 con 0x7fbbc41f6a50 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.852+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc415d670 con 0x7fbbc41f6a50 2024-05-10T03:30:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.854+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc404e290 con 0x7fbbc41f6a50 2024-05-10T03:30:47.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.855+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbbb0007030 con 0x7fbbc41f6a50 2024-05-10T03:30:47.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.856+0000 7fbbbaa00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 0x7fbba406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:47.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.856+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fbbb0089170 con 0x7fbbc41f6a50 2024-05-10T03:30:47.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.856+0000 7fbbc2a00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 0x7fbba406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:47.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.858+0000 7fbbc2a00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 0x7fbba406d110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4071e30 tx=0x7fbbb4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:47.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:47.860+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbbb0053ec0 con 0x7fbbc41f6a50 2024-05-10T03:30:48.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.054+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbbc415ea60 con 0x7fbbc41f6a50 2024-05-10T03:30:48.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.055+0000 7fbbbaa00640 1 -- 172.21.15.55:0/2822612120 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbbb00574e0 con 0x7fbbc41f6a50 2024-05-10T03:30:48.058 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:48.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 msgr2=0x7fbba406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 0x7fbba406d110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4071e30 tx=0x7fbbb4008040 comp rx=0 tx=0).stop 2024-05-10T03:30:48.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 msgr2=0x7fbbc41f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d7d0 tx=0x7fbbb000dc90 comp rx=0 tx=0).stop 2024-05-10T03:30:48.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 shutdown_connections 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbba406ac00 0x7fbba406d110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbbc40ada40 0x7fbbc40ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbc41f6a50 0x7fbbc41f6e60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 --2- 172.21.15.55:0/2822612120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8004d00 0x7fbbc40cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 >> 172.21.15.55:0/2822612120 conn(0x7fbbc4070eb0 msgr2=0x7fbbc40afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.057+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 shutdown_connections 2024-05-10T03:30:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.058+0000 7fbbc3e00640 1 -- 172.21.15.55:0/2822612120 wait complete. 2024-05-10T03:30:48.110 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2024-05-10T03:30:48.110 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:30:48.110 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.381+0000 7f1835a00640 1 Processor -- start 2024-05-10T03:30:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.382+0000 7f1835a00640 1 -- start start 2024-05-10T03:30:48.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f1835a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f183015fe50 0x7f1830141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f1835a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1830142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f1835a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183006ad20 con 0x7f18300b2550 2024-05-10T03:30:48.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f1835a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1830141f50 con 0x7f183015fe50 2024-05-10T03:30:48.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f1835a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183006a110 con 0x7f18300b2140 2024-05-10T03:30:48.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f182f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f18300b2550 0x7f183006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44082/0 (socket says 172.21.15.55:44082) 2024-05-10T03:30:48.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.384+0000 7f182f400640 1 -- 172.21.15.55:0/3465218634 learned_addr learned my addr 172.21.15.55:0/3465218634 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182ea00640 1 --2- 172.21.15.55:0/3465218634 >> v2:172.21.15.162:3300/0 conn(0x7f183015fe50 0x7f1830141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f1827e00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1830142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500347914 0 0) 0x7f183006ad20 con 0x7f18300b2550 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1814003520 con 0x7f18300b2550 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458773129 0 0) 0x7f1814003520 con 0x7f18300b2550 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 msgr2=0x7f1830142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1830142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 >> v2:172.21.15.162:3300/0 conn(0x7f183015fe50 msgr2=0x7f1830141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 --2- 172.21.15.55:0/3465218634 >> v2:172.21.15.162:3300/0 conn(0x7f183015fe50 0x7f1830141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183006a3f0 con 0x7f18300b2550 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.385+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f18280019f0 con 0x7f18300b2550 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 >> v1:172.21.15.55:6789/0 conn(0x7f18300b2550 legacy=0x7f183006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1814003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1814003ef0 0x7f18140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1814004d00 0x7f1814005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830141f50 con 0x7f1814003ef0 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183006a110 con 0x7f1814004d00 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181400ba60 con 0x7f18300b2140 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1814003ef0 0x7f18140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182ea00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1814003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 -- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 msgr2=0x7f1814003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1814003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 -- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1814004d00 msgr2=0x7f1814005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1814004d00 0x7f1814005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f1827e00640 1 -- 172.21.15.55:0/3465218634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814003520 con 0x7f1814003ef0 2024-05-10T03:30:48.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.386+0000 7f182ea00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18300b2140 0x7f1814003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.387+0000 7f1827e00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1814003ef0 0x7f18140047c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1818009a90 tx=0x7f18140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850f7038dce5addc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818031400 con 0x7f1814003ef0 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181802d050 con 0x7f1814003ef0 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f182e000640 1 -- 172.21.15.55:0/3465218634 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818031780 con 0x7f1814003ef0 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f1835a00640 1 -- 172.21.15.55:0/3465218634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1814003ef0 msgr2=0x7f18140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f1835a00640 1 --2- 172.21.15.55:0/3465218634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1814003ef0 0x7f18140047c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1818009a90 tx=0x7f18140057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.388+0000 7f182f400640 1 -- 172.21.15.55:0/3465218634 reap_dead start 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.389+0000 7f1835a00640 1 -- 172.21.15.55:0/3465218634 shutdown_connections 2024-05-10T03:30:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.389+0000 7f1835a00640 1 -- 172.21.15.55:0/3465218634 >> 172.21.15.55:0/3465218634 conn(0x7f18300cbb90 msgr2=0x7f18300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:48.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.389+0000 7f1835a00640 1 -- 172.21.15.55:0/3465218634 shutdown_connections 2024-05-10T03:30:48.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.389+0000 7f1835a00640 1 -- 172.21.15.55:0/3465218634 wait complete. 2024-05-10T03:30:48.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.391+0000 7f1835a00640 1 Processor -- start 2024-05-10T03:30:48.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.391+0000 7f1835a00640 1 -- start start 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1814003ef0 0x7f18301e9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 0x7f18301e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301e9680 con 0x7f18301e5b70 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301e86e0 con 0x7f18301e5760 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f1835a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830143970 con 0x7f1814003ef0 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f182f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f182f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53608/0 (socket says 172.21.15.55:53608) 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.393+0000 7f182f400640 1 -- 172.21.15.55:0/757071856 learned_addr learned my addr 172.21.15.55:0/757071856 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182ea00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1814003ef0 0x7f18301e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1827e00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 0x7f18301e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 -- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1814003ef0 msgr2=0x7f18301e9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1814003ef0 0x7f18301e9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 -- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 msgr2=0x7f18301e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 0x7f18301e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 -- 172.21.15.55:0/757071856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830143bc0 con 0x7f18301e5b70 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1827e00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 0x7f18301e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:48.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f182f400640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f183006d310 tx=0x7f182800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800ce50 con 0x7f18301e5b70 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182800a610 con 0x7f18301e5b70 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800a9c0 con 0x7f18301e5b70 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.394+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830143ea0 con 0x7f18301e5b70 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.395+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830144460 con 0x7f18301e5b70 2024-05-10T03:30:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.396+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830066f60 con 0x7f18301e5b70 2024-05-10T03:30:48.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.397+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f182800ab20 con 0x7f18301e5b70 2024-05-10T03:30:48.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.398+0000 7f1826a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 0x7f180c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.398+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f18280888a0 con 0x7f18301e5b70 2024-05-10T03:30:48.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.398+0000 7f182ea00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 0x7f180c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.399+0000 7f182ea00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 0x7f180c06d110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1820009b20 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.403+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1828053550 con 0x7f18301e5b70 2024-05-10T03:30:48.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.605+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1830067220 con 0x7f18301e5b70 2024-05-10T03:30:48.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.608+0000 7f1826a00640 1 -- 172.21.15.55:0/757071856 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f1828056b70 con 0x7f18301e5b70 2024-05-10T03:30:48.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:48.612 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:48.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.610+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 msgr2=0x7f180c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.610+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 0x7f180c06d110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1820009b20 tx=0x7f1820008040 comp rx=0 tx=0).stop 2024-05-10T03:30:48.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 msgr2=0x7f18301e78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f183006d310 tx=0x7f182800cc00 comp rx=0 tx=0).stop 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 shutdown_connections 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18301e5b70 0x7f18301e78e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f180c06ac00 0x7f180c06d110 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18301e5760 0x7f18301e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 --2- 172.21.15.55:0/757071856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1814003ef0 0x7f18301e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 >> 172.21.15.55:0/757071856 conn(0x7f18300cbb90 msgr2=0x7f18300701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 shutdown_connections 2024-05-10T03:30:48.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.611+0000 7f1835a00640 1 -- 172.21.15.55:0/757071856 wait complete. 2024-05-10T03:30:48.642 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:48.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.923+0000 7f60f5a00640 1 Processor -- start 2024-05-10T03:30:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.923+0000 7f60f5a00640 1 -- start start 2024-05-10T03:30:48.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.925+0000 7f60f5a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 0x7f60f00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60f5a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 0x7f60f015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60f5a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f015b5e0 con 0x7f60f0140900 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60f5a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f00b4380 con 0x7f60f00cc3a0 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60f5a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015acf0 con 0x7f60f006b830 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60ef400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 0x7f60f015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60ee000640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f60f0140900 0x7f60f006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:37568/0 (socket says 172.21.15.55:37568) 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60ee000640 1 -- 172.21.15.55:0/936578257 learned_addr learned my addr 172.21.15.55:0/936578257 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:48.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.926+0000 7f60eea00640 1 --2- 172.21.15.55:0/936578257 >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 0x7f60f00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ed600640 1 -- 172.21.15.55:0/936578257 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804290549 0 0) 0x7f60f015b5e0 con 0x7f60f0140900 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ed600640 1 -- 172.21.15.55:0/936578257 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d4003520 con 0x7f60f0140900 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ef400640 1 -- 172.21.15.55:0/936578257 >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 msgr2=0x7f60f00cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ef400640 1 --2- 172.21.15.55:0/936578257 >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 0x7f60f00cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ef400640 1 -- 172.21.15.55:0/936578257 >> v1:172.21.15.162:6789/0 conn(0x7f60f0140900 legacy=0x7f60f006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60ef400640 1 -- 172.21.15.55:0/936578257 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f015af90 con 0x7f60f006b830 2024-05-10T03:30:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.927+0000 7f60eea00640 1 --2- 172.21.15.55:0/936578257 >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 0x7f60f00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.929+0000 7f60ef400640 1 --2- 172.21.15.55:0/936578257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 0x7f60f015e1d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60e80097a0 tx=0x7f60e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983f97a00cfb1bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.929+0000 7f60ed600640 1 -- 172.21.15.55:0/936578257 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8032440 con 0x7f60f006b830 2024-05-10T03:30:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.929+0000 7f60ed600640 1 -- 172.21.15.55:0/936578257 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e80325a0 con 0x7f60f006b830 2024-05-10T03:30:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.929+0000 7f60ed600640 1 -- 172.21.15.55:0/936578257 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8032930 con 0x7f60f006b830 2024-05-10T03:30:48.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 -- 172.21.15.55:0/936578257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 msgr2=0x7f60f015e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 --2- 172.21.15.55:0/936578257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 0x7f60f015e1d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60e80097a0 tx=0x7f60e802a180 comp rx=0 tx=0).stop 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 -- 172.21.15.55:0/936578257 shutdown_connections 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 --2- 172.21.15.55:0/936578257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f006b830 0x7f60f015e1d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 --2- 172.21.15.55:0/936578257 >> v2:172.21.15.162:3300/0 conn(0x7f60f00cc3a0 0x7f60f00cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 -- 172.21.15.55:0/936578257 >> 172.21.15.55:0/936578257 conn(0x7f60f006c120 msgr2=0x7f60f006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 -- 172.21.15.55:0/936578257 shutdown_connections 2024-05-10T03:30:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.930+0000 7f60f5a00640 1 -- 172.21.15.55:0/936578257 wait complete. 2024-05-10T03:30:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.933+0000 7f60f5a00640 1 Processor -- start 2024-05-10T03:30:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.933+0000 7f60f5a00640 1 -- start start 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 0x7f60f0134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f0140900 0x7f60f013af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0138d50 con 0x7f60f014f670 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f013b7e0 con 0x7f60f00cc3a0 2024-05-10T03:30:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.934+0000 7f60f5a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0137580 con 0x7f60f0140900 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53624/0 (socket says 172.21.15.55:53624) 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 -- 172.21.15.55:0/659852584 learned_addr learned my addr 172.21.15.55:0/659852584 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ee000640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f0140900 0x7f60f013af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60eea00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 0x7f60f0134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 -- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f0140900 msgr2=0x7f60f013af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f0140900 0x7f60f013af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 -- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 msgr2=0x7f60f0134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 0x7f60f0134380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60ef400640 1 -- 172.21.15.55:0/659852584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009a70 con 0x7f60f014f670 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.935+0000 7f60eea00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 0x7f60f0134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:48.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60ef400640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60e8035040 tx=0x7f60e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e803d830 con 0x7f60f014f670 2024-05-10T03:30:48.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e803d9b0 con 0x7f60f014f670 2024-05-10T03:30:48.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e803dd60 con 0x7f60f014f670 2024-05-10T03:30:48.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8009400 con 0x7f60f014f670 2024-05-10T03:30:48.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.936+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f013ba30 con 0x7f60f014f670 2024-05-10T03:30:48.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.937+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0066f60 con 0x7f60f014f670 2024-05-10T03:30:48.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.939+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f60e8039040 con 0x7f60f014f670 2024-05-10T03:30:48.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.940+0000 7f60e6a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 0x7f60cc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:48.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.940+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f60e80a9f80 con 0x7f60f014f670 2024-05-10T03:30:48.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.940+0000 7f60eea00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 0x7f60cc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:48.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.942+0000 7f60eea00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 0x7f60cc06d110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009fd0 tx=0x7f60e000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:48.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:48.944+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f60e80dd900 con 0x7f60f014f670 2024-05-10T03:30:49.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.133+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60f0067220 con 0x7f60f014f670 2024-05-10T03:30:49.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.134+0000 7f60e6a00640 1 -- 172.21.15.55:0/659852584 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60e80370a0 con 0x7f60f014f670 2024-05-10T03:30:49.139 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:49.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 msgr2=0x7f60cc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 0x7f60cc06d110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009fd0 tx=0x7f60e000b040 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 msgr2=0x7f60f014fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f60e8035040 tx=0x7f60e80046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 shutdown_connections 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f014f670 0x7f60f014fa80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f0140900 0x7f60f013af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f60cc06ac00 0x7f60cc06d110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 --2- 172.21.15.55:0/659852584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00cc3a0 0x7f60f0134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 >> 172.21.15.55:0/659852584 conn(0x7f60f006c120 msgr2=0x7f60f00705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 shutdown_connections 2024-05-10T03:30:49.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.137+0000 7f60f5a00640 1 -- 172.21.15.55:0/659852584 wait complete. 2024-05-10T03:30:49.166 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:49.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.446+0000 7f694d600640 1 Processor -- start 2024-05-10T03:30:49.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.446+0000 7f694d600640 1 -- start start 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f694d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f69481405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f694d600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694815fdf0 con 0x7f69480719f0 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f694d600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694815f1e0 con 0x7f6948071e00 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f694d600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694815f4c0 con 0x7f69480710f0 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f6947e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f69480719f0 0x7f694806b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44096/0 (socket says 172.21.15.55:44096) 2024-05-10T03:30:49.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f6947e00640 1 -- 172.21.15.55:0/3562429596 learned_addr learned my addr 172.21.15.55:0/3562429596 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.449+0000 7f693fe00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f69481405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213524065 0 0) 0x7f694815fdf0 con 0x7f69480719f0 2024-05-10T03:30:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6928003520 con 0x7f69480719f0 2024-05-10T03:30:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644615414 0 0) 0x7f6928003520 con 0x7f69480719f0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 msgr2=0x7f69481405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f69481405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 >> v1:172.21.15.162:6789/0 conn(0x7f6948071e00 legacy=0x7f6948140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694815f7a0 con 0x7f69480719f0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f6938001670 con 0x7f69480719f0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 >> v1:172.21.15.55:6789/0 conn(0x7f69480719f0 legacy=0x7f694806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f6928003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69280047c0 0x7f6928004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f692800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694815f4c0 con 0x7f6928003a10 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280054c0 con 0x7f69280047c0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.450+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6928005680 con 0x7f69480710f0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f6928003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f694cc00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f692800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 -- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 msgr2=0x7f692800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:49.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69480710f0 0x7f692800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 -- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69280047c0 msgr2=0x7f6928004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:49.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69280047c0 0x7f6928004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.451+0000 7f6947e00640 1 -- 172.21.15.55:0/3562429596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6928003520 con 0x7f6928003a10 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.452+0000 7f6947e00640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f6928003e80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f693800c240 tx=0x7f69280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a7133d6d27d457 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.452+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693802e070 con 0x7f6928003a10 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.452+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69380058d0 con 0x7f6928003a10 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.452+0000 7f6947400640 1 -- 172.21.15.55:0/3562429596 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938005cc0 con 0x7f6928003a10 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 -- 172.21.15.55:0/3562429596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 msgr2=0x7f6928003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 --2- 172.21.15.55:0/3562429596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f6928003e80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f693800c240 tx=0x7f69280057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694cc00640 1 -- 172.21.15.55:0/3562429596 reap_dead start 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 -- 172.21.15.55:0/3562429596 shutdown_connections 2024-05-10T03:30:49.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 -- 172.21.15.55:0/3562429596 >> 172.21.15.55:0/3562429596 conn(0x7f694806c750 msgr2=0x7f69480b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:49.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 -- 172.21.15.55:0/3562429596 shutdown_connections 2024-05-10T03:30:49.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.453+0000 7f694d600640 1 -- 172.21.15.55:0/3562429596 wait complete. 2024-05-10T03:30:49.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.456+0000 7f694d600640 1 Processor -- start 2024-05-10T03:30:49.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.456+0000 7f694d600640 1 -- start start 2024-05-10T03:30:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 0x7f69481f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 0x7f6948155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948155030 con 0x7f6928003a10 2024-05-10T03:30:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481f2cd0 con 0x7f694815aea0 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.457+0000 7f694d600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69481f1820 con 0x7f69481f2250 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53640/0 (socket says 172.21.15.55:53640) 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 -- 172.21.15.55:0/271976299 learned_addr learned my addr 172.21.15.55:0/271976299 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f693fe00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 0x7f69481f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f694cc00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 0x7f6948155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 -- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 msgr2=0x7f6948155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 0x7f6948155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 -- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 msgr2=0x7f69481f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 0x7f69481f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f6947e00640 1 -- 172.21.15.55:0/271976299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69481f15d0 con 0x7f6928003a10 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f694cc00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 0x7f6948155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.458+0000 7f693fe00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 0x7f69481f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f6947e00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f693802ad70 tx=0x7f6938001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938000ed0 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69380047e0 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938004b90 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69481f2030 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.459+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69481f1d50 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.460+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480cc610 con 0x7f6928003a10 2024-05-10T03:30:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.461+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6938004cf0 con 0x7f6928003a10 2024-05-10T03:30:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.462+0000 7f6944c00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 0x7f692406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.462+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f69380a9560 con 0x7f6928003a10 2024-05-10T03:30:49.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.463+0000 7f693fe00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 0x7f692406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.464+0000 7f693fe00640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 0x7f692406d140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6930009160 tx=0x7f6930008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.467+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f69380742a0 con 0x7f6928003a10 2024-05-10T03:30:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.655+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69481f15d0 con 0x7f6928003a10 2024-05-10T03:30:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.655+0000 7f6944c00640 1 -- 172.21.15.55:0/271976299 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69481f15d0 con 0x7f6928003a10 2024-05-10T03:30:49.660 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 msgr2=0x7f692406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 0x7f692406d140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6930009160 tx=0x7f6930008040 comp rx=0 tx=0).stop 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 msgr2=0x7f69481ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f693802ad70 tx=0x7f6938001a00 comp rx=0 tx=0).stop 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 shutdown_connections 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69481f2250 0x7f6948155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f692406ac30 0x7f692406d140 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f694815aea0 0x7f69481f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 --2- 172.21.15.55:0/271976299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6928003a10 0x7f69481ee710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.658+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 >> 172.21.15.55:0/271976299 conn(0x7f694806c750 msgr2=0x7f694815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.659+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 shutdown_connections 2024-05-10T03:30:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.659+0000 7f694d600640 1 -- 172.21.15.55:0/271976299 wait complete. 2024-05-10T03:30:49.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:49.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.973+0000 7f2fa0800640 1 Processor -- start 2024-05-10T03:30:49.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.973+0000 7f2fa0800640 1 -- start start 2024-05-10T03:30:49.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2fa0800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f9815fe50 0x7f2f98141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2fa0800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f980b2550 0x7f2f9806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2fa0800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f9806ad20 con 0x7f2f980b2140 2024-05-10T03:30:49.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2fa0800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f98141f50 con 0x7f2f980b2550 2024-05-10T03:30:49.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2fa0800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9806a110 con 0x7f2f9815fe50 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2f9d400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2f980b2140 0x7f2f98142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44108/0 (socket says 172.21.15.55:44108) 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.975+0000 7f2f9d400640 1 -- 172.21.15.55:0/1172069504 learned_addr learned my addr 172.21.15.55:0/1172069504 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9e800640 1 --2- 172.21.15.55:0/1172069504 >> v2:172.21.15.55:3300/0 conn(0x7f2f980b2550 0x7f2f9806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9de00640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f9815fe50 0x7f2f98141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9ca00640 1 -- 172.21.15.55:0/1172069504 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188706311 0 0) 0x7f2f9806ad20 con 0x7f2f980b2140 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9ca00640 1 -- 172.21.15.55:0/1172069504 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f80003520 con 0x7f2f980b2140 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9e800640 1 -- 172.21.15.55:0/1172069504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f9815fe50 msgr2=0x7f2f98141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9e800640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f9815fe50 0x7f2f98141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9e800640 1 -- 172.21.15.55:0/1172069504 >> v1:172.21.15.55:6789/0 conn(0x7f2f980b2140 legacy=0x7f2f98142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.976+0000 7f2f9e800640 1 -- 172.21.15.55:0/1172069504 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9806a3f0 con 0x7f2f980b2550 2024-05-10T03:30:49.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.977+0000 7f2f9e800640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2550 0x7f2f9806b1b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009d90 tx=0x7f2f9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe33e17c07a138d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.978+0000 7f2f9ca00640 1 -- 172.21.15.55:0/1172069504 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007740 con 0x7f2f980b2550 2024-05-10T03:30:49.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.978+0000 7f2f9ca00640 1 -- 172.21.15.55:0/1172069504 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9402d050 con 0x7f2f980b2550 2024-05-10T03:30:49.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.978+0000 7f2f9ca00640 1 -- 172.21.15.55:0/1172069504 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007ac0 con 0x7f2f980b2550 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.978+0000 7f2fa0800640 1 -- 172.21.15.55:0/1172069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2550 msgr2=0x7f2f9806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.978+0000 7f2fa0800640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2550 0x7f2f9806b1b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009d90 tx=0x7f2f9402a6f0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 -- 172.21.15.55:0/1172069504 shutdown_connections 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2550 0x7f2f9806b1b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 --2- 172.21.15.55:0/1172069504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f9815fe50 0x7f2f98141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 -- 172.21.15.55:0/1172069504 >> 172.21.15.55:0/1172069504 conn(0x7f2f980cbb90 msgr2=0x7f2f980cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:49.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 -- 172.21.15.55:0/1172069504 shutdown_connections 2024-05-10T03:30:49.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.979+0000 7f2fa0800640 1 -- 172.21.15.55:0/1172069504 wait complete. 2024-05-10T03:30:49.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.981+0000 7f2fa0800640 1 Processor -- start 2024-05-10T03:30:49.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.981+0000 7f2fa0800640 1 -- start start 2024-05-10T03:30:49.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 0x7f2f98144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f9815fe50 0x7f2f9806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9815b030 con 0x7f2f980b2140 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9815d4f0 con 0x7f2f9815fe50 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2fa0800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f981e6770 con 0x7f2f980b2550 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2f9de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53656/0 (socket says 172.21.15.55:53656) 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.983+0000 7f2f9de00640 1 -- 172.21.15.55:0/916688135 learned_addr learned my addr 172.21.15.55:0/916688135 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9d400640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 0x7f2f98144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9e800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f9815fe50 0x7f2f9806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 -- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 msgr2=0x7f2f98144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 0x7f2f98144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 -- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f9815fe50 msgr2=0x7f2f9806aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:49.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f9815fe50 0x7f2f9806aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 -- 172.21.15.55:0/916688135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f94009a70 con 0x7f2f980b2140 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9d400640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 0x7f2f98144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f9de00640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d620 tx=0x7f2f8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00b620 con 0x7f2f980b2140 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00b780 con 0x7f2f980b2140 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00bb30 con 0x7f2f980b2140 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9815bd20 con 0x7f2f980b2140 2024-05-10T03:30:49.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.984+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9815ab00 con 0x7f2f980b2140 2024-05-10T03:30:49.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.986+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98066f60 con 0x7f2f980b2140 2024-05-10T03:30:49.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.987+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f8c00bc90 con 0x7f2f980b2140 2024-05-10T03:30:49.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.988+0000 7f2f92000640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 0x7f2f7c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:49.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.988+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2f8c089d30 con 0x7f2f980b2140 2024-05-10T03:30:49.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.989+0000 7f2f9d400640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 0x7f2f7c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:49.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.990+0000 7f2f9d400640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 0x7f2f7c06d110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007fd0 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:49.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:49.993+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f8c0549e0 con 0x7f2f980b2140 2024-05-10T03:30:50.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.190+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f98067220 con 0x7f2f980b2140 2024-05-10T03:30:50.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.191+0000 7f2f92000640 1 -- 172.21.15.55:0/916688135 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f8c058000 con 0x7f2f980b2140 2024-05-10T03:30:50.198 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 msgr2=0x7f2f7c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 0x7f2f7c06d110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007fd0 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2024-05-10T03:30:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 msgr2=0x7f2f9815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d620 tx=0x7f2f8c00dae0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 shutdown_connections 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f7c06ac00 0x7f2f7c06d110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f9815fe50 0x7f2f9806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.194+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f980b2550 0x7f2f98144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.195+0000 7f2fa0800640 1 --2- 172.21.15.55:0/916688135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f980b2140 0x7f2f9815a5c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.195+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 >> 172.21.15.55:0/916688135 conn(0x7f2f980cbb90 msgr2=0x7f2f98140dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.195+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 shutdown_connections 2024-05-10T03:30:50.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.195+0000 7f2fa0800640 1 -- 172.21.15.55:0/916688135 wait complete. 2024-05-10T03:30:50.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:50.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.507+0000 7f2a9e000640 1 Processor -- start 2024-05-10T03:30:50.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.508+0000 7f2a9e000640 1 -- start start 2024-05-10T03:30:50.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.510+0000 7f2a9e000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a980cc3a0 0x7f2a980cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.510+0000 7f2a9e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a98140900 0x7f2a9806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.510+0000 7f2a9e000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9815b5e0 con 0x7f2a9806b660 2024-05-10T03:30:50.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a9e000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9815f1e0 con 0x7f2a980cc3a0 2024-05-10T03:30:50.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a9e000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980b4380 con 0x7f2a98140900 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a97400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a980cc3a0 0x7f2a980cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a97400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a980cc3a0 0x7f2a980cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53658/0 (socket says 172.21.15.55:53658) 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a97400640 1 -- 172.21.15.55:0/4217562954 learned_addr learned my addr 172.21.15.55:0/4217562954 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.511+0000 7f2a96a00640 1 --2- 172.21.15.55:0/4217562954 >> v2:172.21.15.162:3300/0 conn(0x7f2a98140900 0x7f2a9806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.512+0000 7f2a97400640 1 -- 172.21.15.55:0/4217562954 >> v2:172.21.15.162:3300/0 conn(0x7f2a98140900 msgr2=0x7f2a9806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.512+0000 7f2a97400640 1 --2- 172.21.15.55:0/4217562954 >> v2:172.21.15.162:3300/0 conn(0x7f2a98140900 0x7f2a9806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.512+0000 7f2a97400640 1 -- 172.21.15.55:0/4217562954 >> v1:172.21.15.55:6789/0 conn(0x7f2a9806b660 legacy=0x7f2a98071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.512+0000 7f2a97400640 1 -- 172.21.15.55:0/4217562954 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9806ba70 con 0x7f2a980cc3a0 2024-05-10T03:30:50.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.513+0000 7f2a97400640 1 --2- 172.21.15.55:0/4217562954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a980cc3a0 0x7f2a980cbbd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a9806f820 tx=0x7f2a8802c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f41d5cd99718f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:50.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.513+0000 7f2a96000640 1 -- 172.21.15.55:0/4217562954 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88031710 con 0x7f2a980cc3a0 2024-05-10T03:30:50.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.513+0000 7f2a96000640 1 -- 172.21.15.55:0/4217562954 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8802f050 con 0x7f2a980cc3a0 2024-05-10T03:30:50.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.513+0000 7f2a96000640 1 -- 172.21.15.55:0/4217562954 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88031a90 con 0x7f2a980cc3a0 2024-05-10T03:30:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 -- 172.21.15.55:0/4217562954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a980cc3a0 msgr2=0x7f2a980cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 --2- 172.21.15.55:0/4217562954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a980cc3a0 0x7f2a980cbbd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a9806f820 tx=0x7f2a8802c480 comp rx=0 tx=0).stop 2024-05-10T03:30:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 -- 172.21.15.55:0/4217562954 shutdown_connections 2024-05-10T03:30:50.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 --2- 172.21.15.55:0/4217562954 >> v2:172.21.15.162:3300/0 conn(0x7f2a98140900 0x7f2a9806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 --2- 172.21.15.55:0/4217562954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a980cc3a0 0x7f2a980cbbd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 -- 172.21.15.55:0/4217562954 >> 172.21.15.55:0/4217562954 conn(0x7f2a9806c120 msgr2=0x7f2a9806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:50.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 -- 172.21.15.55:0/4217562954 shutdown_connections 2024-05-10T03:30:50.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.514+0000 7f2a9e000640 1 -- 172.21.15.55:0/4217562954 wait complete. 2024-05-10T03:30:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.516+0000 7f2a9e000640 1 Processor -- start 2024-05-10T03:30:50.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.517+0000 7f2a9e000640 1 -- start start 2024-05-10T03:30:50.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9806b660 0x7f2a980a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a980cc3a0 0x7f2a9813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9813ffe0 con 0x7f2a98140900 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9814b7d0 con 0x7f2a9806b660 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.518+0000 7f2a9e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a981e5cd0 con 0x7f2a980cc3a0 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53664/0 (socket says 172.21.15.55:53664) 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 -- 172.21.15.55:0/2341212314 learned_addr learned my addr 172.21.15.55:0/2341212314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97400640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9806b660 0x7f2a980a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a96a00640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a980cc3a0 0x7f2a9813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 -- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a980cc3a0 msgr2=0x7f2a9813faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a980cc3a0 0x7f2a9813faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 -- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9806b660 msgr2=0x7f2a980a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9806b660 0x7f2a980a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.519+0000 7f2a97e00640 1 -- 172.21.15.55:0/2341212314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880094d0 con 0x7f2a98140900 2024-05-10T03:30:50.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a97e00640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c760 tx=0x7f2a9000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:50.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90003b70 con 0x7f2a98140900 2024-05-10T03:30:50.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a90003cd0 con 0x7f2a98140900 2024-05-10T03:30:50.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000aa90 con 0x7f2a98140900 2024-05-10T03:30:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a981e5fb0 con 0x7f2a98140900 2024-05-10T03:30:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.520+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a981e6570 con 0x7f2a98140900 2024-05-10T03:30:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.521+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a980a8b20 con 0x7f2a98140900 2024-05-10T03:30:50.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.522+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2a9000abf0 con 0x7f2a98140900 2024-05-10T03:30:50.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.523+0000 7f2a8f400640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 0x7f2a7806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:50.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.523+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2a90089540 con 0x7f2a98140900 2024-05-10T03:30:50.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.524+0000 7f2a97400640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 0x7f2a7806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:50.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.525+0000 7f2a97400640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 0x7f2a7806d110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a88035000 tx=0x7f2a88008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:50.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.528+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2a90054280 con 0x7f2a98140900 2024-05-10T03:30:50.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.725+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2a98066f60 con 0x7f2a98140900 2024-05-10T03:30:50.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.727+0000 7f2a8f400640 1 -- 172.21.15.55:0/2341212314 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2a900578a0 con 0x7f2a98140900 2024-05-10T03:30:50.731 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:50.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 msgr2=0x7f2a7806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 0x7f2a7806d110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a88035000 tx=0x7f2a88008030 comp rx=0 tx=0).stop 2024-05-10T03:30:50.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 msgr2=0x7f2a98149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:50.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000c760 tx=0x7f2a9000cc20 comp rx=0 tx=0).stop 2024-05-10T03:30:50.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 shutdown_connections 2024-05-10T03:30:50.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a98140900 0x7f2a98149030 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a980cc3a0 0x7f2a9813faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7806ac00 0x7f2a7806d110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 --2- 172.21.15.55:0/2341212314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9806b660 0x7f2a980a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:50.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.729+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 >> 172.21.15.55:0/2341212314 conn(0x7f2a9806c120 msgr2=0x7f2a98070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:50.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.730+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 shutdown_connections 2024-05-10T03:30:50.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:50.730+0000 7f2a9e000640 1 -- 172.21.15.55:0/2341212314 wait complete. 2024-05-10T03:30:50.759 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-05-10T03:30:51.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.030+0000 7f2633000640 1 Processor -- start 2024-05-10T03:30:51.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.031+0000 7f2633000640 1 -- start start 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.033+0000 7f2633000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.033+0000 7f2633000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f262c15ab30 0x7f262c0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.033+0000 7f2633000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c0cd710 con 0x7f262c15af40 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.033+0000 7f2633000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c0cdfb0 con 0x7f262c15ab30 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.033+0000 7f2633000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c06b4d0 con 0x7f262c0cdba0 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2623e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f262c15ab30 0x7f262c0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2631000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f262c15af40 0x7f262c0b2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44132/0 (socket says 172.21.15.55:44132) 2024-05-10T03:30:51.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2631000640 1 -- 172.21.15.55:0/3675766255 learned_addr learned my addr 172.21.15.55:0/3675766255 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f262be00640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2623e00640 1 -- 172.21.15.55:0/3675766255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 msgr2=0x7f262c06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2623e00640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c06a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2623e00640 1 -- 172.21.15.55:0/3675766255 >> v1:172.21.15.55:6789/0 conn(0x7f262c15af40 legacy=0x7f262c0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f2623e00640 1 -- 172.21.15.55:0/3675766255 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c06b1f0 con 0x7f262c15ab30 2024-05-10T03:30:51.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.034+0000 7f262b400640 1 -- 172.21.15.55:0/3675766255 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293855665 0 0) 0x7f262c0cd710 con 0x7f262c15af40 2024-05-10T03:30:51.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.035+0000 7f2623e00640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c0715e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26140097a0 tx=0x7f261402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5ef6324b6f60da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.036+0000 7f262b400640 1 -- 172.21.15.55:0/3675766255 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261402adb0 con 0x7f262c15ab30 2024-05-10T03:30:51.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.036+0000 7f262b400640 1 -- 172.21.15.55:0/3675766255 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614032440 con 0x7f262c15ab30 2024-05-10T03:30:51.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.036+0000 7f262b400640 1 -- 172.21.15.55:0/3675766255 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140327d0 con 0x7f262c15ab30 2024-05-10T03:30:51.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 -- 172.21.15.55:0/3675766255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 msgr2=0x7f262c0715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c0715e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26140097a0 tx=0x7f261402a060 comp rx=0 tx=0).stop 2024-05-10T03:30:51.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 -- 172.21.15.55:0/3675766255 shutdown_connections 2024-05-10T03:30:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c0715e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 --2- 172.21.15.55:0/3675766255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c06a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 -- 172.21.15.55:0/3675766255 >> 172.21.15.55:0/3675766255 conn(0x7f262c0cbb90 msgr2=0x7f262c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 -- 172.21.15.55:0/3675766255 shutdown_connections 2024-05-10T03:30:51.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.037+0000 7f2633000640 1 -- 172.21.15.55:0/3675766255 wait complete. 2024-05-10T03:30:51.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.039+0000 7f2633000640 1 Processor -- start 2024-05-10T03:30:51.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.039+0000 7f2633000640 1 -- start start 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 0x7f262c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c14f020 con 0x7f262c15ab30 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0af960 con 0x7f262c15af40 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2633000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0afc30 con 0x7f262c0cdba0 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2623e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2623e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53670/0 (socket says 172.21.15.55:53670) 2024-05-10T03:30:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.041+0000 7f2623e00640 1 -- 172.21.15.55:0/287050727 learned_addr learned my addr 172.21.15.55:0/287050727 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f262be00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2631000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 0x7f262c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 -- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 msgr2=0x7f262c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 -- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 msgr2=0x7f262c158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 0x7f262c158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 -- 172.21.15.55:0/287050727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614009400 con 0x7f262c15ab30 2024-05-10T03:30:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2631000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 0x7f262c158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2623e00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f26140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.042+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614004230 con 0x7f262c15ab30 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.043+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614004390 con 0x7f262c15ab30 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.043+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c0aff10 con 0x7f262c15ab30 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.043+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614003850 con 0x7f262c15ab30 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.043+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c0b04d0 con 0x7f262c15ab30 2024-05-10T03:30:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.044+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c066f60 con 0x7f262c15ab30 2024-05-10T03:30:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.045+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2614039050 con 0x7f262c15ab30 2024-05-10T03:30:51.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.046+0000 7f2628c00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 0x7f260806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.046+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f26140a9bf0 con 0x7f262c15ab30 2024-05-10T03:30:51.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.047+0000 7f262be00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 0x7f260806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.048+0000 7f262be00640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 0x7f260806d140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.051+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f26140749c0 con 0x7f262c15ab30 2024-05-10T03:30:51.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.322+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) v1 -- 0x7f262c067220 con 0x7f262c15ab30 2024-05-10T03:30:51.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.471+0000 7f2628c00640 1 -- 172.21.15.55:0/287050727 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v90) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2614037070 con 0x7f262c15ab30 2024-05-10T03:30:51.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 msgr2=0x7f260806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 0x7f260806d140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 msgr2=0x7f262c134710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f26140046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 shutdown_connections 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f260806ac30 0x7f260806d140 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f262c15af40 0x7f262c158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f262c15ab30 0x7f262c134710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 --2- 172.21.15.55:0/287050727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f262c0cdba0 0x7f262c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 >> 172.21.15.55:0/287050727 conn(0x7f262c0cbb90 msgr2=0x7f262c066330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 shutdown_connections 2024-05-10T03:30:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.474+0000 7f2633000640 1 -- 172.21.15.55:0/287050727 wait complete. 2024-05-10T03:30:51.504 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:30:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.773+0000 7f05cb200640 1 Processor -- start 2024-05-10T03:30:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.773+0000 7f05cb200640 1 -- start start 2024-05-10T03:30:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.775+0000 7f05cb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 0x7f05cc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.775+0000 7f05cb200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05cc06b840 0x7f05cc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.776+0000 7f05cb200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc15fdf0 con 0x7f05cc071e00 2024-05-10T03:30:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.776+0000 7f05cb200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc15f1e0 con 0x7f05cc06b840 2024-05-10T03:30:51.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.776+0000 7f05cb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15f4c0 con 0x7f05cc0719f0 2024-05-10T03:30:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.776+0000 7f05c9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 0x7f05cc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.776+0000 7f05c9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 0x7f05cc06b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48138/0 (socket says 172.21.15.55:48138) 2024-05-10T03:30:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9400640 1 --2- 172.21.15.55:0/2930982115 >> v2:172.21.15.162:3300/0 conn(0x7f05cc06b840 0x7f05cc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9e00640 1 -- 172.21.15.55:0/2930982115 learned_addr learned my addr 172.21.15.55:0/2930982115 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9400640 1 -- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 msgr2=0x7f05cc06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9400640 1 --2- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 0x7f05cc06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9400640 1 -- 172.21.15.55:0/2930982115 >> v1:172.21.15.162:6789/0 conn(0x7f05cc071e00 legacy=0x7f05cc070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.777+0000 7f05c9400640 1 -- 172.21.15.55:0/2930982115 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc15f7a0 con 0x7f05cc06b840 2024-05-10T03:30:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.779+0000 7f05c9400640 1 --2- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0700e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05b40097a0 tx=0x7f05b402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac6e67b0570f444 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.779+0000 7f05c8a00640 1 -- 172.21.15.55:0/2930982115 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b402ae40 con 0x7f05cc06b840 2024-05-10T03:30:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.780+0000 7f05c8a00640 1 -- 172.21.15.55:0/2930982115 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4032440 con 0x7f05cc06b840 2024-05-10T03:30:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.780+0000 7f05c8a00640 1 -- 172.21.15.55:0/2930982115 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b40327d0 con 0x7f05cc06b840 2024-05-10T03:30:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.780+0000 7f05cb200640 1 -- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 msgr2=0x7f05cc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.780+0000 7f05cb200640 1 --2- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0700e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05b40097a0 tx=0x7f05b402a0f0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 -- 172.21.15.55:0/2930982115 shutdown_connections 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 --2- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0700e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 --2- 172.21.15.55:0/2930982115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc0719f0 0x7f05cc06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 -- 172.21.15.55:0/2930982115 >> 172.21.15.55:0/2930982115 conn(0x7f05cc06c750 msgr2=0x7f05cc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 -- 172.21.15.55:0/2930982115 shutdown_connections 2024-05-10T03:30:51.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.781+0000 7f05cb200640 1 -- 172.21.15.55:0/2930982115 wait complete. 2024-05-10T03:30:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.783+0000 7f05cb200640 1 Processor -- start 2024-05-10T03:30:51.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.783+0000 7f05cb200640 1 -- start start 2024-05-10T03:30:51.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc071e00 0x7f05cc0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15d290 con 0x7f05cc0719f0 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1f3fa0 con 0x7f05cc06b840 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05cb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15b190 con 0x7f05cc071e00 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05c9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05c9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53678/0 (socket says 172.21.15.55:53678) 2024-05-10T03:30:51.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.785+0000 7f05c9400640 1 -- 172.21.15.55:0/135758938 learned_addr learned my addr 172.21.15.55:0/135758938 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:51.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05ca800640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc071e00 0x7f05cc0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9e00640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 -- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc071e00 msgr2=0x7f05cc0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc071e00 0x7f05cc0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 -- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 msgr2=0x7f05cc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 -- 172.21.15.55:0/135758938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4009400 con 0x7f05cc0719f0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.786+0000 7f05c9400640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05b402a6e0 tx=0x7f05b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.787+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b403e920 con 0x7f05cc0719f0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.787+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b403eaa0 con 0x7f05cc0719f0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.787+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc15a860 con 0x7f05cc0719f0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.787+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b403ee50 con 0x7f05cc0719f0 2024-05-10T03:30:51.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.787+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc15ae20 con 0x7f05cc0719f0 2024-05-10T03:30:51.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.788+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc15afe0 con 0x7f05cc0719f0 2024-05-10T03:30:51.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.789+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f05b403c400 con 0x7f05cc0719f0 2024-05-10T03:30:51.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.790+0000 7f05c2000640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 0x7f05a806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:51.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.791+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f05b40aa490 con 0x7f05cc0719f0 2024-05-10T03:30:51.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.791+0000 7f05c9e00640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 0x7f05a806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:51.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.792+0000 7f05c9e00640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 0x7f05a806d110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009ac0 tx=0x7f05bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:51.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:51.796+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f05b40751f0 con 0x7f05cc0719f0 2024-05-10T03:30:52.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.003+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05cc04e290 con 0x7f05cc0719f0 2024-05-10T03:30:52.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.010+0000 7f05c2000640 1 -- 172.21.15.55:0/135758938 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05b4078810 con 0x7f05cc0719f0 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.012+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 msgr2=0x7f05a806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.012+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 0x7f05a806d110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009ac0 tx=0x7f05bc008040 comp rx=0 tx=0).stop 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.012+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 msgr2=0x7f05cc1f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.012+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05b402a6e0 tx=0x7f05b40046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 shutdown_connections 2024-05-10T03:30:52.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05cc071e00 0x7f05cc0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05cc0719f0 0x7f05cc1f3480 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f05a806ac00 0x7f05a806d110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 --2- 172.21.15.55:0/135758938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05cc06b840 0x7f05cc0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 >> 172.21.15.55:0/135758938 conn(0x7f05cc06c750 msgr2=0x7f05cc0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 shutdown_connections 2024-05-10T03:30:52.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.013+0000 7f05cb200640 1 -- 172.21.15.55:0/135758938 wait complete. 2024-05-10T03:30:52.045 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:30:52.046 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:30:52.147 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:30:52.268 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:30:52.268 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:52.268 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:52.327 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:52.328 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:52.377 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:52.377 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:52.475 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:52.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.746+0000 7f55e1800640 1 Processor -- start 2024-05-10T03:30:52.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.746+0000 7f55e1800640 1 -- start start 2024-05-10T03:30:52.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.748+0000 7f55e1800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55dc15d4b0 0x7f55dc06c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.748+0000 7f55e1800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55dc06c770 0x7f55dc0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.748+0000 7f55e1800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc0b3b20 con 0x7f55dc0ac8e0 2024-05-10T03:30:52.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.748+0000 7f55e1800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc06cb80 con 0x7f55dc15d4b0 2024-05-10T03:30:52.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.748+0000 7f55e1800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc06deb0 con 0x7f55dc06c770 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55d3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55dc15d4b0 0x7f55dc06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55dbe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55dc0ac8e0 0x7f55dc161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44136/0 (socket says 172.21.15.55:44136) 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55dbe00640 1 -- 172.21.15.55:0/2145378117 learned_addr learned my addr 172.21.15.55:0/2145378117 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55e0e00640 1 --2- 172.21.15.55:0/2145378117 >> v2:172.21.15.162:3300/0 conn(0x7f55dc06c770 0x7f55dc0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55d3400640 1 -- 172.21.15.55:0/2145378117 >> v2:172.21.15.162:3300/0 conn(0x7f55dc06c770 msgr2=0x7f55dc0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55d3400640 1 --2- 172.21.15.55:0/2145378117 >> v2:172.21.15.162:3300/0 conn(0x7f55dc06c770 0x7f55dc0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55d3400640 1 -- 172.21.15.55:0/2145378117 >> v1:172.21.15.55:6789/0 conn(0x7f55dc0ac8e0 legacy=0x7f55dc161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55d3400640 1 -- 172.21.15.55:0/2145378117 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc06e190 con 0x7f55dc15d4b0 2024-05-10T03:30:52.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.749+0000 7f55db400640 1 -- 172.21.15.55:0/2145378117 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164662699 0 0) 0x7f55dc0b3b20 con 0x7f55dc0ac8e0 2024-05-10T03:30:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.750+0000 7f55d3400640 1 --2- 172.21.15.55:0/2145378117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc06c230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55dc070c80 tx=0x7f55c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27add780ed87fa4a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.751+0000 7f55db400640 1 -- 172.21.15.55:0/2145378117 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4032440 con 0x7f55dc15d4b0 2024-05-10T03:30:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.751+0000 7f55db400640 1 -- 172.21.15.55:0/2145378117 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c40325a0 con 0x7f55dc15d4b0 2024-05-10T03:30:52.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.751+0000 7f55db400640 1 -- 172.21.15.55:0/2145378117 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4032930 con 0x7f55dc15d4b0 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.751+0000 7f55e1800640 1 -- 172.21.15.55:0/2145378117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 msgr2=0x7f55dc06c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.751+0000 7f55e1800640 1 --2- 172.21.15.55:0/2145378117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc06c230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55dc070c80 tx=0x7f55c402a1a0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 -- 172.21.15.55:0/2145378117 shutdown_connections 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 --2- 172.21.15.55:0/2145378117 >> v2:172.21.15.162:3300/0 conn(0x7f55dc06c770 0x7f55dc0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 --2- 172.21.15.55:0/2145378117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc06c230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 -- 172.21.15.55:0/2145378117 >> 172.21.15.55:0/2145378117 conn(0x7f55dc070eb0 msgr2=0x7f55dc0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 -- 172.21.15.55:0/2145378117 shutdown_connections 2024-05-10T03:30:52.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.752+0000 7f55e1800640 1 -- 172.21.15.55:0/2145378117 wait complete. 2024-05-10T03:30:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.754+0000 7f55e1800640 1 Processor -- start 2024-05-10T03:30:52.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.754+0000 7f55e1800640 1 -- start start 2024-05-10T03:30:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc06c770 0x7f55dc14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 0x7f55dc1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc15e850 con 0x7f55dc15d4b0 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1514f0 con 0x7f55dc0ac8e0 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e1800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0cd4b0 con 0x7f55dc06c770 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53694/0 (socket says 172.21.15.55:53694) 2024-05-10T03:30:52.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.756+0000 7f55e0e00640 1 -- 172.21.15.55:0/127008048 learned_addr learned my addr 172.21.15.55:0/127008048 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:52.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55dbe00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc06c770 0x7f55dc14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55d3400640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 0x7f55dc1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 -- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc06c770 msgr2=0x7f55dc14ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc06c770 0x7f55dc14ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 -- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 msgr2=0x7f55dc1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 0x7f55dc1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 -- 172.21.15.55:0/127008048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009400 con 0x7f55dc15d4b0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55d3400640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 0x7f55dc1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e0e00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55d400d5a0 tx=0x7f55d400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d40125f0 con 0x7f55dc15d4b0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d4012750 con 0x7f55dc15d4b0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4012ae0 con 0x7f55dc15d4b0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.757+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc151740 con 0x7f55dc15d4b0 2024-05-10T03:30:52.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.758+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc150000 con 0x7f55dc15d4b0 2024-05-10T03:30:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.759+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc04e290 con 0x7f55dc15d4b0 2024-05-10T03:30:52.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.760+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f55d4012c40 con 0x7f55dc15d4b0 2024-05-10T03:30:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.761+0000 7f55d8c00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 0x7f55b806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.761+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f55d4089bd0 con 0x7f55dc15d4b0 2024-05-10T03:30:52.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.761+0000 7f55dbe00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 0x7f55b806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:52.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.763+0000 7f55dbe00640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 0x7f55b806d140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0012e0 tx=0x7f55cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:52.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.766+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f55d4054920 con 0x7f55dc15d4b0 2024-05-10T03:30:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.972+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55dc0b5300 con 0x7f55dc15d4b0 2024-05-10T03:30:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.976+0000 7f55d8c00640 1 -- 172.21.15.55:0/127008048 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f55d4057f40 con 0x7f55dc15d4b0 2024-05-10T03:30:52.981 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:52.981 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.978+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 msgr2=0x7f55b806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.978+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 0x7f55b806d140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0012e0 tx=0x7f55cc009040 comp rx=0 tx=0).stop 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.978+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 msgr2=0x7f55dc154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.978+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f55d400d5a0 tx=0x7f55d400da60 comp rx=0 tx=0).stop 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 shutdown_connections 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55dc15d4b0 0x7f55dc154df0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f55b806ac30 0x7f55b806d140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0ac8e0 0x7f55dc1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 --2- 172.21.15.55:0/127008048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc06c770 0x7f55dc14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:52.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 >> 172.21.15.55:0/127008048 conn(0x7f55dc070eb0 msgr2=0x7f55dc0afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:52.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 shutdown_connections 2024-05-10T03:30:52.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:52.979+0000 7f55e1800640 1 -- 172.21.15.55:0/127008048 wait complete. 2024-05-10T03:30:53.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:53.009 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:53.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.293+0000 7f5f50a00640 1 Processor -- start 2024-05-10T03:30:53.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.294+0000 7f5f50a00640 1 -- start start 2024-05-10T03:30:53.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f4c0719f0 0x7f5f4c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f4c071e00 0x7f5f4c070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c15fdf0 con 0x7f5f4c071e00 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c15f1e0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f50a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c15f4c0 con 0x7f5f4c06b840 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f4b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f4c071e00 0x7f5f4c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f4b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f4c071e00 0x7f5f4c070620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33460/0 (socket says 172.21.15.55:33460) 2024-05-10T03:30:53.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.296+0000 7f5f4b200640 1 -- 172.21.15.55:0/3513847758 learned_addr learned my addr 172.21.15.55:0/3513847758 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f41e00640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4a800640 1 --2- 172.21.15.55:0/3513847758 >> v2:172.21.15.162:3300/0 conn(0x7f5f4c0719f0 0x7f5f4c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4b200640 1 -- 172.21.15.55:0/3513847758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 msgr2=0x7f5f4c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4b200640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4b200640 1 -- 172.21.15.55:0/3513847758 >> v2:172.21.15.162:3300/0 conn(0x7f5f4c0719f0 msgr2=0x7f5f4c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4b200640 1 --2- 172.21.15.55:0/3513847758 >> v2:172.21.15.162:3300/0 conn(0x7f5f4c0719f0 0x7f5f4c06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.297+0000 7f5f4b200640 1 -- 172.21.15.55:0/3513847758 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c15f7a0 con 0x7f5f4c071e00 2024-05-10T03:30:53.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.298+0000 7f5f4b200640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c071e00 0x7f5f4c070620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0b41f0 tx=0x7f5f4402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ca4000b346e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.298+0000 7f5f49e00640 1 -- 172.21.15.55:0/3513847758 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44031600 con 0x7f5f4c071e00 2024-05-10T03:30:53.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.298+0000 7f5f49e00640 1 -- 172.21.15.55:0/3513847758 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4402d050 con 0x7f5f4c071e00 2024-05-10T03:30:53.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.298+0000 7f5f49e00640 1 -- 172.21.15.55:0/3513847758 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44031980 con 0x7f5f4c071e00 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 -- 172.21.15.55:0/3513847758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c071e00 msgr2=0x7f5f4c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c071e00 0x7f5f4c070620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0b41f0 tx=0x7f5f4402a610 comp rx=0 tx=0).stop 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 -- 172.21.15.55:0/3513847758 shutdown_connections 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c071e00 0x7f5f4c070620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 --2- 172.21.15.55:0/3513847758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 --2- 172.21.15.55:0/3513847758 >> v2:172.21.15.162:3300/0 conn(0x7f5f4c0719f0 0x7f5f4c06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 -- 172.21.15.55:0/3513847758 >> 172.21.15.55:0/3513847758 conn(0x7f5f4c06c750 msgr2=0x7f5f4c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 -- 172.21.15.55:0/3513847758 shutdown_connections 2024-05-10T03:30:53.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.299+0000 7f5f50a00640 1 -- 172.21.15.55:0/3513847758 wait complete. 2024-05-10T03:30:53.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.302+0000 7f5f50a00640 1 Processor -- start 2024-05-10T03:30:53.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.302+0000 7f5f50a00640 1 -- start start 2024-05-10T03:30:53.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c1f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f4c071e00 0x7f5f4c1fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0c4e90 con 0x7f5f4c0719f0 2024-05-10T03:30:53.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0c6aa0 con 0x7f5f4c071e00 2024-05-10T03:30:53.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f50a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cc6b0 con 0x7f5f4c06b840 2024-05-10T03:30:53.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33474/0 (socket says 172.21.15.55:33474) 2024-05-10T03:30:53.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 -- 172.21.15.55:0/2182330015 learned_addr learned my addr 172.21.15.55:0/2182330015 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 -- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 msgr2=0x7f5f4c1f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f4a800640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c1f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c1f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f4b200640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f4c071e00 0x7f5f4c1fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.304+0000 7f5f41e00640 1 -- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f4c071e00 msgr2=0x7f5f4c1fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f41e00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f4c071e00 0x7f5f4c1fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f41e00640 1 -- 172.21.15.55:0/2182330015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44009b30 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f41e00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400d860 tx=0x7f5f3400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340096a0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34009800 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340099b0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c0c4250 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.305+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c0c47d0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.307+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c04e290 con 0x7f5f4c0719f0 2024-05-10T03:30:53.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.308+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f34009b10 con 0x7f5f4c0719f0 2024-05-10T03:30:53.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.309+0000 7f5f43400640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 0x7f5f2806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.309+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5f34089bb0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.309+0000 7f5f4a800640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 0x7f5f2806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.311+0000 7f5f4a800640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 0x7f5f2806d110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006fd0 tx=0x7f5f3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.314+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f34054990 con 0x7f5f4c0719f0 2024-05-10T03:30:53.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.514+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f4c071500 con 0x7f5f4c0719f0 2024-05-10T03:30:53.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.515+0000 7f5f43400640 1 -- 172.21.15.55:0/2182330015 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f34057fb0 con 0x7f5f4c0719f0 2024-05-10T03:30:53.519 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:53.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.517+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 msgr2=0x7f5f2806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.517+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 0x7f5f2806d110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c006fd0 tx=0x7f5f3c008040 comp rx=0 tx=0).stop 2024-05-10T03:30:53.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 msgr2=0x7f5f4c1fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400d860 tx=0x7f5f3400dd20 comp rx=0 tx=0).stop 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 shutdown_connections 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f2806ac00 0x7f5f2806d110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f4c071e00 0x7f5f4c1fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4c0719f0 0x7f5f4c1fae70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 --2- 172.21.15.55:0/2182330015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f4c06b840 0x7f5f4c1f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 >> 172.21.15.55:0/2182330015 conn(0x7f5f4c06c750 msgr2=0x7f5f4c142840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 shutdown_connections 2024-05-10T03:30:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.518+0000 7f5f50a00640 1 -- 172.21.15.55:0/2182330015 wait complete. 2024-05-10T03:30:53.547 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:53.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.831+0000 7f5717e00640 1 Processor -- start 2024-05-10T03:30:53.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.831+0000 7f5717e00640 1 -- start start 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5717e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f5718141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5717e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f571815f1b0 0x7f571815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5717e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57180b4b90 con 0x7f57180b5020 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5717e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57180d0e80 con 0x7f571815f1b0 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5717e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5718141fd0 con 0x7f571806d430 2024-05-10T03:30:53.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5716000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f57180b5020 0x7f57180b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36844/0 (socket says 172.21.15.55:36844) 2024-05-10T03:30:53.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.834+0000 7f5716000640 1 -- 172.21.15.55:0/2883389924 learned_addr learned my addr 172.21.15.55:0/2883389924 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:53.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5717400640 1 --2- 172.21.15.55:0/2883389924 >> v2:172.21.15.162:3300/0 conn(0x7f571815f1b0 0x7f571815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5716a00640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f5718141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990308338 0 0) 0x7f57180b4b90 con 0x7f57180b5020 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc003520 con 0x7f57180b5020 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832401127 0 0) 0x7f56fc003520 con 0x7f57180b5020 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 msgr2=0x7f5718141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f5718141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 >> v2:172.21.15.162:3300/0 conn(0x7f571815f1b0 msgr2=0x7f571815f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 --2- 172.21.15.55:0/2883389924 >> v2:172.21.15.162:3300/0 conn(0x7f571815f1b0 0x7f571815f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571806d840 con 0x7f57180b5020 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f5704001670 con 0x7f57180b5020 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 >> v1:172.21.15.55:6789/0 conn(0x7f57180b5020 legacy=0x7f57180b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f56fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc003f60 0x7f56fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc004d00 0x7f56fc005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180d0e80 con 0x7f56fc003f60 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5718141fd0 con 0x7f56fc004d00 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.835+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc00ba60 con 0x7f571806d430 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc003f60 0x7f56fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716a00640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f56fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 -- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 msgr2=0x7f56fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571806d430 0x7f56fc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 -- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc004d00 msgr2=0x7f56fc005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc004d00 0x7f56fc005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.836+0000 7f5716000640 1 -- 172.21.15.55:0/2883389924 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc003520 con 0x7f56fc003f60 2024-05-10T03:30:53.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.837+0000 7f5716000640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc003f60 0x7f56fc0047c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f571806f260 tx=0x7f56fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ae61bb366c9029 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.837+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704005ad0 con 0x7f56fc003f60 2024-05-10T03:30:53.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.837+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570402d070 con 0x7f56fc003f60 2024-05-10T03:30:53.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.837+0000 7f5715600640 1 -- 172.21.15.55:0/2883389924 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704007070 con 0x7f56fc003f60 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 -- 172.21.15.55:0/2883389924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc003f60 msgr2=0x7f56fc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 --2- 172.21.15.55:0/2883389924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56fc003f60 0x7f56fc0047c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f571806f260 tx=0x7f56fc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717400640 1 -- 172.21.15.55:0/2883389924 reap_dead start 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 -- 172.21.15.55:0/2883389924 shutdown_connections 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 -- 172.21.15.55:0/2883389924 >> 172.21.15.55:0/2883389924 conn(0x7f57180b6680 msgr2=0x7f57180b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 -- 172.21.15.55:0/2883389924 shutdown_connections 2024-05-10T03:30:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.838+0000 7f5717e00640 1 -- 172.21.15.55:0/2883389924 wait complete. 2024-05-10T03:30:53.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.841+0000 7f5717e00640 1 Processor -- start 2024-05-10T03:30:53.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.841+0000 7f5717e00640 1 -- start start 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.842+0000 7f5717e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc003f60 0x7f571806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.842+0000 7f5717e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.842+0000 7f5717e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57181e5860 0x7f57181f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.842+0000 7f5717e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180ae3d0 con 0x7f571813ca40 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5717e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571806b220 con 0x7f56fc003f60 2024-05-10T03:30:53.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5717e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571806b460 con 0x7f57181e5860 2024-05-10T03:30:53.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33488/0 (socket says 172.21.15.55:33488) 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 -- 172.21.15.55:0/485919343 learned_addr learned my addr 172.21.15.55:0/485919343 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716a00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc003f60 0x7f571806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5717400640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57181e5860 0x7f57181f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 -- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57181e5860 msgr2=0x7f57181f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57181e5860 0x7f57181f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 -- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc003f60 msgr2=0x7f571806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc003f60 0x7f571806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.843+0000 7f5716000640 1 -- 172.21.15.55:0/485919343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57181e5c70 con 0x7f571813ca40 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f5716000640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5704001930 tx=0x7f5704001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040049e0 con 0x7f571813ca40 2024-05-10T03:30:53.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704004b60 con 0x7f571813ca40 2024-05-10T03:30:53.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704007070 con 0x7f571813ca40 2024-05-10T03:30:53.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571806b6b0 con 0x7f571813ca40 2024-05-10T03:30:53.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.844+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571806bc70 con 0x7f571813ca40 2024-05-10T03:30:53.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.845+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571804e290 con 0x7f571813ca40 2024-05-10T03:30:53.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.846+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f570402f030 con 0x7f571813ca40 2024-05-10T03:30:53.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.847+0000 7f570ea00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 0x7f56f806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:53.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.847+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f57040a9040 con 0x7f571813ca40 2024-05-10T03:30:53.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.848+0000 7f5716a00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 0x7f56f806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:53.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.850+0000 7f5716a00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 0x7f56f806d160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5708009670 tx=0x7f570800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:53.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:53.853+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5704073e00 con 0x7f571813ca40 2024-05-10T03:30:54.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.052+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f571804e290 con 0x7f571813ca40 2024-05-10T03:30:54.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.053+0000 7f570ea00640 1 -- 172.21.15.55:0/485919343 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5704007d10 con 0x7f571813ca40 2024-05-10T03:30:54.058 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 msgr2=0x7f56f806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 0x7f56f806d160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5708009670 tx=0x7f570800b040 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 msgr2=0x7f57181e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5704001930 tx=0x7f5704001a00 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 shutdown_connections 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57181e5860 0x7f57181f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f571813ca40 0x7f57181e5320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f56f806ac50 0x7f56f806d160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 --2- 172.21.15.55:0/485919343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56fc003f60 0x7f571806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 >> 172.21.15.55:0/485919343 conn(0x7f57180b6680 msgr2=0x7f5718158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 shutdown_connections 2024-05-10T03:30:54.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.056+0000 7f5717e00640 1 -- 172.21.15.55:0/485919343 wait complete. 2024-05-10T03:30:54.086 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:54.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.363+0000 7feaff200640 1 Processor -- start 2024-05-10T03:30:54.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.363+0000 7feaff200640 1 -- start start 2024-05-10T03:30:54.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.365+0000 7feaff200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.365+0000 7feaff200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 0x7feb0006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.365+0000 7feaff200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0013eae0 con 0x7feb0006b660 2024-05-10T03:30:54.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.365+0000 7feaff200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0013e7a0 con 0x7feb001414c0 2024-05-10T03:30:54.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.365+0000 7feaff200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0006ba70 con 0x7feb0013e390 2024-05-10T03:30:54.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.366+0000 7feafde00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.366+0000 7feafde00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37684/0 (socket says 172.21.15.55:37684) 2024-05-10T03:30:54.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.366+0000 7feafe800640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7feb0006b660 0x7feb0015efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:59258/0 (socket says 172.21.15.55:59258) 2024-05-10T03:30:54.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.366+0000 7feafde00640 1 -- 172.21.15.55:0/1641118821 learned_addr learned my addr 172.21.15.55:0/1641118821 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:54.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.366+0000 7feaf7e00640 1 --2- 172.21.15.55:0/1641118821 >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 0x7feb0006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafd400640 1 -- 172.21.15.55:0/1641118821 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153655633 0 0) 0x7feb0013eae0 con 0x7feb0006b660 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafd400640 1 -- 172.21.15.55:0/1641118821 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae4003520 con 0x7feb0006b660 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafde00640 1 -- 172.21.15.55:0/1641118821 >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 msgr2=0x7feb0006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafde00640 1 --2- 172.21.15.55:0/1641118821 >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 0x7feb0006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafde00640 1 -- 172.21.15.55:0/1641118821 >> v1:172.21.15.162:6789/0 conn(0x7feb0006b660 legacy=0x7feb0015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feafde00640 1 -- 172.21.15.55:0/1641118821 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0015c020 con 0x7feb0013e390 2024-05-10T03:30:54.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.367+0000 7feaf7e00640 1 --2- 172.21.15.55:0/1641118821 >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 0x7feb0006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:54.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.369+0000 7feafde00640 1 --2- 172.21.15.55:0/1641118821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009870 tx=0x7feaf002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f58f883f6e4ba25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.369+0000 7feafd400640 1 -- 172.21.15.55:0/1641118821 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0032440 con 0x7feb0013e390 2024-05-10T03:30:54.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.370+0000 7feafd400640 1 -- 172.21.15.55:0/1641118821 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf00325a0 con 0x7feb0013e390 2024-05-10T03:30:54.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.370+0000 7feafd400640 1 -- 172.21.15.55:0/1641118821 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0032930 con 0x7feb0013e390 2024-05-10T03:30:54.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.370+0000 7feaff200640 1 -- 172.21.15.55:0/1641118821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 msgr2=0x7feb0015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.370+0000 7feaff200640 1 --2- 172.21.15.55:0/1641118821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009870 tx=0x7feaf002a250 comp rx=0 tx=0).stop 2024-05-10T03:30:54.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 -- 172.21.15.55:0/1641118821 shutdown_connections 2024-05-10T03:30:54.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 --2- 172.21.15.55:0/1641118821 >> v2:172.21.15.162:3300/0 conn(0x7feb001414c0 0x7feb0006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 --2- 172.21.15.55:0/1641118821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0013e390 0x7feb0015b9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 -- 172.21.15.55:0/1641118821 >> 172.21.15.55:0/1641118821 conn(0x7feb0006c730 msgr2=0x7feb000b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:54.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 -- 172.21.15.55:0/1641118821 shutdown_connections 2024-05-10T03:30:54.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.371+0000 7feaff200640 1 -- 172.21.15.55:0/1641118821 wait complete. 2024-05-10T03:30:54.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.374+0000 7feaff200640 1 Processor -- start 2024-05-10T03:30:54.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.375+0000 7feaff200640 1 -- start start 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb0006b660 0x7feb0015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 0x7feb001577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0013c320 con 0x7feb0013e390 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00158980 con 0x7feb0006b660 2024-05-10T03:30:54.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaff200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb001f1b40 con 0x7feb001414c0 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33498/0 (socket says 172.21.15.55:33498) 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.377+0000 7feaf7e00640 1 -- 172.21.15.55:0/3605987711 learned_addr learned my addr 172.21.15.55:0/3605987711 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feafde00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb0006b660 0x7feb0015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feafe800640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 0x7feb001577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 -- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 msgr2=0x7feb001577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 0x7feb001577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 -- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb0006b660 msgr2=0x7feb0015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb0006b660 0x7feb0015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 -- 172.21.15.55:0/3605987711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80093a0 con 0x7feb0013e390 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feafe800640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 0x7feb001577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.378+0000 7feaf7e00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feae800e9b0 tx=0x7feae800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.379+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005990 con 0x7feb0013e390 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.379+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae8005af0 con 0x7feb0013e390 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.379+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005e80 con 0x7feb0013e390 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.379+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf00094d0 con 0x7feb0013e390 2024-05-10T03:30:54.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.379+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb001f1d80 con 0x7feb0013e390 2024-05-10T03:30:54.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.380+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0015ab30 con 0x7feb0013e390 2024-05-10T03:30:54.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.381+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feae800c9a0 con 0x7feb0013e390 2024-05-10T03:30:54.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.382+0000 7feaf6000640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 0x7feadc071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.382+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7feae8089ac0 con 0x7feb0013e390 2024-05-10T03:30:54.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.383+0000 7feafde00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 0x7feadc071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.384+0000 7feafde00640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 0x7feadc071520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034000 tx=0x7feaf002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.387+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feae8054930 con 0x7feb0013e390 2024-05-10T03:30:54.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.589+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb001f55a0 con 0x7feb0013e390 2024-05-10T03:30:54.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.591+0000 7feaf6000640 1 -- 172.21.15.55:0/3605987711 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7feae8089e60 con 0x7feb0013e390 2024-05-10T03:30:54.595 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:54.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.593+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 msgr2=0x7feadc071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.593+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 0x7feadc071520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034000 tx=0x7feaf002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:54.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.593+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 msgr2=0x7feb0015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.593+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feae800e9b0 tx=0x7feae800ee70 comp rx=0 tx=0).stop 2024-05-10T03:30:54.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 shutdown_connections 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb001414c0 0x7feb001577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb0013e390 0x7feb0015a5f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feadc06f010 0x7feadc071520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 --2- 172.21.15.55:0/3605987711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb0006b660 0x7feb0015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 >> 172.21.15.55:0/3605987711 conn(0x7feb0006c730 msgr2=0x7feb0006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 shutdown_connections 2024-05-10T03:30:54.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.594+0000 7feaff200640 1 -- 172.21.15.55:0/3605987711 wait complete. 2024-05-10T03:30:54.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.901+0000 7ff3bbe00640 1 Processor -- start 2024-05-10T03:30:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.901+0000 7ff3bbe00640 1 -- start start 2024-05-10T03:30:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.903+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.903+0000 7ff3bbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3bc06b840 0x7ff3bc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3bbe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3bc071e00 0x7ff3bc070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3bbe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc15fdf0 con 0x7ff3bc06b840 2024-05-10T03:30:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3bbe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc15f1e0 con 0x7ff3bc071e00 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3bbe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15f4c0 con 0x7ff3bc0719f0 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3bc06b840 0x7ff3bc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3bc06b840 0x7ff3bc0700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33512/0 (socket says 172.21.15.55:33512) 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 -- 172.21.15.55:0/4093882942 learned_addr learned my addr 172.21.15.55:0/4093882942 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 -- 172.21.15.55:0/4093882942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 msgr2=0x7ff3bc06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 --2- 172.21.15.55:0/4093882942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 -- 172.21.15.55:0/4093882942 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc071e00 msgr2=0x7ff3bc070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 --2- 172.21.15.55:0/4093882942 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc071e00 0x7ff3bc070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.904+0000 7ff3ba000640 1 -- 172.21.15.55:0/4093882942 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc15f7a0 con 0x7ff3bc06b840 2024-05-10T03:30:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.906+0000 7ff3ba000640 1 --2- 172.21.15.55:0/4093882942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0700e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0cc8b0 tx=0x7ff3a802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7236b6ec836da343 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.906+0000 7ff3b9600640 1 -- 172.21.15.55:0/4093882942 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8032440 con 0x7ff3bc06b840 2024-05-10T03:30:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.906+0000 7ff3b9600640 1 -- 172.21.15.55:0/4093882942 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a80325a0 con 0x7ff3bc06b840 2024-05-10T03:30:54.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.906+0000 7ff3b9600640 1 -- 172.21.15.55:0/4093882942 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8032930 con 0x7ff3bc06b840 2024-05-10T03:30:54.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 -- 172.21.15.55:0/4093882942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc06b840 msgr2=0x7ff3bc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/4093882942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0700e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0cc8b0 tx=0x7ff3a802a160 comp rx=0 tx=0).stop 2024-05-10T03:30:54.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 -- 172.21.15.55:0/4093882942 shutdown_connections 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/4093882942 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc071e00 0x7ff3bc070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/4093882942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0700e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/4093882942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 -- 172.21.15.55:0/4093882942 >> 172.21.15.55:0/4093882942 conn(0x7ff3bc06c750 msgr2=0x7ff3bc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 -- 172.21.15.55:0/4093882942 shutdown_connections 2024-05-10T03:30:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.907+0000 7ff3bbe00640 1 -- 172.21.15.55:0/4093882942 wait complete. 2024-05-10T03:30:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.909+0000 7ff3bbe00640 1 Processor -- start 2024-05-10T03:30:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.910+0000 7ff3bbe00640 1 -- start start 2024-05-10T03:30:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0ac910 con 0x7ff3bc071e00 2024-05-10T03:30:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0a9ed0 con 0x7ff3bc06b840 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bbe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0aa110 con 0x7ff3bc0719f0 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.911+0000 7ff3bb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33514/0 (socket says 172.21.15.55:33514) 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 -- 172.21.15.55:0/181635668 learned_addr learned my addr 172.21.15.55:0/181635668 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3ba000640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3baa00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 -- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 msgr2=0x7ff3bc0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 -- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc06b840 msgr2=0x7ff3bc0ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 -- 172.21.15.55:0/181635668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80094d0 con 0x7ff3bc071e00 2024-05-10T03:30:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3bb400640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400b9e0 tx=0x7ff3b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.912+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400cb20 con 0x7ff3bc071e00 2024-05-10T03:30:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.913+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400cca0 con 0x7ff3bc071e00 2024-05-10T03:30:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.913+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc163430 con 0x7ff3bc071e00 2024-05-10T03:30:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.913+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4014680 con 0x7ff3bc071e00 2024-05-10T03:30:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.913+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc0aac00 con 0x7ff3bc071e00 2024-05-10T03:30:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.914+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc1635c0 con 0x7ff3bc071e00 2024-05-10T03:30:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.915+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff3b401e440 con 0x7ff3bc071e00 2024-05-10T03:30:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.916+0000 7ff3b2a00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 0x7ff39c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.916+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff3b40895d0 con 0x7ff3bc071e00 2024-05-10T03:30:54.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.917+0000 7ff3baa00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 0x7ff39c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:54.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.918+0000 7ff3baa00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 0x7ff39c06d140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00a770 tx=0x7ff3ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:54.921+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3b40543a0 con 0x7ff3bc071e00 2024-05-10T03:30:55.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.123+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff3bc163220 con 0x7ff3bc071e00 2024-05-10T03:30:55.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.124+0000 7ff3b2a00640 1 -- 172.21.15.55:0/181635668 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff3b40579c0 con 0x7ff3bc071e00 2024-05-10T03:30:55.129 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 msgr2=0x7ff39c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 0x7ff39c06d140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00a770 tx=0x7ff3ac008040 comp rx=0 tx=0).stop 2024-05-10T03:30:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 msgr2=0x7ff3bc1638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400b9e0 tx=0x7ff3b400bea0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 shutdown_connections 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3bc071e00 0x7ff3bc1638c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc0719f0 0x7ff3bc0b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff39c06ac30 0x7ff39c06d140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 --2- 172.21.15.55:0/181635668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc06b840 0x7ff3bc0ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 >> 172.21.15.55:0/181635668 conn(0x7ff3bc06c750 msgr2=0x7ff3bc143230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:55.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.127+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 shutdown_connections 2024-05-10T03:30:55.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.128+0000 7ff3bbe00640 1 -- 172.21.15.55:0/181635668 wait complete. 2024-05-10T03:30:55.160 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:30:55.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.426+0000 7fc3a9800640 1 Processor -- start 2024-05-10T03:30:55.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.427+0000 7fc3a9800640 1 -- start start 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.428+0000 7fc3a9800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3a4158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.428+0000 7fc3a9800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40708b0 con 0x7fc3a4071db0 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a9800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a415b940 con 0x7fc3a40b3860 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a9800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40b2490 con 0x7fc3a406a4b0 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a2a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc3a4071db0 0x7fc3a40ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36858/0 (socket says 172.21.15.55:36858) 2024-05-10T03:30:55.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a2a00640 1 -- 172.21.15.55:0/3860996168 learned_addr learned my addr 172.21.15.55:0/3860996168 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:55.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a2000640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3a4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.429+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643931454 0 0) 0x7fc3a40708b0 con 0x7fc3a4071db0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc388005730 con 0x7fc3a4071db0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024539297 0 0) 0x7fc388005730 con 0x7fc3a4071db0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 msgr2=0x7fc3a4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3a4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 >> v1:172.21.15.162:6789/0 conn(0x7fc3a40b3860 legacy=0x7fc3a40b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4070260 con 0x7fc3a4071db0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fc394001670 con 0x7fc3a4071db0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 >> v1:172.21.15.55:6789/0 conn(0x7fc3a4071db0 legacy=0x7fc3a40ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc388005bb0 0x7fc388006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc388006de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3880074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38800dd20 con 0x7fc3880069d0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38800de90 con 0x7fc388005bb0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38800e050 con 0x7fc3a406a4b0 2024-05-10T03:30:55.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.430+0000 7fc3a2000640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc388006de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a3400640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3880074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2000640 1 -- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 msgr2=0x7fc3880074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2000640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a406a4b0 0x7fc3880074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2000640 1 -- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc388005bb0 msgr2=0x7fc388006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2a00640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc388005bb0 0x7fc388006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2000640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc388005bb0 0x7fc388006020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.431+0000 7fc3a2000640 1 -- 172.21.15.55:0/3860996168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388005730 con 0x7fc3880069d0 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.432+0000 7fc3a2000640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc388006de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc390009ac0 tx=0x7fc388007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ee378f454c955c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.432+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390031400 con 0x7fc3880069d0 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.432+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39002d050 con 0x7fc3880069d0 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.432+0000 7fc3a1600640 1 -- 172.21.15.55:0/3860996168 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390031780 con 0x7fc3880069d0 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 -- 172.21.15.55:0/3860996168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 msgr2=0x7fc388006de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 --2- 172.21.15.55:0/3860996168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc388006de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc390009ac0 tx=0x7fc388007a00 comp rx=0 tx=0).stop 2024-05-10T03:30:55.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a3400640 1 -- 172.21.15.55:0/3860996168 reap_dead start 2024-05-10T03:30:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 -- 172.21.15.55:0/3860996168 shutdown_connections 2024-05-10T03:30:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 -- 172.21.15.55:0/3860996168 >> 172.21.15.55:0/3860996168 conn(0x7fc3a40cbb90 msgr2=0x7fc3a40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 -- 172.21.15.55:0/3860996168 shutdown_connections 2024-05-10T03:30:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.433+0000 7fc3a9800640 1 -- 172.21.15.55:0/3860996168 wait complete. 2024-05-10T03:30:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.436+0000 7fc3a9800640 1 Processor -- start 2024-05-10T03:30:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.436+0000 7fc3a9800640 1 -- start start 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a41419f0 0x7fc3a4141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a4131140 0x7fc3a4131550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41433a0 con 0x7fc3880069d0 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4132480 con 0x7fc3a41419f0 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.437+0000 7fc3a9800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41424d0 con 0x7fc3a4131140 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33546/0 (socket says 172.21.15.55:33546) 2024-05-10T03:30:55.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 -- 172.21.15.55:0/2536927256 learned_addr learned my addr 172.21.15.55:0/2536927256 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 -- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a4131140 msgr2=0x7fc3a4131550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a4131140 0x7fc3a4131550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 -- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a41419f0 msgr2=0x7fc3a4141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2000640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a41419f0 0x7fc3a4141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a41419f0 0x7fc3a4141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.438+0000 7fc3a2a00640 1 -- 172.21.15.55:0/2536927256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4157770 con 0x7fc3880069d0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc3a2a00640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc39400d840 tx=0x7fc39400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940074e0 con 0x7fc3880069d0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394007640 con 0x7fc3880069d0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a4142720 con 0x7fc3880069d0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940049e0 con 0x7fc3880069d0 2024-05-10T03:30:55.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.439+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a4142ce0 con 0x7fc3880069d0 2024-05-10T03:30:55.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.440+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a41434e0 con 0x7fc3880069d0 2024-05-10T03:30:55.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.442+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc394004b40 con 0x7fc3880069d0 2024-05-10T03:30:55.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.443+0000 7fc39aa00640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 0x7fc38406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:55.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.443+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc394088fe0 con 0x7fc3880069d0 2024-05-10T03:30:55.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.444+0000 7fc3a2000640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 0x7fc38406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:55.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.445+0000 7fc3a2000640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 0x7fc38406d140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc390033000 tx=0x7fc390008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:55.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.448+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc3a41434e0 con 0x7fc3880069d0 2024-05-10T03:30:55.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.664+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3a4157770 con 0x7fc3880069d0 2024-05-10T03:30:55.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.665+0000 7fc39aa00640 1 -- 172.21.15.55:0/2536927256 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc3a4157770 con 0x7fc3880069d0 2024-05-10T03:30:55.670 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:30:55.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 msgr2=0x7fc38406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 0x7fc38406d140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc390033000 tx=0x7fc390008030 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 msgr2=0x7fc3a4140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc39400d840 tx=0x7fc39400dd00 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 shutdown_connections 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3a4131140 0x7fc3a4131550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc38406ac30 0x7fc38406d140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3a41419f0 0x7fc3a4141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 --2- 172.21.15.55:0/2536927256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3880069d0 0x7fc3a4140b80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 >> 172.21.15.55:0/2536927256 conn(0x7fc3a40cbb90 msgr2=0x7fc3a415bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 shutdown_connections 2024-05-10T03:30:55.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:55.668+0000 7fc3a9800640 1 -- 172.21.15.55:0/2536927256 wait complete. 2024-05-10T03:30:55.698 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:30:55.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:55.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-10T03:30:55.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:30:55.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:55.786 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:30:55.840 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:30:55.840 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:55.897 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:30:55.898 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:55.952 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:30:55.952 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:56.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:56.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.292+0000 7f0742800640 1 Processor -- start 2024-05-10T03:30:56.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.292+0000 7f0742800640 1 -- start start 2024-05-10T03:30:56.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f073c144710 0x7f073c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c06d7e0 0x7f073c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2fc0 0x7f073c15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c06d350 con 0x7f073c144710 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c06c740 con 0x7f073c0b2fc0 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0742800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c06ca20 con 0x7f073c06d7e0 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0741400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f073c144710 0x7f073c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0741400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f073c144710 0x7f073c144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33550/0 (socket says 172.21.15.55:33550) 2024-05-10T03:30:56.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.294+0000 7f0741400640 1 -- 172.21.15.55:0/1206583616 learned_addr learned my addr 172.21.15.55:0/1206583616 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0740a00640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c06d7e0 0x7f073c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741e00640 1 --2- 172.21.15.55:0/1206583616 >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2fc0 0x7f073c15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741400640 1 -- 172.21.15.55:0/1206583616 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c06d7e0 msgr2=0x7f073c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741400640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c06d7e0 0x7f073c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741400640 1 -- 172.21.15.55:0/1206583616 >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2fc0 msgr2=0x7f073c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741400640 1 --2- 172.21.15.55:0/1206583616 >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2fc0 0x7f073c15eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.295+0000 7f0741400640 1 -- 172.21.15.55:0/1206583616 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c06cd00 con 0x7f073c144710 2024-05-10T03:30:56.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.296+0000 7f0741400640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c144710 0x7f073c144120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f073c0b0c10 tx=0x7f073002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38919267d8004763 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.297+0000 7f073be00640 1 -- 172.21.15.55:0/1206583616 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073002ae90 con 0x7f073c144710 2024-05-10T03:30:56.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.297+0000 7f073be00640 1 -- 172.21.15.55:0/1206583616 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730032440 con 0x7f073c144710 2024-05-10T03:30:56.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.297+0000 7f073be00640 1 -- 172.21.15.55:0/1206583616 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300327d0 con 0x7f073c144710 2024-05-10T03:30:56.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.297+0000 7f0742800640 1 -- 172.21.15.55:0/1206583616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c144710 msgr2=0x7f073c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.297+0000 7f0742800640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c144710 0x7f073c144120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f073c0b0c10 tx=0x7f073002a140 comp rx=0 tx=0).stop 2024-05-10T03:30:56.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 -- 172.21.15.55:0/1206583616 shutdown_connections 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 --2- 172.21.15.55:0/1206583616 >> v2:172.21.15.162:3300/0 conn(0x7f073c0b2fc0 0x7f073c15eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c06d7e0 0x7f073c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 --2- 172.21.15.55:0/1206583616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c144710 0x7f073c144120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 -- 172.21.15.55:0/1206583616 >> 172.21.15.55:0/1206583616 conn(0x7f073c06a110 msgr2=0x7f073c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 -- 172.21.15.55:0/1206583616 shutdown_connections 2024-05-10T03:30:56.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.298+0000 7f0742800640 1 -- 172.21.15.55:0/1206583616 wait complete. 2024-05-10T03:30:56.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.300+0000 7f0742800640 1 Processor -- start 2024-05-10T03:30:56.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.300+0000 7f0742800640 1 -- start start 2024-05-10T03:30:56.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 0x7f073c1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c144710 0x7f073c137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c142590 con 0x7f073c0b2fc0 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c15e310 con 0x7f073c06d7e0 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0742800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c141d80 con 0x7f073c144710 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0740a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0740a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33556/0 (socket says 172.21.15.55:33556) 2024-05-10T03:30:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.302+0000 7f0740a00640 1 -- 172.21.15.55:0/739708774 learned_addr learned my addr 172.21.15.55:0/739708774 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:56.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0741400640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 0x7f073c1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0741e00640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c144710 0x7f073c137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 -- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c144710 msgr2=0x7f073c137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c144710 0x7f073c137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 -- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 msgr2=0x7f073c1415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 0x7f073c1415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 -- 172.21.15.55:0/739708774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300094d0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0741400640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 0x7f073c1415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0740a00640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f072800d5f0 tx=0x7f072800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280125f0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728012750 con 0x7f073c0b2fc0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728012b00 con 0x7f073c0b2fc0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.303+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c141b30 con 0x7f073c0b2fc0 2024-05-10T03:30:56.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.304+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1422b0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.305+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c066ce0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.306+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0728012c60 con 0x7f073c0b2fc0 2024-05-10T03:30:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.307+0000 7f0739600640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 0x7f071c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.307+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f0728089ab0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.308+0000 7f0741400640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 0x7f071c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.309+0000 7f0741400640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 0x7f071c06d140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0730034000 tx=0x7f073002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.311+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f07280547f0 con 0x7f073c0b2fc0 2024-05-10T03:30:56.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.525+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f073c141b30 con 0x7f073c0b2fc0 2024-05-10T03:30:56.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.528+0000 7f0739600640 1 -- 172.21.15.55:0/739708774 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f073c141b30 con 0x7f073c0b2fc0 2024-05-10T03:30:56.532 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:30:56.533 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:30:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.530+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 msgr2=0x7f071c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.530+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 0x7f071c06d140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0730034000 tx=0x7f073002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.530+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 msgr2=0x7f073c137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f072800d5f0 tx=0x7f072800dab0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 shutdown_connections 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f073c144710 0x7f073c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f073c0b2fc0 0x7f073c137880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f071c06ac30 0x7f071c06d140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 --2- 172.21.15.55:0/739708774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f073c06d7e0 0x7f073c1415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 >> 172.21.15.55:0/739708774 conn(0x7f073c06a110 msgr2=0x7f073c13e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 shutdown_connections 2024-05-10T03:30:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.531+0000 7f0742800640 1 -- 172.21.15.55:0/739708774 wait complete. 2024-05-10T03:30:56.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:30:56.566 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:30:56.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.844+0000 7fe419e00640 1 Processor -- start 2024-05-10T03:30:56.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.844+0000 7fe419e00640 1 -- start start 2024-05-10T03:30:56.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe419e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe414141a30 0x7fe4140cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe419e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe414071ce0 con 0x7fe41415bce0 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe419e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe414070460 con 0x7fe414070050 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe419e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe414070f40 con 0x7fe414141a30 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe412000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe414141a30 0x7fe4140cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe413400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe41415bce0 0x7fe41406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36866/0 (socket says 172.21.15.55:36866) 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.847+0000 7fe413400640 1 -- 172.21.15.55:0/3392713258 learned_addr learned my addr 172.21.15.55:0/3392713258 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:56.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.848+0000 7fe412000640 1 -- 172.21.15.55:0/3392713258 >> v1:172.21.15.162:6789/0 conn(0x7fe414070050 legacy=0x7fe41415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.848+0000 7fe412000640 1 -- 172.21.15.55:0/3392713258 >> v1:172.21.15.55:6789/0 conn(0x7fe41415bce0 legacy=0x7fe41406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.848+0000 7fe411600640 1 -- 172.21.15.55:0/3392713258 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474447366 0 0) 0x7fe40c001490 con 0x7fe41415bce0 2024-05-10T03:30:56.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.848+0000 7fe412000640 1 -- 172.21.15.55:0/3392713258 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414141e40 con 0x7fe414141a30 2024-05-10T03:30:56.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.849+0000 7fe412000640 1 --2- 172.21.15.55:0/3392713258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414141a30 0x7fe4140cd750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4000097a0 tx=0x7fe40002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274c9e669134e8bd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.850+0000 7fe411600640 1 -- 172.21.15.55:0/3392713258 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400032440 con 0x7fe414141a30 2024-05-10T03:30:56.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.850+0000 7fe411600640 1 -- 172.21.15.55:0/3392713258 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4000325a0 con 0x7fe414141a30 2024-05-10T03:30:56.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.850+0000 7fe411600640 1 -- 172.21.15.55:0/3392713258 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400032930 con 0x7fe414141a30 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 -- 172.21.15.55:0/3392713258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414141a30 msgr2=0x7fe4140cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 --2- 172.21.15.55:0/3392713258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414141a30 0x7fe4140cd750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4000097a0 tx=0x7fe40002a230 comp rx=0 tx=0).stop 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 -- 172.21.15.55:0/3392713258 shutdown_connections 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 --2- 172.21.15.55:0/3392713258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414141a30 0x7fe4140cd750 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 -- 172.21.15.55:0/3392713258 >> 172.21.15.55:0/3392713258 conn(0x7fe41406c120 msgr2=0x7fe41406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:56.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 -- 172.21.15.55:0/3392713258 shutdown_connections 2024-05-10T03:30:56.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.851+0000 7fe419e00640 1 -- 172.21.15.55:0/3392713258 wait complete. 2024-05-10T03:30:56.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.853+0000 7fe419e00640 1 Processor -- start 2024-05-10T03:30:56.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.854+0000 7fe419e00640 1 -- start start 2024-05-10T03:30:56.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe414141a30 0x7fe414130840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe41415bce0 0x7fe414158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414131140 con 0x7fe414070050 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141f7850 con 0x7fe41415bce0 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe419e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141f8340 con 0x7fe414141a30 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.855+0000 7fe412a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33580/0 (socket says 172.21.15.55:33580) 2024-05-10T03:30:56.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 -- 172.21.15.55:0/3968499070 learned_addr learned my addr 172.21.15.55:0/3968499070 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe413400640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe41415bce0 0x7fe414158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412000640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe414141a30 0x7fe414130840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 -- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe414141a30 msgr2=0x7fe414130840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe414141a30 0x7fe414130840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 -- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe41415bce0 msgr2=0x7fe414158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe41415bce0 0x7fe414158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 -- 172.21.15.55:0/3968499070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400009400 con 0x7fe414070050 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.856+0000 7fe412a00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c3c0 tx=0x7fe40400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.857+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4040052d0 con 0x7fe414070050 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.857+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404005450 con 0x7fe414070050 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.857+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404003830 con 0x7fe414070050 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.857+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe414132b00 con 0x7fe414070050 2024-05-10T03:30:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.857+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4141f7d80 con 0x7fe414070050 2024-05-10T03:30:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.858+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe414132c90 con 0x7fe414070050 2024-05-10T03:30:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.859+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe404003990 con 0x7fe414070050 2024-05-10T03:30:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.860+0000 7fe40b400640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 0x7fe3f406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.860+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe404088b10 con 0x7fe414070050 2024-05-10T03:30:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.861+0000 7fe412000640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 0x7fe3f406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:56.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.863+0000 7fe412000640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 0x7fe3f406d140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe400034000 tx=0x7fe40002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:56.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:56.865+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe404053860 con 0x7fe414070050 2024-05-10T03:30:57.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.064+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe414070460 con 0x7fe414070050 2024-05-10T03:30:57.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.065+0000 7fe40b400640 1 -- 172.21.15.55:0/3968499070 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe414070460 con 0x7fe414070050 2024-05-10T03:30:57.069 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.067+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 msgr2=0x7fe3f406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.067+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 0x7fe3f406d140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe400034000 tx=0x7fe40002e000 comp rx=0 tx=0).stop 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.067+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 msgr2=0x7fe414156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.067+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c3c0 tx=0x7fe40400c880 comp rx=0 tx=0).stop 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 shutdown_connections 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3f406ac30 0x7fe3f406d140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe41415bce0 0x7fe414158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe414141a30 0x7fe414130840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 --2- 172.21.15.55:0/3968499070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe414070050 0x7fe414156f90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 >> 172.21.15.55:0/3968499070 conn(0x7fe41406c120 msgr2=0x7fe4140aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:57.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 shutdown_connections 2024-05-10T03:30:57.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.068+0000 7fe419e00640 1 -- 172.21.15.55:0/3968499070 wait complete. 2024-05-10T03:30:57.100 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:30:57.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.384+0000 7f03f7400640 1 Processor -- start 2024-05-10T03:30:57.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.384+0000 7f03f7400640 1 -- start start 2024-05-10T03:30:57.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03f00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f7400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03f006b120 0x7f03f006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f7400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f00cee70 con 0x7f03f0070050 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f7400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f00b3590 con 0x7f03f006b120 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f7400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0166830 con 0x7f03f01658d0 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f4a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03f0070050 0x7f03f0161b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36870/0 (socket says 172.21.15.55:36870) 2024-05-10T03:30:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.387+0000 7f03f4a00640 1 -- 172.21.15.55:0/1026269638 learned_addr learned my addr 172.21.15.55:0/1026269638 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03f5400640 1 --2- 172.21.15.55:0/1026269638 >> v2:172.21.15.162:3300/0 conn(0x7f03f006b120 0x7f03f006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03efe00640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03f00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260075336 0 0) 0x7f03f00cee70 con 0x7f03f0070050 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d8003520 con 0x7f03f0070050 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780761005 0 0) 0x7f03d8003520 con 0x7f03f0070050 2024-05-10T03:30:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 msgr2=0x7f03f00cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03f00cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 >> v2:172.21.15.162:3300/0 conn(0x7f03f006b120 msgr2=0x7f03f006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 --2- 172.21.15.55:0/1026269638 >> v2:172.21.15.162:3300/0 conn(0x7f03f006b120 0x7f03f006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f006ba70 con 0x7f03f0070050 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.388+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f03e4001670 con 0x7f03f0070050 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 >> v1:172.21.15.55:6789/0 conn(0x7f03f0070050 legacy=0x7f03f0161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03d8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03d8003ef0 0x7f03d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03d8004d00 0x7f03d8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00b3590 con 0x7f03d8004d00 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0166830 con 0x7f03d8003ef0 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d800b9d0 con 0x7f03f01658d0 2024-05-10T03:30:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03d8004d00 0x7f03d8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03efe00640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03d8003ef0 0x7f03d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f4a00640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03d8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 -- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 msgr2=0x7f03d8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f01658d0 0x7f03d8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 -- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03d8003ef0 msgr2=0x7f03d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03d8003ef0 0x7f03d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.389+0000 7f03f5400640 1 -- 172.21.15.55:0/1026269638 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8003520 con 0x7f03d8004d00 2024-05-10T03:30:57.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.390+0000 7f03efe00640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03d8003ef0 0x7f03d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:57.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.391+0000 7f03f5400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03d8004d00 0x7f03d8005110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f03f00aed10 tx=0x7f03d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7c832b7d1ef609 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.391+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8031610 con 0x7f03d8004d00 2024-05-10T03:30:57.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.391+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e800f040 con 0x7f03d8004d00 2024-05-10T03:30:57.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.391+0000 7f03ef400640 1 -- 172.21.15.55:0/1026269638 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e80319b0 con 0x7f03d8004d00 2024-05-10T03:30:57.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 -- 172.21.15.55:0/1026269638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03d8004d00 msgr2=0x7f03d8005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 --2- 172.21.15.55:0/1026269638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03d8004d00 0x7f03d8005110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f03f00aed10 tx=0x7f03d80057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f5400640 1 -- 172.21.15.55:0/1026269638 reap_dead start 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 -- 172.21.15.55:0/1026269638 shutdown_connections 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 -- 172.21.15.55:0/1026269638 >> 172.21.15.55:0/1026269638 conn(0x7f03f006c120 msgr2=0x7f03f006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 -- 172.21.15.55:0/1026269638 shutdown_connections 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.392+0000 7f03f7400640 1 -- 172.21.15.55:0/1026269638 wait complete. 2024-05-10T03:30:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.394+0000 7f03f7400640 1 Processor -- start 2024-05-10T03:30:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.394+0000 7f03f7400640 1 -- start start 2024-05-10T03:30:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03d8004d00 0x7f03f0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f01577c0 0x7f03f0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f01543f0 con 0x7f03f0158110 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0155b50 con 0x7f03f01577c0 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f7400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f01654c0 con 0x7f03d8004d00 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33590/0 (socket says 172.21.15.55:33590) 2024-05-10T03:30:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.396+0000 7f03f5400640 1 -- 172.21.15.55:0/1984096431 learned_addr learned my addr 172.21.15.55:0/1984096431 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03efe00640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f01577c0 0x7f03f0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f4a00640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03d8004d00 0x7f03f0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 -- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03d8004d00 msgr2=0x7f03f0158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03d8004d00 0x7f03f0158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 -- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f01577c0 msgr2=0x7f03f0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f01577c0 0x7f03f0157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 -- 172.21.15.55:0/1984096431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0165710 con 0x7f03f0158110 2024-05-10T03:30:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03f5400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03e800e230 tx=0x7f03e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.397+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003b90 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.398+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8040070 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.398+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8031e10 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.398+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f01625a0 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.398+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f0153d30 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.399+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0066f60 con 0x7f03f0158110 2024-05-10T03:30:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.400+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f03e8042030 con 0x7f03f0158110 2024-05-10T03:30:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.401+0000 7f03ecc00640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 0x7f03d006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.401+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f03e80aa7d0 con 0x7f03f0158110 2024-05-10T03:30:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.402+0000 7f03f4a00640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 0x7f03d006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.403+0000 7f03f4a00640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 0x7f03d006d140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03e4007950 tx=0x7f03e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.405+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03e80755a0 con 0x7f03f0158110 2024-05-10T03:30:57.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.608+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03f006e260 con 0x7f03f0158110 2024-05-10T03:30:57.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.610+0000 7f03ecc00640 1 -- 172.21.15.55:0/1984096431 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03e8009c10 con 0x7f03f0158110 2024-05-10T03:30:57.614 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:57.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 msgr2=0x7f03d006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 0x7f03d006d140 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03e4007950 tx=0x7f03e4009040 comp rx=0 tx=0).stop 2024-05-10T03:30:57.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 msgr2=0x7f03f0154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03e800e230 tx=0x7f03e8009750 comp rx=0 tx=0).stop 2024-05-10T03:30:57.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 shutdown_connections 2024-05-10T03:30:57.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f0158110 0x7f03f0154810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d006ac30 0x7f03d006d140 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f01577c0 0x7f03f0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 --2- 172.21.15.55:0/1984096431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03d8004d00 0x7f03f0158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.612+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 >> 172.21.15.55:0/1984096431 conn(0x7f03f006c120 msgr2=0x7f03f00cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:57.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.613+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 shutdown_connections 2024-05-10T03:30:57.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.613+0000 7f03f7400640 1 -- 172.21.15.55:0/1984096431 wait complete. 2024-05-10T03:30:57.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:30:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.924+0000 7fc486400640 1 Processor -- start 2024-05-10T03:30:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.925+0000 7fc486400640 1 -- start start 2024-05-10T03:30:57.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc48015fe50 0x7fc480141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4800b2140 0x7fc480142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 0x7fc480142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48006ad20 con 0x7fc4800b2140 2024-05-10T03:30:57.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc480141f50 con 0x7fc4800b2550 2024-05-10T03:30:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.926+0000 7fc486400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006a110 con 0x7fc48015fe50 2024-05-10T03:30:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc476a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4800b2140 0x7fc480142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc476a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4800b2140 0x7fc480142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33606/0 (socket says 172.21.15.55:33606) 2024-05-10T03:30:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc476a00640 1 -- 172.21.15.55:0/1091501625 learned_addr learned my addr 172.21.15.55:0/1091501625 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc47fe00640 1 --2- 172.21.15.55:0/1091501625 >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 0x7fc480142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc476a00640 1 -- 172.21.15.55:0/1091501625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc48015fe50 msgr2=0x7fc480141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.927+0000 7fc476a00640 1 --2- 172.21.15.55:0/1091501625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc48015fe50 0x7fc480141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.928+0000 7fc476a00640 1 -- 172.21.15.55:0/1091501625 >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 msgr2=0x7fc480142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.928+0000 7fc476a00640 1 --2- 172.21.15.55:0/1091501625 >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 0x7fc480142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.928+0000 7fc476a00640 1 -- 172.21.15.55:0/1091501625 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48006a3f0 con 0x7fc4800b2140 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.928+0000 7fc47fe00640 1 --2- 172.21.15.55:0/1091501625 >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 0x7fc480142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.929+0000 7fc476a00640 1 --2- 172.21.15.55:0/1091501625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2140 0x7fc480142440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc48006ffe0 tx=0x7fc46802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1897974cae7102f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.929+0000 7fc47ea00640 1 -- 172.21.15.55:0/1091501625 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468032440 con 0x7fc4800b2140 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.929+0000 7fc47ea00640 1 -- 172.21.15.55:0/1091501625 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4680325a0 con 0x7fc4800b2140 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.929+0000 7fc47ea00640 1 -- 172.21.15.55:0/1091501625 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468032930 con 0x7fc4800b2140 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 -- 172.21.15.55:0/1091501625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2140 msgr2=0x7fc480142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 --2- 172.21.15.55:0/1091501625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2140 0x7fc480142440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc48006ffe0 tx=0x7fc46802a270 comp rx=0 tx=0).stop 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 -- 172.21.15.55:0/1091501625 shutdown_connections 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 --2- 172.21.15.55:0/1091501625 >> v2:172.21.15.162:3300/0 conn(0x7fc4800b2550 0x7fc480142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 --2- 172.21.15.55:0/1091501625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2140 0x7fc480142440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 --2- 172.21.15.55:0/1091501625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc48015fe50 0x7fc480141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 -- 172.21.15.55:0/1091501625 >> 172.21.15.55:0/1091501625 conn(0x7fc4800cbb90 msgr2=0x7fc4800cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 -- 172.21.15.55:0/1091501625 shutdown_connections 2024-05-10T03:30:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.930+0000 7fc486400640 1 -- 172.21.15.55:0/1091501625 wait complete. 2024-05-10T03:30:57.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.932+0000 7fc486400640 1 Processor -- start 2024-05-10T03:30:57.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.933+0000 7fc486400640 1 -- start start 2024-05-10T03:30:57.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4800b2140 0x7fc480132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc48015fe50 0x7fc48012d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48012e1c0 con 0x7fc4800b2550 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015e890 con 0x7fc48015fe50 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc486400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc480133220 con 0x7fc4800b2140 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc476a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.934+0000 7fc476a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33612/0 (socket says 172.21.15.55:33612) 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 -- 172.21.15.55:0/502681763 learned_addr learned my addr 172.21.15.55:0/502681763 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc47f400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4800b2140 0x7fc480132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc47fe00640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc48015fe50 0x7fc48012d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 -- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4800b2140 msgr2=0x7fc480132ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4800b2140 0x7fc480132ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 -- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc48015fe50 msgr2=0x7fc48012d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc48015fe50 0x7fc48012d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 -- 172.21.15.55:0/502681763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680094d0 con 0x7fc4800b2550 2024-05-10T03:30:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.935+0000 7fc476a00640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.936+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468032440 con 0x7fc4800b2550 2024-05-10T03:30:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.936+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc468030b80 con 0x7fc4800b2550 2024-05-10T03:30:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.936+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46803d420 con 0x7fc4800b2550 2024-05-10T03:30:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.936+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc480143630 con 0x7fc4800b2550 2024-05-10T03:30:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.936+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48015eae0 con 0x7fc4800b2550 2024-05-10T03:30:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.937+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4801437c0 con 0x7fc4800b2550 2024-05-10T03:30:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.938+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc468039040 con 0x7fc4800b2550 2024-05-10T03:30:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.939+0000 7fc477e00640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 0x7fc45c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.939+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fc4680aa5d0 con 0x7fc4800b2550 2024-05-10T03:30:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.940+0000 7fc47f400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 0x7fc45c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.941+0000 7fc47f400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 0x7fc45c06d110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc47000b810 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:57.944+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc468075310 con 0x7fc4800b2550 2024-05-10T03:30:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.137+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc480066f60 con 0x7fc4800b2550 2024-05-10T03:30:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.138+0000 7fc477e00640 1 -- 172.21.15.55:0/502681763 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc468078930 con 0x7fc4800b2550 2024-05-10T03:30:58.142 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:30:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.140+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 msgr2=0x7fc45c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.140+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 0x7fc45c06d110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc47000b810 tx=0x7fc470008040 comp rx=0 tx=0).stop 2024-05-10T03:30:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.140+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 msgr2=0x7fc48012d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.140+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680046c0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 shutdown_connections 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc45c06ac00 0x7fc45c06d110 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc48015fe50 0x7fc48012d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4800b2550 0x7fc48012d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 --2- 172.21.15.55:0/502681763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4800b2140 0x7fc480132ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 >> 172.21.15.55:0/502681763 conn(0x7fc4800cbb90 msgr2=0x7fc4800cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 shutdown_connections 2024-05-10T03:30:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.141+0000 7fc486400640 1 -- 172.21.15.55:0/502681763 wait complete. 2024-05-10T03:30:58.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:30:58.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.432+0000 7fa9d3e00640 1 Processor -- start 2024-05-10T03:30:58.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.432+0000 7fa9d3e00640 1 -- start start 2024-05-10T03:30:58.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d3e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9d406b830 0x7fa9d40cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9d40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40b2a30 con 0x7fa9d415e920 2024-05-10T03:30:58.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d3e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d415a900 con 0x7fa9d406b830 2024-05-10T03:30:58.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415eec0 con 0x7fa9d40cdb70 2024-05-10T03:30:58.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d2a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa9d415e920 0x7fa9d406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36876/0 (socket says 172.21.15.55:36876) 2024-05-10T03:30:58.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.435+0000 7fa9d2a00640 1 -- 172.21.15.55:0/4123473361 learned_addr learned my addr 172.21.15.55:0/4123473361 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d3400640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9d40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/4123473361 >> v2:172.21.15.162:3300/0 conn(0x7fa9d406b830 0x7fa9d40cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071384387 0 0) 0x7fa9d40b2a30 con 0x7fa9d415e920 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9b8003520 con 0x7fa9d415e920 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242117151 0 0) 0x7fa9b8003520 con 0x7fa9d415e920 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 msgr2=0x7fa9d40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9d40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 >> v2:172.21.15.162:3300/0 conn(0x7fa9d406b830 msgr2=0x7fa9d40cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 --2- 172.21.15.55:0/4123473361 >> v2:172.21.15.162:3300/0 conn(0x7fa9d406b830 0x7fa9d40cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d415b8b0 con 0x7fa9d415e920 2024-05-10T03:30:58.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa9c4001670 con 0x7fa9d415e920 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 >> v1:172.21.15.55:6789/0 conn(0x7fa9d415e920 legacy=0x7fa9d406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9b8003f60 0x7fa9b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9b8004d00 0x7fa9b8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415eec0 con 0x7fa9b8003f60 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b800dc40 con 0x7fa9b8004d00 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.436+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b800ddb0 con 0x7fa9d40cdb70 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9b8003f60 0x7fa9b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9d2a00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 -- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 msgr2=0x7fa9b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 -- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9b8004d00 msgr2=0x7fa9b8005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:58.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9b8004d00 0x7fa9b8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9cbe00640 1 -- 172.21.15.55:0/4123473361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b8003520 con 0x7fa9b8003f60 2024-05-10T03:30:58.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.437+0000 7fa9d2a00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d40cdb70 0x7fa9b80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.438+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9b8003f60 0x7fa9b80047c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009ac0 tx=0x7fa9b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264ab44dcbdc717a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.438+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc031400 con 0x7fa9b8003f60 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.438+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc02d050 con 0x7fa9b8003f60 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.438+0000 7fa9d2000640 1 -- 172.21.15.55:0/4123473361 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc031780 con 0x7fa9b8003f60 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 -- 172.21.15.55:0/4123473361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9b8003f60 msgr2=0x7fa9b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/4123473361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9b8003f60 0x7fa9b80047c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009ac0 tx=0x7fa9b80057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3400640 1 -- 172.21.15.55:0/4123473361 reap_dead start 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 -- 172.21.15.55:0/4123473361 shutdown_connections 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 -- 172.21.15.55:0/4123473361 >> 172.21.15.55:0/4123473361 conn(0x7fa9d406c750 msgr2=0x7fa9d40b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 -- 172.21.15.55:0/4123473361 shutdown_connections 2024-05-10T03:30:58.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.439+0000 7fa9d3e00640 1 -- 172.21.15.55:0/4123473361 wait complete. 2024-05-10T03:30:58.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.441+0000 7fa9d3e00640 1 Processor -- start 2024-05-10T03:30:58.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.442+0000 7fa9d3e00640 1 -- start start 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9b8003f60 0x7fa9d4159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d4159cd0 0x7fa9d40aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40aaac0 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d41ef3a0 con 0x7fa9d4159cd0 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d413e2c0 con 0x7fa9b8003f60 2024-05-10T03:30:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33634/0 (socket says 172.21.15.55:33634) 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.443+0000 7fa9d3400640 1 -- 172.21.15.55:0/883381152 learned_addr learned my addr 172.21.15.55:0/883381152 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d2a00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9b8003f60 0x7fa9d4159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9cbe00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d4159cd0 0x7fa9d40aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 -- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9b8003f60 msgr2=0x7fa9d4159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9b8003f60 0x7fa9d4159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 -- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d4159cd0 msgr2=0x7fa9d40aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d4159cd0 0x7fa9d40aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 -- 172.21.15.55:0/883381152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d413e510 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9d3400640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cc7a0 tx=0x7fa9cc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.444+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc010040 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.445+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00bb10 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.445+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00bea0 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.445+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d413e7f0 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.445+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d413edb0 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.446+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d404e290 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.447+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa9cc01e410 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.448+0000 7fa9caa00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 0x7fa9b006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.448+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa9cc0899d0 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.449+0000 7fa9d2a00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 0x7fa9b006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:58.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.450+0000 7fa9d2a00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 0x7fa9b006d140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001eb0 tx=0x7fa9c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:58.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.452+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa9cc054710 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.721+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa9d40b3990 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.722+0000 7fa9caa00640 1 -- 172.21.15.55:0/883381152 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa9cc057d30 con 0x7fa9d40aa6b0 2024-05-10T03:30:58.726 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 msgr2=0x7fa9b006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 0x7fa9b006d140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4001eb0 tx=0x7fa9c4009040 comp rx=0 tx=0).stop 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 msgr2=0x7fa9d413f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40cc7a0 tx=0x7fa9cc00b480 comp rx=0 tx=0).stop 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 shutdown_connections 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9d40aa6b0 0x7fa9d413f6f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9b006ac30 0x7fa9b006d140 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d4159cd0 0x7fa9d40aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 --2- 172.21.15.55:0/883381152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9b8003f60 0x7fa9d4159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:58.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.724+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 >> 172.21.15.55:0/883381152 conn(0x7fa9d406c750 msgr2=0x7fa9d4071020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:58.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.725+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 shutdown_connections 2024-05-10T03:30:58.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:58.725+0000 7fa9d3e00640 1 -- 172.21.15.55:0/883381152 wait complete. 2024-05-10T03:30:58.755 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:30:59.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.027+0000 7fd189400640 1 Processor -- start 2024-05-10T03:30:59.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.027+0000 7fd189400640 1 -- start start 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd189400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd18406b830 0x7fd184166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd189400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18415bde0 con 0x7fd18415b9d0 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd189400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18413eae0 con 0x7fd18413e390 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd189400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18413e7a0 con 0x7fd18406b830 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd183400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd18415b9d0 0x7fd18406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36882/0 (socket says 172.21.15.55:36882) 2024-05-10T03:30:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd183400640 1 -- 172.21.15.55:0/3682248265 learned_addr learned my addr 172.21.15.55:0/3682248265 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd188a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd18406b830 0x7fd184166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.030+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234686259 0 0) 0x7fd16c001610 con 0x7fd18415b9d0 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168003520 con 0x7fd18415b9d0 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1175634555 0 0) 0x7fd168003520 con 0x7fd18415b9d0 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd18406b830 msgr2=0x7fd184166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd18406b830 0x7fd184166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 >> v1:172.21.15.162:6789/0 conn(0x7fd18413e390 legacy=0x7fd1841414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18415c020 con 0x7fd18415b9d0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fd16c001d00 con 0x7fd18415b9d0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 >> v1:172.21.15.55:6789/0 conn(0x7fd18415b9d0 legacy=0x7fd18406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1680039a0 0x7fd168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd168004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd168005110 0x7fd16800fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18413e7a0 con 0x7fd1680039a0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd168010380 con 0x7fd168005110 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680104c0 con 0x7fd1680047c0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.031+0000 7fd183e00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1680039a0 0x7fd168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd168004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd188a00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd168005110 0x7fd16800fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 -- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd168005110 msgr2=0x7fd16800fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd168005110 0x7fd16800fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 -- 172.21.15.55:0/3682248265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1680039a0 msgr2=0x7fd168003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1680039a0 0x7fd168003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183400640 1 -- 172.21.15.55:0/3682248265 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168003520 con 0x7fd1680047c0 2024-05-10T03:30:59.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.032+0000 7fd183e00640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1680039a0 0x7fd168003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.033+0000 7fd183400640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd168004bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd18406e290 tx=0x7fd1680057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ec8199dab2c3ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.034+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c005d20 con 0x7fd1680047c0 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.034+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c005e80 con 0x7fd1680047c0 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.034+0000 7fd182a00640 1 -- 172.21.15.55:0/3682248265 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0072b0 con 0x7fd1680047c0 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 -- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 msgr2=0x7fd168004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 --2- 172.21.15.55:0/3682248265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd168004bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd18406e290 tx=0x7fd1680057f0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd188a00640 1 -- 172.21.15.55:0/3682248265 reap_dead start 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 -- 172.21.15.55:0/3682248265 shutdown_connections 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 -- 172.21.15.55:0/3682248265 >> 172.21.15.55:0/3682248265 conn(0x7fd18406c730 msgr2=0x7fd1840b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:59.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 -- 172.21.15.55:0/3682248265 shutdown_connections 2024-05-10T03:30:59.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.035+0000 7fd189400640 1 -- 172.21.15.55:0/3682248265 wait complete. 2024-05-10T03:30:59.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.037+0000 7fd189400640 1 Processor -- start 2024-05-10T03:30:59.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.037+0000 7fd189400640 1 -- start start 2024-05-10T03:30:59.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd1841513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 0x7fd184154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184151df0 con 0x7fd184154e50 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840717c0 con 0x7fd184154500 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd189400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd184152820 con 0x7fd1680047c0 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd188a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd188a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33648/0 (socket says 172.21.15.55:33648) 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd188a00640 1 -- 172.21.15.55:0/833631341 learned_addr learned my addr 172.21.15.55:0/833631341 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd183400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 0x7fd184154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd188a00640 1 -- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 msgr2=0x7fd1841513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.039+0000 7fd183e00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd1841513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd188a00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd1841513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd188a00640 1 -- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 msgr2=0x7fd184154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd188a00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 0x7fd184154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd188a00640 1 -- 172.21.15.55:0/833631341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184134560 con 0x7fd184154e50 2024-05-10T03:30:59.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd183400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 0x7fd184154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd188a00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd18406f0d0 tx=0x7fd17c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004240 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0043c0 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00f9e0 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1840702b0 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.040+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1841518e0 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.041+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184070440 con 0x7fd184154e50 2024-05-10T03:30:59.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.042+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd17c00fb40 con 0x7fd184154e50 2024-05-10T03:30:59.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.043+0000 7fd17be00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 0x7fd16006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.043+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd17c089890 con 0x7fd184154e50 2024-05-10T03:30:59.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.044+0000 7fd183e00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 0x7fd16006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.045+0000 7fd183e00640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 0x7fd16006d140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1840b4350 tx=0x7fd174009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.048+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd17c054540 con 0x7fd184154e50 2024-05-10T03:30:59.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.255+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1840cc250 con 0x7fd184154e50 2024-05-10T03:30:59.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.256+0000 7fd17be00640 1 -- 172.21.15.55:0/833631341 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd17c00fdf0 con 0x7fd184154e50 2024-05-10T03:30:59.261 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:30:59.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 msgr2=0x7fd16006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 0x7fd16006d140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1840b4350 tx=0x7fd174009040 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 msgr2=0x7fd184070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd18406f0d0 tx=0x7fd17c011cf0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 shutdown_connections 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd184154e50 0x7fd184070740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd16006ac30 0x7fd16006d140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd184154500 0x7fd184154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 --2- 172.21.15.55:0/833631341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1680047c0 0x7fd1841513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 >> 172.21.15.55:0/833631341 conn(0x7fd18406c730 msgr2=0x7fd18413afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 shutdown_connections 2024-05-10T03:30:59.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.259+0000 7fd189400640 1 -- 172.21.15.55:0/833631341 wait complete. 2024-05-10T03:30:59.291 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:30:59.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:59.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-10T03:30:59.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:30:59.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:30:59.388 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:30:59.451 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:59.451 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:59.510 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:59.510 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:59.568 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:30:59.568 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:30:59.621 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:30:59.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.901+0000 7f9285a00640 1 Processor -- start 2024-05-10T03:30:59.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.901+0000 7f9285a00640 1 -- start start 2024-05-10T03:30:59.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f9285a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9280143b40 0x7f928015e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f9285a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf880 0x7f92800cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f9285a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92800cefe0 con 0x7f92800cf470 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f9285a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928015edb0 con 0x7f9280143b40 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f9285a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9280143f50 con 0x7f92800cf880 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f927f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f92800cf470 0x7f928015efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36898/0 (socket says 172.21.15.55:36898) 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f927f400640 1 -- 172.21.15.55:0/679222037 learned_addr learned my addr 172.21.15.55:0/679222037 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.904+0000 7f927fe00640 1 --2- 172.21.15.55:0/679222037 >> v2:172.21.15.55:3300/0 conn(0x7f9280143b40 0x7f928015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f9285000640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf880 0x7f92800cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927ea00640 1 -- 172.21.15.55:0/679222037 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636573083 0 0) 0x7f92800cefe0 con 0x7f92800cf470 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927ea00640 1 -- 172.21.15.55:0/679222037 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9264003520 con 0x7f92800cf470 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927fe00640 1 -- 172.21.15.55:0/679222037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf880 msgr2=0x7f92800cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:30:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927fe00640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf880 0x7f92800cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927fe00640 1 -- 172.21.15.55:0/679222037 >> v1:172.21.15.55:6789/0 conn(0x7f92800cf470 legacy=0x7f928015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.905+0000 7f927fe00640 1 -- 172.21.15.55:0/679222037 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92800ce3d0 con 0x7f9280143b40 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.906+0000 7f927fe00640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9280143b40 0x7f928015e870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9270009d90 tx=0x7f927002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8078d652789c3906 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f927ea00640 1 -- 172.21.15.55:0/679222037 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270007740 con 0x7f9280143b40 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f927ea00640 1 -- 172.21.15.55:0/679222037 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927002d050 con 0x7f9280143b40 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f927ea00640 1 -- 172.21.15.55:0/679222037 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270007aa0 con 0x7f9280143b40 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 -- 172.21.15.55:0/679222037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9280143b40 msgr2=0x7f928015e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9280143b40 0x7f928015e870 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9270009d90 tx=0x7f927002a660 comp rx=0 tx=0).stop 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 -- 172.21.15.55:0/679222037 shutdown_connections 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf880 0x7f92800cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 --2- 172.21.15.55:0/679222037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9280143b40 0x7f928015e870 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 -- 172.21.15.55:0/679222037 >> 172.21.15.55:0/679222037 conn(0x7f92800b6680 msgr2=0x7f92800b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:30:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.907+0000 7f9285a00640 1 -- 172.21.15.55:0/679222037 shutdown_connections 2024-05-10T03:30:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.908+0000 7f9285a00640 1 -- 172.21.15.55:0/679222037 wait complete. 2024-05-10T03:30:59.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.910+0000 7f9285a00640 1 Processor -- start 2024-05-10T03:30:59.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.910+0000 7f9285a00640 1 -- start start 2024-05-10T03:30:59.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf470 0x7f928015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9280143b40 0x7f928013d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801f4340 con 0x7f92800cf880 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801f4950 con 0x7f9280143b40 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f9285a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92801415f0 con 0x7f92800cf470 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f927f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f927f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33668/0 (socket says 172.21.15.55:33668) 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.912+0000 7f927f400640 1 -- 172.21.15.55:0/2526346390 learned_addr learned my addr 172.21.15.55:0/2526346390 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f9285000640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9280143b40 0x7f928013d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927fe00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf470 0x7f928015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 -- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf470 msgr2=0x7f928015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf470 0x7f928015a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 -- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9280143b40 msgr2=0x7f928013d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:30:59.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9280143b40 0x7f928013d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 -- 172.21.15.55:0/2526346390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270009a70 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f927f400640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c3c0 tx=0x7f926c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c005ce0 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.913+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c005e40 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.914+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c0072f0 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.914+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92801f3730 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.914+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801f4760 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.915+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92801f4480 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.916+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f926c007450 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.917+0000 7f9277e00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 0x7f926006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.917+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f926c088de0 con 0x7f92800cf880 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.918+0000 7f927fe00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 0x7f926006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:30:59.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.919+0000 7f927fe00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 0x7f926006d110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9270034000 tx=0x7f927002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:30:59.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:30:59.922+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f92801f4480 con 0x7f92800cf880 2024-05-10T03:31:00.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.125+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92801f4480 con 0x7f92800cf880 2024-05-10T03:31:00.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.128+0000 7f9277e00640 1 -- 172.21.15.55:0/2526346390 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f92801f4480 con 0x7f92800cf880 2024-05-10T03:31:00.133 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:00.133 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.131+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 msgr2=0x7f926006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.131+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 0x7f926006d110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9270034000 tx=0x7f927002f000 comp rx=0 tx=0).stop 2024-05-10T03:31:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.131+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 msgr2=0x7f9280140b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.131+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c3c0 tx=0x7f926c00c880 comp rx=0 tx=0).stop 2024-05-10T03:31:00.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 shutdown_connections 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f926006ac00 0x7f926006d110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9280143b40 0x7f928013d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92800cf880 0x7f9280140b50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 --2- 172.21.15.55:0/2526346390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92800cf470 0x7f928015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 >> 172.21.15.55:0/2526346390 conn(0x7f92800b6680 msgr2=0x7f92800b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 shutdown_connections 2024-05-10T03:31:00.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.132+0000 7f9285a00640 1 -- 172.21.15.55:0/2526346390 wait complete. 2024-05-10T03:31:00.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:31:00.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:00.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.447+0000 7f1fa2000640 1 Processor -- start 2024-05-10T03:31:00.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.447+0000 7f1fa2000640 1 -- start start 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.450+0000 7f1fa2000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f9c070050 0x7f1f9c141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.450+0000 7f1fa2000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c071ce0 con 0x7f1f9c0cd750 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.450+0000 7f1fa2000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c070460 con 0x7f1f9c15bce0 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.450+0000 7f1fa2000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c070f40 con 0x7f1f9c070050 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.451+0000 7f1f9aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1f9c0cd750 0x7f1f9c15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36912/0 (socket says 172.21.15.55:36912) 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.451+0000 7f1f9aa00640 1 -- 172.21.15.55:0/2517382845 learned_addr learned my addr 172.21.15.55:0/2517382845 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:00.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.451+0000 7f1f9b400640 1 --2- 172.21.15.55:0/2517382845 >> v2:172.21.15.162:3300/0 conn(0x7f1f9c070050 0x7f1f9c141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.451+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090648762 0 0) 0x7f1f88001110 con 0x7f1f9c0cd750 2024-05-10T03:31:00.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.451+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f80003520 con 0x7f1f9c0cd750 2024-05-10T03:31:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82800465 0 0) 0x7f1f80003520 con 0x7f1f9c0cd750 2024-05-10T03:31:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 >> v2:172.21.15.162:3300/0 conn(0x7f1f9c070050 msgr2=0x7f1f9c141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 --2- 172.21.15.55:0/2517382845 >> v2:172.21.15.162:3300/0 conn(0x7f1f9c070050 0x7f1f9c141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 >> v1:172.21.15.162:6789/0 conn(0x7f1f9c15bce0 legacy=0x7f1f9c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0cdb60 con 0x7f1f9c0cd750 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f1f88001800 con 0x7f1f9c0cd750 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 >> v1:172.21.15.55:6789/0 conn(0x7f1f9c0cd750 legacy=0x7f1f9c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f800047c0 0x7f1f80004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f80005180 0x7f1f8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c070f40 con 0x7f1f80003970 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8000c120 con 0x7f1f80005180 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8000c2e0 con 0x7f1f800047c0 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.452+0000 7f1f9b400640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9aa00640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f800047c0 0x7f1f80004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9be00640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f80005180 0x7f1f8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9b400640 1 -- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f800047c0 msgr2=0x7f1f80004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9b400640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f800047c0 0x7f1f80004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9b400640 1 -- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f80005180 msgr2=0x7f1f8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9b400640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f80005180 0x7f1f8000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9b400640 1 -- 172.21.15.55:0/2517382845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80003520 con 0x7f1f80003970 2024-05-10T03:31:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.453+0000 7f1f9be00640 1 -- 172.21.15.55:0/2517382845 reap_dead start 2024-05-10T03:31:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.454+0000 7f1f9b400640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f80003de0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009a70 tx=0x7f1f800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1693a22892a7f204 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.455+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031400 con 0x7f1f80003970 2024-05-10T03:31:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.455+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c02d050 con 0x7f1f80003970 2024-05-10T03:31:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.455+0000 7f1f9a000640 1 -- 172.21.15.55:0/2517382845 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0317a0 con 0x7f1f80003970 2024-05-10T03:31:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.455+0000 7f1fa2000640 1 -- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 msgr2=0x7f1f80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.455+0000 7f1fa2000640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f80003de0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009a70 tx=0x7f1f800057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.456+0000 7f1fa2000640 1 -- 172.21.15.55:0/2517382845 shutdown_connections 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.456+0000 7f1fa2000640 1 --2- 172.21.15.55:0/2517382845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f80003de0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.456+0000 7f1fa2000640 1 -- 172.21.15.55:0/2517382845 >> 172.21.15.55:0/2517382845 conn(0x7f1f9c06c120 msgr2=0x7f1f9c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.456+0000 7f1fa2000640 1 -- 172.21.15.55:0/2517382845 shutdown_connections 2024-05-10T03:31:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.456+0000 7f1fa2000640 1 -- 172.21.15.55:0/2517382845 wait complete. 2024-05-10T03:31:00.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.458+0000 7f1fa2000640 1 Processor -- start 2024-05-10T03:31:00.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.458+0000 7f1fa2000640 1 -- start start 2024-05-10T03:31:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 0x7f1f9c1e6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 0x7f1f9c1f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1e5ee0 con 0x7f1f80003970 2024-05-10T03:31:00.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1f46a0 con 0x7f1f9c1e6840 2024-05-10T03:31:00.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1fa2000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1e5260 con 0x7f1f9c1f3d50 2024-05-10T03:31:00.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33698/0 (socket says 172.21.15.55:33698) 2024-05-10T03:31:00.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.460+0000 7f1f9b400640 1 -- 172.21.15.55:0/1533017680 learned_addr learned my addr 172.21.15.55:0/1533017680 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:00.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9aa00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 0x7f1f9c1e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9be00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 0x7f1f9c1f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 -- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 msgr2=0x7f1f9c1e6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 0x7f1f9c1e6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 -- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 msgr2=0x7f1f9c1f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 0x7f1f9c1f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 -- 172.21.15.55:0/1533017680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c1e5540 con 0x7f1f80003970 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9be00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 0x7f1f9c1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9aa00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 0x7f1f9c1e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f9b400640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034040 tx=0x7f1f8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c03b670 con 0x7f1f80003970 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c03b7d0 con 0x7f1f80003970 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.461+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c03bb40 con 0x7f1f80003970 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.462+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c1e5820 con 0x7f1f80003970 2024-05-10T03:31:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.462+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c140e20 con 0x7f1f80003970 2024-05-10T03:31:00.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.463+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c1e6150 con 0x7f1f80003970 2024-05-10T03:31:00.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.464+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1f8c038040 con 0x7f1f80003970 2024-05-10T03:31:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.465+0000 7f1f93400640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 0x7f1f7c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.465+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1f8c0a9f50 con 0x7f1f80003970 2024-05-10T03:31:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.466+0000 7f1f9aa00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 0x7f1f7c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.467+0000 7f1f9aa00640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 0x7f1f7c06d110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c15d130 tx=0x7f1f88001150 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:00.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.470+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1f8c078720 con 0x7f1f80003970 2024-05-10T03:31:00.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.667+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f9c066f60 con 0x7f1f80003970 2024-05-10T03:31:00.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.668+0000 7f1f93400640 1 -- 172.21.15.55:0/1533017680 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f8c036020 con 0x7f1f80003970 2024-05-10T03:31:00.672 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.670+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 msgr2=0x7f1f7c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.670+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 0x7f1f7c06d110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c15d130 tx=0x7f1f88001150 comp rx=0 tx=0).stop 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.670+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 msgr2=0x7f1f9c1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.670+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034040 tx=0x7f1f8c004730 comp rx=0 tx=0).stop 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.670+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 shutdown_connections 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1f7c06ac00 0x7f1f7c06d110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f9c1e6840 0x7f1f9c1f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f9c1f3d50 0x7f1f9c1e6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 --2- 172.21.15.55:0/1533017680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f80003970 0x7f1f9c1408e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:00.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 >> 172.21.15.55:0/1533017680 conn(0x7f1f9c06c120 msgr2=0x7f1f9c15df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:00.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 shutdown_connections 2024-05-10T03:31:00.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.671+0000 7f1fa2000640 1 -- 172.21.15.55:0/1533017680 wait complete. 2024-05-10T03:31:00.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.991+0000 7f6f48c00640 1 Processor -- start 2024-05-10T03:31:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.991+0000 7f6f48c00640 1 -- start start 2024-05-10T03:31:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f48c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f48c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6f44071e00 0x7f6f44140b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f48c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4415fdf0 con 0x7f6f440719f0 2024-05-10T03:31:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f48c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4415f1e0 con 0x7f6f44071e00 2024-05-10T03:31:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f48c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4415f4c0 con 0x7f6f440710f0 2024-05-10T03:31:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f42a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f42a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37926/0 (socket says 172.21.15.55:37926) 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.994+0000 7f6f43400640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6f440719f0 0x7f6f4406b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:59298/0 (socket says 172.21.15.55:59298) 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f42a00640 1 -- 172.21.15.55:0/2690519144 learned_addr learned my addr 172.21.15.55:0/2690519144 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f43e00640 1 --2- 172.21.15.55:0/2690519144 >> v2:172.21.15.162:3300/0 conn(0x7f6f44071e00 0x7f6f44140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f43e00640 1 -- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 msgr2=0x7f6f441405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f43e00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f43e00640 1 -- 172.21.15.55:0/2690519144 >> v1:172.21.15.162:6789/0 conn(0x7f6f440719f0 legacy=0x7f6f4406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f43e00640 1 -- 172.21.15.55:0/2690519144 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4415f7a0 con 0x7f6f44071e00 2024-05-10T03:31:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.995+0000 7f6f42a00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.997+0000 7f6f43e00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f44071e00 0x7f6f44140b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0097a0 tx=0x7f6f3c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad42b2c836c6c4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.997+0000 7f6f42000640 1 -- 172.21.15.55:0/2690519144 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c02ae50 con 0x7f6f44071e00 2024-05-10T03:31:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f42000640 1 -- 172.21.15.55:0/2690519144 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c032440 con 0x7f6f44071e00 2024-05-10T03:31:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f42000640 1 -- 172.21.15.55:0/2690519144 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0327d0 con 0x7f6f44071e00 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 -- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f44071e00 msgr2=0x7f6f44140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f44071e00 0x7f6f44140b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0097a0 tx=0x7f6f3c02a100 comp rx=0 tx=0).stop 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 -- 172.21.15.55:0/2690519144 shutdown_connections 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f441405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 --2- 172.21.15.55:0/2690519144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f44071e00 0x7f6f44140b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.998+0000 7f6f48c00640 1 -- 172.21.15.55:0/2690519144 >> 172.21.15.55:0/2690519144 conn(0x7f6f4406c750 msgr2=0x7f6f440b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.999+0000 7f6f48c00640 1 -- 172.21.15.55:0/2690519144 shutdown_connections 2024-05-10T03:31:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:00.999+0000 7f6f48c00640 1 -- 172.21.15.55:0/2690519144 wait complete. 2024-05-10T03:31:01.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.002+0000 7f6f48c00640 1 Processor -- start 2024-05-10T03:31:01.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.002+0000 7f6f48c00640 1 -- start start 2024-05-10T03:31:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f440ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 0x7f6f44155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440ad3b0 con 0x7f6f44071e00 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440ac4e0 con 0x7f6f440719f0 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f48c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440ac730 con 0x7f6f440710f0 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33710/0 (socket says 172.21.15.55:33710) 2024-05-10T03:31:01.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f43e00640 1 -- 172.21.15.55:0/730026695 learned_addr learned my addr 172.21.15.55:0/730026695 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:01.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.004+0000 7f6f42a00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 0x7f6f44155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43400640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f440ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 -- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 msgr2=0x7f6f440ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f440ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 -- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 msgr2=0x7f6f44155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 0x7f6f44155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 -- 172.21.15.55:0/730026695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0094d0 con 0x7f6f44071e00 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f42a00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 0x7f6f44155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.005+0000 7f6f43e00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0097a0 tx=0x7f6f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.006+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03d850 con 0x7f6f44071e00 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.006+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c03d9d0 con 0x7f6f44071e00 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.006+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c009400 con 0x7f6f44071e00 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.006+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03dd80 con 0x7f6f44071e00 2024-05-10T03:31:01.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.006+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f440accf0 con 0x7f6f44071e00 2024-05-10T03:31:01.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.007+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f440ad4f0 con 0x7f6f44071e00 2024-05-10T03:31:01.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.008+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6f3c039040 con 0x7f6f44071e00 2024-05-10T03:31:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.010+0000 7f6f3b400640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 0x7f6f2006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.010+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f6f3c0a9fe0 con 0x7f6f44071e00 2024-05-10T03:31:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.010+0000 7f6f43400640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 0x7f6f2006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.011+0000 7f6f43400640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 0x7f6f2006d110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f340012e0 tx=0x7f6f34005b70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.014+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6f440ad4f0 con 0x7f6f44071e00 2024-05-10T03:31:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.213+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f440ad4f0 con 0x7f6f44071e00 2024-05-10T03:31:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.214+0000 7f6f3b400640 1 -- 172.21.15.55:0/730026695 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f440ad4f0 con 0x7f6f44071e00 2024-05-10T03:31:01.218 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.216+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 msgr2=0x7f6f2006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.216+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 0x7f6f2006d110 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f340012e0 tx=0x7f6f34005b70 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.216+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 msgr2=0x7f6f440c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.216+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0097a0 tx=0x7f6f3c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 shutdown_connections 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f44071e00 0x7f6f440c9a90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6f2006ac00 0x7f6f2006d110 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f440719f0 0x7f6f44155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 --2- 172.21.15.55:0/730026695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f440710f0 0x7f6f440ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 >> 172.21.15.55:0/730026695 conn(0x7f6f4406c750 msgr2=0x7f6f4406b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 shutdown_connections 2024-05-10T03:31:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.217+0000 7f6f48c00640 1 -- 172.21.15.55:0/730026695 wait complete. 2024-05-10T03:31:01.248 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:01.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.524+0000 7f0988a00640 1 Processor -- start 2024-05-10T03:31:01.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.525+0000 7f0988a00640 1 -- start start 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.527+0000 7f0988a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f09840ce880 0x7f098406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.527+0000 7f0988a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098406ba70 con 0x7f098415adc0 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.527+0000 7f0988a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840cec90 con 0x7f098406b660 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.527+0000 7f0988a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098415a0d0 con 0x7f09840ce880 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f0982800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f098415adc0 0x7f098415e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36926/0 (socket says 172.21.15.55:36926) 2024-05-10T03:31:01.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f0982800640 1 -- 172.21.15.55:0/1905375097 learned_addr learned my addr 172.21.15.55:0/1905375097 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:01.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f097be00640 1 --2- 172.21.15.55:0/1905375097 >> v2:172.21.15.162:3300/0 conn(0x7f09840ce880 0x7f098406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860765004 0 0) 0x7f098406ba70 con 0x7f098415adc0 2024-05-10T03:31:01.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0968003520 con 0x7f098415adc0 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.528+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110628654 0 0) 0x7f0968003520 con 0x7f098415adc0 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 >> v2:172.21.15.162:3300/0 conn(0x7f09840ce880 msgr2=0x7f098406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 --2- 172.21.15.55:0/1905375097 >> v2:172.21.15.162:3300/0 conn(0x7f09840ce880 0x7f098406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 >> v1:172.21.15.162:6789/0 conn(0x7f098406b660 legacy=0x7f09840b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840b4330 con 0x7f098415adc0 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0974001670 con 0x7f098415adc0 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 >> v1:172.21.15.55:6789/0 conn(0x7f098415adc0 legacy=0x7f098415e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0968003970 0x7f0968003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09680047c0 0x7f0968004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 0x7f096800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098415a0d0 con 0x7f0968005230 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096800c1c0 con 0x7f09680047c0 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096800d490 con 0x7f0968003970 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0983200640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 0x7f096800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f097be00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09680047c0 0x7f0968004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.529+0000 7f0982800640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0968003970 0x7f0968003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.530+0000 7f0983200640 1 -- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0968003970 msgr2=0x7f0968003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.530+0000 7f0983200640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0968003970 0x7f0968003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.530+0000 7f0983200640 1 -- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09680047c0 msgr2=0x7f0968004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.530+0000 7f0983200640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09680047c0 0x7f0968004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.530+0000 7f0983200640 1 -- 172.21.15.55:0/1905375097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968003520 con 0x7f0968005230 2024-05-10T03:31:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.531+0000 7f0983200640 1 -- 172.21.15.55:0/1905375097 reap_dead start 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.531+0000 7f0983200640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 0x7f096800b980 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f097c001e50 tx=0x7f09680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d07b5c3f677762 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.531+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c02ae70 con 0x7f0968005230 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.531+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c003970 con 0x7f0968005230 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.531+0000 7f0981e00640 1 -- 172.21.15.55:0/1905375097 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c003d20 con 0x7f0968005230 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 -- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 msgr2=0x7f096800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 0x7f096800b980 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f097c001e50 tx=0x7f09680057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 -- 172.21.15.55:0/1905375097 shutdown_connections 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 --2- 172.21.15.55:0/1905375097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0968005230 0x7f096800b980 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 -- 172.21.15.55:0/1905375097 >> 172.21.15.55:0/1905375097 conn(0x7f098406c730 msgr2=0x7f09840b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 -- 172.21.15.55:0/1905375097 shutdown_connections 2024-05-10T03:31:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.532+0000 7f0988a00640 1 -- 172.21.15.55:0/1905375097 wait complete. 2024-05-10T03:31:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.535+0000 7f0988a00640 1 Processor -- start 2024-05-10T03:31:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.535+0000 7f0988a00640 1 -- start start 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0968005230 0x7f098413b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09841f2d60 0x7f09841f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841f3b60 con 0x7f09840c6480 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841f2390 con 0x7f0968005230 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.536+0000 7f0988a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09841f36b0 con 0x7f09841f2d60 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33726/0 (socket says 172.21.15.55:33726) 2024-05-10T03:31:01.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 -- 172.21.15.55:0/3980773180 learned_addr learned my addr 172.21.15.55:0/3980773180 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 -- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09841f2d60 msgr2=0x7f09841f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09841f2d60 0x7f09841f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f0982800640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0968005230 0x7f098413b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 -- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0968005230 msgr2=0x7f098413b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0968005230 0x7f098413b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 -- 172.21.15.55:0/3980773180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098413bd80 con 0x7f09840c6480 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.537+0000 7f097be00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f096c00b550 tx=0x7f096c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.538+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c004020 con 0x7f09840c6480 2024-05-10T03:31:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.538+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c004180 con 0x7f09840c6480 2024-05-10T03:31:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.538+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0106e0 con 0x7f09840c6480 2024-05-10T03:31:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.538+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09841e4620 con 0x7f09840c6480 2024-05-10T03:31:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.538+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09841e4be0 con 0x7f09840c6480 2024-05-10T03:31:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.539+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098404e290 con 0x7f09840c6480 2024-05-10T03:31:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.541+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f096c010840 con 0x7f09840c6480 2024-05-10T03:31:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.541+0000 7f097aa00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 0x7f096006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.542+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f096c089990 con 0x7f09840c6480 2024-05-10T03:31:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.542+0000 7f0982800640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 0x7f096006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:01.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.543+0000 7f0982800640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 0x7f096006d110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0974001200 tx=0x7f0974001150 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.546+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f096c0546e0 con 0x7f09840c6480 2024-05-10T03:31:01.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.748+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0984159610 con 0x7f09840c6480 2024-05-10T03:31:01.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.749+0000 7f097aa00640 1 -- 172.21.15.55:0/3980773180 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f096c021020 con 0x7f09840c6480 2024-05-10T03:31:01.754 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 msgr2=0x7f096006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 0x7f096006d110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0974001200 tx=0x7f0974001150 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 msgr2=0x7f09841f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f096c00b550 tx=0x7f096c00ba10 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 shutdown_connections 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09841f2d60 0x7f09841f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09840c6480 0x7f09841f2820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f096006ac00 0x7f096006d110 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 --2- 172.21.15.55:0/3980773180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0968005230 0x7f098413b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.752+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 >> 172.21.15.55:0/3980773180 conn(0x7f098406c730 msgr2=0x7f098415d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:01.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.753+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 shutdown_connections 2024-05-10T03:31:01.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:01.753+0000 7f0988a00640 1 -- 172.21.15.55:0/3980773180 wait complete. 2024-05-10T03:31:01.785 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:31:02.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.071+0000 7fb0b3800640 1 Processor -- start 2024-05-10T03:31:02.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.071+0000 7fb0b3800640 1 -- start start 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.073+0000 7fb0b3800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac0712f0 0x7fb0ac15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.073+0000 7fb0b3800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac0cd600 con 0x7fb0ac06dd30 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.073+0000 7fb0b3800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac0cc9f0 con 0x7fb0ac145020 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.073+0000 7fb0b3800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac145430 con 0x7fb0ac0712f0 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.074+0000 7fb0abe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb0ac06dd30 0x7fb0ac070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36934/0 (socket says 172.21.15.55:36934) 2024-05-10T03:31:02.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.074+0000 7fb0abe00640 1 -- 172.21.15.55:0/2916985919 learned_addr learned my addr 172.21.15.55:0/2916985919 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:02.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.074+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367388096 0 0) 0x7fb0ac0cd600 con 0x7fb0ac06dd30 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.074+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb090003520 con 0x7fb0ac06dd30 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.074+0000 7fb0b1800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac0712f0 0x7fb0ac15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3444742282 0 0) 0x7fb090003520 con 0x7fb0ac06dd30 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac0712f0 msgr2=0x7fb0ac15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac0712f0 0x7fb0ac15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 >> v1:172.21.15.162:6789/0 conn(0x7fb0ac145020 legacy=0x7fb0ac0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0cccd0 con 0x7fb0ac06dd30 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb094001670 con 0x7fb0ac06dd30 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 >> v1:172.21.15.55:6789/0 conn(0x7fb0ac06dd30 legacy=0x7fb0ac070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0900039a0 0x7fb090003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0900047c0 0x7fb090004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090005110 0x7fb09000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac145430 con 0x7fb090005110 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09000bec0 con 0x7fb0900039a0 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09000c000 con 0x7fb0900047c0 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.075+0000 7fb0b1800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090005110 0x7fb09000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0abe00640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0900047c0 0x7fb090004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b0e00640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0900039a0 0x7fb090003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b1800640 1 -- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0900047c0 msgr2=0x7fb090004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b1800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0900047c0 0x7fb090004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b1800640 1 -- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0900039a0 msgr2=0x7fb090003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b1800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0900039a0 0x7fb090003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.076+0000 7fb0b1800640 1 -- 172.21.15.55:0/2916985919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090003520 con 0x7fb090005110 2024-05-10T03:31:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.077+0000 7fb0b1800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090005110 0x7fb09000b980 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac070480 tx=0x7fb0900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a410a380f07cfbc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.077+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4031400 con 0x7fb090005110 2024-05-10T03:31:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.077+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a402d050 con 0x7fb090005110 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.077+0000 7fb0ab400640 1 -- 172.21.15.55:0/2916985919 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4031780 con 0x7fb090005110 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 -- 172.21.15.55:0/2916985919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090005110 msgr2=0x7fb09000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 --2- 172.21.15.55:0/2916985919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090005110 0x7fb09000b980 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac070480 tx=0x7fb0900057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b1800640 1 -- 172.21.15.55:0/2916985919 reap_dead start 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 -- 172.21.15.55:0/2916985919 shutdown_connections 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 -- 172.21.15.55:0/2916985919 >> 172.21.15.55:0/2916985919 conn(0x7fb0ac0b1ee0 msgr2=0x7fb0ac0704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 -- 172.21.15.55:0/2916985919 shutdown_connections 2024-05-10T03:31:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.078+0000 7fb0b3800640 1 -- 172.21.15.55:0/2916985919 wait complete. 2024-05-10T03:31:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.081+0000 7fb0b3800640 1 Processor -- start 2024-05-10T03:31:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.081+0000 7fb0b3800640 1 -- start start 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac1e3310 0x7fb0ac070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 0x7fb0ac0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac15afb0 con 0x7fb0ac15aba0 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac1f7a20 con 0x7fb0ac070860 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.082+0000 7fb0b3800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac13c030 con 0x7fb0ac1e3310 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33744/0 (socket says 172.21.15.55:33744) 2024-05-10T03:31:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 -- 172.21.15.55:0/1170216109 learned_addr learned my addr 172.21.15.55:0/1170216109 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b1800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 0x7fb0ac0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 -- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac1e3310 msgr2=0x7fb0ac070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac1e3310 0x7fb0ac070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 -- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 msgr2=0x7fb0ac0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 0x7fb0ac0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b0e00640 1 -- 172.21.15.55:0/1170216109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac070c70 con 0x7fb0ac15aba0 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.083+0000 7fb0b1800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 0x7fb0ac0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0b0e00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00eb40 tx=0x7fb09c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c003b80 con 0x7fb0ac15aba0 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c003ce0 con 0x7fb0ac15aba0 2024-05-10T03:31:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00ca20 con 0x7fb0ac15aba0 2024-05-10T03:31:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac1f7180 con 0x7fb0ac15aba0 2024-05-10T03:31:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.084+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac1f7740 con 0x7fb0ac15aba0 2024-05-10T03:31:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.085+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac13be00 con 0x7fb0ac15aba0 2024-05-10T03:31:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.086+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb09c00cb80 con 0x7fb0ac15aba0 2024-05-10T03:31:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.088+0000 7fb0a8c00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 0x7fb08806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.088+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fb09c089a10 con 0x7fb0ac15aba0 2024-05-10T03:31:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.088+0000 7fb0abe00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 0x7fb08806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.090+0000 7fb0abe00640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 0x7fb08806d110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb094000f60 tx=0x7fb094000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.093+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb0ac13be00 con 0x7fb0ac15aba0 2024-05-10T03:31:02.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.294+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb0ac13be00 con 0x7fb0ac15aba0 2024-05-10T03:31:02.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.295+0000 7fb0a8c00640 1 -- 172.21.15.55:0/1170216109 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb0ac13be00 con 0x7fb0ac15aba0 2024-05-10T03:31:02.300 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:02.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 msgr2=0x7fb08806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 0x7fb08806d110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb094000f60 tx=0x7fb094000ed0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 msgr2=0x7fb0ac1f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00eb40 tx=0x7fb09c0020f0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 shutdown_connections 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb08806ac00 0x7fb08806d110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0ac070860 0x7fb0ac0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0ac1e3310 0x7fb0ac070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 --2- 172.21.15.55:0/1170216109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0ac15aba0 0x7fb0ac1f6a80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 >> 172.21.15.55:0/1170216109 conn(0x7fb0ac0b1ee0 msgr2=0x7fb0ac0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 shutdown_connections 2024-05-10T03:31:02.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.298+0000 7fb0b3800640 1 -- 172.21.15.55:0/1170216109 wait complete. 2024-05-10T03:31:02.332 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:31:02.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.611+0000 7f5f1f200640 1 Processor -- start 2024-05-10T03:31:02.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.611+0000 7f5f1f200640 1 -- start start 2024-05-10T03:31:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f200d1ba0 0x7f5f200b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2015d700 0x7f5f2015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200b7b50 con 0x7f5f200cdf70 2024-05-10T03:31:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2015e050 con 0x7f5f200d1ba0 2024-05-10T03:31:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200b8ff0 con 0x7f5f2015d700 2024-05-10T03:31:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f200d1ba0 0x7f5f200b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f200d1ba0 0x7f5f200b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33758/0 (socket says 172.21.15.55:33758) 2024-05-10T03:31:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.613+0000 7f5f1de00640 1 -- 172.21.15.55:0/3622397491 learned_addr learned my addr 172.21.15.55:0/3622397491 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.614+0000 7f5f1d400640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2015d700 0x7f5f2015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.614+0000 7f5f1de00640 1 -- 172.21.15.55:0/3622397491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2015d700 msgr2=0x7f5f2015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.614+0000 7f5f1de00640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2015d700 0x7f5f2015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.614+0000 7f5f1de00640 1 -- 172.21.15.55:0/3622397491 >> v1:172.21.15.162:6789/0 conn(0x7f5f200cdf70 legacy=0x7f5f200ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.614+0000 7f5f1de00640 1 -- 172.21.15.55:0/3622397491 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200b6f40 con 0x7f5f200d1ba0 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.615+0000 7f5f1de00640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f200d1ba0 0x7f5f200b65f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22532bea7b8457cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.615+0000 7f5f1ca00640 1 -- 172.21.15.55:0/3622397491 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10032440 con 0x7f5f200d1ba0 2024-05-10T03:31:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.615+0000 7f5f1ca00640 1 -- 172.21.15.55:0/3622397491 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f100325a0 con 0x7f5f200d1ba0 2024-05-10T03:31:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.615+0000 7f5f1ca00640 1 -- 172.21.15.55:0/3622397491 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10032930 con 0x7f5f200d1ba0 2024-05-10T03:31:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 -- 172.21.15.55:0/3622397491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f200d1ba0 msgr2=0x7f5f200b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f200d1ba0 0x7f5f200b65f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a210 comp rx=0 tx=0).stop 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 -- 172.21.15.55:0/3622397491 shutdown_connections 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f2015d700 0x7f5f2015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 --2- 172.21.15.55:0/3622397491 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f200d1ba0 0x7f5f200b65f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 -- 172.21.15.55:0/3622397491 >> 172.21.15.55:0/3622397491 conn(0x7f5f200d0620 msgr2=0x7f5f200d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 -- 172.21.15.55:0/3622397491 shutdown_connections 2024-05-10T03:31:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.616+0000 7f5f1f200640 1 -- 172.21.15.55:0/3622397491 wait complete. 2024-05-10T03:31:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.619+0000 7f5f1f200640 1 Processor -- start 2024-05-10T03:31:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.619+0000 7f5f1f200640 1 -- start start 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f200cdf70 0x7f5f2014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f200d1ba0 0x7f5f2015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200c8af0 con 0x7f5f2015d700 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2014c490 con 0x7f5f200cdf70 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.620+0000 7f5f1f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2015e8c0 con 0x7f5f200d1ba0 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33772/0 (socket says 172.21.15.55:33772) 2024-05-10T03:31:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 -- 172.21.15.55:0/2086184037 learned_addr learned my addr 172.21.15.55:0/2086184037 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:02.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1d400640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f200d1ba0 0x7f5f2015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 -- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f200d1ba0 msgr2=0x7f5f2015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f200d1ba0 0x7f5f2015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 -- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f200cdf70 msgr2=0x7f5f2014b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f200cdf70 0x7f5f2014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 -- 172.21.15.55:0/2086184037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100094d0 con 0x7f5f2015d700 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.621+0000 7f5f1e800640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b9e0 tx=0x7f5f1800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.622+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003b70 con 0x7f5f2015d700 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.622+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18003cd0 con 0x7f5f2015d700 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.622+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800cb60 con 0x7f5f2015d700 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.622+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200c7e70 con 0x7f5f2015d700 2024-05-10T03:31:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.622+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200c8430 con 0x7f5f2015d700 2024-05-10T03:31:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.623+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f200c8c30 con 0x7f5f2015d700 2024-05-10T03:31:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.624+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f1800ccc0 con 0x7f5f2015d700 2024-05-10T03:31:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.625+0000 7f5f16000640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 0x7f5f0006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.625+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f5f180894b0 con 0x7f5f2015d700 2024-05-10T03:31:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.626+0000 7f5f1de00640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 0x7f5f0006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:02.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.627+0000 7f5f1de00640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 0x7f5f0006d110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:02.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.630+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f200c8c30 con 0x7f5f2015d700 2024-05-10T03:31:02.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.840+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f200c8c30 con 0x7f5f2015d700 2024-05-10T03:31:02.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.842+0000 7f5f16000640 1 -- 172.21.15.55:0/2086184037 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f200c8c30 con 0x7f5f2015d700 2024-05-10T03:31:02.846 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 msgr2=0x7f5f0006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 0x7f5f0006d110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002c000 comp rx=0 tx=0).stop 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 msgr2=0x7f5f2006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b9e0 tx=0x7f5f1800bea0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 shutdown_connections 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f2015d700 0x7f5f2006a5f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f200d1ba0 0x7f5f2015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f0006ac00 0x7f5f0006d110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 --2- 172.21.15.55:0/2086184037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f200cdf70 0x7f5f2014b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 >> 172.21.15.55:0/2086184037 conn(0x7f5f200d0620 msgr2=0x7f5f20066190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.844+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 shutdown_connections 2024-05-10T03:31:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:02.845+0000 7f5f1f200640 1 -- 172.21.15.55:0/2086184037 wait complete. 2024-05-10T03:31:02.876 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-10T03:31:02.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:02.877 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-10T03:31:02.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:31:02.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:02.965 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-10T03:31:03.016 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:03.016 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:03.074 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:03.075 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:03.130 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:03.131 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:03.185 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:03.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.475+0000 7ff7e0400640 1 Processor -- start 2024-05-10T03:31:03.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.475+0000 7ff7e0400640 1 -- start start 2024-05-10T03:31:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.477+0000 7ff7e0400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 0x7ff7d8161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.478+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 0x7ff7d81541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.478+0000 7ff7e0400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d81623a0 con 0x7ff7d8161c70 2024-05-10T03:31:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.478+0000 7ff7e0400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80cdd60 con 0x7ff7d80ad7b0 2024-05-10T03:31:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.478+0000 7ff7e0400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d80ce040 con 0x7ff7d80720f0 2024-05-10T03:31:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.479+0000 7ff7de400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 0x7ff7d81541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.479+0000 7ff7dd000640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff7d8161c70 0x7ff7d80719e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:56712/0 (socket says 172.21.15.55:56712) 2024-05-10T03:31:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.479+0000 7ff7dd000640 1 -- 172.21.15.55:0/1080661032 learned_addr learned my addr 172.21.15.55:0/1080661032 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.479+0000 7ff7dda00640 1 --2- 172.21.15.55:0/1080661032 >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 0x7ff7d8161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.480+0000 7ff7de400640 1 -- 172.21.15.55:0/1080661032 >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 msgr2=0x7ff7d8161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.480+0000 7ff7de400640 1 --2- 172.21.15.55:0/1080661032 >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 0x7ff7d8161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.480+0000 7ff7de400640 1 -- 172.21.15.55:0/1080661032 >> v1:172.21.15.162:6789/0 conn(0x7ff7d8161c70 legacy=0x7ff7d80719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.480+0000 7ff7de400640 1 -- 172.21.15.55:0/1080661032 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d80ce320 con 0x7ff7d80720f0 2024-05-10T03:31:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.480+0000 7ff7dda00640 1 --2- 172.21.15.55:0/1080661032 >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 0x7ff7d8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.481+0000 7ff7de400640 1 --2- 172.21.15.55:0/1080661032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 0x7ff7d81541e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009dc0 tx=0x7ff7d002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5d3f462a5725b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:03.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.482+0000 7ff7d7e00640 1 -- 172.21.15.55:0/1080661032 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0007740 con 0x7ff7d80720f0 2024-05-10T03:31:03.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.482+0000 7ff7d7e00640 1 -- 172.21.15.55:0/1080661032 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d002d050 con 0x7ff7d80720f0 2024-05-10T03:31:03.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.482+0000 7ff7d7e00640 1 -- 172.21.15.55:0/1080661032 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0007ac0 con 0x7ff7d80720f0 2024-05-10T03:31:03.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 -- 172.21.15.55:0/1080661032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 msgr2=0x7ff7d81541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 --2- 172.21.15.55:0/1080661032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 0x7ff7d81541e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009dc0 tx=0x7ff7d002a5f0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 -- 172.21.15.55:0/1080661032 shutdown_connections 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 --2- 172.21.15.55:0/1080661032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80720f0 0x7ff7d81541e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 --2- 172.21.15.55:0/1080661032 >> v2:172.21.15.162:3300/0 conn(0x7ff7d80ad7b0 0x7ff7d8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 -- 172.21.15.55:0/1080661032 >> 172.21.15.55:0/1080661032 conn(0x7ff7d80cbb90 msgr2=0x7ff7d80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 -- 172.21.15.55:0/1080661032 shutdown_connections 2024-05-10T03:31:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.483+0000 7ff7e0400640 1 -- 172.21.15.55:0/1080661032 wait complete. 2024-05-10T03:31:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.486+0000 7ff7e0400640 1 Processor -- start 2024-05-10T03:31:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.487+0000 7ff7e0400640 1 -- start start 2024-05-10T03:31:03.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d80720f0 0x7ff7d8149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80ad7b0 0x7ff7d815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8159960 con 0x7ff7d8161c70 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d814a180 con 0x7ff7d80720f0 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7e0400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d815d1a0 con 0x7ff7d80ad7b0 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7de400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7de400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59394/0 (socket says 172.21.15.55:59394) 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7de400640 1 -- 172.21.15.55:0/4249311497 learned_addr learned my addr 172.21.15.55:0/4249311497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7dda00640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d80720f0 0x7ff7d8149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.489+0000 7ff7dd000640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80ad7b0 0x7ff7d815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 -- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80ad7b0 msgr2=0x7ff7d815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80ad7b0 0x7ff7d815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 -- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d80720f0 msgr2=0x7ff7d8149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d80720f0 0x7ff7d8149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 -- 172.21.15.55:0/4249311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009400 con 0x7ff7d8161c70 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7de400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009ee0 tx=0x7ff7d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.490+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d003b820 con 0x7ff7d8161c70 2024-05-10T03:31:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.491+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d003b980 con 0x7ff7d8161c70 2024-05-10T03:31:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.491+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d003bcf0 con 0x7ff7d8161c70 2024-05-10T03:31:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.491+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d0009a70 con 0x7ff7d8161c70 2024-05-10T03:31:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.491+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d8158670 con 0x7ff7d8161c70 2024-05-10T03:31:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.492+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d8153fd0 con 0x7ff7d8161c70 2024-05-10T03:31:03.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.493+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff7d0031400 con 0x7ff7d8161c70 2024-05-10T03:31:03.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.494+0000 7ff7d5600640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 0x7ff7b806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:03.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.494+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff7d00aaff0 con 0x7ff7d8161c70 2024-05-10T03:31:03.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.495+0000 7ff7dda00640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 0x7ff7b806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:03.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.496+0000 7ff7dda00640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 0x7ff7b806d140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009bc0 tx=0x7ff7cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:03.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.500+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff7d00de900 con 0x7ff7d8161c70 2024-05-10T03:31:03.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.705+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7d8066f60 con 0x7ff7d8161c70 2024-05-10T03:31:03.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.707+0000 7ff7d5600640 1 -- 172.21.15.55:0/4249311497 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7ff7d0049090 con 0x7ff7d8161c70 2024-05-10T03:31:03.712 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:03.712 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.710+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 msgr2=0x7ff7b806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.710+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 0x7ff7b806d140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009bc0 tx=0x7ff7cc008040 comp rx=0 tx=0).stop 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.710+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 msgr2=0x7ff7d8155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.710+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009ee0 tx=0x7ff7d0004a40 comp rx=0 tx=0).stop 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 shutdown_connections 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d8161c70 0x7ff7d8155450 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7d80ad7b0 0x7ff7d815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff7b806ac30 0x7ff7b806d140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 --2- 172.21.15.55:0/4249311497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7d80720f0 0x7ff7d8149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:03.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 >> 172.21.15.55:0/4249311497 conn(0x7ff7d80cbb90 msgr2=0x7ff7d80b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:03.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 shutdown_connections 2024-05-10T03:31:03.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:03.711+0000 7ff7e0400640 1 -- 172.21.15.55:0/4249311497 wait complete. 2024-05-10T03:31:03.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:31:03.744 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.025+0000 7efdce000640 1 Processor -- start 2024-05-10T03:31:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.025+0000 7efdce000640 1 -- start start 2024-05-10T03:31:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.028+0000 7efdce000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.028+0000 7efdce000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efdc80b1bc0 0x7efdc80cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.028+0000 7efdce000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80ceb00 con 0x7efdc80ce090 2024-05-10T03:31:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.028+0000 7efdce000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc815dcd0 con 0x7efdc80b1bc0 2024-05-10T03:31:04.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.028+0000 7efdce000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80d2150 con 0x7efdc80b2110 2024-05-10T03:31:04.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.029+0000 7efdccc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.029+0000 7efdccc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47950/0 (socket says 172.21.15.55:47950) 2024-05-10T03:31:04.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.029+0000 7efdccc00640 1 -- 172.21.15.55:0/3010773953 learned_addr learned my addr 172.21.15.55:0/3010773953 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:04.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.029+0000 7efdcd600640 1 --1- 172.21.15.55:0/3010773953 >> v1:172.21.15.162:6789/0 conn(0x7efdc80ce090 0x7efdc8161310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:56714/0 (socket says 172.21.15.55:56714) 2024-05-10T03:31:04.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.029+0000 7efdc7e00640 1 --2- 172.21.15.55:0/3010773953 >> v2:172.21.15.162:3300/0 conn(0x7efdc80b1bc0 0x7efdc80cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.030+0000 7efdc7400640 1 -- 172.21.15.55:0/3010773953 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742390622 0 0) 0x7efdc80ceb00 con 0x7efdc80ce090 2024-05-10T03:31:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.030+0000 7efdccc00640 1 -- 172.21.15.55:0/3010773953 >> v2:172.21.15.162:3300/0 conn(0x7efdc80b1bc0 msgr2=0x7efdc80cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.030+0000 7efdccc00640 1 --2- 172.21.15.55:0/3010773953 >> v2:172.21.15.162:3300/0 conn(0x7efdc80b1bc0 0x7efdc80cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.030+0000 7efdccc00640 1 -- 172.21.15.55:0/3010773953 >> v1:172.21.15.162:6789/0 conn(0x7efdc80ce090 legacy=0x7efdc8161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.030+0000 7efdccc00640 1 -- 172.21.15.55:0/3010773953 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80b2910 con 0x7efdc80b2110 2024-05-10T03:31:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.031+0000 7efdccc00640 1 --2- 172.21.15.55:0/3010773953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3922f13465885495 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.032+0000 7efdc7400640 1 -- 172.21.15.55:0/3010773953 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8032440 con 0x7efdc80b2110 2024-05-10T03:31:04.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.032+0000 7efdc7400640 1 -- 172.21.15.55:0/3010773953 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb80325a0 con 0x7efdc80b2110 2024-05-10T03:31:04.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.032+0000 7efdc7400640 1 -- 172.21.15.55:0/3010773953 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8032930 con 0x7efdc80b2110 2024-05-10T03:31:04.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 -- 172.21.15.55:0/3010773953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 msgr2=0x7efdc815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 --2- 172.21.15.55:0/3010773953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802a1c0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 -- 172.21.15.55:0/3010773953 shutdown_connections 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 --2- 172.21.15.55:0/3010773953 >> v2:172.21.15.162:3300/0 conn(0x7efdc80b1bc0 0x7efdc80cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 --2- 172.21.15.55:0/3010773953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b2110 0x7efdc815d790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 -- 172.21.15.55:0/3010773953 >> 172.21.15.55:0/3010773953 conn(0x7efdc80d30d0 msgr2=0x7efdc80d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 -- 172.21.15.55:0/3010773953 shutdown_connections 2024-05-10T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.033+0000 7efdce000640 1 -- 172.21.15.55:0/3010773953 wait complete. 2024-05-10T03:31:04.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.036+0000 7efdce000640 1 Processor -- start 2024-05-10T03:31:04.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.036+0000 7efdce000640 1 -- start start 2024-05-10T03:31:04.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b1bc0 0x7efdc80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdc80b2110 0x7efdc8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80cb800 con 0x7efdc80ce090 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80c7a80 con 0x7efdc80b2110 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.037+0000 7efdce000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc814f070 con 0x7efdc80b1bc0 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59404/0 (socket says 172.21.15.55:59404) 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 -- 172.21.15.55:0/1028427762 learned_addr learned my addr 172.21.15.55:0/1028427762 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdc7e00640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdc80b2110 0x7efdc8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdccc00640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b1bc0 0x7efdc80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 -- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b1bc0 msgr2=0x7efdc80c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b1bc0 0x7efdc80c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 -- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdc80b2110 msgr2=0x7efdc8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdc80b2110 0x7efdc8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.038+0000 7efdcd600640 1 -- 172.21.15.55:0/1028427762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80094d0 con 0x7efdc80ce090 2024-05-10T03:31:04.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdcd600640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c440 tx=0x7efdc000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc00074e0 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc0007660 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc80c7cd0 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0004a20 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.039+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc80c8290 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.041+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc804e290 con 0x7efdc80ce090 2024-05-10T03:31:04.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.042+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efdc0004b80 con 0x7efdc80ce090 2024-05-10T03:31:04.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.042+0000 7efdc4c00640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 0x7efda406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.043+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efdc0089640 con 0x7efdc80ce090 2024-05-10T03:31:04.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.043+0000 7efdccc00640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 0x7efda406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.044+0000 7efdccc00640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 0x7efda406d140 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdb8034000 tx=0x7efdb802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.047+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efdc00543b0 con 0x7efdc80ce090 2024-05-10T03:31:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.248+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdc80cd490 con 0x7efdc80ce090 2024-05-10T03:31:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.249+0000 7efdc4c00640 1 -- 172.21.15.55:0/1028427762 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdc00579d0 con 0x7efdc80ce090 2024-05-10T03:31:04.253 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 msgr2=0x7efda406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 0x7efda406d140 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdb8034000 tx=0x7efdb802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 msgr2=0x7efdc815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efdc000c440 tx=0x7efdc000c900 comp rx=0 tx=0).stop 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 shutdown_connections 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdc80ce090 0x7efdc815f290 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efda406ac30 0x7efda406d140 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdc80b2110 0x7efdc8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 --2- 172.21.15.55:0/1028427762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdc80b1bc0 0x7efdc80c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 >> 172.21.15.55:0/1028427762 conn(0x7efdc80d30d0 msgr2=0x7efdc8161070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 shutdown_connections 2024-05-10T03:31:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.251+0000 7efdce000640 1 -- 172.21.15.55:0/1028427762 wait complete. 2024-05-10T03:31:04.286 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.569+0000 7fab94e00640 1 Processor -- start 2024-05-10T03:31:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.569+0000 7fab94e00640 1 -- start start 2024-05-10T03:31:04.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.571+0000 7fab94e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab9015e920 0x7fab900cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab94e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab9006b120 0x7fab9006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab94e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab900710f0 0x7fab90071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab94e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9015a900 con 0x7fab9006b120 2024-05-10T03:31:04.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab94e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab900b2a30 con 0x7fab9015e920 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab94e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015eec0 con 0x7fab900710f0 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab8ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab9006b120 0x7fab9006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab8ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab9006b120 0x7fab9006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59408/0 (socket says 172.21.15.55:59408) 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab8ea00640 1 -- 172.21.15.55:0/2334446539 learned_addr learned my addr 172.21.15.55:0/2334446539 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab8f400640 1 --2- 172.21.15.55:0/2334446539 >> v2:172.21.15.162:3300/0 conn(0x7fab9015e920 0x7fab900cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.572+0000 7fab8fe00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab900710f0 0x7fab90071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.573+0000 7fab8ea00640 1 -- 172.21.15.55:0/2334446539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab900710f0 msgr2=0x7fab90071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.573+0000 7fab8ea00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab900710f0 0x7fab90071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.573+0000 7fab8ea00640 1 -- 172.21.15.55:0/2334446539 >> v2:172.21.15.162:3300/0 conn(0x7fab9015e920 msgr2=0x7fab900cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.573+0000 7fab8ea00640 1 --2- 172.21.15.55:0/2334446539 >> v2:172.21.15.162:3300/0 conn(0x7fab9015e920 0x7fab900cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.573+0000 7fab8ea00640 1 -- 172.21.15.55:0/2334446539 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9006ba70 con 0x7fab9006b120 2024-05-10T03:31:04.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.574+0000 7fab8ea00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9006b120 0x7fab9006b530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab90072190 tx=0x7fab7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525f906eb68d7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.574+0000 7fab8e000640 1 -- 172.21.15.55:0/2334446539 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c032440 con 0x7fab9006b120 2024-05-10T03:31:04.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.575+0000 7fab8e000640 1 -- 172.21.15.55:0/2334446539 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c0325a0 con 0x7fab9006b120 2024-05-10T03:31:04.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.575+0000 7fab8e000640 1 -- 172.21.15.55:0/2334446539 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c032930 con 0x7fab9006b120 2024-05-10T03:31:04.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.575+0000 7fab94e00640 1 -- 172.21.15.55:0/2334446539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9006b120 msgr2=0x7fab9006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.575+0000 7fab94e00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9006b120 0x7fab9006b530 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab90072190 tx=0x7fab7c02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 -- 172.21.15.55:0/2334446539 shutdown_connections 2024-05-10T03:31:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab900710f0 0x7fab90071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 --2- 172.21.15.55:0/2334446539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab9006b120 0x7fab9006b530 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 --2- 172.21.15.55:0/2334446539 >> v2:172.21.15.162:3300/0 conn(0x7fab9015e920 0x7fab900cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 -- 172.21.15.55:0/2334446539 >> 172.21.15.55:0/2334446539 conn(0x7fab9006c750 msgr2=0x7fab900b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:04.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 -- 172.21.15.55:0/2334446539 shutdown_connections 2024-05-10T03:31:04.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.576+0000 7fab94e00640 1 -- 172.21.15.55:0/2334446539 wait complete. 2024-05-10T03:31:04.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.578+0000 7fab94e00640 1 Processor -- start 2024-05-10T03:31:04.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.579+0000 7fab94e00640 1 -- start start 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.580+0000 7fab94e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab9006b120 0x7fab900c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab94e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab94e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 0x7fab90136100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab94e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900c88d0 con 0x7fab900710f0 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab94e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab90136e50 con 0x7fab9015e920 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab94e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900c8d00 con 0x7fab9006b120 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59420/0 (socket says 172.21.15.55:59420) 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 -- 172.21.15.55:0/1582619333 learned_addr learned my addr 172.21.15.55:0/1582619333 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8f400640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab9006b120 0x7fab900c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8fe00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 0x7fab90136100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 -- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab9006b120 msgr2=0x7fab900c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab9006b120 0x7fab900c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 -- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 msgr2=0x7fab90136100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 0x7fab90136100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8ea00640 1 -- 172.21.15.55:0/1582619333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0094d0 con 0x7fab900710f0 2024-05-10T03:31:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.581+0000 7fab8fe00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 0x7fab90136100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab8ea00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab7c02a220 tx=0x7fab7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c032440 con 0x7fab900710f0 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c030ba0 con 0x7fab900710f0 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab90134e10 con 0x7fab900710f0 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c03d440 con 0x7fab900710f0 2024-05-10T03:31:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.582+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab90135340 con 0x7fab900710f0 2024-05-10T03:31:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.584+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9004e290 con 0x7fab900710f0 2024-05-10T03:31:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.585+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab7c039040 con 0x7fab900710f0 2024-05-10T03:31:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.586+0000 7fab87400640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 0x7fab7006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:04.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.586+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fab7c0aa3b0 con 0x7fab900710f0 2024-05-10T03:31:04.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.587+0000 7fab8f400640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 0x7fab7006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:04.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.588+0000 7fab8f400640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 0x7fab7006d110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab80006fd0 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:04.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.591+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab7c0750f0 con 0x7fab900710f0 2024-05-10T03:31:04.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.790+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab900b2810 con 0x7fab900710f0 2024-05-10T03:31:04.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.792+0000 7fab87400640 1 -- 172.21.15.55:0/1582619333 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab7c078710 con 0x7fab900710f0 2024-05-10T03:31:04.796 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:04.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 msgr2=0x7fab7006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 0x7fab7006d110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab80006fd0 tx=0x7fab80008040 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 msgr2=0x7fab901572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab7c02a220 tx=0x7fab7c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 shutdown_connections 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fab7006ac00 0x7fab7006d110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab9015e920 0x7fab90136100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab900710f0 0x7fab901572b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 --2- 172.21.15.55:0/1582619333 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab9006b120 0x7fab900c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 >> 172.21.15.55:0/1582619333 conn(0x7fab9006c750 msgr2=0x7fab90066350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 shutdown_connections 2024-05-10T03:31:04.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:04.795+0000 7fab94e00640 1 -- 172.21.15.55:0/1582619333 wait complete. 2024-05-10T03:31:04.826 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:05.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.109+0000 7f645a800640 1 Processor -- start 2024-05-10T03:31:05.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.109+0000 7f645a800640 1 -- start start 2024-05-10T03:31:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.112+0000 7f645a800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.112+0000 7f645a800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.112+0000 7f645a800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64540708b0 con 0x7f645406a4b0 2024-05-10T03:31:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.112+0000 7f645a800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64540b2490 con 0x7f64540b3860 2024-05-10T03:31:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.112+0000 7f645a800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6454070050 con 0x7f6454071db0 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6453e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6453400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6452a00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f645406a4b0 0x7f64540ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:56728/0 (socket says 172.21.15.55:56728) 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6453400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47978/0 (socket says 172.21.15.55:47978) 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6452a00640 1 -- 172.21.15.55:0/2090747828 learned_addr learned my addr 172.21.15.55:0/2090747828 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.113+0000 7f6453e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:50110/0 (socket says 172.21.15.55:50110) 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.114+0000 7f6453400640 1 -- 172.21.15.55:0/2090747828 >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 msgr2=0x7f64540b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.114+0000 7f6453400640 1 --2- 172.21.15.55:0/2090747828 >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.114+0000 7f6453400640 1 -- 172.21.15.55:0/2090747828 >> v1:172.21.15.162:6789/0 conn(0x7f645406a4b0 legacy=0x7f64540ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.114+0000 7f6453400640 1 -- 172.21.15.55:0/2090747828 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645415b940 con 0x7f6454071db0 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.114+0000 7f6453e00640 1 --2- 172.21.15.55:0/2090747828 >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.115+0000 7f6453400640 1 --2- 172.21.15.55:0/2090747828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64440097a0 tx=0x7f644402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b0867bd9d86a17e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.116+0000 7f6452000640 1 -- 172.21.15.55:0/2090747828 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644402ae50 con 0x7f6454071db0 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.116+0000 7f6452000640 1 -- 172.21.15.55:0/2090747828 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444032440 con 0x7f6454071db0 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.116+0000 7f6452000640 1 -- 172.21.15.55:0/2090747828 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440327d0 con 0x7f6454071db0 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.116+0000 7f645a800640 1 -- 172.21.15.55:0/2090747828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 msgr2=0x7f6454158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.116+0000 7f645a800640 1 --2- 172.21.15.55:0/2090747828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64440097a0 tx=0x7f644402a100 comp rx=0 tx=0).stop 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 -- 172.21.15.55:0/2090747828 shutdown_connections 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 --2- 172.21.15.55:0/2090747828 >> v2:172.21.15.162:3300/0 conn(0x7f64540b3860 0x7f64540b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 --2- 172.21.15.55:0/2090747828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6454071db0 0x7f6454158150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 -- 172.21.15.55:0/2090747828 >> 172.21.15.55:0/2090747828 conn(0x7f64540cbb90 msgr2=0x7f64540cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 -- 172.21.15.55:0/2090747828 shutdown_connections 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.117+0000 7f645a800640 1 -- 172.21.15.55:0/2090747828 wait complete. 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.119+0000 7f645a800640 1 Processor -- start 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.119+0000 7f645a800640 1 -- start start 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 0x7f64541307b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540b3860 0x7f6454130cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541315b0 con 0x7f645406a4b0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64541e58f0 con 0x7f6454071db0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f645a800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64541e5b40 con 0x7f64540b3860 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f6453400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f6453400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59422/0 (socket says 172.21.15.55:59422) 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.121+0000 7f6453400640 1 -- 172.21.15.55:0/4274162082 learned_addr learned my addr 172.21.15.55:0/4274162082 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6452a00640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 0x7f64541307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453e00640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540b3860 0x7f6454130cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 -- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540b3860 msgr2=0x7f6454130cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540b3860 0x7f6454130cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 -- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 msgr2=0x7f64541307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 0x7f64541307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 -- 172.21.15.55:0/4274162082 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c009a70 con 0x7f645406a4b0 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6452a00640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 0x7f64541307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.122+0000 7f6453400640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f64440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.123+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644403c8d0 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.123+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644403ca50 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.123+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6444009400 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.123+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644403ce00 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.123+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64541e6100 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.124+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6454066f60 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.125+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f644403b3a0 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.126+0000 7f644b400640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 0x7f643006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.126+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f64440aa260 con 0x7f645406a4b0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.127+0000 7f6452a00640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 0x7f643006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.128+0000 7f6452a00640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 0x7f643006d110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f64540b0790 tx=0x7f643c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.131+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6444074fc0 con 0x7f645406a4b0 2024-05-10T03:31:05.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.329+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6454067220 con 0x7f645406a4b0 2024-05-10T03:31:05.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.330+0000 7f644b400640 1 -- 172.21.15.55:0/4274162082 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64440785e0 con 0x7f645406a4b0 2024-05-10T03:31:05.334 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 msgr2=0x7f643006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 0x7f643006d110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f64540b0790 tx=0x7f643c009040 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 msgr2=0x7f64540ab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f64440046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 shutdown_connections 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540b3860 0x7f6454130cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f643006ac00 0x7f643006d110 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6454071db0 0x7f64541307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 --2- 172.21.15.55:0/4274162082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f645406a4b0 0x7f64540ab380 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 >> 172.21.15.55:0/4274162082 conn(0x7f64540cbb90 msgr2=0x7f64540afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 shutdown_connections 2024-05-10T03:31:05.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.333+0000 7f645a800640 1 -- 172.21.15.55:0/4274162082 wait complete. 2024-05-10T03:31:05.363 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:31:05.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.641+0000 7f1c91400640 1 Processor -- start 2024-05-10T03:31:05.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.642+0000 7f1c91400640 1 -- start start 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c91400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c8c1612a0 0x7f1c8c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c91400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c161bf0 0x7f1c8c06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c91400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8c06aa00 con 0x7f1c8c0b2e10 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c91400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c0ac8e0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c91400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c06b9f0 con 0x7f1c8c161bf0 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c83400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c8c1612a0 0x7f1c8c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c83400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c8c1612a0 0x7f1c8c1616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59424/0 (socket says 172.21.15.55:59424) 2024-05-10T03:31:05.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.644+0000 7f1c83400640 1 -- 172.21.15.55:0/3750406125 learned_addr learned my addr 172.21.15.55:0/3750406125 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:05.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.645+0000 7f1c83400640 1 -- 172.21.15.55:0/3750406125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c161bf0 msgr2=0x7f1c8c06cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:05.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.645+0000 7f1c83400640 1 --2- 172.21.15.55:0/3750406125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c161bf0 0x7f1c8c06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.645+0000 7f1c83400640 1 -- 172.21.15.55:0/3750406125 >> v1:172.21.15.162:6789/0 conn(0x7f1c8c0b2e10 legacy=0x7f1c8c06a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:05.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.645+0000 7f1c83400640 1 -- 172.21.15.55:0/3750406125 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c06ac30 con 0x7f1c8c1612a0 2024-05-10T03:31:05.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.646+0000 7f1c83400640 1 --2- 172.21.15.55:0/3750406125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1616b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c7402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f0c22a212dd97 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c8b400640 1 -- 172.21.15.55:0/3750406125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74032440 con 0x7f1c8c1612a0 2024-05-10T03:31:05.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c8b400640 1 -- 172.21.15.55:0/3750406125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c740325a0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c8b400640 1 -- 172.21.15.55:0/3750406125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74032930 con 0x7f1c8c1612a0 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 -- 172.21.15.55:0/3750406125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 msgr2=0x7f1c8c1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 --2- 172.21.15.55:0/3750406125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1616b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c7402a250 comp rx=0 tx=0).stop 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 -- 172.21.15.55:0/3750406125 shutdown_connections 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 --2- 172.21.15.55:0/3750406125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c161bf0 0x7f1c8c06cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 --2- 172.21.15.55:0/3750406125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1616b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.647+0000 7f1c91400640 1 -- 172.21.15.55:0/3750406125 >> 172.21.15.55:0/3750406125 conn(0x7f1c8c0cf640 msgr2=0x7f1c8c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.648+0000 7f1c91400640 1 -- 172.21.15.55:0/3750406125 shutdown_connections 2024-05-10T03:31:05.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.648+0000 7f1c91400640 1 -- 172.21.15.55:0/3750406125 wait complete. 2024-05-10T03:31:05.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.650+0000 7f1c91400640 1 Processor -- start 2024-05-10T03:31:05.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.650+0000 7f1c91400640 1 -- start start 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c0b2e10 0x7f1c8c1f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c8c161bf0 0x7f1c8c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0acb80 con 0x7f1c8c1612a0 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c1f38e0 con 0x7f1c8c161bf0 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c91400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c1f5050 con 0x7f1c8c0b2e10 2024-05-10T03:31:05.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c83400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c83400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59436/0 (socket says 172.21.15.55:59436) 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c83400640 1 -- 172.21.15.55:0/3050349719 learned_addr learned my addr 172.21.15.55:0/3050349719 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c8be00640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c0b2e10 0x7f1c8c1f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.652+0000 7f1c90a00640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c8c161bf0 0x7f1c8c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 -- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c0b2e10 msgr2=0x7f1c8c1f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c0b2e10 0x7f1c8c1f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 -- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c8c161bf0 msgr2=0x7f1c8c15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c8c161bf0 0x7f1c8c15a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 -- 172.21.15.55:0/3050349719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740094d0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c83400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c74030b00 con 0x7f1c8c1612a0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c74030c60 con 0x7f1c8c1612a0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7403b400 con 0x7f1c8c1612a0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c1590d0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.653+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c159320 con 0x7f1c8c1612a0 2024-05-10T03:31:05.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.655+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c067740 con 0x7f1c8c1612a0 2024-05-10T03:31:05.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.656+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1c74039040 con 0x7f1c8c1612a0 2024-05-10T03:31:05.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.657+0000 7f1c88c00640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 0x7f1c6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:05.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.658+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f1c740aa180 con 0x7f1c8c1612a0 2024-05-10T03:31:05.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.658+0000 7f1c8be00640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 0x7f1c6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:05.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.660+0000 7f1c8be00640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 0x7f1c6806d140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000f60 tx=0x7f1c7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:05.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.662+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1c74074ec0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.864+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1c8c157860 con 0x7f1c8c1612a0 2024-05-10T03:31:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.865+0000 7f1c88c00640 1 -- 172.21.15.55:0/3050349719 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1c740784e0 con 0x7f1c8c1612a0 2024-05-10T03:31:05.869 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 msgr2=0x7f1c6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 0x7f1c6806d140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c000f60 tx=0x7f1c7c009040 comp rx=0 tx=0).stop 2024-05-10T03:31:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 msgr2=0x7f1c8c1585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009870 tx=0x7f1c740046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 shutdown_connections 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c6806ac30 0x7f1c6806d140 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c8c161bf0 0x7f1c8c15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c8c1612a0 0x7f1c8c1585f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 --2- 172.21.15.55:0/3050349719 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c8c0b2e10 0x7f1c8c1f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 >> 172.21.15.55:0/3050349719 conn(0x7f1c8c0cf640 msgr2=0x7f1c8c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 shutdown_connections 2024-05-10T03:31:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:05.868+0000 7f1c91400640 1 -- 172.21.15.55:0/3050349719 wait complete. 2024-05-10T03:31:05.900 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:31:06.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.173+0000 7ff39fe00640 1 Processor -- start 2024-05-10T03:31:06.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.173+0000 7ff39fe00640 1 -- start start 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3a015d700 0x7ff3a015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00b7b50 con 0x7ff3a00cdf70 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a015e050 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39fe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00b8ff0 con 0x7ff3a015d700 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59452/0 (socket says 172.21.15.55:59452) 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39ea00640 1 -- 172.21.15.55:0/216992076 learned_addr learned my addr 172.21.15.55:0/216992076 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff3a00cdf70 0x7ff3a00ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38694/0 (socket says 172.21.15.55:38694) 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.175+0000 7ff39e000640 1 --2- 172.21.15.55:0/216992076 >> v2:172.21.15.162:3300/0 conn(0x7ff3a015d700 0x7ff3a015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.176+0000 7ff39ea00640 1 -- 172.21.15.55:0/216992076 >> v2:172.21.15.162:3300/0 conn(0x7ff3a015d700 msgr2=0x7ff3a015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.176+0000 7ff39ea00640 1 --2- 172.21.15.55:0/216992076 >> v2:172.21.15.162:3300/0 conn(0x7ff3a015d700 0x7ff3a015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.176+0000 7ff39ea00640 1 -- 172.21.15.55:0/216992076 >> v1:172.21.15.55:6789/0 conn(0x7ff3a00cdf70 legacy=0x7ff3a00ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.176+0000 7ff39ea00640 1 -- 172.21.15.55:0/216992076 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00b6f40 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.177+0000 7ff39ea00640 1 --2- 172.21.15.55:0/216992076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cfa10 tx=0x7ff39002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1094a01dd8e5f555 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.177+0000 7ff39d600640 1 -- 172.21.15.55:0/216992076 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390032440 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.177+0000 7ff39d600640 1 -- 172.21.15.55:0/216992076 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3900325a0 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.177+0000 7ff39d600640 1 -- 172.21.15.55:0/216992076 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390032930 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 -- 172.21.15.55:0/216992076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00d1ba0 msgr2=0x7ff3a00b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 --2- 172.21.15.55:0/216992076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cfa10 tx=0x7ff39002a270 comp rx=0 tx=0).stop 2024-05-10T03:31:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 -- 172.21.15.55:0/216992076 shutdown_connections 2024-05-10T03:31:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 --2- 172.21.15.55:0/216992076 >> v2:172.21.15.162:3300/0 conn(0x7ff3a015d700 0x7ff3a015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 --2- 172.21.15.55:0/216992076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00d1ba0 0x7ff3a00b65f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 -- 172.21.15.55:0/216992076 >> 172.21.15.55:0/216992076 conn(0x7ff3a00d0620 msgr2=0x7ff3a00d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 -- 172.21.15.55:0/216992076 shutdown_connections 2024-05-10T03:31:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.178+0000 7ff39fe00640 1 -- 172.21.15.55:0/216992076 wait complete. 2024-05-10T03:31:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.180+0000 7ff39fe00640 1 Processor -- start 2024-05-10T03:31:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.180+0000 7ff39fe00640 1 -- start start 2024-05-10T03:31:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 0x7ff3a01f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a015d700 0x7ff3a0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01e3880 con 0x7ff3a00cdf70 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01f46b0 con 0x7ff3a015d700 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0156780 con 0x7ff3a00d1ba0 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59466/0 (socket says 172.21.15.55:59466) 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.182+0000 7ff39ea00640 1 -- 172.21.15.55:0/4176592381 learned_addr learned my addr 172.21.15.55:0/4176592381 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39e000640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 0x7ff3a01f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39f400640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a015d700 0x7ff3a0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 -- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 msgr2=0x7ff3a01f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 0x7ff3a01f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 -- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a015d700 msgr2=0x7ff3a0157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a015d700 0x7ff3a0157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 -- 172.21.15.55:0/4176592381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900094d0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39e000640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 0x7ff3a01f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39ea00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff39002a2b0 tx=0x7ff3900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390032440 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390030ba0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39003d440 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.183+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a01575e0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.184+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a0156cb0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.185+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a0157770 con 0x7ff3a00cdf70 2024-05-10T03:31:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.186+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff390039040 con 0x7ff3a00cdf70 2024-05-10T03:31:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.186+0000 7ff396a00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 0x7ff37c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.187+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff3900aa440 con 0x7ff3a00cdf70 2024-05-10T03:31:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.187+0000 7ff39e000640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 0x7ff37c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.188+0000 7ff39e000640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 0x7ff37c06d110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a006aba0 tx=0x7ff38800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.191+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3900750f0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.400+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff3a00d14a0 con 0x7ff3a00cdf70 2024-05-10T03:31:06.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.401+0000 7ff396a00640 1 -- 172.21.15.55:0/4176592381 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff390078710 con 0x7ff3a00cdf70 2024-05-10T03:31:06.405 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:31:06.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.403+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 msgr2=0x7ff37c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.403+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 0x7ff37c06d110 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a006aba0 tx=0x7ff38800b040 comp rx=0 tx=0).stop 2024-05-10T03:31:06.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.403+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 msgr2=0x7ff3a01e7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.403+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff39002a2b0 tx=0x7ff3900046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 shutdown_connections 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff37c06ac00 0x7ff37c06d110 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a015d700 0x7ff3a0157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a00d1ba0 0x7ff3a01f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 --2- 172.21.15.55:0/4176592381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3a00cdf70 0x7ff3a01e7450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 >> 172.21.15.55:0/4176592381 conn(0x7ff3a00d0620 msgr2=0x7ff3a006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 shutdown_connections 2024-05-10T03:31:06.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.404+0000 7ff39fe00640 1 -- 172.21.15.55:0/4176592381 wait complete. 2024-05-10T03:31:06.430 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-10T03:31:06.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:06.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-10T03:31:06.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:31:06.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:06.530 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-10T03:31:06.580 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:31:06.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.859+0000 7ffb4dc00640 1 Processor -- start 2024-05-10T03:31:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.859+0000 7ffb4dc00640 1 -- start start 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 0x7ffb48144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4dc00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb4806d7e0 0x7ffb480b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4dc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4806d350 con 0x7ffb480b2fc0 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4dc00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4806c740 con 0x7ffb4806d7e0 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4806ca20 con 0x7ffb48144710 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4d200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ffb480b2fc0 0x7ffb4815eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38702/0 (socket says 172.21.15.55:38702) 2024-05-10T03:31:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.861+0000 7ffb4d200640 1 -- 172.21.15.55:0/2899164921 learned_addr learned my addr 172.21.15.55:0/2899164921 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb47400640 1 --2- 172.21.15.55:0/2899164921 >> v2:172.21.15.162:3300/0 conn(0x7ffb4806d7e0 0x7ffb480b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb47e00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 0x7ffb48144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348085285 0 0) 0x7ffb4806d350 con 0x7ffb480b2fc0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb28003520 con 0x7ffb480b2fc0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300482241 0 0) 0x7ffb28003520 con 0x7ffb480b2fc0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 msgr2=0x7ffb48144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 0x7ffb48144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 >> v2:172.21.15.162:3300/0 conn(0x7ffb4806d7e0 msgr2=0x7ffb480b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 --2- 172.21.15.55:0/2899164921 >> v2:172.21.15.162:3300/0 conn(0x7ffb4806d7e0 0x7ffb480b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4806cd00 con 0x7ffb480b2fc0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7ffb400019f0 con 0x7ffb480b2fc0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 >> v1:172.21.15.55:6789/0 conn(0x7ffb480b2fc0 legacy=0x7ffb4815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb280039e0 0x7ffb28003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb280047c0 0x7ffb28004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 0x7ffb28005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4806c740 con 0x7ffb280047c0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4806ca20 con 0x7ffb280039e0 2024-05-10T03:31:06.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.862+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2800ba80 con 0x7ffb48144710 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb280047c0 0x7ffb28004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 -- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 msgr2=0x7ffb28005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb48144710 0x7ffb28005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 -- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb280039e0 msgr2=0x7ffb28003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb280039e0 0x7ffb28003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.863+0000 7ffb47400640 1 -- 172.21.15.55:0/2899164921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28003520 con 0x7ffb280047c0 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.864+0000 7ffb47400640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb280047c0 0x7ffb28004bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffb480ae870 tx=0x7ffb280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dc2593e6dea043 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.864+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30031400 con 0x7ffb280047c0 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.864+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3002d050 con 0x7ffb280047c0 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.864+0000 7ffb46a00640 1 -- 172.21.15.55:0/2899164921 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30031780 con 0x7ffb280047c0 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 -- 172.21.15.55:0/2899164921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb280047c0 msgr2=0x7ffb28004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/2899164921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb280047c0 0x7ffb28004bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffb480ae870 tx=0x7ffb280057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4d200640 1 -- 172.21.15.55:0/2899164921 reap_dead start 2024-05-10T03:31:06.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 -- 172.21.15.55:0/2899164921 shutdown_connections 2024-05-10T03:31:06.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 -- 172.21.15.55:0/2899164921 >> 172.21.15.55:0/2899164921 conn(0x7ffb4806a110 msgr2=0x7ffb4806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:06.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 -- 172.21.15.55:0/2899164921 shutdown_connections 2024-05-10T03:31:06.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.865+0000 7ffb4dc00640 1 -- 172.21.15.55:0/2899164921 wait complete. 2024-05-10T03:31:06.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.867+0000 7ffb4dc00640 1 Processor -- start 2024-05-10T03:31:06.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.868+0000 7ffb4dc00640 1 -- start start 2024-05-10T03:31:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb280047c0 0x7ffb4813ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 0x7ffb4815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48142d70 con 0x7ffb4815a710 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4813ee00 con 0x7ffb4813e9f0 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb4dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4815ab20 con 0x7ffb280047c0 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb47400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb47400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59492/0 (socket says 172.21.15.55:59492) 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.869+0000 7ffb47400640 1 -- 172.21.15.55:0/1540889119 learned_addr learned my addr 172.21.15.55:0/1540889119 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47e00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb280047c0 0x7ffb4813ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 -- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb280047c0 msgr2=0x7ffb4813ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb280047c0 0x7ffb4813ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 -- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 msgr2=0x7ffb4815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb4d200640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 0x7ffb4815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 0x7ffb4815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 -- 172.21.15.55:0/1540889119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4815e950 con 0x7ffb4815a710 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb4d200640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 0x7ffb4815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:06.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.870+0000 7ffb47400640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb30034040 tx=0x7ffb30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.871+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30003bb0 con 0x7ffb4815a710 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.871+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30003d10 con 0x7ffb4815a710 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.871+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30044610 con 0x7ffb4815a710 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.871+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48142fc0 con 0x7ffb4815a710 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.871+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4815cdf0 con 0x7ffb4815a710 2024-05-10T03:31:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.872+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb48066ce0 con 0x7ffb4815a710 2024-05-10T03:31:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.873+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb30038070 con 0x7ffb4815a710 2024-05-10T03:31:06.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.874+0000 7ffb3fe00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 0x7ffb2406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:06.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.874+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ffb300a9af0 con 0x7ffb4815a710 2024-05-10T03:31:06.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.875+0000 7ffb47e00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 0x7ffb2406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:06.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.876+0000 7ffb47e00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 0x7ffb2406d110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009160 tx=0x7ffb38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:06.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:06.879+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffb30074850 con 0x7ffb4815a710 2024-05-10T03:31:07.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.121+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb4804e290 con 0x7ffb4815a710 2024-05-10T03:31:07.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.122+0000 7ffb3fe00640 1 -- 172.21.15.55:0/1540889119 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7ffb3003f090 con 0x7ffb4815a710 2024-05-10T03:31:07.127 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:07.127 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:31:07.127 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 msgr2=0x7ffb2406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 0x7ffb2406d110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009160 tx=0x7ffb38008040 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 msgr2=0x7ffb4813e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb30034040 tx=0x7ffb30004730 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 shutdown_connections 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb2406ac00 0x7ffb2406d110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb4813e9f0 0x7ffb4815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb4815a710 0x7ffb4813e4b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 --2- 172.21.15.55:0/1540889119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb280047c0 0x7ffb4813ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.125+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 >> 172.21.15.55:0/1540889119 conn(0x7ffb4806a110 msgr2=0x7ffb4806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:07.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.126+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 shutdown_connections 2024-05-10T03:31:07.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.126+0000 7ffb4dc00640 1 -- 172.21.15.55:0/1540889119 wait complete. 2024-05-10T03:31:07.154 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.422+0000 7f73af200640 1 Processor -- start 2024-05-10T03:31:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.422+0000 7f73af200640 1 -- start start 2024-05-10T03:31:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.424+0000 7f73af200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f73b00b6680 0x7f73b015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.424+0000 7f73af200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b00d25a0 con 0x7f73b00b2170 2024-05-10T03:31:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.424+0000 7f73af200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b00b28d0 con 0x7f73b01618b0 2024-05-10T03:31:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.424+0000 7f73af200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b00d2b20 con 0x7f73b00b6680 2024-05-10T03:31:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.425+0000 7f73a7e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f73b00b2170 0x7f73b01611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38718/0 (socket says 172.21.15.55:38718) 2024-05-10T03:31:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.425+0000 7f73a7e00640 1 -- 172.21.15.55:0/1076458409 learned_addr learned my addr 172.21.15.55:0/1076458409 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:07.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.425+0000 7f73ade00640 1 --2- 172.21.15.55:0/1076458409 >> v2:172.21.15.162:3300/0 conn(0x7f73b00b6680 0x7f73b015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.425+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923947018 0 0) 0x7f73b00d25a0 con 0x7f73b00b2170 2024-05-10T03:31:07.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.425+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7394003520 con 0x7f73b00b2170 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473474629 0 0) 0x7f7394003520 con 0x7f73b00b2170 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 >> v2:172.21.15.162:3300/0 conn(0x7f73b00b6680 msgr2=0x7f73b015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 --2- 172.21.15.55:0/1076458409 >> v2:172.21.15.162:3300/0 conn(0x7f73b00b6680 0x7f73b015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 >> v1:172.21.15.162:6789/0 conn(0x7f73b01618b0 legacy=0x7f73b006aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00d2840 con 0x7f73b00b2170 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7398001670 con 0x7f73b00b2170 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 >> v1:172.21.15.55:6789/0 conn(0x7f73b00b2170 legacy=0x7f73b01611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73940039e0 0x7f7394003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f7394004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73940051e0 0x7f739400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00d2b20 con 0x7f73940047c0 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739400c170 con 0x7f73940039e0 2024-05-10T03:31:07.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739400c330 con 0x7f73940051e0 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.426+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f7394004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73ae800640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73940051e0 0x7f739400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73ade00640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73940039e0 0x7f7394003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73a7e00640 1 -- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73940051e0 msgr2=0x7f739400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73940051e0 0x7f739400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73a7e00640 1 -- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73940039e0 msgr2=0x7f7394003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73940039e0 0x7f7394003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73a7e00640 1 -- 172.21.15.55:0/1076458409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394003520 con 0x7f73940047c0 2024-05-10T03:31:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.427+0000 7f73ae800640 1 -- 172.21.15.55:0/1076458409 reap_dead start 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.428+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f7394004bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f739800c240 tx=0x7f73940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9056e3a8af9f67ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.429+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739802e070 con 0x7f73940047c0 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.429+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980058d0 con 0x7f73940047c0 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.429+0000 7f73ad400640 1 -- 172.21.15.55:0/1076458409 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398005ca0 con 0x7f73940047c0 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.429+0000 7f73af200640 1 -- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 msgr2=0x7f7394004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.429+0000 7f73af200640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f7394004bd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f739800c240 tx=0x7f73940057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.430+0000 7f73af200640 1 -- 172.21.15.55:0/1076458409 shutdown_connections 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.430+0000 7f73af200640 1 --2- 172.21.15.55:0/1076458409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f7394004bd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.430+0000 7f73af200640 1 -- 172.21.15.55:0/1076458409 >> 172.21.15.55:0/1076458409 conn(0x7f73b00b5730 msgr2=0x7f73b00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.430+0000 7f73af200640 1 -- 172.21.15.55:0/1076458409 shutdown_connections 2024-05-10T03:31:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.430+0000 7f73af200640 1 -- 172.21.15.55:0/1076458409 wait complete. 2024-05-10T03:31:07.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.432+0000 7f73af200640 1 Processor -- start 2024-05-10T03:31:07.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.432+0000 7f73af200640 1 -- start start 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73b015dc60 0x7f73b015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73b015e070 0x7f73b0160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b015db20 con 0x7f73940047c0 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0158740 con 0x7f73b015dc60 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73af200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00cb7c0 con 0x7f73b015e070 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59512/0 (socket says 172.21.15.55:59512) 2024-05-10T03:31:07.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 -- 172.21.15.55:0/1492891751 learned_addr learned my addr 172.21.15.55:0/1492891751 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 -- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73b015e070 msgr2=0x7f73b0160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ae800640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73b015e070 0x7f73b0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73b015dc60 0x7f73b015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73b015e070 0x7f73b0160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 -- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73b015dc60 msgr2=0x7f73b015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73b015dc60 0x7f73b015fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.434+0000 7f73ade00640 1 -- 172.21.15.55:0/1492891751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b015e620 con 0x7f73940047c0 2024-05-10T03:31:07.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73ade00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73a0011510 tx=0x7f73a00119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0015870 con 0x7f73940047c0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a00159d0 con 0x7f73940047c0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b015f480 con 0x7f73940047c0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0015d40 con 0x7f73940047c0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.435+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b015eb50 con 0x7f73940047c0 2024-05-10T03:31:07.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.436+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0067740 con 0x7f73940047c0 2024-05-10T03:31:07.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.438+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f73a0013400 con 0x7f73940047c0 2024-05-10T03:31:07.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.439+0000 7f73a6000640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 0x7f738c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.439+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f73a0089fe0 con 0x7f73940047c0 2024-05-10T03:31:07.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.440+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 0x7f738c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.441+0000 7f73a7e00640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 0x7f738c06d140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7398033000 tx=0x7f739802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.445+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f73a0054d40 con 0x7f73940047c0 2024-05-10T03:31:07.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.645+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73b004e290 con 0x7f73940047c0 2024-05-10T03:31:07.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.648+0000 7f73a6000640 1 -- 172.21.15.55:0/1492891751 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f73a0058360 con 0x7f73940047c0 2024-05-10T03:31:07.653 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:07.653 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 msgr2=0x7f738c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 0x7f738c06d140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7398033000 tx=0x7f739802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 msgr2=0x7f73b0157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73a0011510 tx=0x7f73a00119d0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 shutdown_connections 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73b015e070 0x7f73b0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f738c06ac30 0x7f738c06d140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73b015dc60 0x7f73b015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 --2- 172.21.15.55:0/1492891751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73940047c0 0x7f73b0157ad0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 >> 172.21.15.55:0/1492891751 conn(0x7f73b00b5730 msgr2=0x7f73b00d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.651+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 shutdown_connections 2024-05-10T03:31:07.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.652+0000 7f73af200640 1 -- 172.21.15.55:0/1492891751 wait complete. 2024-05-10T03:31:07.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:29:41.917967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:29:43.315048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T03:29:46.815363+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T03:31:07.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:07.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.967+0000 7f9c91000640 1 Processor -- start 2024-05-10T03:31:07.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.968+0000 7f9c91000640 1 -- start start 2024-05-10T03:31:07.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.969+0000 7f9c91000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c8c1546f0 0x7f9c8c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.969+0000 7f9c91000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9c8c06b660 0x7f9c8c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.969+0000 7f9c91000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8c072270 con 0x7f9c8c161e70 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c91000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c161a50 con 0x7f9c8c06b660 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c91000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8c06ba70 con 0x7f9c8c1546f0 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9c8c06b660 0x7f9c8c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8b400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9c8c161e70 0x7f9c8c0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38732/0 (socket says 172.21.15.55:38732) 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8b400640 1 -- 172.21.15.55:0/1639193899 learned_addr learned my addr 172.21.15.55:0/1639193899 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8be00640 1 -- 172.21.15.55:0/1639193899 >> v2:172.21.15.162:3300/0 conn(0x7f9c8c1546f0 msgr2=0x7f9c8c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8be00640 1 --2- 172.21.15.55:0/1639193899 >> v2:172.21.15.162:3300/0 conn(0x7f9c8c1546f0 0x7f9c8c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8be00640 1 -- 172.21.15.55:0/1639193899 >> v1:172.21.15.55:6789/0 conn(0x7f9c8c161e70 legacy=0x7f9c8c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8aa00640 1 -- 172.21.15.55:0/1639193899 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276592967 0 0) 0x7f9c8c072270 con 0x7f9c8c161e70 2024-05-10T03:31:07.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.970+0000 7f9c8be00640 1 -- 172.21.15.55:0/1639193899 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c071f90 con 0x7f9c8c06b660 2024-05-10T03:31:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.971+0000 7f9c8be00640 1 --2- 172.21.15.55:0/1639193899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c06b660 0x7f9c8c15dc10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097a0 tx=0x7f9c8402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32d64ff65d06347 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.972+0000 7f9c8aa00640 1 -- 172.21.15.55:0/1639193899 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8402ae70 con 0x7f9c8c06b660 2024-05-10T03:31:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.972+0000 7f9c8aa00640 1 -- 172.21.15.55:0/1639193899 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84032440 con 0x7f9c8c06b660 2024-05-10T03:31:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.972+0000 7f9c8aa00640 1 -- 172.21.15.55:0/1639193899 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c840327d0 con 0x7f9c8c06b660 2024-05-10T03:31:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.972+0000 7f9c91000640 1 -- 172.21.15.55:0/1639193899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c06b660 msgr2=0x7f9c8c15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.972+0000 7f9c91000640 1 --2- 172.21.15.55:0/1639193899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c06b660 0x7f9c8c15dc10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097a0 tx=0x7f9c8402a160 comp rx=0 tx=0).stop 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 -- 172.21.15.55:0/1639193899 shutdown_connections 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 --2- 172.21.15.55:0/1639193899 >> v2:172.21.15.162:3300/0 conn(0x7f9c8c1546f0 0x7f9c8c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 --2- 172.21.15.55:0/1639193899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c06b660 0x7f9c8c15dc10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 -- 172.21.15.55:0/1639193899 >> 172.21.15.55:0/1639193899 conn(0x7f9c8c06c750 msgr2=0x7f9c8c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 -- 172.21.15.55:0/1639193899 shutdown_connections 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.973+0000 7f9c91000640 1 -- 172.21.15.55:0/1639193899 wait complete. 2024-05-10T03:31:07.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.975+0000 7f9c91000640 1 Processor -- start 2024-05-10T03:31:07.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.975+0000 7f9c91000640 1 -- start start 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 0x7f9c8c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 0x7f9c8c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1f36c0 con 0x7f9c8c1546f0 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1577e0 con 0x7f9c8c06b660 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c91000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c1f3850 con 0x7f9c8c161e70 2024-05-10T03:31:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59530/0 (socket says 172.21.15.55:59530) 2024-05-10T03:31:07.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 -- 172.21.15.55:0/3994207757 learned_addr learned my addr 172.21.15.55:0/3994207757 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:07.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c8b400640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 0x7f9c8c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c8be00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 0x7f9c8c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 -- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 msgr2=0x7f9c8c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 0x7f9c8c150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 -- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 msgr2=0x7f9c8c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 0x7f9c8c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.977+0000 7f9c82a00640 1 -- 172.21.15.55:0/3994207757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84009400 con 0x7f9c8c1546f0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c8b400640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 0x7f9c8c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c8be00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 0x7f9c8c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c82a00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b9c0 tx=0x7f9c7400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400cba0 con 0x7f9c8c1546f0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7400cd00 con 0x7f9c8c1546f0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c74014680 con 0x7f9c8c1546f0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c14cd40 con 0x7f9c8c1546f0 2024-05-10T03:31:07.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.978+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c14d270 con 0x7f9c8c1546f0 2024-05-10T03:31:07.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.979+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c14d5b0 con 0x7f9c8c1546f0 2024-05-10T03:31:07.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.981+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9c7401d3b0 con 0x7f9c8c1546f0 2024-05-10T03:31:07.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.981+0000 7f9c83e00640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 0x7f9c6806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:07.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.982+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f9c74089780 con 0x7f9c8c1546f0 2024-05-10T03:31:07.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.982+0000 7f9c8b400640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 0x7f9c6806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:07.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.984+0000 7f9c8b400640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 0x7f9c6806d110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000f60 tx=0x7f9c7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:07.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:07.988+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9c74054430 con 0x7f9c8c1546f0 2024-05-10T03:31:08.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.182+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c8c04e290 con 0x7f9c8c1546f0 2024-05-10T03:31:08.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.183+0000 7f9c83e00640 1 -- 172.21.15.55:0/3994207757 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c740190a0 con 0x7f9c8c1546f0 2024-05-10T03:31:08.187 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 msgr2=0x7f9c6806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 0x7f9c6806d110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000f60 tx=0x7f9c7c009040 comp rx=0 tx=0).stop 2024-05-10T03:31:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 msgr2=0x7f9c8c14b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b9c0 tx=0x7f9c7400be80 comp rx=0 tx=0).stop 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 shutdown_connections 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c8c161e70 0x7f9c8c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8c1546f0 0x7f9c8c14b980 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9c6806ac00 0x7f9c6806d110 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 --2- 172.21.15.55:0/3994207757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c8c06b660 0x7f9c8c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 >> 172.21.15.55:0/3994207757 conn(0x7f9c8c06c750 msgr2=0x7f9c8c071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 shutdown_connections 2024-05-10T03:31:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.186+0000 7f9c91000640 1 -- 172.21.15.55:0/3994207757 wait complete. 2024-05-10T03:31:08.218 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:08.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.476+0000 7f03f7e00640 1 Processor -- start 2024-05-10T03:31:08.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.476+0000 7f03f7e00640 1 -- start start 2024-05-10T03:31:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.478+0000 7f03f7e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 0x7f03f806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.478+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.479+0000 7f03f7e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f815fdf0 con 0x7f03f8071e00 2024-05-10T03:31:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.479+0000 7f03f7e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f815f1e0 con 0x7f03f80719f0 2024-05-10T03:31:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.479+0000 7f03f7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f815f4c0 con 0x7f03f806b840 2024-05-10T03:31:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.479+0000 7f03efe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.480+0000 7f03efe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48084/0 (socket says 172.21.15.55:48084) 2024-05-10T03:31:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.480+0000 7f03efe00640 1 -- 172.21.15.55:0/214506272 learned_addr learned my addr 172.21.15.55:0/214506272 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.480+0000 7f03f6a00640 1 --2- 172.21.15.55:0/214506272 >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 0x7f03f806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.480+0000 7f03f7400640 1 --1- 172.21.15.55:0/214506272 >> v1:172.21.15.162:6789/0 conn(0x7f03f8071e00 0x7f03f8070620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:56762/0 (socket says 172.21.15.55:56762) 2024-05-10T03:31:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.481+0000 7f03efe00640 1 -- 172.21.15.55:0/214506272 >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 msgr2=0x7f03f806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.481+0000 7f03efe00640 1 --2- 172.21.15.55:0/214506272 >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 0x7f03f806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.481+0000 7f03efe00640 1 -- 172.21.15.55:0/214506272 >> v1:172.21.15.162:6789/0 conn(0x7f03f8071e00 legacy=0x7f03f8070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.481+0000 7f03efe00640 1 -- 172.21.15.55:0/214506272 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f815f7a0 con 0x7f03f806b840 2024-05-10T03:31:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.481+0000 7f03f6a00640 1 --2- 172.21.15.55:0/214506272 >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 0x7f03f806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.482+0000 7f03efe00640 1 --2- 172.21.15.55:0/214506272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9001dc9a94a1769 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:08.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.483+0000 7f03f6000640 1 -- 172.21.15.55:0/214506272 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032440 con 0x7f03f806b840 2024-05-10T03:31:08.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.483+0000 7f03f6000640 1 -- 172.21.15.55:0/214506272 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00325a0 con 0x7f03f806b840 2024-05-10T03:31:08.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.483+0000 7f03f6000640 1 -- 172.21.15.55:0/214506272 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032930 con 0x7f03f806b840 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.483+0000 7f03f7e00640 1 -- 172.21.15.55:0/214506272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 msgr2=0x7f03f80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.483+0000 7f03f7e00640 1 --2- 172.21.15.55:0/214506272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 -- 172.21.15.55:0/214506272 shutdown_connections 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 --2- 172.21.15.55:0/214506272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f806b840 0x7f03f80700e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 --2- 172.21.15.55:0/214506272 >> v2:172.21.15.162:3300/0 conn(0x7f03f80719f0 0x7f03f806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 -- 172.21.15.55:0/214506272 >> 172.21.15.55:0/214506272 conn(0x7f03f806c750 msgr2=0x7f03f80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 -- 172.21.15.55:0/214506272 shutdown_connections 2024-05-10T03:31:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.484+0000 7f03f7e00640 1 -- 172.21.15.55:0/214506272 wait complete. 2024-05-10T03:31:08.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.486+0000 7f03f7e00640 1 Processor -- start 2024-05-10T03:31:08.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.486+0000 7f03f7e00640 1 -- start start 2024-05-10T03:31:08.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 0x7f03f80b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 0x7f03f81f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f815bac0 con 0x7f03f80719f0 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f815a540 con 0x7f03f806b840 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f815e340 con 0x7f03f8071e00 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03efe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03efe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59546/0 (socket says 172.21.15.55:59546) 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03efe00640 1 -- 172.21.15.55:0/4224030623 learned_addr learned my addr 172.21.15.55:0/4224030623 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03efe00640 1 -- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 msgr2=0x7f03f81f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.488+0000 7f03f7400640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 0x7f03f81f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03f6a00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 0x7f03f80b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03efe00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 0x7f03f81f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03efe00640 1 -- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 msgr2=0x7f03f80b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03efe00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 0x7f03f80b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03efe00640 1 -- 172.21.15.55:0/4224030623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8009400 con 0x7f03f80719f0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03f7400640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 0x7f03f81f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03f6a00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 0x7f03f80b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03efe00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032440 con 0x7f03f80719f0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0030980 con 0x7f03f80719f0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e00094d0 con 0x7f03f80719f0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0030d30 con 0x7f03f80719f0 2024-05-10T03:31:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.489+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f815a010 con 0x7f03f80719f0 2024-05-10T03:31:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.491+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f813fec0 con 0x7f03f80719f0 2024-05-10T03:31:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.491+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f03e0039040 con 0x7f03f80719f0 2024-05-10T03:31:08.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.492+0000 7f03ef400640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 0x7f03d406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:08.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.492+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f03e00aa1a0 con 0x7f03f80719f0 2024-05-10T03:31:08.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.493+0000 7f03f6a00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 0x7f03d406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:08.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.494+0000 7f03f6a00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 0x7f03d406d110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009bb0 tx=0x7f03e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:08.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.497+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03f813fec0 con 0x7f03f80719f0 2024-05-10T03:31:08.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.696+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03f813fec0 con 0x7f03f80719f0 2024-05-10T03:31:08.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.697+0000 7f03ef400640 1 -- 172.21.15.55:0/4224030623 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03f813fec0 con 0x7f03f80719f0 2024-05-10T03:31:08.702 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 msgr2=0x7f03d406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 0x7f03d406d110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009bb0 tx=0x7f03e8008040 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 msgr2=0x7f03f81f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e00046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 shutdown_connections 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03f8071e00 0x7f03f81f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03f80719f0 0x7f03f81f7590 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f03d406ac00 0x7f03d406d110 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 --2- 172.21.15.55:0/4224030623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03f806b840 0x7f03f80b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:08.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.700+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 >> 172.21.15.55:0/4224030623 conn(0x7f03f806c750 msgr2=0x7f03f81403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:08.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.701+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 shutdown_connections 2024-05-10T03:31:08.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:08.701+0000 7f03f7e00640 1 -- 172.21.15.55:0/4224030623 wait complete. 2024-05-10T03:31:08.733 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:09.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.004+0000 7f7400c00640 1 Processor -- start 2024-05-10T03:31:09.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.004+0000 7f7400c00640 1 -- start start 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f7400c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f73fc071db0 0x7f73fc158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f7400c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73fc0b3860 0x7f73fc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f7400c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc0708b0 con 0x7f73fc06a4b0 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f7400c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0b2490 con 0x7f73fc0b3860 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f7400c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc070050 con 0x7f73fc071db0 2024-05-10T03:31:09.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f73f9400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f73fc06a4b0 0x7f73fc0ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38746/0 (socket says 172.21.15.55:38746) 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f73f9400640 1 -- 172.21.15.55:0/4088856662 learned_addr learned my addr 172.21.15.55:0/4088856662 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f73fa800640 1 --2- 172.21.15.55:0/4088856662 >> v2:172.21.15.55:3300/0 conn(0x7f73fc0b3860 0x7f73fc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f73f9e00640 1 --2- 172.21.15.55:0/4088856662 >> v2:172.21.15.162:3300/0 conn(0x7f73fc071db0 0x7f73fc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.007+0000 7f73f8a00640 1 -- 172.21.15.55:0/4088856662 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119216733 0 0) 0x7f73fc0708b0 con 0x7f73fc06a4b0 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.008+0000 7f73f8a00640 1 -- 172.21.15.55:0/4088856662 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e0003520 con 0x7f73fc06a4b0 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.008+0000 7f73fa800640 1 -- 172.21.15.55:0/4088856662 >> v2:172.21.15.162:3300/0 conn(0x7f73fc071db0 msgr2=0x7f73fc158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.008+0000 7f73fa800640 1 --2- 172.21.15.55:0/4088856662 >> v2:172.21.15.162:3300/0 conn(0x7f73fc071db0 0x7f73fc158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.008+0000 7f73fa800640 1 -- 172.21.15.55:0/4088856662 >> v1:172.21.15.55:6789/0 conn(0x7f73fc06a4b0 legacy=0x7f73fc0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.008+0000 7f73fa800640 1 -- 172.21.15.55:0/4088856662 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc15b940 con 0x7f73fc0b3860 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.009+0000 7f73fa800640 1 --2- 172.21.15.55:0/4088856662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc0b3860 0x7f73fc0b3c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ceb80 tx=0x7f73f402a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343530804f327f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.009+0000 7f73f8a00640 1 -- 172.21.15.55:0/4088856662 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4007740 con 0x7f73fc0b3860 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.009+0000 7f73f8a00640 1 -- 172.21.15.55:0/4088856662 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f402d050 con 0x7f73fc0b3860 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.009+0000 7f73f8a00640 1 -- 172.21.15.55:0/4088856662 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4007ac0 con 0x7f73fc0b3860 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 -- 172.21.15.55:0/4088856662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc0b3860 msgr2=0x7f73fc0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 --2- 172.21.15.55:0/4088856662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc0b3860 0x7f73fc0b3c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ceb80 tx=0x7f73f402a550 comp rx=0 tx=0).stop 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 -- 172.21.15.55:0/4088856662 shutdown_connections 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 --2- 172.21.15.55:0/4088856662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc0b3860 0x7f73fc0b3c70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 --2- 172.21.15.55:0/4088856662 >> v2:172.21.15.162:3300/0 conn(0x7f73fc071db0 0x7f73fc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 -- 172.21.15.55:0/4088856662 >> 172.21.15.55:0/4088856662 conn(0x7f73fc0cbb90 msgr2=0x7f73fc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 -- 172.21.15.55:0/4088856662 shutdown_connections 2024-05-10T03:31:09.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.010+0000 7f7400c00640 1 -- 172.21.15.55:0/4088856662 wait complete. 2024-05-10T03:31:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.012+0000 7f7400c00640 1 Processor -- start 2024-05-10T03:31:09.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.012+0000 7f7400c00640 1 -- start start 2024-05-10T03:31:09.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73fc071db0 0x7f73fc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73fc0b3860 0x7f73fc129fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc13d820 con 0x7f73fc06a4b0 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc158db0 con 0x7f73fc0b3860 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f7400c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc15a0a0 con 0x7f73fc071db0 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59568/0 (socket says 172.21.15.55:59568) 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 -- 172.21.15.55:0/2551553175 learned_addr learned my addr 172.21.15.55:0/2551553175 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 -- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73fc071db0 msgr2=0x7f73fc156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73fc071db0 0x7f73fc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 -- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73fc0b3860 msgr2=0x7f73fc129fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73fc0b3860 0x7f73fc129fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.014+0000 7f73f9e00640 1 -- 172.21.15.55:0/2551553175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4009a70 con 0x7f73fc06a4b0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f73f9e00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00d5d0 tx=0x7f73ec00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec004240 con 0x7f73fc06a4b0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0043a0 con 0x7f73fc06a4b0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc159000 con 0x7f73fc06a4b0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec003830 con 0x7f73fc06a4b0 2024-05-10T03:31:09.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.015+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc1595c0 con 0x7f73fc06a4b0 2024-05-10T03:31:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.016+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc15b580 con 0x7f73fc06a4b0 2024-05-10T03:31:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.017+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f73ec003990 con 0x7f73fc06a4b0 2024-05-10T03:31:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.018+0000 7f73f2000640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 0x7f73dc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.018+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f73ec089a40 con 0x7f73fc06a4b0 2024-05-10T03:31:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.019+0000 7f73f9400640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 0x7f73dc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.020+0000 7f73f9400640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 0x7f73dc06d110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73fc13c5b0 tx=0x7f73e8007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.023+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f73ec0546f0 con 0x7f73fc06a4b0 2024-05-10T03:31:09.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.212+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73fc066f60 con 0x7f73fc06a4b0 2024-05-10T03:31:09.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.213+0000 7f73f2000640 1 -- 172.21.15.55:0/2551553175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73ec057d10 con 0x7f73fc06a4b0 2024-05-10T03:31:09.217 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 msgr2=0x7f73dc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 0x7f73dc06d110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73fc13c5b0 tx=0x7f73e8007930 comp rx=0 tx=0).stop 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 msgr2=0x7f73fc158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00d5d0 tx=0x7f73ec00da90 comp rx=0 tx=0).stop 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 shutdown_connections 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f73dc06ac00 0x7f73dc06d110 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73fc0b3860 0x7f73fc129fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f73fc071db0 0x7f73fc156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 --2- 172.21.15.55:0/2551553175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73fc06a4b0 0x7f73fc158720 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 >> 172.21.15.55:0/2551553175 conn(0x7f73fc0cbb90 msgr2=0x7f73fc06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.215+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 shutdown_connections 2024-05-10T03:31:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.216+0000 7f7400c00640 1 -- 172.21.15.55:0/2551553175 wait complete. 2024-05-10T03:31:09.244 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T03:31:09.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.524+0000 7f147b800640 1 Processor -- start 2024-05-10T03:31:09.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.525+0000 7f147b800640 1 -- start start 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f147b800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14740cbbd0 0x7f147406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f147b800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147415b5e0 con 0x7f14740cc3a0 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f147b800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14740b4380 con 0x7f147406b660 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f147b800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147406ba70 con 0x7f14740cbbd0 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f1473e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14740cbbd0 0x7f147406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f1478e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f14740cc3a0 0x7f1474140900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38762/0 (socket says 172.21.15.55:38762) 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.527+0000 7f1478e00640 1 -- 172.21.15.55:0/2165785873 learned_addr learned my addr 172.21.15.55:0/2165785873 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.528+0000 7f1473e00640 1 -- 172.21.15.55:0/2165785873 >> v1:172.21.15.162:6789/0 conn(0x7f147406b660 legacy=0x7f147415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.528+0000 7f1473e00640 1 -- 172.21.15.55:0/2165785873 >> v1:172.21.15.55:6789/0 conn(0x7f14740cc3a0 legacy=0x7f1474140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.528+0000 7f1473e00640 1 -- 172.21.15.55:0/2165785873 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147415acf0 con 0x7f14740cbbd0 2024-05-10T03:31:09.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.529+0000 7f1473e00640 1 --2- 172.21.15.55:0/2165785873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f147406b120 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f145c0097a0 tx=0x7f145c02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3530e132635ea9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.529+0000 7f1473400640 1 -- 172.21.15.55:0/2165785873 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c034640 con 0x7f14740cbbd0 2024-05-10T03:31:09.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.529+0000 7f1473400640 1 -- 172.21.15.55:0/2165785873 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c0347a0 con 0x7f14740cbbd0 2024-05-10T03:31:09.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.529+0000 7f1473400640 1 -- 172.21.15.55:0/2165785873 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c034b30 con 0x7f14740cbbd0 2024-05-10T03:31:09.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 -- 172.21.15.55:0/2165785873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 msgr2=0x7f147406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 --2- 172.21.15.55:0/2165785873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f147406b120 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f145c0097a0 tx=0x7f145c02c3f0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 -- 172.21.15.55:0/2165785873 shutdown_connections 2024-05-10T03:31:09.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 --2- 172.21.15.55:0/2165785873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f147406b120 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 -- 172.21.15.55:0/2165785873 >> 172.21.15.55:0/2165785873 conn(0x7f147406c120 msgr2=0x7f147406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:09.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 -- 172.21.15.55:0/2165785873 shutdown_connections 2024-05-10T03:31:09.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.530+0000 7f147b800640 1 -- 172.21.15.55:0/2165785873 wait complete. 2024-05-10T03:31:09.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.533+0000 7f147b800640 1 Processor -- start 2024-05-10T03:31:09.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.533+0000 7f147b800640 1 -- start start 2024-05-10T03:31:09.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f147406b660 0x7f14740c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14740cc3a0 0x7f1474136910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14740c8bb0 con 0x7f14740cbbd0 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14741f4560 con 0x7f147406b660 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.534+0000 7f147b800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14740ca520 con 0x7f14740cc3a0 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59592/0 (socket says 172.21.15.55:59592) 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 -- 172.21.15.55:0/3855037674 learned_addr learned my addr 172.21.15.55:0/3855037674 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1478e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f147406b660 0x7f14740c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 -- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14740cc3a0 msgr2=0x7f1474136910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14740cc3a0 0x7f1474136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 -- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f147406b660 msgr2=0x7f14740c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f147406b660 0x7f14740c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 -- 172.21.15.55:0/3855037674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c009400 con 0x7f14740cbbd0 2024-05-10T03:31:09.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.535+0000 7f1473e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f145c000c00 tx=0x7f145c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.536+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c032b80 con 0x7f14740cbbd0 2024-05-10T03:31:09.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.536+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c032ce0 con 0x7f14740cbbd0 2024-05-10T03:31:09.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.536+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c03e640 con 0x7f14740cbbd0 2024-05-10T03:31:09.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.536+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14741372d0 con 0x7f14740cbbd0 2024-05-10T03:31:09.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.536+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14740c9510 con 0x7f14740cbbd0 2024-05-10T03:31:09.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.537+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14740c8cf0 con 0x7f14740cbbd0 2024-05-10T03:31:09.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.538+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f145c03e7a0 con 0x7f14740cbbd0 2024-05-10T03:31:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.539+0000 7f1470c00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 0x7f145006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.539+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f145c0aa880 con 0x7f14740cbbd0 2024-05-10T03:31:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.540+0000 7f1478e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 0x7f145006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.541+0000 7f1478e00640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 0x7f145006d140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1464005880 tx=0x7f1464009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:09.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.544+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f14740c8cf0 con 0x7f14740cbbd0 2024-05-10T03:31:09.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.746+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f14740c8cf0 con 0x7f14740cbbd0 2024-05-10T03:31:09.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.747+0000 7f1470c00640 1 -- 172.21.15.55:0/3855037674 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f14740c8cf0 con 0x7f14740cbbd0 2024-05-10T03:31:09.752 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:09.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.750+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 msgr2=0x7f145006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.750+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 0x7f145006d140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1464005880 tx=0x7f1464009040 comp rx=0 tx=0).stop 2024-05-10T03:31:09.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.750+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 msgr2=0x7f14740caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:09.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.750+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f145c000c00 tx=0x7f145c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 shutdown_connections 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14740cc3a0 0x7f1474136910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14740cbbd0 0x7f14740caac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f145006ac30 0x7f145006d140 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 --2- 172.21.15.55:0/3855037674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f147406b660 0x7f14740c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 >> 172.21.15.55:0/3855037674 conn(0x7f147406c120 msgr2=0x7f147415d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 shutdown_connections 2024-05-10T03:31:09.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:09.751+0000 7f147b800640 1 -- 172.21.15.55:0/3855037674 wait complete. 2024-05-10T03:31:09.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:10.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.045+0000 7ff9e6200640 1 Processor -- start 2024-05-10T03:31:10.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.045+0000 7ff9e6200640 1 -- start start 2024-05-10T03:31:10.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e6200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9e006d7e0 0x7ff9e00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e6200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e006d350 con 0x7ff9e0144710 2024-05-10T03:31:10.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e6200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e00b36d0 con 0x7ff9e0144120 2024-05-10T03:31:10.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e6200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0144530 con 0x7ff9e006d7e0 2024-05-10T03:31:10.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e4e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff9e0144710 0x7ff9e00b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38778/0 (socket says 172.21.15.55:38778) 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e4e00640 1 -- 172.21.15.55:0/1023940889 learned_addr learned my addr 172.21.15.55:0/1023940889 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.048+0000 7ff9e5800640 1 --2- 172.21.15.55:0/1023940889 >> v2:172.21.15.55:3300/0 conn(0x7ff9e006d7e0 0x7ff9e00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.049+0000 7ff9df400640 1 -- 172.21.15.55:0/1023940889 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574232956 0 0) 0x7ff9e006d350 con 0x7ff9e0144710 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.049+0000 7ff9df400640 1 -- 172.21.15.55:0/1023940889 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c4003520 con 0x7ff9e0144710 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.049+0000 7ff9e5800640 1 -- 172.21.15.55:0/1023940889 >> v1:172.21.15.162:6789/0 conn(0x7ff9e0144120 legacy=0x7ff9e00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.049+0000 7ff9e5800640 1 -- 172.21.15.55:0/1023940889 >> v1:172.21.15.55:6789/0 conn(0x7ff9e0144710 legacy=0x7ff9e00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.049+0000 7ff9e5800640 1 -- 172.21.15.55:0/1023940889 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e006c740 con 0x7ff9e006d7e0 2024-05-10T03:31:10.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.050+0000 7ff9e5800640 1 --2- 172.21.15.55:0/1023940889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e006d7e0 0x7ff9e00b3190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf41574458c444f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.050+0000 7ff9df400640 1 -- 172.21.15.55:0/1023940889 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8007740 con 0x7ff9e006d7e0 2024-05-10T03:31:10.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.050+0000 7ff9df400640 1 -- 172.21.15.55:0/1023940889 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d802d050 con 0x7ff9e006d7e0 2024-05-10T03:31:10.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.050+0000 7ff9df400640 1 -- 172.21.15.55:0/1023940889 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8007ac0 con 0x7ff9e006d7e0 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 -- 172.21.15.55:0/1023940889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e006d7e0 msgr2=0x7ff9e00b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 --2- 172.21.15.55:0/1023940889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e006d7e0 0x7ff9e00b3190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d802a580 comp rx=0 tx=0).stop 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 -- 172.21.15.55:0/1023940889 shutdown_connections 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 --2- 172.21.15.55:0/1023940889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e006d7e0 0x7ff9e00b3190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 -- 172.21.15.55:0/1023940889 >> 172.21.15.55:0/1023940889 conn(0x7ff9e006a110 msgr2=0x7ff9e006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 -- 172.21.15.55:0/1023940889 shutdown_connections 2024-05-10T03:31:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.051+0000 7ff9e6200640 1 -- 172.21.15.55:0/1023940889 wait complete. 2024-05-10T03:31:10.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.054+0000 7ff9e6200640 1 Processor -- start 2024-05-10T03:31:10.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.054+0000 7ff9e6200640 1 -- start start 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9e006d7e0 0x7ff9e013b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9e0144710 0x7ff9e015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01f1740 con 0x7ff9e0144120 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01f56a0 con 0x7ff9e0144710 2024-05-10T03:31:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.055+0000 7ff9e6200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e015b170 con 0x7ff9e006d7e0 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59622/0 (socket says 172.21.15.55:59622) 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 -- 172.21.15.55:0/3405712426 learned_addr learned my addr 172.21.15.55:0/3405712426 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9e5800640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9e0144710 0x7ff9e015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9e4e00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9e006d7e0 0x7ff9e013b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 -- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9e006d7e0 msgr2=0x7ff9e013b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9e006d7e0 0x7ff9e013b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 -- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9e0144710 msgr2=0x7ff9e015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9e0144710 0x7ff9e015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.056+0000 7ff9dfe00640 1 -- 172.21.15.55:0/3405712426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009a70 con 0x7ff9e0144120 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9dfe00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b3c0 tx=0x7ff9c800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011830 con 0x7ff9e0144120 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8011990 con 0x7ff9e0144120 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8011d80 con 0x7ff9e0144120 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e01f58f0 con 0x7ff9e0144120 2024-05-10T03:31:10.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.057+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e0141270 con 0x7ff9e0144120 2024-05-10T03:31:10.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.059+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e0066ce0 con 0x7ff9e0144120 2024-05-10T03:31:10.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.060+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9c801b460 con 0x7ff9e0144120 2024-05-10T03:31:10.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.060+0000 7ff9dcc00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 0x7ff9bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.061+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff9c80892c0 con 0x7ff9e0144120 2024-05-10T03:31:10.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.061+0000 7ff9e4e00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 0x7ff9bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.062+0000 7ff9e4e00640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 0x7ff9bc06d140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005880 tx=0x7ff9d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.065+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff9c8053f70 con 0x7ff9e0144120 2024-05-10T03:31:10.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.302+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9e004e290 con 0x7ff9e0144120 2024-05-10T03:31:10.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.304+0000 7ff9dcc00640 1 -- 172.21.15.55:0/3405712426 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7ff9c8057590 con 0x7ff9e0144120 2024-05-10T03:31:10.308 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:10.308 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:31:10.309 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.306+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 msgr2=0x7ff9bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.306+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 0x7ff9bc06d140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005880 tx=0x7ff9d0009040 comp rx=0 tx=0).stop 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 msgr2=0x7ff9e01f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b3c0 tx=0x7ff9c800b880 comp rx=0 tx=0).stop 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 shutdown_connections 2024-05-10T03:31:10.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff9bc06ac30 0x7ff9bc06d140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9e0144710 0x7ff9e015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9e0144120 0x7ff9e01f4d30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 --2- 172.21.15.55:0/3405712426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff9e006d7e0 0x7ff9e013b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 >> 172.21.15.55:0/3405712426 conn(0x7ff9e006a110 msgr2=0x7ff9e00af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 shutdown_connections 2024-05-10T03:31:10.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.307+0000 7ff9e6200640 1 -- 172.21.15.55:0/3405712426 wait complete. 2024-05-10T03:31:10.338 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:10.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.614+0000 7f2209800640 1 Processor -- start 2024-05-10T03:31:10.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.614+0000 7f2209800640 1 -- start start 2024-05-10T03:31:10.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220415a5d0 0x7f22040cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22040cb730 0x7f22040b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22040b46b0 0x7f220415df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040b4ac0 con 0x7f22040b46b0 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220415ad30 con 0x7f22040cb730 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.616+0000 7f2209800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220415b010 con 0x7f220415a5d0 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22040b46b0 0x7f220415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22040b46b0 0x7f220415df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59630/0 (socket says 172.21.15.55:59630) 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 -- 172.21.15.55:0/1216764173 learned_addr learned my addr 172.21.15.55:0/1216764173 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2203400640 1 --2- 172.21.15.55:0/1216764173 >> v2:172.21.15.162:3300/0 conn(0x7f22040cb730 0x7f22040b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 -- 172.21.15.55:0/1216764173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220415a5d0 msgr2=0x7f22040cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2203e00640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220415a5d0 0x7f22040cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220415a5d0 0x7f22040cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 -- 172.21.15.55:0/1216764173 >> v2:172.21.15.162:3300/0 conn(0x7f22040cb730 msgr2=0x7f22040b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 --2- 172.21.15.55:0/1216764173 >> v2:172.21.15.162:3300/0 conn(0x7f22040cb730 0x7f22040b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.617+0000 7f2208e00640 1 -- 172.21.15.55:0/1216764173 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220415b2f0 con 0x7f22040b46b0 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.618+0000 7f2208e00640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22040b46b0 0x7f220415df40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f220406e3d0 tx=0x7f21fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4333e8e4fef29f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2202a00640 1 -- 172.21.15.55:0/1216764173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc032440 con 0x7f22040b46b0 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2202a00640 1 -- 172.21.15.55:0/1216764173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0325a0 con 0x7f22040b46b0 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2202a00640 1 -- 172.21.15.55:0/1216764173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc032930 con 0x7f22040b46b0 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 -- 172.21.15.55:0/1216764173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22040b46b0 msgr2=0x7f220415df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22040b46b0 0x7f220415df40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f220406e3d0 tx=0x7f21fc02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 -- 172.21.15.55:0/1216764173 shutdown_connections 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22040b46b0 0x7f220415df40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 --2- 172.21.15.55:0/1216764173 >> v2:172.21.15.162:3300/0 conn(0x7f22040cb730 0x7f22040b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 --2- 172.21.15.55:0/1216764173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220415a5d0 0x7f22040cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.619+0000 7f2209800640 1 -- 172.21.15.55:0/1216764173 >> 172.21.15.55:0/1216764173 conn(0x7f220406a110 msgr2=0x7f220406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.620+0000 7f2209800640 1 -- 172.21.15.55:0/1216764173 shutdown_connections 2024-05-10T03:31:10.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.620+0000 7f2209800640 1 -- 172.21.15.55:0/1216764173 wait complete. 2024-05-10T03:31:10.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.622+0000 7f2209800640 1 Processor -- start 2024-05-10T03:31:10.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.622+0000 7f2209800640 1 -- start start 2024-05-10T03:31:10.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22040b46b0 0x7f220413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22040cb730 0x7f2204155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204159270 con 0x7f220415a5d0 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204071060 con 0x7f22040b46b0 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2209800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220413efc0 con 0x7f22040cb730 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59632/0 (socket says 172.21.15.55:59632) 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 -- 172.21.15.55:0/374929707 learned_addr learned my addr 172.21.15.55:0/374929707 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 -- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22040cb730 msgr2=0x7f2204155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22040cb730 0x7f2204155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 -- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22040b46b0 msgr2=0x7f220413ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22040b46b0 0x7f220413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.624+0000 7f2208e00640 1 -- 172.21.15.55:0/374929707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0094d0 con 0x7f220415a5d0 2024-05-10T03:31:10.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f2208e00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f21fc02a220 tx=0x7f21fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc032440 con 0x7f220415a5d0 2024-05-10T03:31:10.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc032e90 con 0x7f220415a5d0 2024-05-10T03:31:10.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc030de0 con 0x7f220415a5d0 2024-05-10T03:31:10.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22041550e0 con 0x7f220415a5d0 2024-05-10T03:31:10.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.625+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2204155380 con 0x7f220415a5d0 2024-05-10T03:31:10.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.626+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204066ce0 con 0x7f220415a5d0 2024-05-10T03:31:10.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.628+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f21fc039040 con 0x7f220415a5d0 2024-05-10T03:31:10.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.629+0000 7f21fbe00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 0x7f21e006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:10.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.629+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f21fc0aa190 con 0x7f220415a5d0 2024-05-10T03:31:10.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.629+0000 7f2203e00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 0x7f21e006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:10.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.631+0000 7f2203e00640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 0x7f21e006d140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2204072370 tx=0x7f21f4007310 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:10.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.633+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f21fc074ed0 con 0x7f220415a5d0 2024-05-10T03:31:10.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.825+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f220404e290 con 0x7f220415a5d0 2024-05-10T03:31:10.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.827+0000 7f21fbe00640 1 -- 172.21.15.55:0/374929707 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f21fc0784f0 con 0x7f220415a5d0 2024-05-10T03:31:10.832 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:10.832 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:10.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 msgr2=0x7f21e006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 0x7f21e006d140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2204072370 tx=0x7f21f4007310 comp rx=0 tx=0).stop 2024-05-10T03:31:10.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 msgr2=0x7f220406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:10.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f21fc02a220 tx=0x7f21fc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 shutdown_connections 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f220415a5d0 0x7f220406fe80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22040cb730 0x7f2204155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f21e006ac30 0x7f21e006d140 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 --2- 172.21.15.55:0/374929707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22040b46b0 0x7f220413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 >> 172.21.15.55:0/374929707 conn(0x7f220406a110 msgr2=0x7f22040afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 shutdown_connections 2024-05-10T03:31:10.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:10.830+0000 7f2209800640 1 -- 172.21.15.55:0/374929707 wait complete. 2024-05-10T03:31:10.861 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:31:10.861 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:11.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.134+0000 7fda2e000640 1 Processor -- start 2024-05-10T03:31:11.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.134+0000 7fda2e000640 1 -- start start 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda2e000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda280cdba0 0x7fda2806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda2e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda28071cf0 0x7fda280b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda2e000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280cd710 con 0x7fda2815ab30 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda2e000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda280cdfb0 con 0x7fda280cdba0 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda2e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2806b4d0 con 0x7fda28071cf0 2024-05-10T03:31:11.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda27400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda280cdba0 0x7fda2806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda27400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda280cdba0 0x7fda2806a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59634/0 (socket says 172.21.15.55:59634) 2024-05-10T03:31:11.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda27400640 1 -- 172.21.15.55:0/3006263808 learned_addr learned my addr 172.21.15.55:0/3006263808 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:11.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.137+0000 7fda27e00640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda28071cf0 0x7fda280b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.138+0000 7fda27400640 1 -- 172.21.15.55:0/3006263808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda28071cf0 msgr2=0x7fda280b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.138+0000 7fda27400640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda28071cf0 0x7fda280b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.138+0000 7fda27400640 1 -- 172.21.15.55:0/3006263808 >> v1:172.21.15.162:6789/0 conn(0x7fda2815ab30 legacy=0x7fda280715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.138+0000 7fda27400640 1 -- 172.21.15.55:0/3006263808 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2806b1f0 con 0x7fda280cdba0 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.139+0000 7fda27400640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda280cdba0 0x7fda2806a9a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fda18009870 tx=0x7fda1802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9add0213dbf5402d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.139+0000 7fda26000640 1 -- 172.21.15.55:0/3006263808 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1802ae10 con 0x7fda280cdba0 2024-05-10T03:31:11.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.139+0000 7fda26000640 1 -- 172.21.15.55:0/3006263808 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18032440 con 0x7fda280cdba0 2024-05-10T03:31:11.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.139+0000 7fda26000640 1 -- 172.21.15.55:0/3006263808 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda180327d0 con 0x7fda280cdba0 2024-05-10T03:31:11.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 -- 172.21.15.55:0/3006263808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda280cdba0 msgr2=0x7fda2806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda280cdba0 0x7fda2806a9a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fda18009870 tx=0x7fda1802a100 comp rx=0 tx=0).stop 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 -- 172.21.15.55:0/3006263808 shutdown_connections 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda28071cf0 0x7fda280b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 --2- 172.21.15.55:0/3006263808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda280cdba0 0x7fda2806a9a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 -- 172.21.15.55:0/3006263808 >> 172.21.15.55:0/3006263808 conn(0x7fda280cbb90 msgr2=0x7fda280cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 -- 172.21.15.55:0/3006263808 shutdown_connections 2024-05-10T03:31:11.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.140+0000 7fda2e000640 1 -- 172.21.15.55:0/3006263808 wait complete. 2024-05-10T03:31:11.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.144+0000 7fda2e000640 1 Processor -- start 2024-05-10T03:31:11.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.144+0000 7fda2e000640 1 -- start start 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda280cdba0 0x7fda281e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 0x7fda2813da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2813dfa0 con 0x7fda2815ab30 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2815e670 con 0x7fda2813d650 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.146+0000 7fda2e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2813e3e0 con 0x7fda280cdba0 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59636/0 (socket says 172.21.15.55:59636) 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 -- 172.21.15.55:0/4252643706 learned_addr learned my addr 172.21.15.55:0/4252643706 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:11.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda27400640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda280cdba0 0x7fda281e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda27e00640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 0x7fda2813da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 -- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda280cdba0 msgr2=0x7fda281e5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda280cdba0 0x7fda281e5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 -- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 msgr2=0x7fda2813da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 0x7fda2813da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda26a00640 1 -- 172.21.15.55:0/4252643706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180094d0 con 0x7fda2815ab30 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.147+0000 7fda27e00640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 0x7fda2813da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda26a00640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b420 tx=0x7fda1000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10011830 con 0x7fda2815ab30 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10011990 con 0x7fda2815ab30 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10011d40 con 0x7fda2815ab30 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda281e4dd0 con 0x7fda2815ab30 2024-05-10T03:31:11.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.148+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda281f3ae0 con 0x7fda2815ab30 2024-05-10T03:31:11.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.149+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda281e4f60 con 0x7fda2815ab30 2024-05-10T03:31:11.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.151+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fda1001b460 con 0x7fda2815ab30 2024-05-10T03:31:11.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.152+0000 7fda1f400640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 0x7fda0406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.152+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fda10089430 con 0x7fda2815ab30 2024-05-10T03:31:11.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.152+0000 7fda27400640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 0x7fda0406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.154+0000 7fda27400640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 0x7fda0406d140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda18034000 tx=0x7fda1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.157+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fda100540e0 con 0x7fda2815ab30 2024-05-10T03:31:11.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.396+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda28066f60 con 0x7fda2815ab30 2024-05-10T03:31:11.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.397+0000 7fda1f400640 1 -- 172.21.15.55:0/4252643706 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7fda10057700 con 0x7fda2815ab30 2024-05-10T03:31:11.401 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:11.402 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:31:11.402 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 msgr2=0x7fda0406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 0x7fda0406d140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda18034000 tx=0x7fda1802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 msgr2=0x7fda281f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b420 tx=0x7fda1000b8e0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 shutdown_connections 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fda0406ac30 0x7fda0406d140 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda2813d650 0x7fda2813da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda2815ab30 0x7fda281f35a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 --2- 172.21.15.55:0/4252643706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda280cdba0 0x7fda281e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 >> 172.21.15.55:0/4252643706 conn(0x7fda280cbb90 msgr2=0x7fda28158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 shutdown_connections 2024-05-10T03:31:11.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.401+0000 7fda2e000640 1 -- 172.21.15.55:0/4252643706 wait complete. 2024-05-10T03:31:11.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:11.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.703+0000 7f3939200640 1 Processor -- start 2024-05-10T03:31:11.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.704+0000 7f3939200640 1 -- start start 2024-05-10T03:31:11.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.705+0000 7f3939200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 0x7f39340cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.705+0000 7f3939200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39340b4880 0x7f39340ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.705+0000 7f3939200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393415ad30 con 0x7f39340cb730 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.705+0000 7f3939200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393415b010 con 0x7f39340b4880 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.705+0000 7f3939200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393415b2f0 con 0x7f393415a5d0 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f39340b4880 0x7f39340ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f392be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f39340cb730 0x7f39340b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38788/0 (socket says 172.21.15.55:38788) 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f392be00640 1 -- 172.21.15.55:0/1876920447 learned_addr learned my addr 172.21.15.55:0/1876920447 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933400640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 0x7f39340cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933e00640 1 -- 172.21.15.55:0/1876920447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 msgr2=0x7f39340cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933e00640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 0x7f39340cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933e00640 1 -- 172.21.15.55:0/1876920447 >> v1:172.21.15.55:6789/0 conn(0x7f39340cb730 legacy=0x7f39340b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933e00640 1 -- 172.21.15.55:0/1876920447 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393415b5d0 con 0x7f39340b4880 2024-05-10T03:31:11.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.706+0000 7f3933400640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 0x7f39340cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.707+0000 7f3933e00640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39340b4880 0x7f39340ccc80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f392c00bfa0 tx=0x7f392c02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98b0689468b0bc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.708+0000 7f3932a00640 1 -- 172.21.15.55:0/1876920447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c009540 con 0x7f39340b4880 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.708+0000 7f3932a00640 1 -- 172.21.15.55:0/1876920447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c0096a0 con 0x7f39340b4880 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.708+0000 7f3932a00640 1 -- 172.21.15.55:0/1876920447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c009a30 con 0x7f39340b4880 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.708+0000 7f3939200640 1 -- 172.21.15.55:0/1876920447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39340b4880 msgr2=0x7f39340ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.708+0000 7f3939200640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39340b4880 0x7f39340ccc80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f392c00bfa0 tx=0x7f392c02c870 comp rx=0 tx=0).stop 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 -- 172.21.15.55:0/1876920447 shutdown_connections 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39340b4880 0x7f39340ccc80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 --2- 172.21.15.55:0/1876920447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f393415a5d0 0x7f39340cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 -- 172.21.15.55:0/1876920447 >> 172.21.15.55:0/1876920447 conn(0x7f393406a110 msgr2=0x7f393406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:11.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 -- 172.21.15.55:0/1876920447 shutdown_connections 2024-05-10T03:31:11.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.709+0000 7f3939200640 1 -- 172.21.15.55:0/1876920447 wait complete. 2024-05-10T03:31:11.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.711+0000 7f3939200640 1 Processor -- start 2024-05-10T03:31:11.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.711+0000 7f3939200640 1 -- start start 2024-05-10T03:31:11.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 0x7f3934144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3934148620 0x7f393415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934148200 con 0x7f393415a5d0 2024-05-10T03:31:11.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934147940 con 0x7f3934148620 2024-05-10T03:31:11.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39341467a0 con 0x7f39340cb730 2024-05-10T03:31:11.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59668/0 (socket says 172.21.15.55:59668) 2024-05-10T03:31:11.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 -- 172.21.15.55:0/3390168320 learned_addr learned my addr 172.21.15.55:0/3390168320 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3933400640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 0x7f3934144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 -- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 msgr2=0x7f3934144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 0x7f3934144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 -- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3934148620 msgr2=0x7f393415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3934148620 0x7f393415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 -- 172.21.15.55:0/3390168320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c00bc80 con 0x7f393415a5d0 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3933400640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 0x7f3934144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392be00640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c390 tx=0x7f391c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c005ce0 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c005e60 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c007310 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393415e690 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.714+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393415ebc0 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.715+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934066ce0 con 0x7f393415a5d0 2024-05-10T03:31:11.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.716+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f391c007470 con 0x7f393415a5d0 2024-05-10T03:31:11.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.718+0000 7f392b400640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 0x7f391006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:11.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.718+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f391c088d00 con 0x7f393415a5d0 2024-05-10T03:31:11.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.718+0000 7f3933400640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 0x7f391006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:11.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.719+0000 7f3933400640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 0x7f391006d110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3924006fd0 tx=0x7f3924008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:11.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.722+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f391c053a40 con 0x7f393415a5d0 2024-05-10T03:31:11.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.960+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f393415e940 con 0x7f393415a5d0 2024-05-10T03:31:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.962+0000 7f392b400640 1 -- 172.21.15.55:0/3390168320 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4736 (secure 0 0 0) 0x7f391c017090 con 0x7f393415a5d0 2024-05-10T03:31:11.966 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:11.966 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3650948924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3650948924},{"type":"v1","addr":"172.21.15.55:6835","nonce":3650948924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3664624296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3664624296},{"type":"v1","addr":"172.21.15.55:6834","nonce":3664624296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/2958284106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2958284106},{"type":"v1","addr":"172.21.15.162:6836","nonce":2958284106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:29:44.296174+0000","modified":"2024-05-10T03:30:08.098152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/2047857054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2047857054},{"type":"v1","addr":"172.21.15.162:6837","nonce":2047857054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T03:31:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 msgr2=0x7f391006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 0x7f391006d110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3924006fd0 tx=0x7f3924008040 comp rx=0 tx=0).stop 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 msgr2=0x7f393415dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c390 tx=0x7f391c00c850 comp rx=0 tx=0).stop 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 shutdown_connections 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f391006ac00 0x7f391006d110 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3934148620 0x7f393415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f393415a5d0 0x7f393415dcd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 --2- 172.21.15.55:0/3390168320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f39340cb730 0x7f3934144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.964+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 >> 172.21.15.55:0/3390168320 conn(0x7f393406a110 msgr2=0x7f393406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.965+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 shutdown_connections 2024-05-10T03:31:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:11.965+0000 7f3939200640 1 -- 172.21.15.55:0/3390168320 wait complete. 2024-05-10T03:31:11.994 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:12.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.273+0000 7f2ff7600640 1 Processor -- start 2024-05-10T03:31:12.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.273+0000 7f2ff7600640 1 -- start start 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.275+0000 7f2ff7600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2ff0161c70 0x7f2ff00cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.275+0000 7f2ff7600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00d0110 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2ff7600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00cf330 con 0x7f2ff00b5df0 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2ff7600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff00b6200 con 0x7f2ff0161c70 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2fefe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2ff00cf7c0 0x7f2ff0161560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38794/0 (socket says 172.21.15.55:38794) 2024-05-10T03:31:12.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2fefe00640 1 -- 172.21.15.55:0/1708823165 learned_addr learned my addr 172.21.15.55:0/1708823165 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:12.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2ff5600640 1 --2- 172.21.15.55:0/1708823165 >> v2:172.21.15.162:3300/0 conn(0x7f2ff0161c70 0x7f2ff00cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245608969 0 0) 0x7f2fd8001110 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.276+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd4003520 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891886156 0 0) 0x7f2fd4003520 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 >> v2:172.21.15.162:3300/0 conn(0x7f2ff0161c70 msgr2=0x7f2ff00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 --2- 172.21.15.55:0/1708823165 >> v2:172.21.15.162:3300/0 conn(0x7f2ff0161c70 0x7f2ff00cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 >> v1:172.21.15.162:6789/0 conn(0x7f2ff00b5df0 legacy=0x7f2ff00b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0143ab0 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2fd8001800 con 0x7f2ff00cf7c0 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 >> v1:172.21.15.55:6789/0 conn(0x7f2ff00cf7c0 legacy=0x7f2ff0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd4003970 0x7f2fd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b6200 con 0x7f2fd40047c0 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd400c120 con 0x7f2fd4005180 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd400c2e0 con 0x7f2fd4003970 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2ff4c00640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd4003970 0x7f2fd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2ff5600640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 -- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd4003970 msgr2=0x7f2fd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd4003970 0x7f2fd4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 -- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd4005180 msgr2=0x7f2fd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd4005180 0x7f2fd400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2fefe00640 1 -- 172.21.15.55:0/1708823165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4003520 con 0x7f2fd40047c0 2024-05-10T03:31:12.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.278+0000 7f2ff5600640 1 -- 172.21.15.55:0/1708823165 reap_dead start 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.279+0000 7f2fefe00640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004bd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c240 tx=0x7f2fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe71522a09d156eb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.279+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8005ad0 con 0x7f2fd40047c0 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.279+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd802d070 con 0x7f2fd40047c0 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.279+0000 7f2fef400640 1 -- 172.21.15.55:0/1708823165 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8007070 con 0x7f2fd40047c0 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 -- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 msgr2=0x7f2fd4004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004bd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c240 tx=0x7f2fd40057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 -- 172.21.15.55:0/1708823165 shutdown_connections 2024-05-10T03:31:12.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 --2- 172.21.15.55:0/1708823165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fd40047c0 0x7f2fd4004bd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 -- 172.21.15.55:0/1708823165 >> 172.21.15.55:0/1708823165 conn(0x7f2ff00b2c10 msgr2=0x7f2ff00b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:12.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 -- 172.21.15.55:0/1708823165 shutdown_connections 2024-05-10T03:31:12.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.280+0000 7f2ff7600640 1 -- 172.21.15.55:0/1708823165 wait complete. 2024-05-10T03:31:12.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.283+0000 7f2ff7600640 1 Processor -- start 2024-05-10T03:31:12.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.283+0000 7f2ff7600640 1 -- start start 2024-05-10T03:31:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 0x7f2ff0144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 0x7f2ff015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0144850 con 0x7f2ff015f5e0 2024-05-10T03:31:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01f3a50 con 0x7f2ff01f7180 2024-05-10T03:31:12.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff7600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015f9f0 con 0x7f2ff013ebb0 2024-05-10T03:31:12.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59680/0 (socket says 172.21.15.55:59680) 2024-05-10T03:31:12.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff5600640 1 -- 172.21.15.55:0/4026369004 learned_addr learned my addr 172.21.15.55:0/4026369004 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff4c00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 0x7f2ff0144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2fefe00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 0x7f2ff015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.285+0000 7f2ff5600640 1 -- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 msgr2=0x7f2ff015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff5600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 0x7f2ff015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff5600640 1 -- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 msgr2=0x7f2ff0144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff5600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 0x7f2ff0144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff5600640 1 -- 172.21.15.55:0/4026369004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0144a70 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2fefe00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 0x7f2ff015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff4c00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 0x7f2ff0144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff5600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0071970 tx=0x7f2fe8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004230 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80043b0 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004520 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff01f3400 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.286+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff01f30f0 con 0x7f2ff015f5e0 2024-05-10T03:31:12.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.288+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff0067220 con 0x7f2ff015f5e0 2024-05-10T03:31:12.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.289+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2fe800e8b0 con 0x7f2ff015f5e0 2024-05-10T03:31:12.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.290+0000 7f2fecc00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 0x7f2fcc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.290+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2fe8089680 con 0x7f2ff015f5e0 2024-05-10T03:31:12.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.290+0000 7f2ff4c00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 0x7f2fcc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.292+0000 7f2ff4c00640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 0x7f2fcc06d110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0007fd0 tx=0x7f2fe0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.296+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2fe80544f0 con 0x7f2ff015f5e0 2024-05-10T03:31:12.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.501+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ff00b31c0 con 0x7f2ff015f5e0 2024-05-10T03:31:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.503+0000 7f2fecc00640 1 -- 172.21.15.55:0/4026369004 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f2fe8057b10 con 0x7f2ff015f5e0 2024-05-10T03:31:12.507 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:12.507 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":27,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:30:24.259666+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:29:41.917967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T03:29:43.315048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T03:29:46.815363+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2c","mappings":[{"from":4,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 msgr2=0x7f2fcc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 0x7f2fcc06d110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0007fd0 tx=0x7f2fe0009040 comp rx=0 tx=0).stop 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 msgr2=0x7f2ff0141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0071970 tx=0x7f2fe8010c20 comp rx=0 tx=0).stop 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 shutdown_connections 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ff015f5e0 0x7f2ff0141380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff013ebb0 0x7f2ff015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2fcc06ac00 0x7f2fcc06d110 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 --2- 172.21.15.55:0/4026369004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff01f7180 0x7f2ff0144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 >> 172.21.15.55:0/4026369004 conn(0x7f2ff00b2c10 msgr2=0x7f2ff00d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:12.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 shutdown_connections 2024-05-10T03:31:12.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.506+0000 7f2ff7600640 1 -- 172.21.15.55:0/4026369004 wait complete. 2024-05-10T03:31:12.536 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:31:12.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.805+0000 7f2ccea00640 1 Processor -- start 2024-05-10T03:31:12.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.806+0000 7f2ccea00640 1 -- start start 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.808+0000 7f2ccea00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cc815b9d0 0x7f2cc806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.808+0000 7f2ccea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cc806b660 0x7f2cc813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.808+0000 7f2ccea00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc815bde0 con 0x7f2cc813e390 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.808+0000 7f2ccea00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc813e7a0 con 0x7f2cc806b660 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.808+0000 7f2ccea00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc813eae0 con 0x7f2cc815b9d0 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2ccd600640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2cc813e390 0x7f2cc81414c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38808/0 (socket says 172.21.15.55:38808) 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2ccd600640 1 -- 172.21.15.55:0/3051003268 learned_addr learned my addr 172.21.15.55:0/3051003268 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cce000640 1 --2- 172.21.15.55:0/3051003268 >> v2:172.21.15.55:3300/0 conn(0x7f2cc806b660 0x7f2cc813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cccc00640 1 --2- 172.21.15.55:0/3051003268 >> v2:172.21.15.162:3300/0 conn(0x7f2cc815b9d0 0x7f2cc806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cce000640 1 -- 172.21.15.55:0/3051003268 >> v2:172.21.15.162:3300/0 conn(0x7f2cc815b9d0 msgr2=0x7f2cc806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cce000640 1 --2- 172.21.15.55:0/3051003268 >> v2:172.21.15.162:3300/0 conn(0x7f2cc815b9d0 0x7f2cc806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cce000640 1 -- 172.21.15.55:0/3051003268 >> v1:172.21.15.55:6789/0 conn(0x7f2cc813e390 legacy=0x7f2cc81414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cce000640 1 -- 172.21.15.55:0/3051003268 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc806ba70 con 0x7f2cc806b660 2024-05-10T03:31:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.809+0000 7f2cc7e00640 1 -- 172.21.15.55:0/3051003268 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221193785 0 0) 0x7f2cbc001110 con 0x7f2cc813e390 2024-05-10T03:31:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.810+0000 7f2cce000640 1 --2- 172.21.15.55:0/3051003268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc813ef70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80ae8f0 tx=0x7f2cc002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113c9d327ff69b6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.811+0000 7f2cc7e00640 1 -- 172.21.15.55:0/3051003268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0007740 con 0x7f2cc806b660 2024-05-10T03:31:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.811+0000 7f2cc7e00640 1 -- 172.21.15.55:0/3051003268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc002d050 con 0x7f2cc806b660 2024-05-10T03:31:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.811+0000 7f2cc7e00640 1 -- 172.21.15.55:0/3051003268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0007ac0 con 0x7f2cc806b660 2024-05-10T03:31:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.811+0000 7f2ccea00640 1 -- 172.21.15.55:0/3051003268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 msgr2=0x7f2cc813ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.811+0000 7f2ccea00640 1 --2- 172.21.15.55:0/3051003268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc813ef70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80ae8f0 tx=0x7f2cc002a5f0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 -- 172.21.15.55:0/3051003268 shutdown_connections 2024-05-10T03:31:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 --2- 172.21.15.55:0/3051003268 >> v2:172.21.15.162:3300/0 conn(0x7f2cc815b9d0 0x7f2cc806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 --2- 172.21.15.55:0/3051003268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc813ef70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 -- 172.21.15.55:0/3051003268 >> 172.21.15.55:0/3051003268 conn(0x7f2cc806c730 msgr2=0x7f2cc80b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 -- 172.21.15.55:0/3051003268 shutdown_connections 2024-05-10T03:31:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.812+0000 7f2ccea00640 1 -- 172.21.15.55:0/3051003268 wait complete. 2024-05-10T03:31:12.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.814+0000 7f2ccea00640 1 Processor -- start 2024-05-10T03:31:12.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.814+0000 7f2ccea00640 1 -- start start 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 0x7f2cc80af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815b9d0 0x7f2cc8136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80af090 con 0x7f2cc806b660 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc814eb70 con 0x7f2cc813e390 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccea00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc81543a0 con 0x7f2cc815b9d0 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccd600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccd600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59690/0 (socket says 172.21.15.55:59690) 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2ccd600640 1 -- 172.21.15.55:0/575472891 learned_addr learned my addr 172.21.15.55:0/575472891 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.816+0000 7f2cccc00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 0x7f2cc80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 -- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815b9d0 msgr2=0x7f2cc8136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2cce000640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815b9d0 0x7f2cc8136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815b9d0 0x7f2cc8136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 -- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 msgr2=0x7f2cc80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 0x7f2cc80af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 -- 172.21.15.55:0/575472891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0009a70 con 0x7f2cc806b660 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2cccc00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 0x7f2cc80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccd600640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c410 tx=0x7f2cbc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0074e0 con 0x7f2cc806b660 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc007660 con 0x7f2cc806b660 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc004a20 con 0x7f2cc806b660 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc81522d0 con 0x7f2cc806b660 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.817+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81548d0 con 0x7f2cc806b660 2024-05-10T03:31:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.819+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8071c80 con 0x7f2cc806b660 2024-05-10T03:31:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.820+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2cbc004b80 con 0x7f2cc806b660 2024-05-10T03:31:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.820+0000 7f2cc5600640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 0x7f2ca806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.821+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f2cbc0894b0 con 0x7f2cc806b660 2024-05-10T03:31:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.821+0000 7f2cccc00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 0x7f2ca806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:12.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.822+0000 7f2cccc00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 0x7f2ca806d110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8151b70 tx=0x7f2cb4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:12.825+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cbc057c80 con 0x7f2cc806b660 2024-05-10T03:31:13.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.063+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2cc815bde0 con 0x7f2cc806b660 2024-05-10T03:31:13.068 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:31:13.064+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:31:13.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.084+0000 7f2cc5600640 1 -- 172.21.15.55:0/575472891 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2cc815bde0 con 0x7f2cc806b660 2024-05-10T03:31:13.088 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:31:13.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 msgr2=0x7f2ca806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 0x7f2ca806d110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8151b70 tx=0x7f2cb4008040 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 msgr2=0x7f2cc814e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c410 tx=0x7f2cbc00c8d0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 shutdown_connections 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815b9d0 0x7f2cc8136820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca806ac00 0x7f2ca806d110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc813e390 0x7f2cc80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 --2- 172.21.15.55:0/575472891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b660 0x7f2cc814e0f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 >> 172.21.15.55:0/575472891 conn(0x7f2cc806c730 msgr2=0x7f2cc80b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.087+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 shutdown_connections 2024-05-10T03:31:13.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.088+0000 7f2ccea00640 1 -- 172.21.15.55:0/575472891 wait complete. 2024-05-10T03:31:13.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:31:13.123 INFO:tasks.ceph.mds.d.smithi162.stderr: -6086> 2024-05-10T03:29:49.396+0000 7f3d33800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:31:13.290 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:31:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.400+0000 7f8a6fe00640 1 Processor -- start 2024-05-10T03:31:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.401+0000 7f8a6fe00640 1 -- start start 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.403+0000 7f8a6fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a7015d7c0 0x7f8a700ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.403+0000 7f8a6fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a700ceaa0 con 0x7f8a70161370 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.403+0000 7f8a6fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7006a480 con 0x7f8a700ce690 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.403+0000 7f8a6fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700cfa80 con 0x7f8a7015d7c0 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a7015d7c0 0x7f8a700ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a7015d7c0 0x7f8a700ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36108/0 (socket says 172.21.15.55:36108) 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a6ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8a70161370 0x7f8a7006b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39392/0 (socket says 172.21.15.55:39392) 2024-05-10T03:31:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 -- 172.21.15.55:0/4079367647 learned_addr learned my addr 172.21.15.55:0/4079367647 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:13.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 -- 172.21.15.55:0/4079367647 >> v1:172.21.15.162:6789/0 conn(0x7f8a700ce690 legacy=0x7f8a700b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:13.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 -- 172.21.15.55:0/4079367647 >> v1:172.21.15.55:6789/0 conn(0x7f8a70161370 legacy=0x7f8a7006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a67e00640 1 -- 172.21.15.55:0/4079367647 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7006a760 con 0x7f8a7015d7c0 2024-05-10T03:31:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.404+0000 7f8a6e000640 1 -- 172.21.15.55:0/4079367647 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936815136 0 0) 0x7f8a700ceaa0 con 0x7f8a70161370 2024-05-10T03:31:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.405+0000 7f8a67e00640 1 --2- 172.21.15.55:0/4079367647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ce150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14090b79eb7d1578 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.406+0000 7f8a6e000640 1 -- 172.21.15.55:0/4079367647 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58032440 con 0x7f8a7015d7c0 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.406+0000 7f8a6e000640 1 -- 172.21.15.55:0/4079367647 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a580325a0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.406+0000 7f8a6e000640 1 -- 172.21.15.55:0/4079367647 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58032930 con 0x7f8a7015d7c0 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.406+0000 7f8a6fe00640 1 -- 172.21.15.55:0/4079367647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 msgr2=0x7f8a700ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.406+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/4079367647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ce150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a5802a2a0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.407+0000 7f8a6fe00640 1 -- 172.21.15.55:0/4079367647 shutdown_connections 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.407+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/4079367647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ce150 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.407+0000 7f8a6fe00640 1 -- 172.21.15.55:0/4079367647 >> 172.21.15.55:0/4079367647 conn(0x7f8a700d08b0 msgr2=0x7f8a700d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.407+0000 7f8a6fe00640 1 -- 172.21.15.55:0/4079367647 shutdown_connections 2024-05-10T03:31:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.407+0000 7f8a6fe00640 1 -- 172.21.15.55:0/4079367647 wait complete. 2024-05-10T03:31:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.409+0000 7f8a6fe00640 1 Processor -- start 2024-05-10T03:31:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.409+0000 7f8a6fe00640 1 -- start start 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 0x7f8a701484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 0x7f8a700cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015f4f0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700cab50 con 0x7f8a700ce690 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70148a20 con 0x7f8a70161370 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36112/0 (socket says 172.21.15.55:36112) 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.411+0000 7f8a67e00640 1 -- 172.21.15.55:0/2520596941 learned_addr learned my addr 172.21.15.55:0/2520596941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a6ea00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 0x7f8a701484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a6f400640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 0x7f8a700cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 -- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 msgr2=0x7f8a700cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 0x7f8a700cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 -- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 msgr2=0x7f8a701484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 0x7f8a701484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 -- 172.21.15.55:0/2520596941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580094d0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a6f400640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 0x7f8a700cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a6ea00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 0x7f8a701484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a67e00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.412+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58032440 con 0x7f8a7015d7c0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.413+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58030ba0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.413+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5803b400 con 0x7f8a7015d7c0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.413+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a700cada0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.413+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a700cb360 con 0x7f8a7015d7c0 2024-05-10T03:31:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.414+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7004e290 con 0x7f8a7015d7c0 2024-05-10T03:31:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.415+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8a58039040 con 0x7f8a7015d7c0 2024-05-10T03:31:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.417+0000 7f8a66a00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 0x7f8a4c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.417+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f8a580aa1b0 con 0x7f8a7015d7c0 2024-05-10T03:31:13.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.418+0000 7f8a6ea00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 0x7f8a4c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:13.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.419+0000 7f8a6ea00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 0x7f8a4c06d140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a60000f60 tx=0x7f8a60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.422+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8a58074e10 con 0x7f8a7015d7c0 2024-05-10T03:31:13.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:13.649+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a700d1710 con 0x7f8a7015d7c0 2024-05-10T03:31:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.089+0000 7f8a66a00640 1 -- 172.21.15.55:0/2520596941 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a58078430 con 0x7f8a7015d7c0 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.092+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 msgr2=0x7f8a4c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.092+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 0x7f8a4c06d140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a60000f60 tx=0x7f8a60009040 comp rx=0 tx=0).stop 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.092+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 msgr2=0x7f8a700ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009870 tx=0x7f8a580046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 shutdown_connections 2024-05-10T03:31:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a70161370 0x7f8a700cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a7015d7c0 0x7f8a700ca230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a4c06ac30 0x7f8a4c06d140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 --2- 172.21.15.55:0/2520596941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a700ce690 0x7f8a701484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 >> 172.21.15.55:0/2520596941 conn(0x7f8a700d08b0 msgr2=0x7f8a700d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 shutdown_connections 2024-05-10T03:31:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.093+0000 7f8a6fe00640 1 -- 172.21.15.55:0/2520596941 wait complete. 2024-05-10T03:31:14.125 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:31:14.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.407+0000 7f12c8e00640 1 Processor -- start 2024-05-10T03:31:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.407+0000 7f12c8e00640 1 -- start start 2024-05-10T03:31:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.409+0000 7f12c8e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f12c4154300 0x7f12c406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.409+0000 7f12c8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c8e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4071080 con 0x7f12c40cb9d0 2024-05-10T03:31:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c8e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c40ce3a0 con 0x7f12c4154300 2024-05-10T03:31:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4154710 con 0x7f12c413fb70 2024-05-10T03:31:14.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:36048/0 (socket says 172.21.15.55:36048) 2024-05-10T03:31:14.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c3e00640 1 -- 172.21.15.55:0/2927775381 learned_addr learned my addr 172.21.15.55:0/2927775381 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:14.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.410+0000 7f12c2a00640 1 --1- 172.21.15.55:0/2927775381 >> v1:172.21.15.162:6789/0 conn(0x7f12c40cb9d0 0x7f12c413f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:38038/0 (socket says 172.21.15.55:38038) 2024-05-10T03:31:14.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c3400640 1 --2- 172.21.15.55:0/2927775381 >> v2:172.21.15.162:3300/0 conn(0x7f12c4154300 0x7f12c406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c3e00640 1 -- 172.21.15.55:0/2927775381 >> v2:172.21.15.162:3300/0 conn(0x7f12c4154300 msgr2=0x7f12c406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c3e00640 1 --2- 172.21.15.55:0/2927775381 >> v2:172.21.15.162:3300/0 conn(0x7f12c4154300 0x7f12c406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c3e00640 1 -- 172.21.15.55:0/2927775381 >> v1:172.21.15.162:6789/0 conn(0x7f12c40cb9d0 legacy=0x7f12c413f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c3e00640 1 -- 172.21.15.55:0/2927775381 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40cd4f0 con 0x7f12c413fb70 2024-05-10T03:31:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.411+0000 7f12c2000640 1 -- 172.21.15.55:0/2927775381 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330008462 0 0) 0x7f12c4071080 con 0x7f12c40cb9d0 2024-05-10T03:31:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.413+0000 7f12c3e00640 1 --2- 172.21.15.55:0/2927775381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009e50 tx=0x7f12bc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6fb23f9481be61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:14.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.413+0000 7f12c2000640 1 -- 172.21.15.55:0/2927775381 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc031600 con 0x7f12c413fb70 2024-05-10T03:31:14.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c2000640 1 -- 172.21.15.55:0/2927775381 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc02d050 con 0x7f12c413fb70 2024-05-10T03:31:14.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c2000640 1 -- 172.21.15.55:0/2927775381 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc031980 con 0x7f12c413fb70 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 -- 172.21.15.55:0/2927775381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 msgr2=0x7f12c4164bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2927775381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009e50 tx=0x7f12bc02a5f0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 -- 172.21.15.55:0/2927775381 shutdown_connections 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2927775381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413fb70 0x7f12c4164bc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2927775381 >> v2:172.21.15.162:3300/0 conn(0x7f12c4154300 0x7f12c406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.414+0000 7f12c8e00640 1 -- 172.21.15.55:0/2927775381 >> 172.21.15.55:0/2927775381 conn(0x7f12c406c070 msgr2=0x7f12c406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.415+0000 7f12c8e00640 1 -- 172.21.15.55:0/2927775381 shutdown_connections 2024-05-10T03:31:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.415+0000 7f12c8e00640 1 -- 172.21.15.55:0/2927775381 wait complete. 2024-05-10T03:31:14.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.417+0000 7f12c8e00640 1 Processor -- start 2024-05-10T03:31:14.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.417+0000 7f12c8e00640 1 -- start start 2024-05-10T03:31:14.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12c40cb9d0 0x7f12c4153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413be90 0x7f12c40b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40ace70 con 0x7f12c4154300 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c413c470 con 0x7f12c40cb9d0 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40ab0e0 con 0x7f12c413be90 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.419+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36126/0 (socket says 172.21.15.55:36126) 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 -- 172.21.15.55:0/2655563888 learned_addr learned my addr 172.21.15.55:0/2655563888 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c3400640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12c40cb9d0 0x7f12c4153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c3e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413be90 0x7f12c40b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 -- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413be90 msgr2=0x7f12c40b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413be90 0x7f12c40b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 -- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12c40cb9d0 msgr2=0x7f12c4153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12c40cb9d0 0x7f12c4153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:14.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 -- 172.21.15.55:0/2655563888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc009b30 con 0x7f12c4154300 2024-05-10T03:31:14.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.420+0000 7f12c2a00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c3e0 tx=0x7f12ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:14.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.421+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0074e0 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.421+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac007660 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.421+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac004a20 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.421+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c413c6c0 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.421+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c40ab330 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.422+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4067220 con 0x7f12c4154300 2024-05-10T03:31:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.423+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f12ac004b80 con 0x7f12c4154300 2024-05-10T03:31:14.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.424+0000 7f12bb400640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 0x7f12a006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:14.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.424+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6807+0+0 (secure 0 0 0) 0x7f12ac089540 con 0x7f12c4154300 2024-05-10T03:31:14.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.425+0000 7f12c3400640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 0x7f12a006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:14.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.426+0000 7f12c3400640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 0x7f12a006d110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12c40b1c10 tx=0x7f12b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:14.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.429+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f12ac057d60 con 0x7f12c4154300 2024-05-10T03:31:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:14.629+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12c4072070 con 0x7f12c4154300 2024-05-10T03:31:15.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.096+0000 7f12bb400640 1 -- 172.21.15.55:0/2655563888 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f12ac0578f0 con 0x7f12c4154300 2024-05-10T03:31:15.100 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:31:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 msgr2=0x7f12a006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 0x7f12a006d110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12c40b1c10 tx=0x7f12b4008040 comp rx=0 tx=0).stop 2024-05-10T03:31:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 msgr2=0x7f12c413b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c3e0 tx=0x7f12ac00c8a0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 shutdown_connections 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12c413be90 0x7f12c40b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12c4154300 0x7f12c413b950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f12a006ac00 0x7f12a006d110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 --2- 172.21.15.55:0/2655563888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12c40cb9d0 0x7f12c4153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 >> 172.21.15.55:0/2655563888 conn(0x7f12c406c070 msgr2=0x7f12c415ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.102+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 shutdown_connections 2024-05-10T03:31:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.103+0000 7f12c8e00640 1 -- 172.21.15.55:0/2655563888 wait complete. 2024-05-10T03:31:15.139 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:31:15.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.421+0000 7fd177200640 1 Processor -- start 2024-05-10T03:31:15.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.421+0000 7fd177200640 1 -- start start 2024-05-10T03:31:15.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.423+0000 7fd177200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd1780d2e00 0x7fd17815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.423+0000 7fd177200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.423+0000 7fd177200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780d2970 con 0x7fd1780ce1b0 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.423+0000 7fd177200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780b4710 con 0x7fd1780d2e00 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.423+0000 7fd177200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780d1a90 con 0x7fd1780d1680 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.424+0000 7fd175400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.424+0000 7fd175400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:36066/0 (socket says 172.21.15.55:36066) 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.424+0000 7fd175400640 1 -- 172.21.15.55:0/117250166 learned_addr learned my addr 172.21.15.55:0/117250166 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:15.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.424+0000 7fd175e00640 1 --2- 172.21.15.55:0/117250166 >> v2:172.21.15.162:3300/0 conn(0x7fd1780d2e00 0x7fd17815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.425+0000 7fd175400640 1 -- 172.21.15.55:0/117250166 >> v2:172.21.15.162:3300/0 conn(0x7fd1780d2e00 msgr2=0x7fd17815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:15.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.425+0000 7fd175400640 1 --2- 172.21.15.55:0/117250166 >> v2:172.21.15.162:3300/0 conn(0x7fd1780d2e00 0x7fd17815dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.425+0000 7fd175400640 1 -- 172.21.15.55:0/117250166 >> v1:172.21.15.162:6789/0 conn(0x7fd1780ce1b0 legacy=0x7fd1780ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.425+0000 7fd175400640 1 -- 172.21.15.55:0/117250166 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780d1d60 con 0x7fd1780d1680 2024-05-10T03:31:15.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.426+0000 7fd175400640 1 --2- 172.21.15.55:0/117250166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5417d73a82088344 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:15.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.427+0000 7fd174a00640 1 -- 172.21.15.55:0/117250166 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160032440 con 0x7fd1780d1680 2024-05-10T03:31:15.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.427+0000 7fd174a00640 1 -- 172.21.15.55:0/117250166 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1600325a0 con 0x7fd1780d1680 2024-05-10T03:31:15.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.427+0000 7fd174a00640 1 -- 172.21.15.55:0/117250166 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160032930 con 0x7fd1780d1680 2024-05-10T03:31:15.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 -- 172.21.15.55:0/117250166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 msgr2=0x7fd1780b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 --2- 172.21.15.55:0/117250166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 -- 172.21.15.55:0/117250166 shutdown_connections 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 --2- 172.21.15.55:0/117250166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780d1680 0x7fd1780b41d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 --2- 172.21.15.55:0/117250166 >> v2:172.21.15.162:3300/0 conn(0x7fd1780d2e00 0x7fd17815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 -- 172.21.15.55:0/117250166 >> 172.21.15.55:0/117250166 conn(0x7fd1780b59d0 msgr2=0x7fd1780d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:15.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 -- 172.21.15.55:0/117250166 shutdown_connections 2024-05-10T03:31:15.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.428+0000 7fd177200640 1 -- 172.21.15.55:0/117250166 wait complete. 2024-05-10T03:31:15.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.430+0000 7fd177200640 1 Processor -- start 2024-05-10T03:31:15.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.430+0000 7fd177200640 1 -- start start 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780ce1b0 0x7fd178154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd178140ff0 0x7fd178141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178141940 con 0x7fd1780d2e00 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178155380 con 0x7fd178140ff0 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd177200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd178153270 con 0x7fd1780ce1b0 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd175400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd175400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36138/0 (socket says 172.21.15.55:36138) 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.432+0000 7fd175400640 1 -- 172.21.15.55:0/3607395697 learned_addr learned my addr 172.21.15.55:0/3607395697 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175e00640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780ce1b0 0x7fd178154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd176800640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd178140ff0 0x7fd178141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 -- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780ce1b0 msgr2=0x7fd178154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780ce1b0 0x7fd178154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 -- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd178140ff0 msgr2=0x7fd178141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd178140ff0 0x7fd178141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 -- 172.21.15.55:0/3607395697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600094d0 con 0x7fd1780d2e00 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.433+0000 7fd175400640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.434+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160032440 con 0x7fd1780d2e00 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.434+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160030980 con 0x7fd1780d2e00 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.434+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160030d30 con 0x7fd1780d2e00 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.434+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178142ee0 con 0x7fd1780d2e00 2024-05-10T03:31:15.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.434+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178140b60 con 0x7fd1780d2e00 2024-05-10T03:31:15.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.435+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178067220 con 0x7fd1780d2e00 2024-05-10T03:31:15.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.436+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd160039040 con 0x7fd1780d2e00 2024-05-10T03:31:15.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.437+0000 7fd16e000640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 0x7fd15406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:15.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.437+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6369+0+0 (secure 0 0 0) 0x7fd1600a9dc0 con 0x7fd1780d2e00 2024-05-10T03:31:15.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.438+0000 7fd175e00640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 0x7fd15406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:15.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.439+0000 7fd175e00640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 0x7fd15406d110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd168009a30 tx=0x7fd168008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:15.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.442+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd160074d10 con 0x7fd1780d2e00 2024-05-10T03:31:15.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:15.641+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd17806a1b0 con 0x7fd1780d2e00 2024-05-10T03:31:16.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.100+0000 7fd16e000640 1 -- 172.21.15.55:0/3607395697 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd160078330 con 0x7fd1780d2e00 2024-05-10T03:31:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:31:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.108+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 msgr2=0x7fd15406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.108+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 0x7fd15406d110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd168009a30 tx=0x7fd168008040 comp rx=0 tx=0).stop 2024-05-10T03:31:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.108+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 msgr2=0x7fd1780c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.108+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd160000c00 tx=0x7fd1600046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 shutdown_connections 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd15406ac00 0x7fd15406d110 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd178140ff0 0x7fd178141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1780d2e00 0x7fd1780c8b90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 --2- 172.21.15.55:0/3607395697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1780ce1b0 0x7fd178154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 >> 172.21.15.55:0/3607395697 conn(0x7fd1780b59d0 msgr2=0x7fd178159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 shutdown_connections 2024-05-10T03:31:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.109+0000 7fd177200640 1 -- 172.21.15.55:0/3607395697 wait complete. 2024-05-10T03:31:16.145 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T03:31:16.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.431+0000 7ff760800640 1 Processor -- start 2024-05-10T03:31:16.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.431+0000 7ff760800640 1 -- start start 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff760800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 0x7ff758143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff760800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff758162400 con 0x7ff7580cf350 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff760800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff758161880 con 0x7ff7580b3b20 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff760800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff758161b60 con 0x7ff758140700 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff75d400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff7580cf350 0x7ff7580b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39404/0 (socket says 172.21.15.55:39404) 2024-05-10T03:31:16.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.434+0000 7ff75d400640 1 -- 172.21.15.55:0/1599115023 learned_addr learned my addr 172.21.15.55:0/1599115023 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75de00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 0x7ff758143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625784528 0 0) 0x7ff758162400 con 0x7ff7580cf350 2024-05-10T03:31:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff740003520 con 0x7ff7580cf350 2024-05-10T03:31:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722872992 0 0) 0x7ff740003520 con 0x7ff7580cf350 2024-05-10T03:31:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 msgr2=0x7ff758143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 0x7ff758143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 >> v1:172.21.15.162:6789/0 conn(0x7ff7580b3b20 legacy=0x7ff75815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758161e40 con 0x7ff7580cf350 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7ff748001670 con 0x7ff7580cf350 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 >> v1:172.21.15.55:6789/0 conn(0x7ff7580cf350 legacy=0x7ff7580b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7400039a0 0x7ff740003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 0x7ff7400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff740004d00 0x7ff740005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758161b60 con 0x7ff740004d00 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74000bb40 con 0x7ff7400039a0 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.435+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74000bd00 con 0x7ff758140700 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff740004d00 0x7ff740005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75de00640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7400039a0 0x7ff740003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 -- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 msgr2=0x7ff7400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758140700 0x7ff7400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 -- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7400039a0 msgr2=0x7ff740003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7400039a0 0x7ff740003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.436+0000 7ff75e800640 1 -- 172.21.15.55:0/1599115023 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff740003520 con 0x7ff740004d00 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.437+0000 7ff75e800640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff740004d00 0x7ff740005110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff7580d3390 tx=0x7ff7400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fe82d76086c72b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75402ae10 con 0x7ff740004d00 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754003970 con 0x7ff740004d00 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff75ca00640 1 -- 172.21.15.55:0/1599115023 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754003d20 con 0x7ff740004d00 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff760800640 1 -- 172.21.15.55:0/1599115023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff740004d00 msgr2=0x7ff740005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff760800640 1 --2- 172.21.15.55:0/1599115023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff740004d00 0x7ff740005110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff7580d3390 tx=0x7ff7400057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.438+0000 7ff75e800640 1 -- 172.21.15.55:0/1599115023 reap_dead start 2024-05-10T03:31:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.439+0000 7ff760800640 1 -- 172.21.15.55:0/1599115023 shutdown_connections 2024-05-10T03:31:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.439+0000 7ff760800640 1 -- 172.21.15.55:0/1599115023 >> 172.21.15.55:0/1599115023 conn(0x7ff7580b5390 msgr2=0x7ff7580b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.439+0000 7ff760800640 1 -- 172.21.15.55:0/1599115023 shutdown_connections 2024-05-10T03:31:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.439+0000 7ff760800640 1 -- 172.21.15.55:0/1599115023 wait complete. 2024-05-10T03:31:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.441+0000 7ff760800640 1 Processor -- start 2024-05-10T03:31:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.441+0000 7ff760800640 1 -- start start 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 0x7ff7581585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 0x7ff758155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75815a120 con 0x7ff758155580 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758159c90 con 0x7ff75813efa0 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff760800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff758155990 con 0x7ff758158b20 2024-05-10T03:31:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff75e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff75e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36158/0 (socket says 172.21.15.55:36158) 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.443+0000 7ff75e800640 1 -- 172.21.15.55:0/953850512 learned_addr learned my addr 172.21.15.55:0/953850512 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75de00640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 0x7ff7581585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75d400640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 0x7ff758155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 -- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 msgr2=0x7ff758155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 0x7ff758155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 -- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 msgr2=0x7ff7581585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 0x7ff7581585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 -- 172.21.15.55:0/953850512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758159080 con 0x7ff758155580 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75de00640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 0x7ff7581585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75d400640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 0x7ff758155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff75e800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff75402a8c0 tx=0x7ff7540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754003cf0 con 0x7ff758155580 2024-05-10T03:31:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754003e50 con 0x7ff758155580 2024-05-10T03:31:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7540052d0 con 0x7ff758155580 2024-05-10T03:31:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.444+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff758159360 con 0x7ff758155580 2024-05-10T03:31:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.445+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff758159920 con 0x7ff758155580 2024-05-10T03:31:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.446+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758159e40 con 0x7ff758155580 2024-05-10T03:31:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.448+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff754005430 con 0x7ff758155580 2024-05-10T03:31:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.449+0000 7ff752000640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 0x7ff73c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.449+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7ff7540a9210 con 0x7ff758155580 2024-05-10T03:31:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.450+0000 7ff75de00640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 0x7ff73c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.451+0000 7ff75de00640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 0x7ff73c06d110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff758155010 tx=0x7ff74c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:16.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.454+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff758159e40 con 0x7ff758155580 2024-05-10T03:31:16.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:16.653+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff758159e40 con 0x7ff758155580 2024-05-10T03:31:16.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:31:16.798 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:31:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.125+0000 7ff752000640 1 -- 172.21.15.55:0/953850512 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff758159e40 con 0x7ff758155580 2024-05-10T03:31:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data_ec' removed 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.129+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 msgr2=0x7ff73c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.129+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 0x7ff73c06d110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff758155010 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.129+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 msgr2=0x7ff75815a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.129+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff75402a8c0 tx=0x7ff7540014a0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 shutdown_connections 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff758155580 0x7ff75815a4a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff758158b20 0x7ff758155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73c06ac00 0x7ff73c06d110 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff758155010 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2024-05-10T03:31:17.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 --2- 172.21.15.55:0/953850512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff75813efa0 0x7ff7581585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.130+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 >> 172.21.15.55:0/953850512 conn(0x7ff7580b5390 msgr2=0x7ff75815c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:17.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.131+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 shutdown_connections 2024-05-10T03:31:17.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.131+0000 7ff760800640 1 -- 172.21.15.55:0/953850512 wait complete. 2024-05-10T03:31:17.166 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:31:17.166 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:31:17.166 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:31:17.166 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:31:17.166 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:31:17.167 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:31:17.167 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:31:17.167 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:31:17.167 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:31:17.167 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:31:17.167 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:31:17.167 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:31:17.167 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:31:17.167 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:31:17.168 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:31:17.168 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:31:17.195 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:31:17.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:31:17.197 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:31:17.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:31:17.198 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:31:17.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:31:17.199 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:31:17.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:31:17.244 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:31:17.244 DEBUG:teuthology.parallel:result is None 2024-05-10T03:31:17.246 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:31:17.247 DEBUG:teuthology.parallel:result is None 2024-05-10T03:31:17.249 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:31:17.249 DEBUG:teuthology.parallel:result is None 2024-05-10T03:31:17.253 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:31:17.253 DEBUG:teuthology.parallel:result is None 2024-05-10T03:31:17.253 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:31:17.464 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:31:17.472 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:31:17.490 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:31:17.490 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:31:17.492 INFO:tasks.ceph.mds.b.smithi162.stderr:2024-05-10T03:31:17.488+0000 7f4101c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-10T03:31:17.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.514+0000 7f58d3200640 1 Processor -- start 2024-05-10T03:31:17.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.515+0000 7f58d3200640 1 -- start start 2024-05-10T03:31:17.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.517+0000 7f58d3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d40d1ba0 0x7f58d40b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.517+0000 7f58d3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58d40ce680 0x7f58d415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.517+0000 7f58d3200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d40b7b50 con 0x7f58d415d700 2024-05-10T03:31:17.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.517+0000 7f58d3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d415e050 con 0x7f58d40ce680 2024-05-10T03:31:17.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.517+0000 7f58d3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40b8ff0 con 0x7f58d40d1ba0 2024-05-10T03:31:17.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.518+0000 7f58d2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58d40ce680 0x7f58d415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.518+0000 7f58d2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58d40ce680 0x7f58d415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36196/0 (socket says 172.21.15.55:36196) 2024-05-10T03:31:17.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.518+0000 7f58d2800640 1 -- 172.21.15.55:0/3004276476 learned_addr learned my addr 172.21.15.55:0/3004276476 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:17.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.518+0000 7f58d1e00640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d40d1ba0 0x7f58d40b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.519+0000 7f58d2800640 1 -- 172.21.15.55:0/3004276476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d40d1ba0 msgr2=0x7f58d40b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.519+0000 7f58d2800640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d40d1ba0 0x7f58d40b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.519+0000 7f58d2800640 1 -- 172.21.15.55:0/3004276476 >> v1:172.21.15.162:6789/0 conn(0x7f58d415d700 legacy=0x7f58d40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.519+0000 7f58d2800640 1 -- 172.21.15.55:0/3004276476 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40b6f40 con 0x7f58d40ce680 2024-05-10T03:31:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.521+0000 7f58d2800640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d40ce680 0x7f58d415db10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009e50 tx=0x7f58cc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4958d961aece3d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:17.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.521+0000 7f58d0a00640 1 -- 172.21.15.55:0/3004276476 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc031600 con 0x7f58d40ce680 2024-05-10T03:31:17.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.522+0000 7f58d0a00640 1 -- 172.21.15.55:0/3004276476 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc02d050 con 0x7f58d40ce680 2024-05-10T03:31:17.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.522+0000 7f58d0a00640 1 -- 172.21.15.55:0/3004276476 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc031980 con 0x7f58d40ce680 2024-05-10T03:31:17.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 -- 172.21.15.55:0/3004276476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d40ce680 msgr2=0x7f58d415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:17.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d40ce680 0x7f58d415db10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009e50 tx=0x7f58cc02a6d0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 -- 172.21.15.55:0/3004276476 shutdown_connections 2024-05-10T03:31:17.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d40ce680 0x7f58d415db10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 --2- 172.21.15.55:0/3004276476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d40d1ba0 0x7f58d40b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 -- 172.21.15.55:0/3004276476 >> 172.21.15.55:0/3004276476 conn(0x7f58d40d0620 msgr2=0x7f58d40d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:17.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 -- 172.21.15.55:0/3004276476 shutdown_connections 2024-05-10T03:31:17.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.523+0000 7f58d3200640 1 -- 172.21.15.55:0/3004276476 wait complete. 2024-05-10T03:31:17.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.526+0000 7f58d3200640 1 Processor -- start 2024-05-10T03:31:17.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.526+0000 7f58d3200640 1 -- start start 2024-05-10T03:31:17.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40d1ba0 0x7f58d41510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d41f4510 0x7f58d40cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f7ca0 con 0x7f58d415d700 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41549e0 con 0x7f58d40d1ba0 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.528+0000 7f58d3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d4154730 con 0x7f58d41f4510 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36202/0 (socket says 172.21.15.55:36202) 2024-05-10T03:31:17.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 -- 172.21.15.55:0/2838332915 learned_addr learned my addr 172.21.15.55:0/2838332915 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:17.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1e00640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40d1ba0 0x7f58d41510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d2800640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d41f4510 0x7f58d40cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 -- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d41f4510 msgr2=0x7f58d40cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:17.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d41f4510 0x7f58d40cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 -- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40d1ba0 msgr2=0x7f58d41510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:17.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40d1ba0 0x7f58d41510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:17.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.529+0000 7f58d1400640 1 -- 172.21.15.55:0/2838332915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc009b30 con 0x7f58d415d700 2024-05-10T03:31:17.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.530+0000 7f58d1400640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b390 tx=0x7f58c000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:17.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.530+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0004230 con 0x7f58d415d700 2024-05-10T03:31:17.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.531+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c0004390 con 0x7f58d415d700 2024-05-10T03:31:17.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.531+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0004500 con 0x7f58d415d700 2024-05-10T03:31:17.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.531+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d41f3d50 con 0x7f58d415d700 2024-05-10T03:31:17.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.531+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d4154c30 con 0x7f58d415d700 2024-05-10T03:31:17.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.533+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d404e290 con 0x7f58d415d700 2024-05-10T03:31:17.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.534+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f58c0011850 con 0x7f58d415d700 2024-05-10T03:31:17.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.535+0000 7f58ca000640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 0x7f58b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:17.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.535+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f58c00890f0 con 0x7f58d415d700 2024-05-10T03:31:17.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.536+0000 7f58d1e00640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 0x7f58b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:17.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.537+0000 7f58d1e00640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 0x7f58b406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009aa0 tx=0x7f58c4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:17.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.540+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f58c00543e0 con 0x7f58d415d700 2024-05-10T03:31:17.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:17.752+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f58d40b3ec0 con 0x7f58d415d700 2024-05-10T03:31:18.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.530+0000 7f58ca000640 1 -- 172.21.15.55:0/2838332915 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f58c0057a00 con 0x7f58d415d700 2024-05-10T03:31:18.534 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 msgr2=0x7f58b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 0x7f58b406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009aa0 tx=0x7f58c4009190 comp rx=0 tx=0).stop 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 msgr2=0x7f58d41f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b390 tx=0x7f58c000b850 comp rx=0 tx=0).stop 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 shutdown_connections 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58d41f4510 0x7f58d40cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58d415d700 0x7f58d41f71f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f58b406ac80 0x7f58b406d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 --2- 172.21.15.55:0/2838332915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58d40d1ba0 0x7f58d41510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.534+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 >> 172.21.15.55:0/2838332915 conn(0x7f58d40d0620 msgr2=0x7f58d40cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.535+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 shutdown_connections 2024-05-10T03:31:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.535+0000 7f58d3200640 1 -- 172.21.15.55:0/2838332915 wait complete. 2024-05-10T03:31:18.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:31:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.843+0000 7f6657e00640 1 Processor -- start 2024-05-10T03:31:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.843+0000 7f6657e00640 1 -- start start 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.845+0000 7f6657e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6658161940 0x7f665815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.845+0000 7f6657e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580d2ee0 con 0x7f66580d20e0 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.845+0000 7f6657e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665806a240 con 0x7f66580b2540 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.845+0000 7f6657e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66580b2950 con 0x7f6658161940 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.846+0000 7f6656a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f66580d20e0 0x7f6658161230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39436/0 (socket says 172.21.15.55:39436) 2024-05-10T03:31:18.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.846+0000 7f6656a00640 1 -- 172.21.15.55:0/2191207161 learned_addr learned my addr 172.21.15.55:0/2191207161 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.846+0000 7f664fe00640 1 --2- 172.21.15.55:0/2191207161 >> v2:172.21.15.162:3300/0 conn(0x7f6658161940 0x7f665815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.846+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157915418 0 0) 0x7f66580d2ee0 con 0x7f66580d20e0 2024-05-10T03:31:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.846+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f663c003520 con 0x7f66580d20e0 2024-05-10T03:31:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1831955605 0 0) 0x7f663c003520 con 0x7f66580d20e0 2024-05-10T03:31:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 >> v2:172.21.15.162:3300/0 conn(0x7f6658161940 msgr2=0x7f665815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 --2- 172.21.15.55:0/2191207161 >> v2:172.21.15.162:3300/0 conn(0x7f6658161940 0x7f665815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 >> v1:172.21.15.162:6789/0 conn(0x7f66580b2540 legacy=0x7f665806a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580d2640 con 0x7f66580d20e0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f6648001670 con 0x7f66580d20e0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 >> v1:172.21.15.55:6789/0 conn(0x7f66580d20e0 legacy=0x7f6658161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f663c0039e0 0x7f663c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f663c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f663c005360 0x7f663c00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580b2950 con 0x7f663c0047c0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c00c2c0 con 0x7f663c005360 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c00c480 con 0x7f663c0039e0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.847+0000 7f664fe00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f663c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f6656a00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f663c0039e0 0x7f663c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f664fe00640 1 -- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f663c0039e0 msgr2=0x7f663c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f664fe00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f663c0039e0 0x7f663c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f664fe00640 1 -- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f663c005360 msgr2=0x7f663c00b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f6657400640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f663c005360 0x7f663c00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f664fe00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f663c005360 0x7f663c00b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f6657400640 1 -- 172.21.15.55:0/2191207161 reap_dead start 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f664fe00640 1 -- 172.21.15.55:0/2191207161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c003520 con 0x7f663c0047c0 2024-05-10T03:31:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.848+0000 7f6656a00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f663c0039e0 0x7f663c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:18.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.849+0000 7f664fe00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f663c004bd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6640009ac0 tx=0x7f663c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c41370aea082de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:18.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.849+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640031400 con 0x7f663c0047c0 2024-05-10T03:31:18.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.849+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664002d050 con 0x7f663c0047c0 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.849+0000 7f6656000640 1 -- 172.21.15.55:0/2191207161 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640031780 con 0x7f663c0047c0 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 -- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 msgr2=0x7f663c004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f663c004bd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6640009ac0 tx=0x7f663c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 -- 172.21.15.55:0/2191207161 shutdown_connections 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 --2- 172.21.15.55:0/2191207161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f663c004bd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 -- 172.21.15.55:0/2191207161 >> 172.21.15.55:0/2191207161 conn(0x7f66580d30d0 msgr2=0x7f66580d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 -- 172.21.15.55:0/2191207161 shutdown_connections 2024-05-10T03:31:18.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.850+0000 7f6657e00640 1 -- 172.21.15.55:0/2191207161 wait complete. 2024-05-10T03:31:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.852+0000 7f6657e00640 1 Processor -- start 2024-05-10T03:31:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.853+0000 7f6657e00640 1 -- start start 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66580c9e60 0x7f66580ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66580ca7b0 0x7f66580cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580a9ab0 con 0x7f663c0047c0 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580ce680 con 0x7f66580c9e60 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6657e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66580a9cb0 con 0x7f66580ca7b0 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6656a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.854+0000 7f6656a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36226/0 (socket says 172.21.15.55:36226) 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 -- 172.21.15.55:0/1324252415 learned_addr learned my addr 172.21.15.55:0/1324252415 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f664fe00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66580c9e60 0x7f66580ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6657400640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66580ca7b0 0x7f66580cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 -- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66580ca7b0 msgr2=0x7f66580cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66580ca7b0 0x7f66580cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 -- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66580c9e60 msgr2=0x7f66580ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66580c9e60 0x7f66580ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 -- 172.21.15.55:0/1324252415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580cc610 con 0x7f663c0047c0 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.855+0000 7f6656a00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f664800d770 tx=0x7f664800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.856+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480072d0 con 0x7f663c0047c0 2024-05-10T03:31:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.856+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648007430 con 0x7f663c0047c0 2024-05-10T03:31:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.856+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480047e0 con 0x7f663c0047c0 2024-05-10T03:31:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.856+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66580cceb0 con 0x7f663c0047c0 2024-05-10T03:31:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.856+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66580cc860 con 0x7f663c0047c0 2024-05-10T03:31:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.857+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665804e290 con 0x7f663c0047c0 2024-05-10T03:31:18.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.858+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6648004940 con 0x7f663c0047c0 2024-05-10T03:31:18.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.859+0000 7f664ea00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 0x7f663406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:18.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.859+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6648088970 con 0x7f663c0047c0 2024-05-10T03:31:18.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.860+0000 7f664fe00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 0x7f663406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:18.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.861+0000 7f664fe00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 0x7f663406d140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6640033000 tx=0x7f6640008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:18.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:18.863+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6648053cc0 con 0x7f663c0047c0 2024-05-10T03:31:19.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.132+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f66580cf1a0 con 0x7f663c0047c0 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.133+0000 7f664ea00640 1 -- 172.21.15.55:0/1324252415 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f6648019020 con 0x7f663c0047c0 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.137 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.138 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:31:19.139 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.141 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.142 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.143 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:31:19.144 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.145 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:31:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mds", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mSjQqLBAAPHvhR8DdaEgSPON/XYECtw==", 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mds" 2024-05-10T03:31:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mgr", 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mP2oqLBAAPyI1YnXFgJB2QuueC2iriA==", 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-osd", 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mWo0qLBAAYpRSbn389YFPC5x9fygC8w==", 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-osd" 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.149 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd", 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mOagqLBAAJn2yZtsrtnIzsYKccWxRJQ==", 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mwsIqLBAAZw7vHIebhuET22aG+xzhpA==", 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.150 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rgw", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQCNlD1mO90qLBAAwDSY6sSCojQA6bEnAGTxSA==", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:31:19.152 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 msgr2=0x7f663406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 0x7f663406d140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6640033000 tx=0x7f6640008030 comp rx=0 tx=0).stop 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 msgr2=0x7f66580aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f664800d770 tx=0x7f664800dc30 comp rx=0 tx=0).stop 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 shutdown_connections 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66580ca7b0 0x7f66580cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f663406ac30 0x7f663406d140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66580c9e60 0x7f66580ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 --2- 172.21.15.55:0/1324252415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f663c0047c0 0x7f66580aa430 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 >> 172.21.15.55:0/1324252415 conn(0x7f66580d30d0 msgr2=0x7f66580b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:19.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.136+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 shutdown_connections 2024-05-10T03:31:19.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.137+0000 7f6657e00640 1 -- 172.21.15.55:0/1324252415 wait complete. 2024-05-10T03:31:19.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-10T03:31:19.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.445+0000 7f1072600640 1 Processor -- start 2024-05-10T03:31:19.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.446+0000 7f1072600640 1 -- start start 2024-05-10T03:31:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.448+0000 7f1072600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 0x7f106c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.448+0000 7f1072600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f106c0b2550 0x7f106c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.448+0000 7f1072600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c06ad20 con 0x7f106c0b2140 2024-05-10T03:31:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.448+0000 7f1072600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c141f50 con 0x7f106c0b2550 2024-05-10T03:31:19.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.448+0000 7f1072600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c06a110 con 0x7f106c15fe50 2024-05-10T03:31:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.449+0000 7f106b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 0x7f106c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.449+0000 7f1063e00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f106c0b2140 0x7f106c142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:38104/0 (socket says 172.21.15.55:38104) 2024-05-10T03:31:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.449+0000 7f106be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f106c0b2550 0x7f106c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.449+0000 7f1063e00640 1 -- 172.21.15.55:0/2617089197 learned_addr learned my addr 172.21.15.55:0/2617089197 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.450+0000 7f106be00640 1 -- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 msgr2=0x7f106c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.450+0000 7f106be00640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 0x7f106c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.450+0000 7f106be00640 1 -- 172.21.15.55:0/2617089197 >> v1:172.21.15.162:6789/0 conn(0x7f106c0b2140 legacy=0x7f106c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.450+0000 7f106be00640 1 -- 172.21.15.55:0/2617089197 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c06a3f0 con 0x7f106c0b2550 2024-05-10T03:31:19.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.450+0000 7f106b400640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 0x7f106c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:19.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.452+0000 7f106be00640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c0b2550 0x7f106c06b1b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1064009d90 tx=0x7f106402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff94f65d1bf9d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:19.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.452+0000 7f106aa00640 1 -- 172.21.15.55:0/2617089197 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007740 con 0x7f106c0b2550 2024-05-10T03:31:19.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.452+0000 7f106aa00640 1 -- 172.21.15.55:0/2617089197 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106402d050 con 0x7f106c0b2550 2024-05-10T03:31:19.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.452+0000 7f106aa00640 1 -- 172.21.15.55:0/2617089197 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007ac0 con 0x7f106c0b2550 2024-05-10T03:31:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.453+0000 7f1072600640 1 -- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c0b2550 msgr2=0x7f106c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.453+0000 7f1072600640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c0b2550 0x7f106c06b1b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1064009d90 tx=0x7f106402a6f0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.453+0000 7f1072600640 1 -- 172.21.15.55:0/2617089197 shutdown_connections 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.453+0000 7f1072600640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c0b2550 0x7f106c06b1b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.453+0000 7f1072600640 1 --2- 172.21.15.55:0/2617089197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c15fe50 0x7f106c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.454+0000 7f1072600640 1 -- 172.21.15.55:0/2617089197 >> 172.21.15.55:0/2617089197 conn(0x7f106c0cbb90 msgr2=0x7f106c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.454+0000 7f1072600640 1 -- 172.21.15.55:0/2617089197 shutdown_connections 2024-05-10T03:31:19.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.454+0000 7f1072600640 1 -- 172.21.15.55:0/2617089197 wait complete. 2024-05-10T03:31:19.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.456+0000 7f1072600640 1 Processor -- start 2024-05-10T03:31:19.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.456+0000 7f1072600640 1 -- start start 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c0b2550 0x7f106c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c15fe50 0x7f106c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c13d150 con 0x7f106c0b2140 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15e120 con 0x7f106c15fe50 2024-05-10T03:31:19.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f1072600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c13c280 con 0x7f106c0b2550 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f106b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f106b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36238/0 (socket says 172.21.15.55:36238) 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f106b400640 1 -- 172.21.15.55:0/861388208 learned_addr learned my addr 172.21.15.55:0/861388208 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.458+0000 7f106be00640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c15fe50 0x7f106c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f1063e00640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c0b2550 0x7f106c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 -- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c0b2550 msgr2=0x7f106c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c0b2550 0x7f106c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 -- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c15fe50 msgr2=0x7f106c143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c15fe50 0x7f106c143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 -- 172.21.15.55:0/861388208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009400 con 0x7f106c0b2140 2024-05-10T03:31:19.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f106b400640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e960 tx=0x7f105c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:19.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00ca00 con 0x7f106c0b2140 2024-05-10T03:31:19.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.460+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1064009a70 con 0x7f106c0b2140 2024-05-10T03:31:19.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.459+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c00cb60 con 0x7f106c0b2140 2024-05-10T03:31:19.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.460+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c013470 con 0x7f106c0b2140 2024-05-10T03:31:19.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.460+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c13c4d0 con 0x7f106c0b2140 2024-05-10T03:31:19.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.461+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c13d3c0 con 0x7f106c0b2140 2024-05-10T03:31:19.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.462+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f105c0135d0 con 0x7f106c0b2140 2024-05-10T03:31:19.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.463+0000 7f1063400640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 0x7f104806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:19.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.463+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f105c089910 con 0x7f106c0b2140 2024-05-10T03:31:19.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.464+0000 7f1063e00640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 0x7f104806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:19.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.465+0000 7f1063e00640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 0x7f104806d110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f106c0cd0a0 tx=0x7f1054009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:19.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.468+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f105c054b50 con 0x7f106c0b2140 2024-05-10T03:31:19.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.734+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f106c066f60 con 0x7f106c0b2140 2024-05-10T03:31:19.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.742+0000 7f1063400640 1 -- 172.21.15.55:0/861388208 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f105c058170 con 0x7f106c0b2140 2024-05-10T03:31:19.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 msgr2=0x7f104806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 0x7f104806d110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f106c0cd0a0 tx=0x7f1054009040 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 msgr2=0x7f106c1f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e960 tx=0x7f105c00ee20 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 shutdown_connections 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f104806ac00 0x7f104806d110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f106c15fe50 0x7f106c143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f106c0b2550 0x7f106c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 --2- 172.21.15.55:0/861388208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f106c0b2140 0x7f106c1f8360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.744+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 >> 172.21.15.55:0/861388208 conn(0x7f106c0cbb90 msgr2=0x7f106c0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.745+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 shutdown_connections 2024-05-10T03:31:19.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:19.745+0000 7f1072600640 1 -- 172.21.15.55:0/861388208 wait complete. 2024-05-10T03:31:19.772 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-10T03:31:20.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.036+0000 7f2297800640 1 Processor -- start 2024-05-10T03:31:20.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.036+0000 7f2297800640 1 -- start start 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.038+0000 7f2297800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f229015fe50 0x7f2290141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.038+0000 7f2297800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b2550 0x7f229006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.038+0000 7f2297800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229006ad20 con 0x7f22900b2140 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.038+0000 7f2297800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2290141f50 con 0x7f22900b2550 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.038+0000 7f2297800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229006a110 con 0x7f229015fe50 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b2550 0x7f229006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22900b2550 0x7f229006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36254/0 (socket says 172.21.15.55:36254) 2024-05-10T03:31:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 -- 172.21.15.55:0/626767776 learned_addr learned my addr 172.21.15.55:0/626767776 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:20.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2294e00640 1 --2- 172.21.15.55:0/626767776 >> v2:172.21.15.162:3300/0 conn(0x7f229015fe50 0x7f2290141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 -- 172.21.15.55:0/626767776 >> v2:172.21.15.162:3300/0 conn(0x7f229015fe50 msgr2=0x7f2290141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:20.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 --2- 172.21.15.55:0/626767776 >> v2:172.21.15.162:3300/0 conn(0x7f229015fe50 0x7f2290141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 -- 172.21.15.55:0/626767776 >> v1:172.21.15.162:6789/0 conn(0x7f22900b2140 legacy=0x7f2290142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:20.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.039+0000 7f2295800640 1 -- 172.21.15.55:0/626767776 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229006a3f0 con 0x7f22900b2550 2024-05-10T03:31:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.040+0000 7f2295800640 1 --2- 172.21.15.55:0/626767776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2550 0x7f229006b1b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f229006e520 tx=0x7f228802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530fa4b483cf5579 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.041+0000 7f228f400640 1 -- 172.21.15.55:0/626767776 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288031600 con 0x7f22900b2550 2024-05-10T03:31:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.041+0000 7f228f400640 1 -- 172.21.15.55:0/626767776 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228802d050 con 0x7f22900b2550 2024-05-10T03:31:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.041+0000 7f228f400640 1 -- 172.21.15.55:0/626767776 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288031980 con 0x7f22900b2550 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.041+0000 7f2297800640 1 -- 172.21.15.55:0/626767776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2550 msgr2=0x7f229006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.041+0000 7f2297800640 1 --2- 172.21.15.55:0/626767776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2550 0x7f229006b1b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f229006e520 tx=0x7f228802a780 comp rx=0 tx=0).stop 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 -- 172.21.15.55:0/626767776 shutdown_connections 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 --2- 172.21.15.55:0/626767776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2550 0x7f229006b1b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 --2- 172.21.15.55:0/626767776 >> v2:172.21.15.162:3300/0 conn(0x7f229015fe50 0x7f2290141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 -- 172.21.15.55:0/626767776 >> 172.21.15.55:0/626767776 conn(0x7f22900cbb90 msgr2=0x7f22900cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 -- 172.21.15.55:0/626767776 shutdown_connections 2024-05-10T03:31:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.042+0000 7f2297800640 1 -- 172.21.15.55:0/626767776 wait complete. 2024-05-10T03:31:20.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.045+0000 7f2297800640 1 Processor -- start 2024-05-10T03:31:20.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.045+0000 7f2297800640 1 -- start start 2024-05-10T03:31:20.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22900b2550 0x7f229015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f229015fe50 0x7f229015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229015b560 con 0x7f22900b2140 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229012f650 con 0x7f229015fe50 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2297800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229015ebb0 con 0x7f22900b2550 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2294e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2294e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36262/0 (socket says 172.21.15.55:36262) 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.047+0000 7f2294e00640 1 -- 172.21.15.55:0/3897111101 learned_addr learned my addr 172.21.15.55:0/3897111101 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f228fe00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22900b2550 0x7f229015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2295800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f229015fe50 0x7f229015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 -- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22900b2550 msgr2=0x7f229015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22900b2550 0x7f229015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 -- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f229015fe50 msgr2=0x7f229015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f229015fe50 0x7f229015b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 -- 172.21.15.55:0/3897111101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009b30 con 0x7f22900b2140 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2294e00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f228400c7f0 tx=0x7f228400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284004220 con 0x7f22900b2140 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2284004380 con 0x7f22900b2140 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900acdb0 con 0x7f22900b2140 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.048+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284007840 con 0x7f22900b2140 2024-05-10T03:31:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.049+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22900ad370 con 0x7f22900b2140 2024-05-10T03:31:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.050+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290066f60 con 0x7f22900b2140 2024-05-10T03:31:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.051+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2284007bb0 con 0x7f22900b2140 2024-05-10T03:31:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.051+0000 7f228cc00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 0x7f227006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.052+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f22840890b0 con 0x7f22900b2140 2024-05-10T03:31:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.052+0000 7f228fe00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 0x7f227006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.054+0000 7f228fe00640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 0x7f227006d140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.056+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2284054410 con 0x7f22900b2140 2024-05-10T03:31:20.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.326+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2290067220 con 0x7f22900b2140 2024-05-10T03:31:20.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.336+0000 7f228cc00640 1 -- 172.21.15.55:0/3897111101 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f228401e0a0 con 0x7f22900b2140 2024-05-10T03:31:20.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.338+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 msgr2=0x7f227006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.338+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 0x7f227006d140 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).stop 2024-05-10T03:31:20.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 msgr2=0x7f229015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f228400c7f0 tx=0x7f228400ccb0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 shutdown_connections 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f227006ac30 0x7f227006d140 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f229015fe50 0x7f229015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22900b2550 0x7f229015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 --2- 172.21.15.55:0/3897111101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22900b2140 0x7f229015e640 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f228400c7f0 tx=0x7f228400ccb0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 >> 172.21.15.55:0/3897111101 conn(0x7f22900cbb90 msgr2=0x7f22900afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 shutdown_connections 2024-05-10T03:31:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.339+0000 7f2297800640 1 -- 172.21.15.55:0/3897111101 wait complete. 2024-05-10T03:31:20.366 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-10T03:31:20.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.641+0000 7fcd96800640 1 Processor -- start 2024-05-10T03:31:20.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.641+0000 7fcd96800640 1 -- start start 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd9015a5d0 0x7fcd900cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcd900cb730 0x7fcd900b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd900b46b0 0x7fcd9015df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900b4ac0 con 0x7fcd9015a5d0 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9015ad30 con 0x7fcd900cb730 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.643+0000 7fcd96800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9015b010 con 0x7fcd900b46b0 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd9015a5d0 0x7fcd900cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd9015a5d0 0x7fcd900cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36274/0 (socket says 172.21.15.55:36274) 2024-05-10T03:31:20.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 -- 172.21.15.55:0/1190854147 learned_addr learned my addr 172.21.15.55:0/1190854147 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:20.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95e00640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd900b46b0 0x7fcd9015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 -- 172.21.15.55:0/1190854147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd900b46b0 msgr2=0x7fcd9015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:20.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd900b46b0 0x7fcd9015df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 -- 172.21.15.55:0/1190854147 >> v2:172.21.15.162:3300/0 conn(0x7fcd900cb730 msgr2=0x7fcd900b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 --2- 172.21.15.55:0/1190854147 >> v2:172.21.15.162:3300/0 conn(0x7fcd900cb730 0x7fcd900b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.644+0000 7fcd95400640 1 -- 172.21.15.55:0/1190854147 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9015b2f0 con 0x7fcd9015a5d0 2024-05-10T03:31:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.645+0000 7fcd95400640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9015a5d0 0x7fcd900cb1f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006edf0 tx=0x7fcd8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc640fa528f104c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.646+0000 7fcd8fe00640 1 -- 172.21.15.55:0/1190854147 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84032440 con 0x7fcd9015a5d0 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.646+0000 7fcd8fe00640 1 -- 172.21.15.55:0/1190854147 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840325a0 con 0x7fcd9015a5d0 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.646+0000 7fcd8fe00640 1 -- 172.21.15.55:0/1190854147 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84032930 con 0x7fcd9015a5d0 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.646+0000 7fcd96800640 1 -- 172.21.15.55:0/1190854147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9015a5d0 msgr2=0x7fcd900cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.646+0000 7fcd96800640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9015a5d0 0x7fcd900cb1f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006edf0 tx=0x7fcd8402a1e0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 -- 172.21.15.55:0/1190854147 shutdown_connections 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd900b46b0 0x7fcd9015df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 --2- 172.21.15.55:0/1190854147 >> v2:172.21.15.162:3300/0 conn(0x7fcd900cb730 0x7fcd900b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 --2- 172.21.15.55:0/1190854147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9015a5d0 0x7fcd900cb1f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 -- 172.21.15.55:0/1190854147 >> 172.21.15.55:0/1190854147 conn(0x7fcd9006a110 msgr2=0x7fcd9006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 -- 172.21.15.55:0/1190854147 shutdown_connections 2024-05-10T03:31:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.647+0000 7fcd96800640 1 -- 172.21.15.55:0/1190854147 wait complete. 2024-05-10T03:31:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.649+0000 7fcd96800640 1 Processor -- start 2024-05-10T03:31:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.649+0000 7fcd96800640 1 -- start start 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd900cb730 0x7fcd900afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 0x7fcd9013f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90147a90 con 0x7fcd900b46b0 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900ce4c0 con 0x7fcd900cb730 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.650+0000 7fcd96800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9013ebd0 con 0x7fcd9015a5d0 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36276/0 (socket says 172.21.15.55:36276) 2024-05-10T03:31:20.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 -- 172.21.15.55:0/3453920949 learned_addr learned my addr 172.21.15.55:0/3453920949 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 -- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 msgr2=0x7fcd9013f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95e00640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 0x7fcd9013f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd94a00640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd900cb730 0x7fcd900afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 0x7fcd9013f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 -- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd900cb730 msgr2=0x7fcd900afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd900cb730 0x7fcd900afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95400640 1 -- 172.21.15.55:0/3453920949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840094d0 con 0x7fcd900b46b0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.651+0000 7fcd95e00640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 0x7fcd9013f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd95400640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84032440 con 0x7fcd900b46b0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84030b80 con 0x7fcd900b46b0 2024-05-10T03:31:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8403d420 con 0x7fcd900b46b0 2024-05-10T03:31:20.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9013e5c0 con 0x7fcd900b46b0 2024-05-10T03:31:20.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.652+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd900ce9f0 con 0x7fcd900b46b0 2024-05-10T03:31:20.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.653+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd90066ce0 con 0x7fcd900b46b0 2024-05-10T03:31:20.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.654+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcd84039040 con 0x7fcd900b46b0 2024-05-10T03:31:20.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.655+0000 7fcd8d600640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 0x7fcd7006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:20.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.655+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fcd840a9ed0 con 0x7fcd900b46b0 2024-05-10T03:31:20.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.656+0000 7fcd94a00640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 0x7fcd7006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:20.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.657+0000 7fcd94a00640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 0x7fcd7006d110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd90070480 tx=0x7fcd7c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:20.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.660+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcd840752c0 con 0x7fcd900b46b0 2024-05-10T03:31:20.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.927+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcd9004e290 con 0x7fcd900b46b0 2024-05-10T03:31:20.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.936+0000 7fcd8d600640 1 -- 172.21.15.55:0/3453920949 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcd840788e0 con 0x7fcd900b46b0 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.938+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 msgr2=0x7fcd7006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.938+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 0x7fcd7006d110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd90070480 tx=0x7fcd7c008040 comp rx=0 tx=0).stop 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.938+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 msgr2=0x7fcd90147190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.938+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd840046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 shutdown_connections 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9015a5d0 0x7fcd9013f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd7006ac00 0x7fcd7006d110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd900cb730 0x7fcd900afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 --2- 172.21.15.55:0/3453920949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd900b46b0 0x7fcd90147190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:20.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 >> 172.21.15.55:0/3453920949 conn(0x7fcd9006a110 msgr2=0x7fcd901495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:20.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 shutdown_connections 2024-05-10T03:31:20.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:20.939+0000 7fcd96800640 1 -- 172.21.15.55:0/3453920949 wait complete. 2024-05-10T03:31:20.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-10T03:31:21.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.242+0000 7f485ce00640 1 Processor -- start 2024-05-10T03:31:21.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.242+0000 7f485ce00640 1 -- start start 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f485ce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f4858158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f485ce00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48580708b0 con 0x7f485806a4b0 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f485ce00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485815b940 con 0x7f4858071db0 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f485ce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580b2490 con 0x7f48580b3f70 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f484d400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f485806a4b0 0x7f48580b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39444/0 (socket says 172.21.15.55:39444) 2024-05-10T03:31:21.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f484d400640 1 -- 172.21.15.55:0/360479193 learned_addr learned my addr 172.21.15.55:0/360479193 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.245+0000 7f4856800640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f4858158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694778711 0 0) 0x7f48580708b0 con 0x7f485806a4b0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c005730 con 0x7f485806a4b0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365065907 0 0) 0x7f483c005730 con 0x7f485806a4b0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 msgr2=0x7f4858158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f4858158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 >> v1:172.21.15.162:6789/0 conn(0x7f4858071db0 legacy=0x7f48580ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858070260 con 0x7f485806a4b0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f4840001670 con 0x7f485806a4b0 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 >> v1:172.21.15.55:6789/0 conn(0x7f485806a4b0 legacy=0x7f48580b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f483c005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006110 0x7f483c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.246+0000 7f4855400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c006f10 0x7f483c007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00dd20 con 0x7f483c006f10 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00de90 con 0x7f483c006110 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00e050 con 0x7f48580b3f70 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c006f10 0x7f483c007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f484d400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006110 0x7f483c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4855e00640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f483c005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 -- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 msgr2=0x7f483c005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580b3f70 0x7f483c005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 -- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006110 msgr2=0x7f483c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006110 0x7f483c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f4856800640 1 -- 172.21.15.55:0/360479193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c005730 con 0x7f483c006f10 2024-05-10T03:31:21.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.247+0000 7f484d400640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006110 0x7f483c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:21.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.248+0000 7f4856800640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c006f10 0x7f483c007320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4850009f90 tx=0x7f483c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83e3b3fda524ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.249+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850031610 con 0x7f483c006f10 2024-05-10T03:31:21.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.249+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000f040 con 0x7f483c006f10 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.249+0000 7f4855400640 1 -- 172.21.15.55:0/360479193 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850031970 con 0x7f483c006f10 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 -- 172.21.15.55:0/360479193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c006f10 msgr2=0x7f483c007320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 --2- 172.21.15.55:0/360479193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483c006f10 0x7f483c007320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4850009f90 tx=0x7f483c007a00 comp rx=0 tx=0).stop 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f4856800640 1 -- 172.21.15.55:0/360479193 reap_dead start 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 -- 172.21.15.55:0/360479193 shutdown_connections 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 -- 172.21.15.55:0/360479193 >> 172.21.15.55:0/360479193 conn(0x7f48580cbb90 msgr2=0x7f48580cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 -- 172.21.15.55:0/360479193 shutdown_connections 2024-05-10T03:31:21.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.250+0000 7f485ce00640 1 -- 172.21.15.55:0/360479193 wait complete. 2024-05-10T03:31:21.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.253+0000 7f485ce00640 1 Processor -- start 2024-05-10T03:31:21.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.253+0000 7f485ce00640 1 -- start start 2024-05-10T03:31:21.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 0x7f4858138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580cdd40 0x7f485815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48581375e0 con 0x7f48580ada20 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485813f770 con 0x7f483c006f10 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.254+0000 7f485ce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580ce150 con 0x7f48580cdd40 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36294/0 (socket says 172.21.15.55:36294) 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 -- 172.21.15.55:0/2706172836 learned_addr learned my addr 172.21.15.55:0/2706172836 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f4856800640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580cdd40 0x7f485815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f4855e00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 0x7f4858138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 -- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580cdd40 msgr2=0x7f485815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580cdd40 0x7f485815ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 -- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 msgr2=0x7f4858138ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 0x7f4858138ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f484d400640 1 -- 172.21.15.55:0/2706172836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485813f550 con 0x7f48580ada20 2024-05-10T03:31:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.255+0000 7f4855e00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 0x7f4858138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f484d400640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f484000d800 tx=0x7f484000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48400074e0 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4840007640 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840004a00 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4858140d80 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.256+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485813fca0 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.257+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4858066f60 con 0x7f48580ada20 2024-05-10T03:31:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.258+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4840004b60 con 0x7f48580ada20 2024-05-10T03:31:21.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.259+0000 7f484ea00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 0x7f483406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.260+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4840088e80 con 0x7f48580ada20 2024-05-10T03:31:21.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.260+0000 7f4855e00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 0x7f483406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.261+0000 7f4855e00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 0x7f483406d110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4848000f60 tx=0x7f4848009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.264+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f48400541f0 con 0x7f48580ada20 2024-05-10T03:31:21.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.534+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f48580b5430 con 0x7f48580ada20 2024-05-10T03:31:21.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.547+0000 7f484ea00640 1 -- 172.21.15.55:0/2706172836 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4840019070 con 0x7f48580ada20 2024-05-10T03:31:21.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.549+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 msgr2=0x7f483406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.549+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 0x7f483406d110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4848000f60 tx=0x7f4848009040 comp rx=0 tx=0).stop 2024-05-10T03:31:21.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.549+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 msgr2=0x7f48580cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.549+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f484000d800 tx=0x7f484000dcc0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 shutdown_connections 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48580cdd40 0x7f485815ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48580ada20 0x7f48580cd800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f483406ac00 0x7f483406d110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 --2- 172.21.15.55:0/2706172836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f483c006f10 0x7f4858138ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 >> 172.21.15.55:0/2706172836 conn(0x7f48580cbb90 msgr2=0x7f48580ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 shutdown_connections 2024-05-10T03:31:21.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.550+0000 7f485ce00640 1 -- 172.21.15.55:0/2706172836 wait complete. 2024-05-10T03:31:21.579 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-10T03:31:21.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.855+0000 7f0dd4e00640 1 Processor -- start 2024-05-10T03:31:21.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.855+0000 7f0dd4e00640 1 -- start start 2024-05-10T03:31:21.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.857+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 0x7f0dd0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.857+0000 7f0dd4e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0dd006d7e0 0x7f0dd00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.857+0000 7f0dd4e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd006d350 con 0x7f0dd00b2fc0 2024-05-10T03:31:21.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.857+0000 7f0dd4e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd006c740 con 0x7f0dd006d7e0 2024-05-10T03:31:21.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.857+0000 7f0dd4e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd006ca20 con 0x7f0dd0144710 2024-05-10T03:31:21.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.858+0000 7f0dcf400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 0x7f0dd0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.858+0000 7f0dcfe00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0dd00b2fc0 0x7f0dd015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:38128/0 (socket says 172.21.15.55:38128) 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.858+0000 7f0dcfe00640 1 -- 172.21.15.55:0/621438265 learned_addr learned my addr 172.21.15.55:0/621438265 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.858+0000 7f0dc7e00640 1 --2- 172.21.15.55:0/621438265 >> v2:172.21.15.162:3300/0 conn(0x7f0dd006d7e0 0x7f0dd00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.859+0000 7f0dcf400640 1 -- 172.21.15.55:0/621438265 >> v2:172.21.15.162:3300/0 conn(0x7f0dd006d7e0 msgr2=0x7f0dd00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.859+0000 7f0dcf400640 1 --2- 172.21.15.55:0/621438265 >> v2:172.21.15.162:3300/0 conn(0x7f0dd006d7e0 0x7f0dd00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.859+0000 7f0dcf400640 1 -- 172.21.15.55:0/621438265 >> v1:172.21.15.162:6789/0 conn(0x7f0dd00b2fc0 legacy=0x7f0dd015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.859+0000 7f0dcf400640 1 -- 172.21.15.55:0/621438265 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd006cd00 con 0x7f0dd0144710 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.859+0000 7f0dcea00640 1 -- 172.21.15.55:0/621438265 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482917268 0 0) 0x7f0dd006d350 con 0x7f0dd00b2fc0 2024-05-10T03:31:21.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.861+0000 7f0dcf400640 1 --2- 172.21.15.55:0/621438265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 0x7f0dd0144120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097a0 tx=0x7f0dc002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba9a5910ec6e331 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.861+0000 7f0dcea00640 1 -- 172.21.15.55:0/621438265 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc002ae40 con 0x7f0dd0144710 2024-05-10T03:31:21.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.861+0000 7f0dcea00640 1 -- 172.21.15.55:0/621438265 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc0032440 con 0x7f0dd0144710 2024-05-10T03:31:21.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dcea00640 1 -- 172.21.15.55:0/621438265 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00327d0 con 0x7f0dd0144710 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 -- 172.21.15.55:0/621438265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 msgr2=0x7f0dd0144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/621438265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 0x7f0dd0144120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097a0 tx=0x7f0dc002a0f0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 -- 172.21.15.55:0/621438265 shutdown_connections 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/621438265 >> v2:172.21.15.162:3300/0 conn(0x7f0dd006d7e0 0x7f0dd00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/621438265 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd0144710 0x7f0dd0144120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.862+0000 7f0dd4e00640 1 -- 172.21.15.55:0/621438265 >> 172.21.15.55:0/621438265 conn(0x7f0dd006a110 msgr2=0x7f0dd006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.863+0000 7f0dd4e00640 1 -- 172.21.15.55:0/621438265 shutdown_connections 2024-05-10T03:31:21.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.863+0000 7f0dd4e00640 1 -- 172.21.15.55:0/621438265 wait complete. 2024-05-10T03:31:21.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.865+0000 7f0dd4e00640 1 Processor -- start 2024-05-10T03:31:21.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.865+0000 7f0dd4e00640 1 -- start start 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 0x7f0dd0136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd00b2fc0 0x7f0dd015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0137330 con 0x7f0dd00714d0 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00701e0 con 0x7f0dd006d7e0 2024-05-10T03:31:21.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dd4e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd01f3d30 con 0x7f0dd00b2fc0 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36306/0 (socket says 172.21.15.55:36306) 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dcfe00640 1 -- 172.21.15.55:0/1967128329 learned_addr learned my addr 172.21.15.55:0/1967128329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dcf400640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 0x7f0dd0136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.867+0000 7f0dc7e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd00b2fc0 0x7f0dd015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 -- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd00b2fc0 msgr2=0x7f0dd015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd00b2fc0 0x7f0dd015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 -- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 msgr2=0x7f0dd0136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 0x7f0dd0136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 -- 172.21.15.55:0/1967128329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc0009400 con 0x7f0dd00714d0 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcf400640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 0x7f0dd0136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:21.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dcfe00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c700 tx=0x7f0dc800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800ce10 con 0x7f0dd00714d0 2024-05-10T03:31:21.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800a640 con 0x7f0dd00714d0 2024-05-10T03:31:21.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0136e60 con 0x7f0dd00714d0 2024-05-10T03:31:21.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.868+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800a9f0 con 0x7f0dd00714d0 2024-05-10T03:31:21.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.869+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0070710 con 0x7f0dd00714d0 2024-05-10T03:31:21.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.870+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0066ce0 con 0x7f0dd00714d0 2024-05-10T03:31:21.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.871+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0dc800ab50 con 0x7f0dd00714d0 2024-05-10T03:31:21.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.872+0000 7f0dc7400640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 0x7f0dac06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:21.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.872+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0dc8088620 con 0x7f0dd00714d0 2024-05-10T03:31:21.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.872+0000 7f0dcf400640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 0x7f0dac06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:21.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.874+0000 7f0dcf400640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 0x7f0dac06d140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0034000 tx=0x7f0dc002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:21.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:21.877+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0dc8053970 con 0x7f0dd00714d0 2024-05-10T03:31:22.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.144+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f0dd004e290 con 0x7f0dd00714d0 2024-05-10T03:31:22.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.155+0000 7f0dc7400640 1 -- 172.21.15.55:0/1967128329 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0dc80180a0 con 0x7f0dd00714d0 2024-05-10T03:31:22.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 msgr2=0x7f0dac06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 0x7f0dac06d140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0034000 tx=0x7f0dc002e000 comp rx=0 tx=0).stop 2024-05-10T03:31:22.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 msgr2=0x7f0dd00718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c700 tx=0x7f0dc800cbc0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 shutdown_connections 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dd00714d0 0x7f0dd00718e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dd00b2fc0 0x7f0dd015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dac06ac30 0x7f0dac06d140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 --2- 172.21.15.55:0/1967128329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dd006d7e0 0x7f0dd0136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 >> 172.21.15.55:0/1967128329 conn(0x7f0dd006a110 msgr2=0x7f0dd00cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 shutdown_connections 2024-05-10T03:31:22.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.158+0000 7f0dd4e00640 1 -- 172.21.15.55:0/1967128329 wait complete. 2024-05-10T03:31:22.184 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-10T03:31:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.461+0000 7f8652600640 1 Processor -- start 2024-05-10T03:31:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.462+0000 7f8652600640 1 -- start start 2024-05-10T03:31:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f8652600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f864c15fe50 0x7f864c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f8652600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f864c0b2140 0x7f864c142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f8652600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c06ad20 con 0x7f864c0b2550 2024-05-10T03:31:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f8652600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c141f50 con 0x7f864c0b2140 2024-05-10T03:31:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f8652600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c06a110 con 0x7f864c15fe50 2024-05-10T03:31:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f864aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f864c0b2140 0x7f864c142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f864be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f864c0b2550 0x7f864c06b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39450/0 (socket says 172.21.15.55:39450) 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.464+0000 7f864be00640 1 -- 172.21.15.55:0/273638088 learned_addr learned my addr 172.21.15.55:0/273638088 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.465+0000 7f864b400640 1 --2- 172.21.15.55:0/273638088 >> v2:172.21.15.162:3300/0 conn(0x7f864c15fe50 0x7f864c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.465+0000 7f864aa00640 1 -- 172.21.15.55:0/273638088 >> v2:172.21.15.162:3300/0 conn(0x7f864c15fe50 msgr2=0x7f864c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.465+0000 7f864aa00640 1 --2- 172.21.15.55:0/273638088 >> v2:172.21.15.162:3300/0 conn(0x7f864c15fe50 0x7f864c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.465+0000 7f864aa00640 1 -- 172.21.15.55:0/273638088 >> v1:172.21.15.55:6789/0 conn(0x7f864c0b2550 legacy=0x7f864c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.465+0000 7f864aa00640 1 -- 172.21.15.55:0/273638088 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c06a3f0 con 0x7f864c0b2140 2024-05-10T03:31:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.466+0000 7f864aa00640 1 --2- 172.21.15.55:0/273638088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c142440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f864c06e520 tx=0x7f863802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9378ec0f81c28dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.466+0000 7f864a000640 1 -- 172.21.15.55:0/273638088 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638032440 con 0x7f864c0b2140 2024-05-10T03:31:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f864a000640 1 -- 172.21.15.55:0/273638088 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380325a0 con 0x7f864c0b2140 2024-05-10T03:31:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f864a000640 1 -- 172.21.15.55:0/273638088 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638032930 con 0x7f864c0b2140 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 -- 172.21.15.55:0/273638088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 msgr2=0x7f864c142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 --2- 172.21.15.55:0/273638088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c142440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f864c06e520 tx=0x7f863802a250 comp rx=0 tx=0).stop 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 -- 172.21.15.55:0/273638088 shutdown_connections 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 --2- 172.21.15.55:0/273638088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c142440 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 --2- 172.21.15.55:0/273638088 >> v2:172.21.15.162:3300/0 conn(0x7f864c15fe50 0x7f864c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 -- 172.21.15.55:0/273638088 >> 172.21.15.55:0/273638088 conn(0x7f864c0cbb90 msgr2=0x7f864c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 -- 172.21.15.55:0/273638088 shutdown_connections 2024-05-10T03:31:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.467+0000 7f8652600640 1 -- 172.21.15.55:0/273638088 wait complete. 2024-05-10T03:31:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.470+0000 7f8652600640 1 Processor -- start 2024-05-10T03:31:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.470+0000 7f8652600640 1 -- start start 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f864c0b2550 0x7f864c143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 0x7f864c1e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1e6870 con 0x7f864c0b2140 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1e50a0 con 0x7f864c15fe50 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f8652600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c070590 con 0x7f864c0b2550 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f864b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f864b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36320/0 (socket says 172.21.15.55:36320) 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.472+0000 7f864b400640 1 -- 172.21.15.55:0/605419810 learned_addr learned my addr 172.21.15.55:0/605419810 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864be00640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 0x7f864c1e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 -- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f864c0b2550 msgr2=0x7f864c143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f864c0b2550 0x7f864c143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 -- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 msgr2=0x7f864c1e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 0x7f864c1e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 -- 172.21.15.55:0/605419810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8638009400 con 0x7f864c0b2140 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864be00640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 0x7f864c1e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f864b400640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f863c00d5f0 tx=0x7f863c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0125f0 con 0x7f864c0b2140 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c012750 con 0x7f864c0b2140 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c070840 con 0x7f864c0b2140 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.473+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c012b00 con 0x7f864c0b2140 2024-05-10T03:31:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.474+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c070e00 con 0x7f864c0b2140 2024-05-10T03:31:22.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.475+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c066f60 con 0x7f864c0b2140 2024-05-10T03:31:22.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.476+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f863c012c60 con 0x7f864c0b2140 2024-05-10T03:31:22.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.477+0000 7f8643400640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 0x7f862c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:22.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.477+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f863c0897b0 con 0x7f864c0b2140 2024-05-10T03:31:22.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.478+0000 7f864aa00640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 0x7f862c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:22.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.479+0000 7f864aa00640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 0x7f862c06d110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f863802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:22.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.485+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f863c054a70 con 0x7f864c0b2140 2024-05-10T03:31:22.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.758+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f864c071140 con 0x7f864c0b2140 2024-05-10T03:31:22.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.768+0000 7f8643400640 1 -- 172.21.15.55:0/605419810 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f863c058090 con 0x7f864c0b2140 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 msgr2=0x7f862c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 0x7f862c06d110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f863802c000 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 msgr2=0x7f864c070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f863c00d5f0 tx=0x7f863c00dab0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 shutdown_connections 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f862c06ac00 0x7f862c06d110 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f864c15fe50 0x7f864c1e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f864c0b2550 0x7f864c143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 --2- 172.21.15.55:0/605419810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f864c0b2140 0x7f864c070050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 >> 172.21.15.55:0/605419810 conn(0x7f864c0cbb90 msgr2=0x7f864c0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 shutdown_connections 2024-05-10T03:31:22.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:22.771+0000 7f8652600640 1 -- 172.21.15.55:0/605419810 wait complete. 2024-05-10T03:31:22.803 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:23.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.085+0000 7fc2e3400640 1 Processor -- start 2024-05-10T03:31:23.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.085+0000 7fc2e3400640 1 -- start start 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2dc15fe50 0x7fc2dc141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e3400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc2dc0b2140 0x7fc2dc142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e3400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc06ad20 con 0x7fc2dc0b2550 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e3400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc141f50 con 0x7fc2dc15fe50 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e3400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc06a110 con 0x7fc2dc0b2140 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2dc15fe50 0x7fc2dc141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2dc15fe50 0x7fc2dc141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40140/0 (socket says 172.21.15.55:40140) 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e0a00640 1 -- 172.21.15.55:0/2188635060 learned_addr learned my addr 172.21.15.55:0/2188635060 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e0a00640 1 -- 172.21.15.55:0/2188635060 >> v2:172.21.15.162:3300/0 conn(0x7fc2dc0b2140 msgr2=0x7fc2dc142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.088+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/2188635060 >> v2:172.21.15.162:3300/0 conn(0x7fc2dc0b2140 0x7fc2dc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.089+0000 7fc2e0a00640 1 -- 172.21.15.55:0/2188635060 >> v1:172.21.15.162:6789/0 conn(0x7fc2dc0b2550 legacy=0x7fc2dc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.089+0000 7fc2e0a00640 1 -- 172.21.15.55:0/2188635060 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc06a3f0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.090+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/2188635060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc141a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc1454b0 tx=0x7fc2d402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4755ac7976075764 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.090+0000 7fc2db400640 1 -- 172.21.15.55:0/2188635060 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031600 con 0x7fc2dc15fe50 2024-05-10T03:31:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.090+0000 7fc2db400640 1 -- 172.21.15.55:0/2188635060 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d402d050 con 0x7fc2dc15fe50 2024-05-10T03:31:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.090+0000 7fc2db400640 1 -- 172.21.15.55:0/2188635060 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031980 con 0x7fc2dc15fe50 2024-05-10T03:31:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 -- 172.21.15.55:0/2188635060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 msgr2=0x7fc2dc141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 --2- 172.21.15.55:0/2188635060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc141a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc1454b0 tx=0x7fc2d402a780 comp rx=0 tx=0).stop 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 -- 172.21.15.55:0/2188635060 shutdown_connections 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 --2- 172.21.15.55:0/2188635060 >> v2:172.21.15.162:3300/0 conn(0x7fc2dc0b2140 0x7fc2dc142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 --2- 172.21.15.55:0/2188635060 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc141a10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 -- 172.21.15.55:0/2188635060 >> 172.21.15.55:0/2188635060 conn(0x7fc2dc0cbb90 msgr2=0x7fc2dc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 -- 172.21.15.55:0/2188635060 shutdown_connections 2024-05-10T03:31:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.091+0000 7fc2e3400640 1 -- 172.21.15.55:0/2188635060 wait complete. 2024-05-10T03:31:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.094+0000 7fc2e3400640 1 Processor -- start 2024-05-10T03:31:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.094+0000 7fc2e3400640 1 -- start start 2024-05-10T03:31:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 0x7fc2dc1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2dc0b2550 0x7fc2dc1e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc15e150 con 0x7fc2dc15fe50 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc1e6670 con 0x7fc2dc0b2550 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.095+0000 7fc2e3400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc1e5f80 con 0x7fc2dc0b2140 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40152/0 (socket says 172.21.15.55:40152) 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 -- 172.21.15.55:0/359564433 learned_addr learned my addr 172.21.15.55:0/359564433 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2dbe00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2dc0b2550 0x7fc2dc1e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 0x7fc2dc1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 -- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 msgr2=0x7fc2dc1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 0x7fc2dc1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 -- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2dc0b2550 msgr2=0x7fc2dc1e5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2dc0b2550 0x7fc2dc1e5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e1400640 1 -- 172.21.15.55:0/359564433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4009b30 con 0x7fc2dc15fe50 2024-05-10T03:31:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.096+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 0x7fc2dc1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2e1400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00b490 tx=0x7fc2cc00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011830 con 0x7fc2dc15fe50 2024-05-10T03:31:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc0119b0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011d60 con 0x7fc2dc15fe50 2024-05-10T03:31:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc1e7dd0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.097+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc15db90 con 0x7fc2dc15fe50 2024-05-10T03:31:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.098+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc1e67b0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.099+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc2cc01a400 con 0x7fc2dc15fe50 2024-05-10T03:31:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.100+0000 7fc2d8c00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 0x7fc2b806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.100+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc2cc089310 con 0x7fc2dc15fe50 2024-05-10T03:31:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.101+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 0x7fc2b806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.102+0000 7fc2e0a00640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 0x7fc2b806d110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4033000 tx=0x7fc2d4009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.105+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc2dc1e67b0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.308+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2dc1e67b0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.309+0000 7fc2d8c00640 1 -- 172.21.15.55:0/359564433 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12594 (secure 0 0 0) 0x7fc2dc1e67b0 con 0x7fc2dc15fe50 2024-05-10T03:31:23.313 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:23.313 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":36,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:18.516770+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 msgr2=0x7fc2b806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 0x7fc2b806d110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4033000 tx=0x7fc2d4009000 comp rx=0 tx=0).stop 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 msgr2=0x7fc2dc1f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00b490 tx=0x7fc2cc00b950 comp rx=0 tx=0).stop 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 shutdown_connections 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2dc15fe50 0x7fc2dc1f74e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc2b806ac00 0x7fc2b806d110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2dc0b2550 0x7fc2dc1e5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.312+0000 7fc2e3400640 1 --2- 172.21.15.55:0/359564433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc2dc0b2140 0x7fc2dc1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.313+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 >> 172.21.15.55:0/359564433 conn(0x7fc2dc0cbb90 msgr2=0x7fc2dc141000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.313+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 shutdown_connections 2024-05-10T03:31:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.313+0000 7fc2e3400640 1 -- 172.21.15.55:0/359564433 wait complete. 2024-05-10T03:31:23.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:31:23.345 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:23.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.630+0000 7f260ce00640 1 Processor -- start 2024-05-10T03:31:23.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.631+0000 7f260ce00640 1 -- start start 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f260806b770 0x7f260815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 0x7f26080ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26080b62e0 con 0x7f260806b770 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260806b2e0 con 0x7f2608161310 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.632+0000 7f260ce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260815dbc0 con 0x7f260806a110 2024-05-10T03:31:23.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.633+0000 7f2607400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f260806b770 0x7f260815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.633+0000 7f2607400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f260806b770 0x7f260815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40156/0 (socket says 172.21.15.55:40156) 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.633+0000 7f2607400640 1 -- 172.21.15.55:0/3499939318 learned_addr learned my addr 172.21.15.55:0/3499939318 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.633+0000 7f2606a00640 1 --2- 172.21.15.55:0/3499939318 >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 0x7f26080ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.633+0000 7f2607e00640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2607400640 1 -- 172.21.15.55:0/3499939318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 msgr2=0x7f26080ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2607400640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2607400640 1 -- 172.21.15.55:0/3499939318 >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 msgr2=0x7f26080ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:23.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2607400640 1 --2- 172.21.15.55:0/3499939318 >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 0x7f26080ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2607400640 1 -- 172.21.15.55:0/3499939318 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260806a6d0 con 0x7f260806b770 2024-05-10T03:31:23.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.634+0000 7f2606a00640 1 --2- 172.21.15.55:0/3499939318 >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 0x7f26080ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:23.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.635+0000 7f2607400640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f260815d680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f260806c940 tx=0x7f25f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30a60a61078ad6b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.636+0000 7f2606000640 1 -- 172.21.15.55:0/3499939318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8032440 con 0x7f260806b770 2024-05-10T03:31:23.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.636+0000 7f2606000640 1 -- 172.21.15.55:0/3499939318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f80325a0 con 0x7f260806b770 2024-05-10T03:31:23.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.636+0000 7f2606000640 1 -- 172.21.15.55:0/3499939318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8032930 con 0x7f260806b770 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.636+0000 7f260ce00640 1 -- 172.21.15.55:0/3499939318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 msgr2=0x7f260815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.636+0000 7f260ce00640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f260815d680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f260806c940 tx=0x7f25f802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 -- 172.21.15.55:0/3499939318 shutdown_connections 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 --2- 172.21.15.55:0/3499939318 >> v2:172.21.15.162:3300/0 conn(0x7f2608161310 0x7f26080ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 --2- 172.21.15.55:0/3499939318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f260815d680 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 -- 172.21.15.55:0/3499939318 >> 172.21.15.55:0/3499939318 conn(0x7f26080d3140 msgr2=0x7f26080d3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 -- 172.21.15.55:0/3499939318 shutdown_connections 2024-05-10T03:31:23.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.637+0000 7f260ce00640 1 -- 172.21.15.55:0/3499939318 wait complete. 2024-05-10T03:31:23.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.639+0000 7f260ce00640 1 Processor -- start 2024-05-10T03:31:23.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.639+0000 7f260ce00640 1 -- start start 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 0x7f26080cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608155220 con 0x7f260806b770 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608157070 con 0x7f2608161310 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f260ce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2608147260 con 0x7f260806a110 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f2606a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f2606a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40172/0 (socket says 172.21.15.55:40172) 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.641+0000 7f2606a00640 1 -- 172.21.15.55:0/4285544749 learned_addr learned my addr 172.21.15.55:0/4285544749 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2607400640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 -- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 msgr2=0x7f26080aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2607e00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 0x7f26080cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 -- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 msgr2=0x7f26080cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 0x7f26080cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 -- 172.21.15.55:0/4285544749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80094d0 con 0x7f260806b770 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2607400640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2607e00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 0x7f26080cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f2606a00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25f000d7d0 tx=0x7f25f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0004030 con 0x7f260806b770 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0004190 con 0x7f260806b770 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0004520 con 0x7f260806b770 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.642+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26080ab010 con 0x7f260806b770 2024-05-10T03:31:23.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.643+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081474b0 con 0x7f260806b770 2024-05-10T03:31:23.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.644+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2608155490 con 0x7f260806b770 2024-05-10T03:31:23.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.645+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f25f00096a0 con 0x7f260806b770 2024-05-10T03:31:23.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.646+0000 7f25ff400640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 0x7f25e406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:23.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.646+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f25f0089330 con 0x7f260806b770 2024-05-10T03:31:23.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.646+0000 7f2607400640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 0x7f25e406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:23.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.648+0000 7f2607400640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 0x7f25e406d110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25f8034000 tx=0x7f25f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:23.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.650+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f25f0054720 con 0x7f260806b770 2024-05-10T03:31:23.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.852+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f260804e290 con 0x7f260806b770 2024-05-10T03:31:23.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.853+0000 7f25ff400640 1 -- 172.21.15.55:0/4285544749 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25f0057d40 con 0x7f260806b770 2024-05-10T03:31:23.857 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:23.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 msgr2=0x7f25e406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 0x7f25e406d110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25f8034000 tx=0x7f25f802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 msgr2=0x7f26080a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25f000d7d0 tx=0x7f25f000dc90 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 shutdown_connections 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f25e406ac00 0x7f25e406d110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608161310 0x7f26080cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f260806b770 0x7f26080a9f40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 --2- 172.21.15.55:0/4285544749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f260806a110 0x7f26080aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 >> 172.21.15.55:0/4285544749 conn(0x7f26080d3140 msgr2=0x7f260815c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 shutdown_connections 2024-05-10T03:31:23.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:23.856+0000 7f260ce00640 1 -- 172.21.15.55:0/4285544749 wait complete. 2024-05-10T03:31:23.886 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:31:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.167+0000 7efeb5600640 1 Processor -- start 2024-05-10T03:31:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.167+0000 7efeb5600640 1 -- start start 2024-05-10T03:31:24.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.169+0000 7efeb5600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efeb015fe50 0x7efeb0141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.169+0000 7efeb5600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efeb00b2140 0x7efeb0142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.169+0000 7efeb5600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb006ad20 con 0x7efeb00b2550 2024-05-10T03:31:24.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.169+0000 7efeb5600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb0141f50 con 0x7efeb00b2140 2024-05-10T03:31:24.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.169+0000 7efeb5600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb006a110 con 0x7efeb015fe50 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.170+0000 7efeae000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efeb00b2140 0x7efeb0142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.170+0000 7efeaf400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efeb00b2550 0x7efeb006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35388/0 (socket says 172.21.15.55:35388) 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.170+0000 7efeaf400640 1 -- 172.21.15.55:0/56484167 learned_addr learned my addr 172.21.15.55:0/56484167 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.170+0000 7efeaea00640 1 --2- 172.21.15.55:0/56484167 >> v2:172.21.15.162:3300/0 conn(0x7efeb015fe50 0x7efeb0141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.171+0000 7efeae000640 1 -- 172.21.15.55:0/56484167 >> v2:172.21.15.162:3300/0 conn(0x7efeb015fe50 msgr2=0x7efeb0141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.171+0000 7efeae000640 1 --2- 172.21.15.55:0/56484167 >> v2:172.21.15.162:3300/0 conn(0x7efeb015fe50 0x7efeb0141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.171+0000 7efeae000640 1 -- 172.21.15.55:0/56484167 >> v1:172.21.15.55:6789/0 conn(0x7efeb00b2550 legacy=0x7efeb006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.171+0000 7efeae000640 1 -- 172.21.15.55:0/56484167 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb006a3f0 con 0x7efeb00b2140 2024-05-10T03:31:24.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.171+0000 7efead600640 1 -- 172.21.15.55:0/56484167 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488536976 0 0) 0x7efeb006ad20 con 0x7efeb00b2550 2024-05-10T03:31:24.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.172+0000 7efeae000640 1 --2- 172.21.15.55:0/56484167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb0142440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efeb006e520 tx=0x7efe9802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d11300972a0bce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.172+0000 7efead600640 1 -- 172.21.15.55:0/56484167 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98007740 con 0x7efeb00b2140 2024-05-10T03:31:24.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.172+0000 7efead600640 1 -- 172.21.15.55:0/56484167 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9802d050 con 0x7efeb00b2140 2024-05-10T03:31:24.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.172+0000 7efead600640 1 -- 172.21.15.55:0/56484167 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98007ac0 con 0x7efeb00b2140 2024-05-10T03:31:24.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 -- 172.21.15.55:0/56484167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 msgr2=0x7efeb0142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 --2- 172.21.15.55:0/56484167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb0142440 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efeb006e520 tx=0x7efe9802a710 comp rx=0 tx=0).stop 2024-05-10T03:31:24.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 -- 172.21.15.55:0/56484167 shutdown_connections 2024-05-10T03:31:24.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 --2- 172.21.15.55:0/56484167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb0142440 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 --2- 172.21.15.55:0/56484167 >> v2:172.21.15.162:3300/0 conn(0x7efeb015fe50 0x7efeb0141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 -- 172.21.15.55:0/56484167 >> 172.21.15.55:0/56484167 conn(0x7efeb00cbb90 msgr2=0x7efeb00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:24.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 -- 172.21.15.55:0/56484167 shutdown_connections 2024-05-10T03:31:24.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.173+0000 7efeb5600640 1 -- 172.21.15.55:0/56484167 wait complete. 2024-05-10T03:31:24.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.176+0000 7efeb5600640 1 Processor -- start 2024-05-10T03:31:24.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.176+0000 7efeb5600640 1 -- start start 2024-05-10T03:31:24.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.177+0000 7efeb5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.177+0000 7efeb5600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 0x7efeb00ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.177+0000 7efeb5600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb015fe50 0x7efeb01e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeb5600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00ac270 con 0x7efeb00b2140 2024-05-10T03:31:24.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeb5600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01e5890 con 0x7efeb00b2550 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeb5600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01e60a0 con 0x7efeb015fe50 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40194/0 (socket says 172.21.15.55:40194) 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 -- 172.21.15.55:0/3540985457 learned_addr learned my addr 172.21.15.55:0/3540985457 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeae000640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 0x7efeb00ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 -- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb015fe50 msgr2=0x7efeb01e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb015fe50 0x7efeb01e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 -- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 msgr2=0x7efeb00ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 0x7efeb00ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeaea00640 1 -- 172.21.15.55:0/3540985457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98009a70 con 0x7efeb00b2140 2024-05-10T03:31:24.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.178+0000 7efeae000640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 0x7efeb00ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efeaea00640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efea000e870 tx=0x7efea000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00094f0 con 0x7efeb00b2140 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea000c940 con 0x7efeb00b2140 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000ccf0 con 0x7efeb00b2140 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb00a9c90 con 0x7efeb00b2140 2024-05-10T03:31:24.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.179+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01e5dc0 con 0x7efeb00b2140 2024-05-10T03:31:24.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.181+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0066f60 con 0x7efeb00b2140 2024-05-10T03:31:24.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.182+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efea0013470 con 0x7efeb00b2140 2024-05-10T03:31:24.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.183+0000 7efea6a00640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 0x7efe8c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.183+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7efea0089230 con 0x7efeb00b2140 2024-05-10T03:31:24.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.184+0000 7efeae000640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 0x7efe8c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.186+0000 7efeae000640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 0x7efe8c06d110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe98033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.188+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efea0054620 con 0x7efeb00b2140 2024-05-10T03:31:24.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.392+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efeb0067220 con 0x7efeb00b2140 2024-05-10T03:31:24.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.393+0000 7efea6a00640 1 -- 172.21.15.55:0/3540985457 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7efea0057c40 con 0x7efeb00b2140 2024-05-10T03:31:24.396 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:31:24.397 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:31:24.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 msgr2=0x7efe8c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 0x7efe8c06d110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe98033040 comp rx=0 tx=0).stop 2024-05-10T03:31:24.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 msgr2=0x7efeb013ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efea000e870 tx=0x7efea000ed30 comp rx=0 tx=0).stop 2024-05-10T03:31:24.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 shutdown_connections 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb015fe50 0x7efeb01e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe8c06ac00 0x7efe8c06d110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb00b2550 0x7efeb00ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.396+0000 7efeb5600640 1 --2- 172.21.15.55:0/3540985457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeb00b2140 0x7efeb013ee80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.397+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 >> 172.21.15.55:0/3540985457 conn(0x7efeb00cbb90 msgr2=0x7efeb015efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.397+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 shutdown_connections 2024-05-10T03:31:24.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.397+0000 7efeb5600640 1 -- 172.21.15.55:0/3540985457 wait complete. 2024-05-10T03:31:24.429 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:31:24.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:31:24.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.714+0000 7f8e85600640 1 Processor -- start 2024-05-10T03:31:24.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.715+0000 7f8e85600640 1 -- start start 2024-05-10T03:31:24.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e85600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e8015f210 0x7f8e8006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e85600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e8006b840 0x7f8e800cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e85600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e80071b80 con 0x7f8e80071e70 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e85600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80072280 con 0x7f8e8006b840 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e85600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8015bc20 con 0x7f8e8015f210 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.717+0000 7f8e84c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e8006b840 0x7f8e800cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e8006b840 0x7f8e800cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40196/0 (socket says 172.21.15.55:40196) 2024-05-10T03:31:24.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 -- 172.21.15.55:0/1508063408 learned_addr learned my addr 172.21.15.55:0/1508063408 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e77e00640 1 --2- 172.21.15.55:0/1508063408 >> v2:172.21.15.162:3300/0 conn(0x7f8e8015f210 0x7f8e8006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 -- 172.21.15.55:0/1508063408 >> v2:172.21.15.162:3300/0 conn(0x7f8e8015f210 msgr2=0x7f8e8006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 --2- 172.21.15.55:0/1508063408 >> v2:172.21.15.162:3300/0 conn(0x7f8e8015f210 0x7f8e8006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 -- 172.21.15.55:0/1508063408 >> v1:172.21.15.162:6789/0 conn(0x7f8e80071e70 legacy=0x7f8e8015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.718+0000 7f8e84c00640 1 -- 172.21.15.55:0/1508063408 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8015ca60 con 0x7f8e8006b840 2024-05-10T03:31:24.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.720+0000 7f8e84c00640 1 --2- 172.21.15.55:0/1508063408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800cb900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e800cee80 tx=0x7f8e7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae44bb658c1a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.720+0000 7f8e7f400640 1 -- 172.21.15.55:0/1508063408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78031600 con 0x7f8e8006b840 2024-05-10T03:31:24.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.720+0000 7f8e7f400640 1 -- 172.21.15.55:0/1508063408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7802d050 con 0x7f8e8006b840 2024-05-10T03:31:24.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.720+0000 7f8e7f400640 1 -- 172.21.15.55:0/1508063408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78031980 con 0x7f8e8006b840 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 -- 172.21.15.55:0/1508063408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 msgr2=0x7f8e800cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 --2- 172.21.15.55:0/1508063408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800cb900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e800cee80 tx=0x7f8e7802a6a0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 -- 172.21.15.55:0/1508063408 shutdown_connections 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 --2- 172.21.15.55:0/1508063408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800cb900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 --2- 172.21.15.55:0/1508063408 >> v2:172.21.15.162:3300/0 conn(0x7f8e8015f210 0x7f8e8006b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 -- 172.21.15.55:0/1508063408 >> 172.21.15.55:0/1508063408 conn(0x7f8e8006c070 msgr2=0x7f8e8006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 -- 172.21.15.55:0/1508063408 shutdown_connections 2024-05-10T03:31:24.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.721+0000 7f8e85600640 1 -- 172.21.15.55:0/1508063408 wait complete. 2024-05-10T03:31:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.723+0000 7f8e85600640 1 Processor -- start 2024-05-10T03:31:24.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.724+0000 7f8e85600640 1 -- start start 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e80071e70 0x7f8e801f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8015f210 0x7f8e8013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801406b0 con 0x7f8e8006b840 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801f41d0 con 0x7f8e8015f210 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e85600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800a8840 con 0x7f8e80071e70 2024-05-10T03:31:24.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e7fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e7fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40204/0 (socket says 172.21.15.55:40204) 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.725+0000 7f8e7fe00640 1 -- 172.21.15.55:0/3276926038 learned_addr learned my addr 172.21.15.55:0/3276926038 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e84c00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8015f210 0x7f8e8013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 -- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e80071e70 msgr2=0x7f8e801f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e80071e70 0x7f8e801f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 -- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8015f210 msgr2=0x7f8e8013fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8015f210 0x7f8e8013fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 -- 172.21.15.55:0/3276926038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009b30 con 0x7f8e8006b840 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7fe00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c400 tx=0x7f8e7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.726+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700052d0 con 0x7f8e8006b840 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.727+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e70005430 con 0x7f8e8006b840 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.727+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70003830 con 0x7f8e8006b840 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.727+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e800a9460 con 0x7f8e8006b840 2024-05-10T03:31:24.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.727+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801f3d30 con 0x7f8e8006b840 2024-05-10T03:31:24.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.728+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e80067220 con 0x7f8e8006b840 2024-05-10T03:31:24.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.729+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e70003990 con 0x7f8e8006b840 2024-05-10T03:31:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.731+0000 7f8e7cc00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 0x7f8e5c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.731+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8e70088660 con 0x7f8e8006b840 2024-05-10T03:31:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.731+0000 7f8e77e00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 0x7f8e5c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.733+0000 7f8e77e00640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 0x7f8e5c06d110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e80140a10 tx=0x7f8e68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.735+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8e70053920 con 0x7f8e8006b840 2024-05-10T03:31:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:24.935+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e800a9460 con 0x7f8e8006b840 2024-05-10T03:31:25.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.793+0000 7f8e7cc00640 1 -- 172.21.15.55:0/3276926038 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e800a9460 con 0x7f8e8006b840 2024-05-10T03:31:25.797 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:31:25.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 msgr2=0x7f8e5c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:25.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 0x7f8e5c06d110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e80140a10 tx=0x7f8e68008040 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 msgr2=0x7f8e800a8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000c400 tx=0x7f8e7000c8c0 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 shutdown_connections 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e5c06ac00 0x7f8e5c06d110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8015f210 0x7f8e8013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e80071e70 0x7f8e801f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 --2- 172.21.15.55:0/3276926038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8006b840 0x7f8e800a8f20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 >> 172.21.15.55:0/3276926038 conn(0x7f8e8006c070 msgr2=0x7f8e80070230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.798+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 shutdown_connections 2024-05-10T03:31:25.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:25.799+0000 7f8e85600640 1 -- 172.21.15.55:0/3276926038 wait complete. 2024-05-10T03:31:25.836 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:31:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.105+0000 7fba1e200640 1 Processor -- start 2024-05-10T03:31:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.105+0000 7fba1e200640 1 -- start start 2024-05-10T03:31:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba1e200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fba1815e260 0x7fba180ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba1e200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba180cc620 0x7fba180706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba1e200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1815ad80 con 0x7fba1806b120 2024-05-10T03:31:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba1e200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba180cee60 con 0x7fba180cc620 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba1e200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba18072250 con 0x7fba1815e260 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba16a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba180cc620 0x7fba180706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba16a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba180cc620 0x7fba180706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40218/0 (socket says 172.21.15.55:40218) 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.108+0000 7fba16a00640 1 -- 172.21.15.55:0/38522525 learned_addr learned my addr 172.21.15.55:0/38522525 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.109+0000 7fba16a00640 1 -- 172.21.15.55:0/38522525 >> v2:172.21.15.162:3300/0 conn(0x7fba1815e260 msgr2=0x7fba180ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.109+0000 7fba16a00640 1 --2- 172.21.15.55:0/38522525 >> v2:172.21.15.162:3300/0 conn(0x7fba1815e260 0x7fba180ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.109+0000 7fba16a00640 1 -- 172.21.15.55:0/38522525 >> v1:172.21.15.162:6789/0 conn(0x7fba1806b120 legacy=0x7fba1806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.109+0000 7fba16a00640 1 -- 172.21.15.55:0/38522525 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1815e840 con 0x7fba180cc620 2024-05-10T03:31:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.110+0000 7fba16a00640 1 --2- 172.21.15.55:0/38522525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba180706e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fba180b4440 tx=0x7fba0002c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ebf5b516f3fdc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.110+0000 7fba16000640 1 -- 172.21.15.55:0/38522525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00034640 con 0x7fba180cc620 2024-05-10T03:31:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.110+0000 7fba16000640 1 -- 172.21.15.55:0/38522525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba000347a0 con 0x7fba180cc620 2024-05-10T03:31:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.110+0000 7fba16000640 1 -- 172.21.15.55:0/38522525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00034b30 con 0x7fba180cc620 2024-05-10T03:31:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 -- 172.21.15.55:0/38522525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 msgr2=0x7fba180706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 --2- 172.21.15.55:0/38522525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba180706e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fba180b4440 tx=0x7fba0002c3d0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 -- 172.21.15.55:0/38522525 shutdown_connections 2024-05-10T03:31:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 --2- 172.21.15.55:0/38522525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba180706e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 --2- 172.21.15.55:0/38522525 >> v2:172.21.15.162:3300/0 conn(0x7fba1815e260 0x7fba180ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 -- 172.21.15.55:0/38522525 >> 172.21.15.55:0/38522525 conn(0x7fba1806c120 msgr2=0x7fba1806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:26.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 -- 172.21.15.55:0/38522525 shutdown_connections 2024-05-10T03:31:26.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.111+0000 7fba1e200640 1 -- 172.21.15.55:0/38522525 wait complete. 2024-05-10T03:31:26.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.114+0000 7fba1e200640 1 Processor -- start 2024-05-10T03:31:26.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.114+0000 7fba1e200640 1 -- start start 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.115+0000 7fba1e200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba1806b120 0x7fba1814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.115+0000 7fba1e200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.115+0000 7fba1e200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba1815e260 0x7fba1815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba1e200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba181f52c0 con 0x7fba180cc620 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba1e200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba181e54e0 con 0x7fba1815e260 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba1e200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1814e4b0 con 0x7fba1806b120 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40220/0 (socket says 172.21.15.55:40220) 2024-05-10T03:31:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 -- 172.21.15.55:0/865823713 learned_addr learned my addr 172.21.15.55:0/865823713 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:26.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba17400640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba1806b120 0x7fba1814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 -- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba1806b120 msgr2=0x7fba1814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba1806b120 0x7fba1814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba17e00640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba1815e260 0x7fba1815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 -- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba1815e260 msgr2=0x7fba1815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba1815e260 0x7fba1815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.116+0000 7fba16a00640 1 -- 172.21.15.55:0/865823713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000094d0 con 0x7fba180cc620 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba16a00640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba000097f0 tx=0x7fba000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00004230 con 0x7fba180cc620 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba00004390 con 0x7fba180cc620 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0003f8a0 con 0x7fba180cc620 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba181e5730 con 0x7fba180cc620 2024-05-10T03:31:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.117+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba18150540 con 0x7fba180cc620 2024-05-10T03:31:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.119+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba18066f60 con 0x7fba180cc620 2024-05-10T03:31:26.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.120+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fba0003b070 con 0x7fba180cc620 2024-05-10T03:31:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.121+0000 7fba0f400640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 0x7fb9f406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.121+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fba000aa570 con 0x7fba180cc620 2024-05-10T03:31:26.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.122+0000 7fba17400640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 0x7fb9f406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:26.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.123+0000 7fba17400640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 0x7fb9f406d140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba08007dc0 tx=0x7fba080072f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:26.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.126+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fba00075750 con 0x7fba180cc620 2024-05-10T03:31:26.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.331+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fba18066f60 con 0x7fba180cc620 2024-05-10T03:31:26.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.813+0000 7fba0f400640 1 -- 172.21.15.55:0/865823713 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7fba00078d70 con 0x7fba180cc620 2024-05-10T03:31:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:31:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.820+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 msgr2=0x7fb9f406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.820+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 0x7fb9f406d140 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba08007dc0 tx=0x7fba080072f0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.820+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 msgr2=0x7fba181e49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.820+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fba000097f0 tx=0x7fba000046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 shutdown_connections 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9f406ac30 0x7fb9f406d140 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba08007dc0 tx=0x7fba080072f0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba1815e260 0x7fba1815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba180cc620 0x7fba181e49c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 --2- 172.21.15.55:0/865823713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba1806b120 0x7fba1814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.821+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 >> 172.21.15.55:0/865823713 conn(0x7fba1806c120 msgr2=0x7fba180b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.822+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 shutdown_connections 2024-05-10T03:31:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:26.822+0000 7fba1e200640 1 -- 172.21.15.55:0/865823713 wait complete. 2024-05-10T03:31:26.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:31:27.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.122+0000 7f5a83800640 1 Processor -- start 2024-05-10T03:31:27.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.122+0000 7f5a83800640 1 -- start start 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.124+0000 7f5a83800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5a7c1418c0 0x7f5a7c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.124+0000 7f5a83800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c06d840 0x7f5a7c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.124+0000 7f5a83800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c141cd0 con 0x7f5a7c15ba10 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.124+0000 7f5a83800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c15ca50 con 0x7f5a7c1418c0 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.124+0000 7f5a83800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c15cd10 con 0x7f5a7c06d840 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5a7c1418c0 0x7f5a7c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5a7c1418c0 0x7f5a7c15fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40226/0 (socket says 172.21.15.55:40226) 2024-05-10T03:31:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 -- 172.21.15.55:0/1728646220 learned_addr learned my addr 172.21.15.55:0/1728646220 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 -- 172.21.15.55:0/1728646220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c06d840 msgr2=0x7f5a7c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 --2- 172.21.15.55:0/1728646220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c06d840 0x7f5a7c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 -- 172.21.15.55:0/1728646220 >> v1:172.21.15.162:6789/0 conn(0x7f5a7c15ba10 legacy=0x7f5a7c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.125+0000 7f5a80e00640 1 -- 172.21.15.55:0/1728646220 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c15c1b0 con 0x7f5a7c1418c0 2024-05-10T03:31:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.126+0000 7f5a80e00640 1 --2- 172.21.15.55:0/1728646220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c15fa20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06f670 tx=0x7f5a6c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3e9bc3b1d0d1f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.127+0000 7f5a7b400640 1 -- 172.21.15.55:0/1728646220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c032440 con 0x7f5a7c1418c0 2024-05-10T03:31:27.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.127+0000 7f5a7b400640 1 -- 172.21.15.55:0/1728646220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0325a0 con 0x7f5a7c1418c0 2024-05-10T03:31:27.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.127+0000 7f5a7b400640 1 -- 172.21.15.55:0/1728646220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c032930 con 0x7f5a7c1418c0 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 -- 172.21.15.55:0/1728646220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1418c0 msgr2=0x7f5a7c15fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 --2- 172.21.15.55:0/1728646220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c15fa20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06f670 tx=0x7f5a6c02a210 comp rx=0 tx=0).stop 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 -- 172.21.15.55:0/1728646220 shutdown_connections 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 --2- 172.21.15.55:0/1728646220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c06d840 0x7f5a7c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 --2- 172.21.15.55:0/1728646220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c15fa20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 -- 172.21.15.55:0/1728646220 >> 172.21.15.55:0/1728646220 conn(0x7f5a7c06a710 msgr2=0x7f5a7c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:27.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 -- 172.21.15.55:0/1728646220 shutdown_connections 2024-05-10T03:31:27.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.128+0000 7f5a83800640 1 -- 172.21.15.55:0/1728646220 wait complete. 2024-05-10T03:31:27.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.130+0000 7f5a83800640 1 Processor -- start 2024-05-10T03:31:27.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.131+0000 7f5a83800640 1 -- start start 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c1e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c15ba10 0x7f5a7c1f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0a8600 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1e6e80 con 0x7f5a7c1418c0 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a83800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1f41f0 con 0x7f5a7c15ba10 2024-05-10T03:31:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.132+0000 7f5a81800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40242/0 (socket says 172.21.15.55:40242) 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 -- 172.21.15.55:0/3153092415 learned_addr learned my addr 172.21.15.55:0/3153092415 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a80e00640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c1e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a7be00640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c15ba10 0x7f5a7c1f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 -- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c15ba10 msgr2=0x7f5a7c1f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c15ba10 0x7f5a7c1f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 -- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a7c1418c0 msgr2=0x7f5a7c1e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c1e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.133+0000 7f5a81800640 1 -- 172.21.15.55:0/3153092415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0094d0 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a81800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400ba10 tx=0x7f5a7400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400cb20 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7400cca0 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74014680 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c1e7010 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.134+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c13e680 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.136+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c067740 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.137+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a7401d3b0 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.138+0000 7f5a78c00640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 0x7f5a5806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.138+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f5a74089390 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.139+0000 7f5a80e00640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 0x7f5a5806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.140+0000 7f5a80e00640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 0x7f5a5806d140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c034000 tx=0x7f5a6c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.143+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5a740543f0 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.381+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5a7c071dd0 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.390+0000 7f5a78c00640 1 -- 172.21.15.55:0/3153092415 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5a74057a10 con 0x7f5a7c1e5b00 2024-05-10T03:31:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 5 and data pool 6 2024-05-10T03:31:27.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.397+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 msgr2=0x7f5a5806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.397+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 0x7f5a5806d140 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c034000 tx=0x7f5a6c02e000 comp rx=0 tx=0).stop 2024-05-10T03:31:27.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.397+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 msgr2=0x7f5a7c1e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.397+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400ba10 tx=0x7f5a7400bed0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.398+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 shutdown_connections 2024-05-10T03:31:27.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.398+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a7c1e5b00 0x7f5a7c1e7e20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.398+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a7c15ba10 0x7f5a7c1f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.398+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a5806ac30 0x7f5a5806d140 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.398+0000 7f5a83800640 1 --2- 172.21.15.55:0/3153092415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a7c1418c0 0x7f5a7c1e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.399+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 >> 172.21.15.55:0/3153092415 conn(0x7f5a7c06a710 msgr2=0x7f5a7c15efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:27.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.399+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 shutdown_connections 2024-05-10T03:31:27.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.400+0000 7f5a83800640 1 -- 172.21.15.55:0/3153092415 wait complete. 2024-05-10T03:31:27.459 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:27.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.696+0000 7fdd49e00640 1 Processor -- start 2024-05-10T03:31:27.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.696+0000 7fdd49e00640 1 -- start start 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd4406b830 0x7fdd440aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd49e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440cf580 con 0x7fdd441614b0 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd49e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4415e5c0 con 0x7fdd4415dec0 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd49e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415ee60 con 0x7fdd4406b830 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd42a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdd441614b0 0x7fdd4406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35390/0 (socket says 172.21.15.55:35390) 2024-05-10T03:31:27.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd42a00640 1 -- 172.21.15.55:0/3930491116 learned_addr learned my addr 172.21.15.55:0/3930491116 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:27.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090217448 0 0) 0x7fdd440cf580 con 0x7fdd441614b0 2024-05-10T03:31:27.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd4406b830 0x7fdd440aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.698+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd28003520 con 0x7fdd441614b0 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348524890 0 0) 0x7fdd28003520 con 0x7fdd441614b0 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd4406b830 msgr2=0x7fdd440aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd4406b830 0x7fdd440aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 >> v1:172.21.15.162:6789/0 conn(0x7fdd4415dec0 legacy=0x7fdd440b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4415e8a0 con 0x7fdd441614b0 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fdd34001670 con 0x7fdd441614b0 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd4406b830 0x7fdd440aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 >> v1:172.21.15.55:6789/0 conn(0x7fdd441614b0 legacy=0x7fdd4406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd280039a0 0x7fdd28003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd28004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd28005110 0x7fdd2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415ee60 con 0x7fdd280047c0 2024-05-10T03:31:27.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2800bfa0 con 0x7fdd28005110 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2800c160 con 0x7fdd280039a0 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.699+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd28004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42a00640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd280039a0 0x7fdd28003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42000640 1 -- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd280039a0 msgr2=0x7fdd28003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd280039a0 0x7fdd28003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42000640 1 -- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd28005110 msgr2=0x7fdd2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd28005110 0x7fdd2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.700+0000 7fdd42000640 1 -- 172.21.15.55:0/3930491116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd28003520 con 0x7fdd280047c0 2024-05-10T03:31:27.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.701+0000 7fdd42000640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd28004bd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd300098a0 tx=0x7fdd280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64337f1bd0ef7f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3002ad00 con 0x7fdd280047c0 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3002ae60 con 0x7fdd280047c0 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd41600640 1 -- 172.21.15.55:0/3930491116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30004250 con 0x7fdd280047c0 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd49e00640 1 -- 172.21.15.55:0/3930491116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 msgr2=0x7fdd28004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd49e00640 1 --2- 172.21.15.55:0/3930491116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd28004bd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd300098a0 tx=0x7fdd280057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.702+0000 7fdd43400640 1 -- 172.21.15.55:0/3930491116 reap_dead start 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.703+0000 7fdd49e00640 1 -- 172.21.15.55:0/3930491116 shutdown_connections 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.703+0000 7fdd49e00640 1 -- 172.21.15.55:0/3930491116 >> 172.21.15.55:0/3930491116 conn(0x7fdd44070900 msgr2=0x7fdd44070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.703+0000 7fdd49e00640 1 -- 172.21.15.55:0/3930491116 shutdown_connections 2024-05-10T03:31:27.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.703+0000 7fdd49e00640 1 -- 172.21.15.55:0/3930491116 wait complete. 2024-05-10T03:31:27.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.705+0000 7fdd49e00640 1 Processor -- start 2024-05-10T03:31:27.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.705+0000 7fdd49e00640 1 -- start start 2024-05-10T03:31:27.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd440c5010 0x7fdd440afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd440aff70 0x7fdd4406c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440b1450 con 0x7fdd280047c0 2024-05-10T03:31:27.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441f50a0 con 0x7fdd440c5010 2024-05-10T03:31:27.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd49e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440b0960 con 0x7fdd440aff70 2024-05-10T03:31:27.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40266/0 (socket says 172.21.15.55:40266) 2024-05-10T03:31:27.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 -- 172.21.15.55:0/2147935832 learned_addr learned my addr 172.21.15.55:0/2147935832 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42000640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd440c5010 0x7fdd440afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 -- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd440aff70 msgr2=0x7fdd4406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd440aff70 0x7fdd4406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.707+0000 7fdd42a00640 1 -- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd440c5010 msgr2=0x7fdd440afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd42a00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd440c5010 0x7fdd440afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd42a00640 1 -- 172.21.15.55:0/2147935832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4414e740 con 0x7fdd280047c0 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd42a00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400d810 tx=0x7fdd3400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340074e0 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34007640 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340049e0 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd441f35d0 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.708+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440b1170 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.709+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44067220 con 0x7fdd280047c0 2024-05-10T03:31:27.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.711+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdd34004b40 con 0x7fdd280047c0 2024-05-10T03:31:27.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.711+0000 7fdd3b400640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 0x7fdd2406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:27.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.712+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdd34088ef0 con 0x7fdd280047c0 2024-05-10T03:31:27.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.712+0000 7fdd42000640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 0x7fdd2406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:27.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.713+0000 7fdd42000640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 0x7fdd2406d140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd30033000 tx=0x7fdd3002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:27.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.717+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd34053f00 con 0x7fdd280047c0 2024-05-10T03:31:27.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.911+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd440af810 con 0x7fdd280047c0 2024-05-10T03:31:27.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.913+0000 7fdd3b400640 1 -- 172.21.15.55:0/2147935832 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16319 (secure 0 0 0) 0x7fdd34019070 con 0x7fdd280047c0 2024-05-10T03:31:27.917 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:27.917 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:27.382801+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:27.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 msgr2=0x7fdd2406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 0x7fdd2406d140 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd30033000 tx=0x7fdd3002e000 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 msgr2=0x7fdd4414e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400d810 tx=0x7fdd3400dcd0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 shutdown_connections 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd440aff70 0x7fdd4406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdd2406ac30 0x7fdd2406d140 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd440c5010 0x7fdd440afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 --2- 172.21.15.55:0/2147935832 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd280047c0 0x7fdd4414e200 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 >> 172.21.15.55:0/2147935832 conn(0x7fdd44070900 msgr2=0x7fdd4415b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 shutdown_connections 2024-05-10T03:31:27.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:27.916+0000 7fdd49e00640 1 -- 172.21.15.55:0/2147935832 wait complete. 2024-05-10T03:31:27.946 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:28.162 INFO:tasks.ceph.mds.a.smithi055.stderr:2024-05-10T03:31:28.159+0000 7f4e4ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:31:28.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.210+0000 7f9e43600640 1 Processor -- start 2024-05-10T03:31:28.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.210+0000 7f9e43600640 1 -- start start 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.212+0000 7f9e43600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e3c15ab30 0x7f9e3c0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.212+0000 7f9e43600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0cd710 con 0x7f9e3c15af40 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.212+0000 7f9e43600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0cdfb0 con 0x7f9e3c0cdba0 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.212+0000 7f9e43600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c06b4d0 con 0x7f9e3c15ab30 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.213+0000 7f9e41600640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9e3c15af40 0x7f9e3c0b2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35398/0 (socket says 172.21.15.55:35398) 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.213+0000 7f9e41600640 1 -- 172.21.15.55:0/2338886425 learned_addr learned my addr 172.21.15.55:0/2338886425 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.213+0000 7f9e3be00640 1 --2- 172.21.15.55:0/2338886425 >> v2:172.21.15.162:3300/0 conn(0x7f9e3c15ab30 0x7f9e3c0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.213+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614194514 0 0) 0x7f9e3c0cd710 con 0x7f9e3c15af40 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.213+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e1c003520 con 0x7f9e3c15af40 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 566907469 0 0) 0x7f9e1c003520 con 0x7f9e3c15af40 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 >> v2:172.21.15.162:3300/0 conn(0x7f9e3c15ab30 msgr2=0x7f9e3c0715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 --2- 172.21.15.55:0/2338886425 >> v2:172.21.15.162:3300/0 conn(0x7f9e3c15ab30 0x7f9e3c0715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 >> v1:172.21.15.162:6789/0 conn(0x7f9e3c0cdba0 legacy=0x7f9e3c06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c06b1f0 con 0x7f9e3c15af40 2024-05-10T03:31:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f9e34001670 con 0x7f9e3c15af40 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 >> v1:172.21.15.55:6789/0 conn(0x7f9e3c15af40 legacy=0x7f9e3c0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 0x7f9e1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e1c0047c0 0x7f9e1c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c005310 0x7f9e1c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06b4d0 con 0x7f9e1c003970 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c00e450 con 0x7f9e1c0047c0 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c00e610 con 0x7f9e1c005310 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.214+0000 7f9e40c00640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 0x7f9e1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e40c00640 1 -- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c005310 msgr2=0x7f9e1c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e41600640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c005310 0x7f9e1c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e40c00640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c005310 0x7f9e1c00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e40c00640 1 -- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e1c0047c0 msgr2=0x7f9e1c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e3be00640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e1c0047c0 0x7f9e1c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e40c00640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e1c0047c0 0x7f9e1c004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e40c00640 1 -- 172.21.15.55:0/2338886425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c003520 con 0x7f9e1c003970 2024-05-10T03:31:28.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.215+0000 7f9e41600640 1 -- 172.21.15.55:0/2338886425 reap_dead start 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.216+0000 7f9e40c00640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 0x7f9e1c003de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c001750 tx=0x7f9e1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0af2f1d37c237ce server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.216+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0052d0 con 0x7f9e1c003970 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.216+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c005430 con 0x7f9e1c003970 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.216+0000 7f9e3b400640 1 -- 172.21.15.55:0/2338886425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c02abc0 con 0x7f9e1c003970 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 -- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 msgr2=0x7f9e1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 0x7f9e1c003de0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c001750 tx=0x7f9e1c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 -- 172.21.15.55:0/2338886425 shutdown_connections 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 --2- 172.21.15.55:0/2338886425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e1c003970 0x7f9e1c003de0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 -- 172.21.15.55:0/2338886425 >> 172.21.15.55:0/2338886425 conn(0x7f9e3c0cbb90 msgr2=0x7f9e3c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 -- 172.21.15.55:0/2338886425 shutdown_connections 2024-05-10T03:31:28.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.217+0000 7f9e43600640 1 -- 172.21.15.55:0/2338886425 wait complete. 2024-05-10T03:31:28.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.219+0000 7f9e43600640 1 Processor -- start 2024-05-10T03:31:28.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.220+0000 7f9e43600640 1 -- start start 2024-05-10T03:31:28.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c003970 0x7f9e3c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e3c0a84d0 0x7f9e3c0a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0a91f0 con 0x7f9e3c153830 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c153cf0 con 0x7f9e3c0a84d0 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e43600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c14f320 con 0x7f9e1c003970 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40272/0 (socket says 172.21.15.55:40272) 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.221+0000 7f9e3be00640 1 -- 172.21.15.55:0/3591560213 learned_addr learned my addr 172.21.15.55:0/3591560213 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:28.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e40c00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c003970 0x7f9e3c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 -- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c003970 msgr2=0x7f9e3c1531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c003970 0x7f9e3c1531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 -- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e3c0a84d0 msgr2=0x7f9e3c0a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e3c0a84d0 0x7f9e3c0a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 -- 172.21.15.55:0/3591560213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c152d60 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e3be00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ae80 tx=0x7f9e2400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.222+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400fe60 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.223+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24010040 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.223+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e240156c0 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.223+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c152430 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.223+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1529f0 con 0x7f9e3c153830 2024-05-10T03:31:28.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.224+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c152bb0 con 0x7f9e3c153830 2024-05-10T03:31:28.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.225+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e24015820 con 0x7f9e3c153830 2024-05-10T03:31:28.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.226+0000 7f9e38c00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 0x7f9e1806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.226+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9e24089df0 con 0x7f9e3c153830 2024-05-10T03:31:28.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.226+0000 7f9e40c00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 0x7f9e1806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.228+0000 7f9e40c00640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 0x7f9e1806d110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c032000 tx=0x7f9e2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.230+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9e24054df0 con 0x7f9e3c153830 2024-05-10T03:31:28.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.424+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e3c066f60 con 0x7f9e3c153830 2024-05-10T03:31:28.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.427+0000 7f9e38c00640 1 -- 172.21.15.55:0/3591560213 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f9e24058410 con 0x7f9e3c153830 2024-05-10T03:31:28.431 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:28.431 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 msgr2=0x7f9e1806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 0x7f9e1806d110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c032000 tx=0x7f9e2c02e000 comp rx=0 tx=0).stop 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 msgr2=0x7f9e3c1f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ae80 tx=0x7f9e2400f510 comp rx=0 tx=0).stop 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 shutdown_connections 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1806ac00 0x7f9e1806d110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e3c0a84d0 0x7f9e3c0a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e3c153830 0x7f9e3c1f77b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 --2- 172.21.15.55:0/3591560213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e1c003970 0x7f9e3c1531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 >> 172.21.15.55:0/3591560213 conn(0x7f9e3c0cbb90 msgr2=0x7f9e3c0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 shutdown_connections 2024-05-10T03:31:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.430+0000 7f9e43600640 1 -- 172.21.15.55:0/3591560213 wait complete. 2024-05-10T03:31:28.458 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:31:28.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.730+0000 7fb9d1e00640 1 Processor -- start 2024-05-10T03:31:28.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.730+0000 7fb9d1e00640 1 -- start start 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.732+0000 7fb9d1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9cc0b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.732+0000 7fb9d1e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0d29f0 con 0x7fb9cc0ce030 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9d1e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0b92b0 con 0x7fb9cc0b23f0 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9d1e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0d1de0 con 0x7fb9cc0b3930 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9cb400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb9cc0ce030 0x7fb9cc0ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35412/0 (socket says 172.21.15.55:35412) 2024-05-10T03:31:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9cb400640 1 -- 172.21.15.55:0/4167776970 learned_addr learned my addr 172.21.15.55:0/4167776970 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:28.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9ca000640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9cc0b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546015246 0 0) 0x7fb9cc0d29f0 con 0x7fb9cc0ce030 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.733+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b0003520 con 0x7fb9cc0ce030 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735893850 0 0) 0x7fb9b0003520 con 0x7fb9cc0ce030 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 msgr2=0x7fb9cc0b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9cc0b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 >> v1:172.21.15.162:6789/0 conn(0x7fb9cc0b23f0 legacy=0x7fb9cc161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0d20c0 con 0x7fb9cc0ce030 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb9c40019f0 con 0x7fb9cc0ce030 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 >> v1:172.21.15.55:6789/0 conn(0x7fb9cc0ce030 legacy=0x7fb9cc0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9b00039a0 0x7fb9b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00047c0 0x7fb9b0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9b0005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0d1de0 con 0x7fb9b00039a0 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b000bb90 con 0x7fb9b00047c0 2024-05-10T03:31:28.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b000bd50 con 0x7fb9cc0b3930 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9caa00640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9b00039a0 0x7fb9b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.734+0000 7fb9ca000640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00047c0 0x7fb9b0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9cb400640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9b0005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9caa00640 1 -- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 msgr2=0x7fb9b0005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9caa00640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9b0005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9caa00640 1 -- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00047c0 msgr2=0x7fb9b0004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9caa00640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00047c0 0x7fb9b0004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9caa00640 1 -- 172.21.15.55:0/4167776970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0003520 con 0x7fb9b00039a0 2024-05-10T03:31:28.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.735+0000 7fb9cb400640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc0b3930 0x7fb9b0005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:28.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.736+0000 7fb9caa00640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9b00039a0 0x7fb9b0003e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001750 tx=0x7fb9b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d247317136e4c77e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.736+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0052d0 con 0x7fb9b00039a0 2024-05-10T03:31:28.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.736+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc005430 con 0x7fb9b00039a0 2024-05-10T03:31:28.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.736+0000 7fb9c9600640 1 -- 172.21.15.55:0/4167776970 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc02abe0 con 0x7fb9b00039a0 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 -- 172.21.15.55:0/4167776970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9b00039a0 msgr2=0x7fb9b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/4167776970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9b00039a0 0x7fb9b0003e10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001750 tx=0x7fb9b00057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9cb400640 1 -- 172.21.15.55:0/4167776970 reap_dead start 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 -- 172.21.15.55:0/4167776970 shutdown_connections 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 -- 172.21.15.55:0/4167776970 >> 172.21.15.55:0/4167776970 conn(0x7fb9cc0d0850 msgr2=0x7fb9cc0d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 -- 172.21.15.55:0/4167776970 shutdown_connections 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.737+0000 7fb9d1e00640 1 -- 172.21.15.55:0/4167776970 wait complete. 2024-05-10T03:31:28.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.739+0000 7fb9d1e00640 1 Processor -- start 2024-05-10T03:31:28.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.739+0000 7fb9d1e00640 1 -- start start 2024-05-10T03:31:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00039a0 0x7fb9cc14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc15c1f0 0x7fb9cc1e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc14ea00 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc14ee30 con 0x7fb9b00039a0 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9d1e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15c720 con 0x7fb9cc15c1f0 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9ca000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9ca000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40286/0 (socket says 172.21.15.55:40286) 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.741+0000 7fb9ca000640 1 -- 172.21.15.55:0/497073849 learned_addr learned my addr 172.21.15.55:0/497073849 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 -- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc15c1f0 msgr2=0x7fb9cc1e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc15c1f0 0x7fb9cc1e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 -- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00039a0 msgr2=0x7fb9cc14e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00039a0 0x7fb9cc14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:28.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 -- 172.21.15.55:0/497073849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc150870 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9ca000640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800f580 tx=0x7fb9b800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8014870 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b80149d0 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8014d80 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc1e7a60 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.742+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc1e6e50 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.743+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc067220 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.745+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb9b8015410 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.746+0000 7fb9c3400640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 0x7fb9ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.746+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9b8089eb0 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.746+0000 7fb9caa00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 0x7fb9ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:28.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.748+0000 7fb9caa00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 0x7fb9ac06d110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc032000 tx=0x7fb9bc02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:28.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.750+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb9b8054e30 con 0x7fb9cc15b3c0 2024-05-10T03:31:28.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:28.979+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9cc150850 con 0x7fb9cc15b3c0 2024-05-10T03:31:29.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.840+0000 7fb9c3400640 1 -- 172.21.15.55:0/497073849 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb9b8058450 con 0x7fb9cc15b3c0 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 msgr2=0x7fb9ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 0x7fb9ac06d110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc032000 tx=0x7fb9bc02c000 comp rx=0 tx=0).stop 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 msgr2=0x7fb9cc15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800f580 tx=0x7fb9b800fa40 comp rx=0 tx=0).stop 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 shutdown_connections 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9cc15c1f0 0x7fb9cc1e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9cc15b3c0 0x7fb9cc15bcb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb9ac06ac00 0x7fb9ac06d110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:29.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 --2- 172.21.15.55:0/497073849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9b00039a0 0x7fb9cc14e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:29.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.844+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 >> 172.21.15.55:0/497073849 conn(0x7fb9cc0d0850 msgr2=0x7fb9cc0b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:29.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.845+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 shutdown_connections 2024-05-10T03:31:29.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:29.845+0000 7fb9d1e00640 1 -- 172.21.15.55:0/497073849 wait complete. 2024-05-10T03:31:29.875 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:31:29.875 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:31:30.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.161+0000 7f145b000640 1 Processor -- start 2024-05-10T03:31:30.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.161+0000 7f145b000640 1 -- start start 2024-05-10T03:31:30.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.163+0000 7f145b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14540cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.163+0000 7f145b000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f145406b830 0x7f145415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.163+0000 7f145b000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145415b5e0 con 0x7f1454140900 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.163+0000 7f145b000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14540b4380 con 0x7f145406b830 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.163+0000 7f145b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145415acf0 con 0x7f14540cc3a0 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f144be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1454140900 0x7f145406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35416/0 (socket says 172.21.15.55:35416) 2024-05-10T03:31:30.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f144be00640 1 -- 172.21.15.55:0/2930734780 learned_addr learned my addr 172.21.15.55:0/2930734780 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:30.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f1459000640 1 --2- 172.21.15.55:0/2930734780 >> v2:172.21.15.162:3300/0 conn(0x7f145406b830 0x7f145415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f1453e00640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14540cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594288009 0 0) 0x7f145415b5e0 con 0x7f1454140900 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.164+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1438003520 con 0x7f1454140900 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901762399 0 0) 0x7f1438003520 con 0x7f1454140900 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 msgr2=0x7f14540cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14540cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 >> v2:172.21.15.162:3300/0 conn(0x7f145406b830 msgr2=0x7f145415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 --2- 172.21.15.55:0/2930734780 >> v2:172.21.15.162:3300/0 conn(0x7f145406b830 0x7f145415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145415af90 con 0x7f1454140900 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f143c001670 con 0x7f1454140900 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 >> v1:172.21.15.55:6789/0 conn(0x7f1454140900 legacy=0x7f145406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1438003970 0x7f1438003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1438004d00 0x7f1438005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14540b4380 con 0x7f1438004d00 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145415acf0 con 0x7f1438003970 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143800ba30 con 0x7f14540cc3a0 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.165+0000 7f1459000640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1438004d00 0x7f1438005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1453e00640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1438003970 0x7f1438003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f144be00640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1459000640 1 -- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 msgr2=0x7f14380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1459000640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1459000640 1 -- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1438003970 msgr2=0x7f1438003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1459000640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1438003970 0x7f1438003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f1459000640 1 -- 172.21.15.55:0/2930734780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438003520 con 0x7f1438004d00 2024-05-10T03:31:30.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.166+0000 7f144be00640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14540cc3a0 0x7f14380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:30.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.167+0000 7f1459000640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1438004d00 0x7f1438005110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f144c009fc0 tx=0x7f14380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff783be85f95241 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.167+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c031670 con 0x7f1438004d00 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.167+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00f040 con 0x7f1438004d00 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.167+0000 7f1453400640 1 -- 172.21.15.55:0/2930734780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c031a10 con 0x7f1438004d00 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 -- 172.21.15.55:0/2930734780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1438004d00 msgr2=0x7f1438005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 --2- 172.21.15.55:0/2930734780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1438004d00 0x7f1438005110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f144c009fc0 tx=0x7f14380057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f1459000640 1 -- 172.21.15.55:0/2930734780 reap_dead start 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 -- 172.21.15.55:0/2930734780 shutdown_connections 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 -- 172.21.15.55:0/2930734780 >> 172.21.15.55:0/2930734780 conn(0x7f145406c120 msgr2=0x7f145406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 -- 172.21.15.55:0/2930734780 shutdown_connections 2024-05-10T03:31:30.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.168+0000 7f145b000640 1 -- 172.21.15.55:0/2930734780 wait complete. 2024-05-10T03:31:30.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.171+0000 7f145b000640 1 Processor -- start 2024-05-10T03:31:30.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.171+0000 7f145b000640 1 -- start start 2024-05-10T03:31:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1438004d00 0x7f1454132280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14540a84d0 0x7f14541586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14540ca860 con 0x7f14540cac80 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454132b10 con 0x7f14540a84d0 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.172+0000 7f145b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14540c9990 con 0x7f1438004d00 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40300/0 (socket says 172.21.15.55:40300) 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 -- 172.21.15.55:0/282948378 learned_addr learned my addr 172.21.15.55:0/282948378 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f144be00640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14540a84d0 0x7f14541586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1453e00640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1438004d00 0x7f1454132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 -- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1438004d00 msgr2=0x7f1454132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1438004d00 0x7f1454132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 -- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14540a84d0 msgr2=0x7f14541586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14540a84d0 0x7f14541586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.173+0000 7f1459000640 1 -- 172.21.15.55:0/282948378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540cb5d0 con 0x7f14540cac80 2024-05-10T03:31:30.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f1459000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f144c033040 tx=0x7f144c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:30.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c003990 con 0x7f14540cac80 2024-05-10T03:31:30.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c040070 con 0x7f14540cac80 2024-05-10T03:31:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c003d10 con 0x7f14540cac80 2024-05-10T03:31:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14540c9be0 con 0x7f14540cac80 2024-05-10T03:31:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.174+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14540ca1a0 con 0x7f14540cac80 2024-05-10T03:31:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.175+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1454066f60 con 0x7f14540cac80 2024-05-10T03:31:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.176+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f144c042030 con 0x7f14540cac80 2024-05-10T03:31:30.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.177+0000 7f1450c00640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 0x7f143006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:30.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.177+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f144c0aa710 con 0x7f14540cac80 2024-05-10T03:31:30.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.178+0000 7f1453e00640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 0x7f143006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.179+0000 7f1453e00640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 0x7f143006d140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1444006fd0 tx=0x7f1444008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.182+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f144c075690 con 0x7f14540cac80 2024-05-10T03:31:30.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.419+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f145415bd90 con 0x7f14540cac80 2024-05-10T03:31:30.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.843+0000 7f1450c00640 1 -- 172.21.15.55:0/282948378 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f144c009c10 con 0x7f14540cac80 2024-05-10T03:31:30.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 msgr2=0x7f143006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 0x7f143006d140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1444006fd0 tx=0x7f1444008040 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 msgr2=0x7f14540cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f144c033040 tx=0x7f144c009750 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 shutdown_connections 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14540cac80 0x7f14540cb090 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f143006ac30 0x7f143006d140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14540a84d0 0x7f14541586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 --2- 172.21.15.55:0/282948378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1438004d00 0x7f1454132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 >> 172.21.15.55:0/282948378 conn(0x7f145406c120 msgr2=0x7f14540cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 shutdown_connections 2024-05-10T03:31:30.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:30.847+0000 7f145b000640 1 -- 172.21.15.55:0/282948378 wait complete. 2024-05-10T03:31:30.882 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:31:30.883 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:31.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.165+0000 7f597da00640 1 Processor -- start 2024-05-10T03:31:31.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.165+0000 7f597da00640 1 -- start start 2024-05-10T03:31:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597da00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 0x7f59781471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5978162840 0x7f5978147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597da00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5978162c50 con 0x7f5978144b60 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597da00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780b1f60 con 0x7f5978162840 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597da00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5978162ed0 con 0x7f59780b2d00 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5978162840 0x7f5978147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5978162840 0x7f5978147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40308/0 (socket says 172.21.15.55:40308) 2024-05-10T03:31:31.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.168+0000 7f597d000640 1 -- 172.21.15.55:0/884611827 learned_addr learned my addr 172.21.15.55:0/884611827 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f597d000640 1 -- 172.21.15.55:0/884611827 >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 msgr2=0x7f59781471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f5977e00640 1 --2- 172.21.15.55:0/884611827 >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 0x7f59781471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f597d000640 1 --2- 172.21.15.55:0/884611827 >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 0x7f59781471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f597d000640 1 -- 172.21.15.55:0/884611827 >> v1:172.21.15.162:6789/0 conn(0x7f5978144b60 legacy=0x7f5978162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f597d000640 1 -- 172.21.15.55:0/884611827 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780b2220 con 0x7f5978162840 2024-05-10T03:31:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.169+0000 7f5977e00640 1 --2- 172.21.15.55:0/884611827 >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 0x7f59781471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.170+0000 7f597d000640 1 --2- 172.21.15.55:0/884611827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978147730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f59780aeb80 tx=0x7f597002a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2c145e9d2cfda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.171+0000 7f5977400640 1 -- 172.21.15.55:0/884611827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970031400 con 0x7f5978162840 2024-05-10T03:31:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.171+0000 7f5977400640 1 -- 172.21.15.55:0/884611827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597002d050 con 0x7f5978162840 2024-05-10T03:31:31.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.171+0000 7f5977400640 1 -- 172.21.15.55:0/884611827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970031780 con 0x7f5978162840 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.171+0000 7f597da00640 1 -- 172.21.15.55:0/884611827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 msgr2=0x7f5978147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.171+0000 7f597da00640 1 --2- 172.21.15.55:0/884611827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978147730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f59780aeb80 tx=0x7f597002a620 comp rx=0 tx=0).stop 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 -- 172.21.15.55:0/884611827 shutdown_connections 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 --2- 172.21.15.55:0/884611827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978147730 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 --2- 172.21.15.55:0/884611827 >> v2:172.21.15.162:3300/0 conn(0x7f59780b2d00 0x7f59781471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 -- 172.21.15.55:0/884611827 >> 172.21.15.55:0/884611827 conn(0x7f597806a110 msgr2=0x7f597806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 -- 172.21.15.55:0/884611827 shutdown_connections 2024-05-10T03:31:31.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.172+0000 7f597da00640 1 -- 172.21.15.55:0/884611827 wait complete. 2024-05-10T03:31:31.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.174+0000 7f597da00640 1 Processor -- start 2024-05-10T03:31:31.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.174+0000 7f597da00640 1 -- start start 2024-05-10T03:31:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59780b2d00 0x7f5978140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978144b60 0x7f597814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780ab610 con 0x7f5978162840 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597814cbe0 con 0x7f59780b2d00 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597814da40 con 0x7f5978144b60 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40316/0 (socket says 172.21.15.55:40316) 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f597d000640 1 -- 172.21.15.55:0/2199538783 learned_addr learned my addr 172.21.15.55:0/2199538783 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f5977e00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59780b2d00 0x7f5978140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.176+0000 7f596fe00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978144b60 0x7f597814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 -- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978144b60 msgr2=0x7f597814c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978144b60 0x7f597814c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 -- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59780b2d00 msgr2=0x7f5978140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59780b2d00 0x7f5978140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 -- 172.21.15.55:0/2199538783 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009b30 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f597d000640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5970034040 tx=0x7f5970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.177+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597003d650 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.178+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597003d7f0 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.178+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597003dba0 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.178+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597814ce30 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.178+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597814d3f0 con 0x7f5978162840 2024-05-10T03:31:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.179+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978066ce0 con 0x7f5978162840 2024-05-10T03:31:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.180+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f597003dd00 con 0x7f5978162840 2024-05-10T03:31:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.181+0000 7f5974c00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 0x7f595406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.181+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59700aa3b0 con 0x7f5978162840 2024-05-10T03:31:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.182+0000 7f5977e00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 0x7f595406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.183+0000 7f5977e00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 0x7f595406d110 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f596800a950 tx=0x7f5968008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.186+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f59700753b0 con 0x7f5978162840 2024-05-10T03:31:31.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.425+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f597804e290 con 0x7f5978162840 2024-05-10T03:31:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.426+0000 7f5974c00640 1 -- 172.21.15.55:0/2199538783 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f5970009510 con 0x7f5978162840 2024-05-10T03:31:31.429 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:31.430 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:31.430 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:31.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 msgr2=0x7f595406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 0x7f595406d110 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f596800a950 tx=0x7f5968008040 comp rx=0 tx=0).stop 2024-05-10T03:31:31.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 msgr2=0x7f5978165440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5970034040 tx=0x7f5970004a40 comp rx=0 tx=0).stop 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 shutdown_connections 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5978162840 0x7f5978165440 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978144b60 0x7f597814c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595406ac00 0x7f595406d110 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 --2- 172.21.15.55:0/2199538783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59780b2d00 0x7f5978140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 >> 172.21.15.55:0/2199538783 conn(0x7f597806a110 msgr2=0x7f597806b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 shutdown_connections 2024-05-10T03:31:31.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.429+0000 7f597da00640 1 -- 172.21.15.55:0/2199538783 wait complete. 2024-05-10T03:31:31.458 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:31.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.733+0000 7f51b8e00640 1 Processor -- start 2024-05-10T03:31:31.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.733+0000 7f51b8e00640 1 -- start start 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.735+0000 7f51b8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f51b4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.735+0000 7f51b8e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51b41613c0 0x7f51b40cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.735+0000 7f51b8e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415edc0 con 0x7f51b40ce420 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.735+0000 7f51b8e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b41623d0 con 0x7f51b41613c0 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.735+0000 7f51b8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415e1e0 con 0x7f51b415ddd0 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b3e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f51b40ce420 0x7f51b40b1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35432/0 (socket says 172.21.15.55:35432) 2024-05-10T03:31:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b3e00640 1 -- 172.21.15.55:0/653532728 learned_addr learned my addr 172.21.15.55:0/653532728 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:31.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b3400640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f51b4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b2a00640 1 --2- 172.21.15.55:0/653532728 >> v2:172.21.15.162:3300/0 conn(0x7f51b41613c0 0x7f51b40cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845855525 0 0) 0x7f51b415edc0 con 0x7f51b40ce420 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.736+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5198003520 con 0x7f51b40ce420 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496995659 0 0) 0x7f5198003520 con 0x7f51b40ce420 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 msgr2=0x7f51b4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f51b4157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 >> v2:172.21.15.162:3300/0 conn(0x7f51b41613c0 msgr2=0x7f51b40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 --2- 172.21.15.55:0/653532728 >> v2:172.21.15.162:3300/0 conn(0x7f51b41613c0 0x7f51b40cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40cecd0 con 0x7f51b40ce420 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f51ac001670 con 0x7f51b40ce420 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 >> v1:172.21.15.55:6789/0 conn(0x7f51b40ce420 legacy=0x7f51b40b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51980039e0 0x7f5198003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51980047c0 0x7f5198004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f5198005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41623d0 con 0x7f51980039e0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415e1e0 con 0x7f51980047c0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519800ba30 con 0x7f51b415ddd0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.737+0000 7f51b3400640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51980039e0 0x7f5198003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3400640 1 -- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 msgr2=0x7f5198005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3e00640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f5198005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b2a00640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51980047c0 0x7f5198004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3400640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f5198005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3400640 1 -- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51980047c0 msgr2=0x7f5198004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3400640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51980047c0 0x7f5198004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3400640 1 -- 172.21.15.55:0/653532728 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198003520 con 0x7f51980039e0 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b3e00640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51b415ddd0 0x7f5198005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.738+0000 7f51b2a00640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51980047c0 0x7f5198004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.739+0000 7f51b3400640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51980039e0 0x7f5198003e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009fc0 tx=0x7f51980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f250d17366a99f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.739+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40305f0 con 0x7f51980039e0 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.739+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a400f040 con 0x7f51980039e0 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.739+0000 7f51b2000640 1 -- 172.21.15.55:0/653532728 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4030950 con 0x7f51980039e0 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 -- 172.21.15.55:0/653532728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51980039e0 msgr2=0x7f5198003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 --2- 172.21.15.55:0/653532728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51980039e0 0x7f5198003e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009fc0 tx=0x7f51980057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b3e00640 1 -- 172.21.15.55:0/653532728 reap_dead start 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 -- 172.21.15.55:0/653532728 shutdown_connections 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 -- 172.21.15.55:0/653532728 >> 172.21.15.55:0/653532728 conn(0x7f51b40d0370 msgr2=0x7f51b40d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 -- 172.21.15.55:0/653532728 shutdown_connections 2024-05-10T03:31:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.740+0000 7f51b8e00640 1 -- 172.21.15.55:0/653532728 wait complete. 2024-05-10T03:31:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.743+0000 7f51b8e00640 1 Processor -- start 2024-05-10T03:31:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.743+0000 7f51b8e00640 1 -- start start 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 0x7f51b415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 0x7f51b4158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41f4c20 con 0x7f51b4156900 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41e7fa0 con 0x7f51b40b8190 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40b85a0 con 0x7f51980039e0 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.744+0000 7f51b2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40330/0 (socket says 172.21.15.55:40330) 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 -- 172.21.15.55:0/2038790476 learned_addr learned my addr 172.21.15.55:0/2038790476 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b3400640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 0x7f51b415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 -- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 msgr2=0x7f51b415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b3e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 0x7f51b4158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 0x7f51b415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 -- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 msgr2=0x7f51b4158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 0x7f51b4158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 -- 172.21.15.55:0/2038790476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b41e9290 con 0x7f51b4156900 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b3400640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 0x7f51b415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b3e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 0x7f51b4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.745+0000 7f51b2a00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f519c0109d0 tx=0x7f519c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.746+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c00ec20 con 0x7f51b4156900 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.746+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c00ed80 con 0x7f51b4156900 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.746+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c01c620 con 0x7f51b4156900 2024-05-10T03:31:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.746+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b41e81f0 con 0x7f51b4156900 2024-05-10T03:31:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.746+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b41e87b0 con 0x7f51b4156900 2024-05-10T03:31:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.747+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b404e290 con 0x7f51b4156900 2024-05-10T03:31:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.748+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f519c01c780 con 0x7f51b4156900 2024-05-10T03:31:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.749+0000 7f51ab400640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 0x7f519006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.749+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f519c089b70 con 0x7f51b4156900 2024-05-10T03:31:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.750+0000 7f51b3400640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 0x7f519006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.751+0000 7f51b3400640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 0x7f519006d140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51a4035000 tx=0x7f51a402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:31.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.754+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f519c054b90 con 0x7f51b4156900 2024-05-10T03:31:31.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.954+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51b40b27e0 con 0x7f51b4156900 2024-05-10T03:31:31.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.958+0000 7f51ab400640 1 -- 172.21.15.55:0/2038790476 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f519c0581b0 con 0x7f51b4156900 2024-05-10T03:31:31.961 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:31.962 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 msgr2=0x7f519006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 0x7f519006d140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f51a4035000 tx=0x7f51a402f000 comp rx=0 tx=0).stop 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 msgr2=0x7f51b40b7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f519c0109d0 tx=0x7f519c010e90 comp rx=0 tx=0).stop 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 shutdown_connections 2024-05-10T03:31:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f519006ac30 0x7f519006d140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51b40b8190 0x7f51b4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51b4156900 0x7f51b40b7c50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 --2- 172.21.15.55:0/2038790476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51980039e0 0x7f51b415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 >> 172.21.15.55:0/2038790476 conn(0x7f51b40d0370 msgr2=0x7f51b40d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 shutdown_connections 2024-05-10T03:31:31.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:31.961+0000 7f51b8e00640 1 -- 172.21.15.55:0/2038790476 wait complete. 2024-05-10T03:31:31.989 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:31:31.989 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:31:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.256+0000 7f6339e00640 1 Processor -- start 2024-05-10T03:31:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.256+0000 7f6339e00640 1 -- start start 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6339e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f633415a5d0 0x7f63340cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6339e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63340b4880 0x7f63340ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6339e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633415ad30 con 0x7f63340cb730 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6339e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633415b010 con 0x7f633415a5d0 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6339e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633415b2f0 con 0x7f63340b4880 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6338a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f633415a5d0 0x7f63340cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6338a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f633415a5d0 0x7f63340cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40344/0 (socket says 172.21.15.55:40344) 2024-05-10T03:31:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.259+0000 7f6338a00640 1 -- 172.21.15.55:0/1764548263 learned_addr learned my addr 172.21.15.55:0/1764548263 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.260+0000 7f6338a00640 1 -- 172.21.15.55:0/1764548263 >> v2:172.21.15.162:3300/0 conn(0x7f63340b4880 msgr2=0x7f63340ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.260+0000 7f6338a00640 1 --2- 172.21.15.55:0/1764548263 >> v2:172.21.15.162:3300/0 conn(0x7f63340b4880 0x7f63340ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.260+0000 7f6338a00640 1 -- 172.21.15.55:0/1764548263 >> v1:172.21.15.162:6789/0 conn(0x7f63340cb730 legacy=0x7f63340b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.260+0000 7f6338a00640 1 -- 172.21.15.55:0/1764548263 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633415b5d0 con 0x7f633415a5d0 2024-05-10T03:31:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.261+0000 7f6338a00640 1 --2- 172.21.15.55:0/1764548263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f633415a5d0 0x7f63340cb1f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63340caff0 tx=0x7f632802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a461c5850c58dd40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.261+0000 7f6333400640 1 -- 172.21.15.55:0/1764548263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328032440 con 0x7f633415a5d0 2024-05-10T03:31:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.261+0000 7f6333400640 1 -- 172.21.15.55:0/1764548263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280325a0 con 0x7f633415a5d0 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.261+0000 7f6333400640 1 -- 172.21.15.55:0/1764548263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328032930 con 0x7f633415a5d0 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 -- 172.21.15.55:0/1764548263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f633415a5d0 msgr2=0x7f63340cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 --2- 172.21.15.55:0/1764548263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f633415a5d0 0x7f63340cb1f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63340caff0 tx=0x7f632802a230 comp rx=0 tx=0).stop 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 -- 172.21.15.55:0/1764548263 shutdown_connections 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 --2- 172.21.15.55:0/1764548263 >> v2:172.21.15.162:3300/0 conn(0x7f63340b4880 0x7f63340ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 --2- 172.21.15.55:0/1764548263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f633415a5d0 0x7f63340cb1f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 -- 172.21.15.55:0/1764548263 >> 172.21.15.55:0/1764548263 conn(0x7f633406a110 msgr2=0x7f633406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 -- 172.21.15.55:0/1764548263 shutdown_connections 2024-05-10T03:31:32.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.262+0000 7f6339e00640 1 -- 172.21.15.55:0/1764548263 wait complete. 2024-05-10T03:31:32.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.264+0000 7f6339e00640 1 Processor -- start 2024-05-10T03:31:32.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.264+0000 7f6339e00640 1 -- start start 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633415a5d0 0x7f6334154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6334154c80 0x7f6334158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341496a0 con 0x7f63340cb730 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334155260 con 0x7f633415a5d0 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6339e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6334147ed0 con 0x7f6334154c80 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40354/0 (socket says 172.21.15.55:40354) 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 -- 172.21.15.55:0/3228879058 learned_addr learned my addr 172.21.15.55:0/3228879058 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 -- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6334154c80 msgr2=0x7f6334158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6334154c80 0x7f6334158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 -- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633415a5d0 msgr2=0x7f6334154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633415a5d0 0x7f6334154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.266+0000 7f6338a00640 1 -- 172.21.15.55:0/3228879058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280094d0 con 0x7f63340cb730 2024-05-10T03:31:32.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6338a00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6328000c00 tx=0x7f63280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328032440 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328030b80 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632803b400 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63341554b0 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.267+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334155a70 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.268+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334066ce0 con 0x7f63340cb730 2024-05-10T03:31:32.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.270+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6328039040 con 0x7f63340cb730 2024-05-10T03:31:32.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.270+0000 7f6330c00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 0x7f631406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.270+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f63280a9de0 con 0x7f63340cb730 2024-05-10T03:31:32.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.271+0000 7f6333e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 0x7f631406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.272+0000 7f6333e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 0x7f631406d110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f63340cbd40 tx=0x7f6320006ba0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.275+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6328074df0 con 0x7f63340cb730 2024-05-10T03:31:32.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.470+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f633415a9e0 con 0x7f631406ac00 2024-05-10T03:31:32.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.499+0000 7f6330c00640 1 -- 172.21.15.55:0/3228879058 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283965 (secure 0 0 0) 0x7f6320012390 con 0x7f631406ac00 2024-05-10T03:31:32.506 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:32.507 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:31:32.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 msgr2=0x7f631406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 0x7f631406d110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f63340cbd40 tx=0x7f6320006ba0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 msgr2=0x7f6334156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6328000c00 tx=0x7f63280046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 shutdown_connections 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6334154c80 0x7f6334158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631406ac00 0x7f631406d110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633415a5d0 0x7f6334154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 --2- 172.21.15.55:0/3228879058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63340cb730 0x7f6334156cf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 >> 172.21.15.55:0/3228879058 conn(0x7f633406a110 msgr2=0x7f633415ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 shutdown_connections 2024-05-10T03:31:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.504+0000 7f6339e00640 1 -- 172.21.15.55:0/3228879058 wait complete. 2024-05-10T03:31:32.532 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-05-10T03:31:30.990600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232948,"kb_used_data":4500,"kb_used_omap":12,"kb_used_meta":228339,"kb_avail":754741772,"statfs":{"total":773094113280,"available":772855574528,"internally_reserved":0,"allocated":4608000,"data_stored":2617859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":233819704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.201767"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.826569+0000","last_change":"2024-05-10T03:31:26.826569+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.826569+0000","last_undegraded":"2024-05-10T03:31:26.826569+0000","last_fullsized":"2024-05-10T03:31:26.826569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:39.335145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.824803+0000","last_change":"2024-05-10T03:31:26.824803+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.824803+0000","last_undegraded":"2024-05-10T03:31:26.824803+0000","last_fullsized":"2024-05-10T03:31:26.824803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:54.811349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.829970+0000","last_change":"2024-05-10T03:31:26.829970+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.829970+0000","last_undegraded":"2024-05-10T03:31:26.829970+0000","last_fullsized":"2024-05-10T03:31:26.829970+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:17.715674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.822188+0000","last_change":"2024-05-10T03:31:26.822188+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.822188+0000","last_undegraded":"2024-05-10T03:31:26.822188+0000","last_fullsized":"2024-05-10T03:31:26.822188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:19.464687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820249+0000","last_change":"2024-05-10T03:31:26.820249+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820249+0000","last_undegraded":"2024-05-10T03:31:26.820249+0000","last_fullsized":"2024-05-10T03:31:26.820249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:19.132985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.825041+0000","last_change":"2024-05-10T03:31:26.825041+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.825041+0000","last_undegraded":"2024-05-10T03:31:26.825041+0000","last_fullsized":"2024-05-10T03:31:26.825041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.838956+0000","last_change":"2024-05-10T03:31:26.838956+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.838956+0000","last_undegraded":"2024-05-10T03:31:26.838956+0000","last_fullsized":"2024-05-10T03:31:26.838956+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:55:32.871492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.826755+0000","last_change":"2024-05-10T03:31:26.826755+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.826755+0000","last_undegraded":"2024-05-10T03:31:26.826755+0000","last_fullsized":"2024-05-10T03:31:26.826755+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:23.429272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.827918+0000","last_change":"2024-05-10T03:31:26.827918+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.827918+0000","last_undegraded":"2024-05-10T03:31:26.827918+0000","last_fullsized":"2024-05-10T03:31:26.827918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:51.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.835225+0000","last_change":"2024-05-10T03:31:26.835225+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.835225+0000","last_undegraded":"2024-05-10T03:31:26.835225+0000","last_fullsized":"2024-05-10T03:31:26.835225+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:38:15.955193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.833507+0000","last_change":"2024-05-10T03:31:26.833507+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.833507+0000","last_undegraded":"2024-05-10T03:31:26.833507+0000","last_fullsized":"2024-05-10T03:31:26.833507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:54.811349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.826987+0000","last_change":"2024-05-10T03:31:26.826987+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.826987+0000","last_undegraded":"2024-05-10T03:31:26.826987+0000","last_fullsized":"2024-05-10T03:31:26.826987+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.832347+0000","last_change":"2024-05-10T03:31:26.832347+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.832347+0000","last_undegraded":"2024-05-10T03:31:26.832347+0000","last_fullsized":"2024-05-10T03:31:26.832347+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:46:53.551974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.840944+0000","last_change":"2024-05-10T03:31:26.840944+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.840944+0000","last_undegraded":"2024-05-10T03:31:26.840944+0000","last_fullsized":"2024-05-10T03:31:26.840944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:51.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.838054+0000","last_change":"2024-05-10T03:31:26.838054+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.838054+0000","last_undegraded":"2024-05-10T03:31:26.838054+0000","last_fullsized":"2024-05-10T03:31:26.838054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:01.205590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.822783+0000","last_change":"2024-05-10T03:31:26.822783+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.822783+0000","last_undegraded":"2024-05-10T03:31:26.822783+0000","last_fullsized":"2024-05-10T03:31:26.822783+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.825273+0000","last_change":"2024-05-10T03:31:26.825273+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.825273+0000","last_undegraded":"2024-05-10T03:31:26.825273+0000","last_fullsized":"2024-05-10T03:31:26.825273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:17.715674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.815719+0000","last_change":"2024-05-10T03:31:26.815719+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.815719+0000","last_undegraded":"2024-05-10T03:31:26.815719+0000","last_fullsized":"2024-05-10T03:31:26.815719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.821570+0000","last_change":"2024-05-10T03:31:26.821570+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.821570+0000","last_undegraded":"2024-05-10T03:31:26.821570+0000","last_fullsized":"2024-05-10T03:31:26.821570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:52.920514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.815559+0000","last_change":"2024-05-10T03:31:26.815559+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.815559+0000","last_undegraded":"2024-05-10T03:31:26.815559+0000","last_fullsized":"2024-05-10T03:31:26.815559+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.831161+0000","last_change":"2024-05-10T03:31:26.831161+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.831161+0000","last_undegraded":"2024-05-10T03:31:26.831161+0000","last_fullsized":"2024-05-10T03:31:26.831161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:20.956387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.839781+0000","last_change":"2024-05-10T03:31:26.839781+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.839781+0000","last_undegraded":"2024-05-10T03:31:26.839781+0000","last_fullsized":"2024-05-10T03:31:26.839781+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:00:57.396408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.824060+0000","last_change":"2024-05-10T03:31:26.824060+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.824060+0000","last_undegraded":"2024-05-10T03:31:26.824060+0000","last_fullsized":"2024-05-10T03:31:26.824060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:24.950508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.817518+0000","last_change":"2024-05-10T03:31:26.817518+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.817518+0000","last_undegraded":"2024-05-10T03:31:26.817518+0000","last_fullsized":"2024-05-10T03:31:26.817518+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:51:36.759651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.816486+0000","last_change":"2024-05-10T03:31:26.816486+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.816486+0000","last_undegraded":"2024-05-10T03:31:26.816486+0000","last_fullsized":"2024-05-10T03:31:26.816486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.824385+0000","last_change":"2024-05-10T03:31:26.824385+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.824385+0000","last_undegraded":"2024-05-10T03:31:26.824385+0000","last_fullsized":"2024-05-10T03:31:26.824385+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:43.991795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.817956+0000","last_change":"2024-05-10T03:31:26.817956+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.817956+0000","last_undegraded":"2024-05-10T03:31:26.817956+0000","last_fullsized":"2024-05-10T03:31:26.817956+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:24.950508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.830335+0000","last_change":"2024-05-10T03:31:26.830335+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.830335+0000","last_undegraded":"2024-05-10T03:31:26.830335+0000","last_fullsized":"2024-05-10T03:31:26.830335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:00:57.396408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.818945+0000","last_change":"2024-05-10T03:31:26.818945+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.818945+0000","last_undegraded":"2024-05-10T03:31:26.818945+0000","last_fullsized":"2024-05-10T03:31:26.818945+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:09.174922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.815537+0000","last_change":"2024-05-10T03:31:26.815537+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.815537+0000","last_undegraded":"2024-05-10T03:31:26.815537+0000","last_fullsized":"2024-05-10T03:31:26.815537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:34:45.541326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820456+0000","last_change":"2024-05-10T03:31:26.820456+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820456+0000","last_undegraded":"2024-05-10T03:31:26.820456+0000","last_fullsized":"2024-05-10T03:31:26.820456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:09.174922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.823930+0000","last_change":"2024-05-10T03:31:26.823930+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.823930+0000","last_undegraded":"2024-05-10T03:31:26.823930+0000","last_fullsized":"2024-05-10T03:31:26.823930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820682+0000","last_change":"2024-05-10T03:31:26.820682+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820682+0000","last_undegraded":"2024-05-10T03:31:26.820682+0000","last_fullsized":"2024-05-10T03:31:26.820682+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:23.509798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.851846+0000","last_change":"2024-05-10T03:31:26.847390+0000","last_active":"2024-05-10T03:31:26.851846+0000","last_peered":"2024-05-10T03:31:26.851846+0000","last_clean":"2024-05-10T03:31:26.851846+0000","last_became_active":"2024-05-10T03:31:26.847046+0000","last_became_peered":"2024-05-10T03:31:26.847046+0000","last_unstale":"2024-05-10T03:31:26.851846+0000","last_undegraded":"2024-05-10T03:31:26.851846+0000","last_fullsized":"2024-05-10T03:31:26.851846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:18.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820212+0000","last_change":"2024-05-10T03:31:26.820212+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820212+0000","last_undegraded":"2024-05-10T03:31:26.820212+0000","last_fullsized":"2024-05-10T03:31:26.820212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849224+0000","last_change":"2024-05-10T03:31:26.843441+0000","last_active":"2024-05-10T03:31:26.849224+0000","last_peered":"2024-05-10T03:31:26.849224+0000","last_clean":"2024-05-10T03:31:26.849224+0000","last_became_active":"2024-05-10T03:31:26.843089+0000","last_became_peered":"2024-05-10T03:31:26.843089+0000","last_unstale":"2024-05-10T03:31:26.849224+0000","last_undegraded":"2024-05-10T03:31:26.849224+0000","last_fullsized":"2024-05-10T03:31:26.849224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:03:32.235889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.834286+0000","last_change":"2024-05-10T03:31:26.834286+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.834286+0000","last_undegraded":"2024-05-10T03:31:26.834286+0000","last_fullsized":"2024-05-10T03:31:26.834286+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:34.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.850887+0000","last_change":"2024-05-10T03:31:26.847776+0000","last_active":"2024-05-10T03:31:26.850887+0000","last_peered":"2024-05-10T03:31:26.850887+0000","last_clean":"2024-05-10T03:31:26.850887+0000","last_became_active":"2024-05-10T03:31:26.845100+0000","last_became_peered":"2024-05-10T03:31:26.845100+0000","last_unstale":"2024-05-10T03:31:26.850887+0000","last_undegraded":"2024-05-10T03:31:26.850887+0000","last_fullsized":"2024-05-10T03:31:26.850887+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:06:21.809607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.832489+0000","last_change":"2024-05-10T03:31:26.832489+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.832489+0000","last_undegraded":"2024-05-10T03:31:26.832489+0000","last_fullsized":"2024-05-10T03:31:26.832489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:58.044652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.853842+0000","last_change":"2024-05-10T03:31:26.849723+0000","last_active":"2024-05-10T03:31:26.853842+0000","last_peered":"2024-05-10T03:31:26.853842+0000","last_clean":"2024-05-10T03:31:26.853842+0000","last_became_active":"2024-05-10T03:31:26.848734+0000","last_became_peered":"2024-05-10T03:31:26.848734+0000","last_unstale":"2024-05-10T03:31:26.853842+0000","last_undegraded":"2024-05-10T03:31:26.853842+0000","last_fullsized":"2024-05-10T03:31:26.853842+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:41:10.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.831232+0000","last_change":"2024-05-10T03:31:26.831232+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.831232+0000","last_undegraded":"2024-05-10T03:31:26.831232+0000","last_fullsized":"2024-05-10T03:31:26.831232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849309+0000","last_change":"2024-05-10T03:31:26.844933+0000","last_active":"2024-05-10T03:31:26.849309+0000","last_peered":"2024-05-10T03:31:26.849309+0000","last_clean":"2024-05-10T03:31:26.849309+0000","last_became_active":"2024-05-10T03:31:26.844536+0000","last_became_peered":"2024-05-10T03:31:26.844536+0000","last_unstale":"2024-05-10T03:31:26.849309+0000","last_undegraded":"2024-05-10T03:31:26.849309+0000","last_fullsized":"2024-05-10T03:31:26.849309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:20.235961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.837283+0000","last_change":"2024-05-10T03:31:26.837283+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.837283+0000","last_undegraded":"2024-05-10T03:31:26.837283+0000","last_fullsized":"2024-05-10T03:31:26.837283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:08.633258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.853391+0000","last_change":"2024-05-10T03:31:26.848362+0000","last_active":"2024-05-10T03:31:26.853391+0000","last_peered":"2024-05-10T03:31:26.853391+0000","last_clean":"2024-05-10T03:31:26.853391+0000","last_became_active":"2024-05-10T03:31:26.848023+0000","last_became_peered":"2024-05-10T03:31:26.848023+0000","last_unstale":"2024-05-10T03:31:26.853391+0000","last_undegraded":"2024-05-10T03:31:26.853391+0000","last_fullsized":"2024-05-10T03:31:26.853391+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:51:35.745245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.834664+0000","last_change":"2024-05-10T03:31:26.834664+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.834664+0000","last_undegraded":"2024-05-10T03:31:26.834664+0000","last_fullsized":"2024-05-10T03:31:26.834664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:24:40.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.850920+0000","last_change":"2024-05-10T03:31:26.848438+0000","last_active":"2024-05-10T03:31:26.850920+0000","last_peered":"2024-05-10T03:31:26.850920+0000","last_clean":"2024-05-10T03:31:26.850920+0000","last_became_active":"2024-05-10T03:31:26.848160+0000","last_became_peered":"2024-05-10T03:31:26.848160+0000","last_unstale":"2024-05-10T03:31:26.850920+0000","last_undegraded":"2024-05-10T03:31:26.850920+0000","last_fullsized":"2024-05-10T03:31:26.850920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:50.010449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.829361+0000","last_change":"2024-05-10T03:31:26.829361+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.829361+0000","last_undegraded":"2024-05-10T03:31:26.829361+0000","last_fullsized":"2024-05-10T03:31:26.829361+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:56.137036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.853874+0000","last_change":"2024-05-10T03:31:26.850037+0000","last_active":"2024-05-10T03:31:26.853874+0000","last_peered":"2024-05-10T03:31:26.853874+0000","last_clean":"2024-05-10T03:31:26.853874+0000","last_became_active":"2024-05-10T03:31:26.849717+0000","last_became_peered":"2024-05-10T03:31:26.849717+0000","last_unstale":"2024-05-10T03:31:26.853874+0000","last_undegraded":"2024-05-10T03:31:26.853874+0000","last_fullsized":"2024-05-10T03:31:26.853874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:43.257447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.821254+0000","last_change":"2024-05-10T03:31:26.821254+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.821254+0000","last_undegraded":"2024-05-10T03:31:26.821254+0000","last_fullsized":"2024-05-10T03:31:26.821254+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:17.322835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.850309+0000","last_change":"2024-05-10T03:31:26.848053+0000","last_active":"2024-05-10T03:31:26.850309+0000","last_peered":"2024-05-10T03:31:26.850309+0000","last_clean":"2024-05-10T03:31:26.850309+0000","last_became_active":"2024-05-10T03:31:26.847746+0000","last_became_peered":"2024-05-10T03:31:26.847746+0000","last_unstale":"2024-05-10T03:31:26.850309+0000","last_undegraded":"2024-05-10T03:31:26.850309+0000","last_fullsized":"2024-05-10T03:31:26.850309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:33.667505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.824738+0000","last_change":"2024-05-10T03:31:26.824738+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.824738+0000","last_undegraded":"2024-05-10T03:31:26.824738+0000","last_fullsized":"2024-05-10T03:31:26.824738+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:43.991795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849258+0000","last_change":"2024-05-10T03:31:26.844807+0000","last_active":"2024-05-10T03:31:26.849258+0000","last_peered":"2024-05-10T03:31:26.849258+0000","last_clean":"2024-05-10T03:31:26.849258+0000","last_became_active":"2024-05-10T03:31:26.844488+0000","last_became_peered":"2024-05-10T03:31:26.844488+0000","last_unstale":"2024-05-10T03:31:26.849258+0000","last_undegraded":"2024-05-10T03:31:26.849258+0000","last_fullsized":"2024-05-10T03:31:26.849258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:03:32.235889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.828792+0000","last_change":"2024-05-10T03:31:26.828792+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.828792+0000","last_undegraded":"2024-05-10T03:31:26.828792+0000","last_fullsized":"2024-05-10T03:31:26.828792+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:06.010217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849205+0000","last_change":"2024-05-10T03:31:26.844675+0000","last_active":"2024-05-10T03:31:26.849205+0000","last_peered":"2024-05-10T03:31:26.849205+0000","last_clean":"2024-05-10T03:31:26.849205+0000","last_became_active":"2024-05-10T03:31:26.844168+0000","last_became_peered":"2024-05-10T03:31:26.844168+0000","last_unstale":"2024-05-10T03:31:26.849205+0000","last_undegraded":"2024-05-10T03:31:26.849205+0000","last_fullsized":"2024-05-10T03:31:26.849205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:08.160516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.838790+0000","last_change":"2024-05-10T03:31:26.838790+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.838790+0000","last_undegraded":"2024-05-10T03:31:26.838790+0000","last_fullsized":"2024-05-10T03:31:26.838790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:06.010217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.846432+0000","last_change":"2024-05-10T03:31:26.841092+0000","last_active":"2024-05-10T03:31:26.846432+0000","last_peered":"2024-05-10T03:31:26.846432+0000","last_clean":"2024-05-10T03:31:26.846432+0000","last_became_active":"2024-05-10T03:31:26.839870+0000","last_became_peered":"2024-05-10T03:31:26.839870+0000","last_unstale":"2024-05-10T03:31:26.846432+0000","last_undegraded":"2024-05-10T03:31:26.846432+0000","last_fullsized":"2024-05-10T03:31:26.846432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:42.977389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.828983+0000","last_change":"2024-05-10T03:31:26.828983+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.828983+0000","last_undegraded":"2024-05-10T03:31:26.828983+0000","last_fullsized":"2024-05-10T03:31:26.828983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:34.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.847627+0000","last_change":"2024-05-10T03:31:26.843203+0000","last_active":"2024-05-10T03:31:26.847627+0000","last_peered":"2024-05-10T03:31:26.847627+0000","last_clean":"2024-05-10T03:31:26.847627+0000","last_became_active":"2024-05-10T03:31:26.842873+0000","last_became_peered":"2024-05-10T03:31:26.842873+0000","last_unstale":"2024-05-10T03:31:26.847627+0000","last_undegraded":"2024-05-10T03:31:26.847627+0000","last_fullsized":"2024-05-10T03:31:26.847627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:22.495392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.826215+0000","last_change":"2024-05-10T03:31:26.826215+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.826215+0000","last_undegraded":"2024-05-10T03:31:26.826215+0000","last_fullsized":"2024-05-10T03:31:26.826215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:41.040971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.850706+0000","last_change":"2024-05-10T03:31:26.846098+0000","last_active":"2024-05-10T03:31:26.850706+0000","last_peered":"2024-05-10T03:31:26.850706+0000","last_clean":"2024-05-10T03:31:26.850706+0000","last_became_active":"2024-05-10T03:31:26.845408+0000","last_became_peered":"2024-05-10T03:31:26.845408+0000","last_unstale":"2024-05-10T03:31:26.850706+0000","last_undegraded":"2024-05-10T03:31:26.850706+0000","last_fullsized":"2024-05-10T03:31:26.850706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:25:59.627178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.821651+0000","last_change":"2024-05-10T03:31:26.821651+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.821651+0000","last_undegraded":"2024-05-10T03:31:26.821651+0000","last_fullsized":"2024-05-10T03:31:26.821651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.851890+0000","last_change":"2024-05-10T03:31:26.847776+0000","last_active":"2024-05-10T03:31:26.851890+0000","last_peered":"2024-05-10T03:31:26.851890+0000","last_clean":"2024-05-10T03:31:26.851890+0000","last_became_active":"2024-05-10T03:31:26.846754+0000","last_became_peered":"2024-05-10T03:31:26.846754+0000","last_unstale":"2024-05-10T03:31:26.851890+0000","last_undegraded":"2024-05-10T03:31:26.851890+0000","last_fullsized":"2024-05-10T03:31:26.851890+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:25:59.627178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.828851+0000","last_change":"2024-05-10T03:31:26.828851+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.828851+0000","last_undegraded":"2024-05-10T03:31:26.828851+0000","last_fullsized":"2024-05-10T03:31:26.828851+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:03.580906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.853348+0000","last_change":"2024-05-10T03:31:26.845895+0000","last_active":"2024-05-10T03:31:26.853348+0000","last_peered":"2024-05-10T03:31:26.853348+0000","last_clean":"2024-05-10T03:31:26.853348+0000","last_became_active":"2024-05-10T03:31:26.845210+0000","last_became_peered":"2024-05-10T03:31:26.845210+0000","last_unstale":"2024-05-10T03:31:26.853348+0000","last_undegraded":"2024-05-10T03:31:26.853348+0000","last_fullsized":"2024-05-10T03:31:26.853348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.118579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.825501+0000","last_change":"2024-05-10T03:31:26.825501+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.825501+0000","last_undegraded":"2024-05-10T03:31:26.825501+0000","last_fullsized":"2024-05-10T03:31:26.825501+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:08.633258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.851911+0000","last_change":"2024-05-10T03:31:26.848467+0000","last_active":"2024-05-10T03:31:26.851911+0000","last_peered":"2024-05-10T03:31:26.851911+0000","last_clean":"2024-05-10T03:31:26.851911+0000","last_became_active":"2024-05-10T03:31:26.847896+0000","last_became_peered":"2024-05-10T03:31:26.847896+0000","last_unstale":"2024-05-10T03:31:26.851911+0000","last_undegraded":"2024-05-10T03:31:26.851911+0000","last_fullsized":"2024-05-10T03:31:26.851911+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:41:10.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.833002+0000","last_change":"2024-05-10T03:31:26.833002+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.833002+0000","last_undegraded":"2024-05-10T03:31:26.833002+0000","last_fullsized":"2024-05-10T03:31:26.833002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:49.589365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.853819+0000","last_change":"2024-05-10T03:31:26.849184+0000","last_active":"2024-05-10T03:31:26.853819+0000","last_peered":"2024-05-10T03:31:26.853819+0000","last_clean":"2024-05-10T03:31:26.853819+0000","last_became_active":"2024-05-10T03:31:26.848854+0000","last_became_peered":"2024-05-10T03:31:26.848854+0000","last_unstale":"2024-05-10T03:31:26.853819+0000","last_undegraded":"2024-05-10T03:31:26.853819+0000","last_fullsized":"2024-05-10T03:31:26.853819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:08.160516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.835772+0000","last_change":"2024-05-10T03:31:26.835772+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.835772+0000","last_undegraded":"2024-05-10T03:31:26.835772+0000","last_fullsized":"2024-05-10T03:31:26.835772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.839279+0000","last_change":"2024-05-10T03:31:26.834653+0000","last_active":"2024-05-10T03:31:26.839279+0000","last_peered":"2024-05-10T03:31:26.839279+0000","last_clean":"2024-05-10T03:31:26.839279+0000","last_became_active":"2024-05-10T03:31:26.834272+0000","last_became_peered":"2024-05-10T03:31:26.834272+0000","last_unstale":"2024-05-10T03:31:26.839279+0000","last_undegraded":"2024-05-10T03:31:26.839279+0000","last_fullsized":"2024-05-10T03:31:26.839279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:22.652785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.817498+0000","last_change":"2024-05-10T03:31:26.817498+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.817498+0000","last_undegraded":"2024-05-10T03:31:26.817498+0000","last_fullsized":"2024-05-10T03:31:26.817498+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:45.345427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.851599+0000","last_change":"2024-05-10T03:31:26.845483+0000","last_active":"2024-05-10T03:31:26.851599+0000","last_peered":"2024-05-10T03:31:26.851599+0000","last_clean":"2024-05-10T03:31:26.851599+0000","last_became_active":"2024-05-10T03:31:26.845147+0000","last_became_peered":"2024-05-10T03:31:26.845147+0000","last_unstale":"2024-05-10T03:31:26.851599+0000","last_undegraded":"2024-05-10T03:31:26.851599+0000","last_fullsized":"2024-05-10T03:31:26.851599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:18.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820922+0000","last_change":"2024-05-10T03:31:26.820922+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820922+0000","last_undegraded":"2024-05-10T03:31:26.820922+0000","last_fullsized":"2024-05-10T03:31:26.820922+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:34:45.541326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.845884+0000","last_change":"2024-05-10T03:31:26.844097+0000","last_active":"2024-05-10T03:31:26.845884+0000","last_peered":"2024-05-10T03:31:26.845884+0000","last_clean":"2024-05-10T03:31:26.845884+0000","last_became_active":"2024-05-10T03:31:26.840267+0000","last_became_peered":"2024-05-10T03:31:26.840267+0000","last_unstale":"2024-05-10T03:31:26.845884+0000","last_undegraded":"2024-05-10T03:31:26.845884+0000","last_fullsized":"2024-05-10T03:31:26.845884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:18.634658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.827683+0000","last_change":"2024-05-10T03:31:26.827683+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.827683+0000","last_undegraded":"2024-05-10T03:31:26.827683+0000","last_fullsized":"2024-05-10T03:31:26.827683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:23.509798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.850335+0000","last_change":"2024-05-10T03:31:26.848815+0000","last_active":"2024-05-10T03:31:26.850335+0000","last_peered":"2024-05-10T03:31:26.850335+0000","last_clean":"2024-05-10T03:31:26.850335+0000","last_became_active":"2024-05-10T03:31:26.845792+0000","last_became_peered":"2024-05-10T03:31:26.845792+0000","last_unstale":"2024-05-10T03:31:26.850335+0000","last_undegraded":"2024-05-10T03:31:26.850335+0000","last_fullsized":"2024-05-10T03:31:26.850335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:51.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.834048+0000","last_change":"2024-05-10T03:31:26.834048+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.834048+0000","last_undegraded":"2024-05-10T03:31:26.834048+0000","last_fullsized":"2024-05-10T03:31:26.834048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:04:30.420174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.846409+0000","last_change":"2024-05-10T03:31:26.841418+0000","last_active":"2024-05-10T03:31:26.846409+0000","last_peered":"2024-05-10T03:31:26.846409+0000","last_clean":"2024-05-10T03:31:26.846409+0000","last_became_active":"2024-05-10T03:31:26.840974+0000","last_became_peered":"2024-05-10T03:31:26.840974+0000","last_unstale":"2024-05-10T03:31:26.846409+0000","last_undegraded":"2024-05-10T03:31:26.846409+0000","last_fullsized":"2024-05-10T03:31:26.846409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:51.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.828269+0000","last_change":"2024-05-10T03:31:26.828269+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.828269+0000","last_undegraded":"2024-05-10T03:31:26.828269+0000","last_fullsized":"2024-05-10T03:31:26.828269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:55:32.871492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.842449+0000","last_change":"2024-05-10T03:31:26.835674+0000","last_active":"2024-05-10T03:31:26.842449+0000","last_peered":"2024-05-10T03:31:26.842449+0000","last_clean":"2024-05-10T03:31:26.842449+0000","last_became_active":"2024-05-10T03:31:26.832501+0000","last_became_peered":"2024-05-10T03:31:26.832501+0000","last_unstale":"2024-05-10T03:31:26.842449+0000","last_undegraded":"2024-05-10T03:31:26.842449+0000","last_fullsized":"2024-05-10T03:31:26.842449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:19.915231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820054+0000","last_change":"2024-05-10T03:31:26.820054+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820054+0000","last_undegraded":"2024-05-10T03:31:26.820054+0000","last_fullsized":"2024-05-10T03:31:26.820054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.854224+0000","last_change":"2024-05-10T03:31:26.852059+0000","last_active":"2024-05-10T03:31:26.854224+0000","last_peered":"2024-05-10T03:31:26.854224+0000","last_clean":"2024-05-10T03:31:26.854224+0000","last_became_active":"2024-05-10T03:31:26.851696+0000","last_became_peered":"2024-05-10T03:31:26.851696+0000","last_unstale":"2024-05-10T03:31:26.854224+0000","last_undegraded":"2024-05-10T03:31:26.854224+0000","last_fullsized":"2024-05-10T03:31:26.854224+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:43.257447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.823378+0000","last_change":"2024-05-10T03:31:26.823378+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.823378+0000","last_undegraded":"2024-05-10T03:31:26.823378+0000","last_fullsized":"2024-05-10T03:31:26.823378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849033+0000","last_change":"2024-05-10T03:31:26.846345+0000","last_active":"2024-05-10T03:31:26.849033+0000","last_peered":"2024-05-10T03:31:26.849033+0000","last_clean":"2024-05-10T03:31:26.849033+0000","last_became_active":"2024-05-10T03:31:26.845936+0000","last_became_peered":"2024-05-10T03:31:26.845936+0000","last_unstale":"2024-05-10T03:31:26.849033+0000","last_undegraded":"2024-05-10T03:31:26.849033+0000","last_fullsized":"2024-05-10T03:31:26.849033+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:28.175997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.824213+0000","last_change":"2024-05-10T03:31:26.824213+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.824213+0000","last_undegraded":"2024-05-10T03:31:26.824213+0000","last_fullsized":"2024-05-10T03:31:26.824213+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:09:56.820377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849961+0000","last_change":"2024-05-10T03:31:26.840909+0000","last_active":"2024-05-10T03:31:26.849961+0000","last_peered":"2024-05-10T03:31:26.849961+0000","last_clean":"2024-05-10T03:31:26.849961+0000","last_became_active":"2024-05-10T03:31:26.840728+0000","last_became_peered":"2024-05-10T03:31:26.840728+0000","last_unstale":"2024-05-10T03:31:26.849961+0000","last_undegraded":"2024-05-10T03:31:26.849961+0000","last_fullsized":"2024-05-10T03:31:26.849961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:37:34.541792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.826133+0000","last_change":"2024-05-10T03:31:26.826133+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.826133+0000","last_undegraded":"2024-05-10T03:31:26.826133+0000","last_fullsized":"2024-05-10T03:31:26.826133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:24:40.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849160+0000","last_change":"2024-05-10T03:31:26.841896+0000","last_active":"2024-05-10T03:31:26.849160+0000","last_peered":"2024-05-10T03:31:26.849160+0000","last_clean":"2024-05-10T03:31:26.849160+0000","last_became_active":"2024-05-10T03:31:26.841326+0000","last_became_peered":"2024-05-10T03:31:26.841326+0000","last_unstale":"2024-05-10T03:31:26.849160+0000","last_undegraded":"2024-05-10T03:31:26.849160+0000","last_fullsized":"2024-05-10T03:31:26.849160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.820582+0000","last_change":"2024-05-10T03:31:26.820582+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.820582+0000","last_undegraded":"2024-05-10T03:31:26.820582+0000","last_fullsized":"2024-05-10T03:31:26.820582+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:44.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.849030+0000","last_change":"2024-05-10T03:31:26.842721+0000","last_active":"2024-05-10T03:31:26.849030+0000","last_peered":"2024-05-10T03:31:26.849030+0000","last_clean":"2024-05-10T03:31:26.849030+0000","last_became_active":"2024-05-10T03:31:26.841642+0000","last_became_peered":"2024-05-10T03:31:26.841642+0000","last_unstale":"2024-05-10T03:31:26.849030+0000","last_undegraded":"2024-05-10T03:31:26.849030+0000","last_fullsized":"2024-05-10T03:31:26.849030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:06:21.809607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.840357+0000","last_change":"2024-05-10T03:31:26.840357+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.840357+0000","last_undegraded":"2024-05-10T03:31:26.840357+0000","last_fullsized":"2024-05-10T03:31:26.840357+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.856412+0000","last_change":"2024-05-10T03:31:26.852845+0000","last_active":"2024-05-10T03:31:26.856412+0000","last_peered":"2024-05-10T03:31:26.856412+0000","last_clean":"2024-05-10T03:31:26.856412+0000","last_became_active":"2024-05-10T03:31:26.852546+0000","last_became_peered":"2024-05-10T03:31:26.852546+0000","last_unstale":"2024-05-10T03:31:26.856412+0000","last_undegraded":"2024-05-10T03:31:26.856412+0000","last_fullsized":"2024-05-10T03:31:26.856412+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:20.235961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.822909+0000","last_change":"2024-05-10T03:31:26.822909+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.822909+0000","last_undegraded":"2024-05-10T03:31:26.822909+0000","last_fullsized":"2024-05-10T03:31:26.822909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.847195+0000","last_change":"2024-05-10T03:31:26.843177+0000","last_active":"2024-05-10T03:31:26.847195+0000","last_peered":"2024-05-10T03:31:26.847195+0000","last_clean":"2024-05-10T03:31:26.847195+0000","last_became_active":"2024-05-10T03:31:26.842849+0000","last_became_peered":"2024-05-10T03:31:26.842849+0000","last_unstale":"2024-05-10T03:31:26.847195+0000","last_undegraded":"2024-05-10T03:31:26.847195+0000","last_fullsized":"2024-05-10T03:31:26.847195+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:16.701268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-05-10T03:31:26.836888+0000","last_change":"2024-05-10T03:31:26.836888+0000","last_active":"2024-05-10T03:31:26.786622+0000","last_peered":"2024-05-10T03:31:26.786622+0000","last_clean":"2024-05-10T03:31:26.786622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:31:26.836888+0000","last_undegraded":"2024-05-10T03:31:26.836888+0000","last_fullsized":"2024-05-10T03:31:26.836888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-10T03:31:26.851501+0000","last_change":"2024-05-10T03:31:26.844431+0000","last_active":"2024-05-10T03:31:26.851501+0000","last_peered":"2024-05-10T03:31:26.851501+0000","last_clean":"2024-05-10T03:31:26.851501+0000","last_became_active":"2024-05-10T03:31:26.843146+0000","last_became_peered":"2024-05-10T03:31:26.843146+0000","last_unstale":"2024-05-10T03:31:26.851501+0000","last_undegraded":"2024-05-10T03:31:26.851501+0000","last_fullsized":"2024-05-10T03:31:26.851501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:44.331021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-05-10T03:31:25.808014+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:31:25.808014+0000","last_peered":"2024-05-10T03:31:25.808014+0000","last_clean":"2024-05-10T03:31:25.808014+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:31:25.808014+0000","last_undegraded":"2024-05-10T03:31:25.808014+0000","last_fullsized":"2024-05-10T03:31:25.808014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":769184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.4930000000000001}]},{"osd":1,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.333},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.94999999999999996}]},{"osd":2,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.022},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.4630000000000001}]},{"osd":3,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.5629999999999999}]},{"osd":4,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.208}]},{"osd":5,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.6060000000000001}]},{"osd":6,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.4279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":185411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.876},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2470000000000001}]},{"osd":1,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.4139999999999999}]},{"osd":2,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":1.3680000000000001}]},{"osd":3,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.647,"15min":1.647},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.536},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.615}]},{"osd":4,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.339,"5min":1.339,"15min":1.339},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.7649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.2430000000000001}]},{"osd":1,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1659999999999999}]},{"osd":2,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.631},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.3220000000000001}]},{"osd":3,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.4390000000000001}]},{"osd":4,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.349}]},{"osd":6,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.899},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.7509999999999999}]},{"osd":7,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.819,"5min":1.819,"15min":1.819},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.8049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.069},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.131}]},{"osd":1,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.649},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.335}]},{"osd":2,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.216},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.3979999999999999}]},{"osd":3,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.571},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.7789999999999999}]},{"osd":5,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.804,"5min":1.804,"15min":1.804},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":5.3529999999999998,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.863}]},{"osd":7,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":2.012}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":425984,"data_stored":178816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.496}]},{"osd":1,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.751},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.383}]},{"osd":5,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.542},{"interface":"front","average":{"1min":1.819,"5min":1.819,"15min":1.819},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.2390000000000001}]},{"osd":7,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.089,"5min":1.089,"15min":1.089},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.4450000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":178816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.431}]},{"osd":1,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.514}]},{"osd":3,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.353},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.9650000000000001}]},{"osd":4,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.6830000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":2.036},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.1190000000000002}]},{"osd":6,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.569}]},{"osd":7,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.849}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":425984,"data_stored":178816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.194}]},{"osd":3,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.4570000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":3.8029999999999999,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.143},{"interface":"front","average":{"1min":1.661,"5min":1.661,"15min":1.661},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.5449999999999999}]},{"osd":6,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":3.48,"5min":3.48,"15min":3.48},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.627}]},{"osd":7,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.222,"5min":1.222,"15min":1.222},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.591}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":1019904,"data_stored":769184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.214}]},{"osd":2,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.4550000000000001}]},{"osd":4,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":3.1040000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.395},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.95699999999999996}]},{"osd":5,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.022},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.357}]},{"osd":6,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.532},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.591}]},{"osd":7,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.177,"5min":1.177,"15min":1.177},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.26}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:31:32.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:31:32.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.858+0000 7ff4d3600640 1 Processor -- start 2024-05-10T03:31:32.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.858+0000 7ff4d3600640 1 -- start start 2024-05-10T03:31:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.861+0000 7ff4d3600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff4cc15ab30 0x7ff4cc0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.861+0000 7ff4d3600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.861+0000 7ff4d3600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0cd710 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.861+0000 7ff4d3600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc0cdfb0 con 0x7ff4cc15ab30 2024-05-10T03:31:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.861+0000 7ff4d3600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc06b4d0 con 0x7ff4cc15af40 2024-05-10T03:31:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.862+0000 7ff4d1600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.862+0000 7ff4d0c00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff4cc0cdba0 0x7ff4cc06a9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:33578/0 (socket says 172.21.15.55:33578) 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.862+0000 7ff4d0c00640 1 -- 172.21.15.55:0/2630013809 learned_addr learned my addr 172.21.15.55:0/2630013809 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.862+0000 7ff4cbe00640 1 --2- 172.21.15.55:0/2630013809 >> v2:172.21.15.162:3300/0 conn(0x7ff4cc15ab30 0x7ff4cc0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.863+0000 7ff4d1600640 1 -- 172.21.15.55:0/2630013809 >> v2:172.21.15.162:3300/0 conn(0x7ff4cc15ab30 msgr2=0x7ff4cc0715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.863+0000 7ff4d1600640 1 --2- 172.21.15.55:0/2630013809 >> v2:172.21.15.162:3300/0 conn(0x7ff4cc15ab30 0x7ff4cc0715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.863+0000 7ff4d1600640 1 -- 172.21.15.55:0/2630013809 >> v1:172.21.15.162:6789/0 conn(0x7ff4cc0cdba0 legacy=0x7ff4cc06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.863+0000 7ff4d1600640 1 -- 172.21.15.55:0/2630013809 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc06b1f0 con 0x7ff4cc15af40 2024-05-10T03:31:32.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.864+0000 7ff4d1600640 1 --2- 172.21.15.55:0/2630013809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc071b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d90 tx=0x7ff4c402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0465c0ed574b948 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.865+0000 7ff4cb400640 1 -- 172.21.15.55:0/2630013809 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4007740 con 0x7ff4cc15af40 2024-05-10T03:31:32.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.865+0000 7ff4cb400640 1 -- 172.21.15.55:0/2630013809 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c402d050 con 0x7ff4cc15af40 2024-05-10T03:31:32.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.865+0000 7ff4cb400640 1 -- 172.21.15.55:0/2630013809 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4007ac0 con 0x7ff4cc15af40 2024-05-10T03:31:32.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 -- 172.21.15.55:0/2630013809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 msgr2=0x7ff4cc071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2630013809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc071b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009d90 tx=0x7ff4c402a530 comp rx=0 tx=0).stop 2024-05-10T03:31:32.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 -- 172.21.15.55:0/2630013809 shutdown_connections 2024-05-10T03:31:32.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2630013809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc071b20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2630013809 >> v2:172.21.15.162:3300/0 conn(0x7ff4cc15ab30 0x7ff4cc0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 -- 172.21.15.55:0/2630013809 >> 172.21.15.55:0/2630013809 conn(0x7ff4cc0cbb90 msgr2=0x7ff4cc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:32.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.866+0000 7ff4d3600640 1 -- 172.21.15.55:0/2630013809 shutdown_connections 2024-05-10T03:31:32.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.867+0000 7ff4d3600640 1 -- 172.21.15.55:0/2630013809 wait complete. 2024-05-10T03:31:32.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.870+0000 7ff4d3600640 1 Processor -- start 2024-05-10T03:31:32.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.870+0000 7ff4d3600640 1 -- start start 2024-05-10T03:31:32.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc15ab30 0x7ff4cc14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc13f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc14b060 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc14d270 con 0x7ff4cc15ab30 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d3600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc14d4c0 con 0x7ff4cc15af40 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37026/0 (socket says 172.21.15.55:37026) 2024-05-10T03:31:32.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.872+0000 7ff4d0c00640 1 -- 172.21.15.55:0/2675158018 learned_addr learned my addr 172.21.15.55:0/2675158018 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:32.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 -- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 msgr2=0x7ff4cc13f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d1600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4cbe00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc15ab30 0x7ff4cc14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc13f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 -- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc15ab30 msgr2=0x7ff4cc14e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc15ab30 0x7ff4cc14e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 -- 172.21.15.55:0/2675158018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009a70 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.873+0000 7ff4d0c00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c4b0 tx=0x7ff4bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.874+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc005ce0 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.874+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc005e40 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.874+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc007270 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.874+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc14d7a0 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.874+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc14dd60 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.875+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc066f60 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.876+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4bc0073d0 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.877+0000 7ff4c8c00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 0x7ff4ac06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:32.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.877+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff4bc0888d0 con 0x7ff4cc0cdba0 2024-05-10T03:31:32.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.878+0000 7ff4cbe00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 0x7ff4ac06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:32.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.879+0000 7ff4cbe00640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 0x7ff4ac06d140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ae650 tx=0x7ff4b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:32.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:32.883+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff4bc053a10 con 0x7ff4cc0cdba0 2024-05-10T03:31:33.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.079+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4cc0cd3a0 con 0x7ff4ac06ac30 2024-05-10T03:31:33.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.103+0000 7ff4c8c00640 1 -- 172.21.15.55:0/2675158018 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286930 (secure 0 0 0) 0x7ff4b80095e0 con 0x7ff4ac06ac30 2024-05-10T03:31:33.109 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:33.112 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:31:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 msgr2=0x7ff4ac06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 0x7ff4ac06d140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ae650 tx=0x7ff4b8008040 comp rx=0 tx=0).stop 2024-05-10T03:31:33.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 msgr2=0x7ff4cc14cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c4b0 tx=0x7ff4bc00c970 comp rx=0 tx=0).stop 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 shutdown_connections 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4cc15af40 0x7ff4cc13f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4ac06ac30 0x7ff4ac06d140 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc15ab30 0x7ff4cc14e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 --2- 172.21.15.55:0/2675158018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4cc0cdba0 0x7ff4cc14cc00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 >> 172.21.15.55:0/2675158018 conn(0x7ff4cc0cbb90 msgr2=0x7ff4cc1506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.110+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 shutdown_connections 2024-05-10T03:31:33.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.111+0000 7ff4d3600640 1 -- 172.21.15.55:0/2675158018 wait complete. 2024-05-10T03:31:33.142 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-05-10T03:31:32.991474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237424,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737296,"statfs":{"total":773094113280,"available":772850991104,"internally_reserved":0,"allocated":4956160,"data_stored":2778468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183478"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434517+0000","last_change":"2024-05-10T03:31:27.430839+0000","last_active":"2024-05-10T03:31:27.434517+0000","last_peered":"2024-05-10T03:31:27.434517+0000","last_clean":"2024-05-10T03:31:27.434517+0000","last_became_active":"2024-05-10T03:31:27.430293+0000","last_became_peered":"2024-05-10T03:31:27.430293+0000","last_unstale":"2024-05-10T03:31:27.434517+0000","last_undegraded":"2024-05-10T03:31:27.434517+0000","last_fullsized":"2024-05-10T03:31:27.434517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:39.335145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.415765+0000","last_change":"2024-05-10T03:31:27.405662+0000","last_active":"2024-05-10T03:31:27.415765+0000","last_peered":"2024-05-10T03:31:27.415765+0000","last_clean":"2024-05-10T03:31:27.415765+0000","last_became_active":"2024-05-10T03:31:27.405336+0000","last_became_peered":"2024-05-10T03:31:27.405336+0000","last_unstale":"2024-05-10T03:31:27.415765+0000","last_undegraded":"2024-05-10T03:31:27.415765+0000","last_fullsized":"2024-05-10T03:31:27.415765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:54.811349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.443965+0000","last_change":"2024-05-10T03:31:27.442236+0000","last_active":"2024-05-10T03:31:27.443965+0000","last_peered":"2024-05-10T03:31:27.443965+0000","last_clean":"2024-05-10T03:31:27.443965+0000","last_became_active":"2024-05-10T03:31:27.441676+0000","last_became_peered":"2024-05-10T03:31:27.441676+0000","last_unstale":"2024-05-10T03:31:27.443965+0000","last_undegraded":"2024-05-10T03:31:27.443965+0000","last_fullsized":"2024-05-10T03:31:27.443965+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:17.715674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433891+0000","last_change":"2024-05-10T03:31:27.417522+0000","last_active":"2024-05-10T03:31:27.433891+0000","last_peered":"2024-05-10T03:31:27.433891+0000","last_clean":"2024-05-10T03:31:27.433891+0000","last_became_active":"2024-05-10T03:31:27.417245+0000","last_became_peered":"2024-05-10T03:31:27.417245+0000","last_unstale":"2024-05-10T03:31:27.433891+0000","last_undegraded":"2024-05-10T03:31:27.433891+0000","last_fullsized":"2024-05-10T03:31:27.433891+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:19.464687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433854+0000","last_change":"2024-05-10T03:31:27.420318+0000","last_active":"2024-05-10T03:31:27.433854+0000","last_peered":"2024-05-10T03:31:27.433854+0000","last_clean":"2024-05-10T03:31:27.433854+0000","last_became_active":"2024-05-10T03:31:27.419992+0000","last_became_peered":"2024-05-10T03:31:27.419992+0000","last_unstale":"2024-05-10T03:31:27.433854+0000","last_undegraded":"2024-05-10T03:31:27.433854+0000","last_fullsized":"2024-05-10T03:31:27.433854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:19.132985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.421448+0000","last_change":"2024-05-10T03:31:27.413915+0000","last_active":"2024-05-10T03:31:27.421448+0000","last_peered":"2024-05-10T03:31:27.421448+0000","last_clean":"2024-05-10T03:31:27.421448+0000","last_became_active":"2024-05-10T03:31:27.413569+0000","last_became_peered":"2024-05-10T03:31:27.413569+0000","last_unstale":"2024-05-10T03:31:27.421448+0000","last_undegraded":"2024-05-10T03:31:27.421448+0000","last_fullsized":"2024-05-10T03:31:27.421448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.438375+0000","last_change":"2024-05-10T03:31:27.430257+0000","last_active":"2024-05-10T03:31:27.438375+0000","last_peered":"2024-05-10T03:31:27.438375+0000","last_clean":"2024-05-10T03:31:27.438375+0000","last_became_active":"2024-05-10T03:31:27.417302+0000","last_became_peered":"2024-05-10T03:31:27.417302+0000","last_unstale":"2024-05-10T03:31:27.438375+0000","last_undegraded":"2024-05-10T03:31:27.438375+0000","last_fullsized":"2024-05-10T03:31:27.438375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:55:32.871492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.445491+0000","last_change":"2024-05-10T03:31:27.444200+0000","last_active":"2024-05-10T03:31:27.445491+0000","last_peered":"2024-05-10T03:31:27.445491+0000","last_clean":"2024-05-10T03:31:27.445491+0000","last_became_active":"2024-05-10T03:31:27.442708+0000","last_became_peered":"2024-05-10T03:31:27.442708+0000","last_unstale":"2024-05-10T03:31:27.445491+0000","last_undegraded":"2024-05-10T03:31:27.445491+0000","last_fullsized":"2024-05-10T03:31:27.445491+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:09:23.429272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.438001+0000","last_change":"2024-05-10T03:31:27.436571+0000","last_active":"2024-05-10T03:31:27.438001+0000","last_peered":"2024-05-10T03:31:27.438001+0000","last_clean":"2024-05-10T03:31:27.438001+0000","last_became_active":"2024-05-10T03:31:27.436219+0000","last_became_peered":"2024-05-10T03:31:27.436219+0000","last_unstale":"2024-05-10T03:31:27.438001+0000","last_undegraded":"2024-05-10T03:31:27.438001+0000","last_fullsized":"2024-05-10T03:31:27.438001+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:51.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447661+0000","last_change":"2024-05-10T03:31:27.442481+0000","last_active":"2024-05-10T03:31:27.447661+0000","last_peered":"2024-05-10T03:31:27.447661+0000","last_clean":"2024-05-10T03:31:27.447661+0000","last_became_active":"2024-05-10T03:31:27.442280+0000","last_became_peered":"2024-05-10T03:31:27.442280+0000","last_unstale":"2024-05-10T03:31:27.447661+0000","last_undegraded":"2024-05-10T03:31:27.447661+0000","last_fullsized":"2024-05-10T03:31:27.447661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:38:15.955193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.439437+0000","last_change":"2024-05-10T03:31:27.422910+0000","last_active":"2024-05-10T03:31:27.439437+0000","last_peered":"2024-05-10T03:31:27.439437+0000","last_clean":"2024-05-10T03:31:27.439437+0000","last_became_active":"2024-05-10T03:31:27.422544+0000","last_became_peered":"2024-05-10T03:31:27.422544+0000","last_unstale":"2024-05-10T03:31:27.439437+0000","last_undegraded":"2024-05-10T03:31:27.439437+0000","last_fullsized":"2024-05-10T03:31:27.439437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:54.811349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434168+0000","last_change":"2024-05-10T03:31:27.425984+0000","last_active":"2024-05-10T03:31:27.434168+0000","last_peered":"2024-05-10T03:31:27.434168+0000","last_clean":"2024-05-10T03:31:27.434168+0000","last_became_active":"2024-05-10T03:31:27.425000+0000","last_became_peered":"2024-05-10T03:31:27.425000+0000","last_unstale":"2024-05-10T03:31:27.434168+0000","last_undegraded":"2024-05-10T03:31:27.434168+0000","last_fullsized":"2024-05-10T03:31:27.434168+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447651+0000","last_change":"2024-05-10T03:31:27.442329+0000","last_active":"2024-05-10T03:31:27.447651+0000","last_peered":"2024-05-10T03:31:27.447651+0000","last_clean":"2024-05-10T03:31:27.447651+0000","last_became_active":"2024-05-10T03:31:27.442011+0000","last_became_peered":"2024-05-10T03:31:27.442011+0000","last_unstale":"2024-05-10T03:31:27.447651+0000","last_undegraded":"2024-05-10T03:31:27.447651+0000","last_fullsized":"2024-05-10T03:31:27.447651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:46:53.551974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.437442+0000","last_change":"2024-05-10T03:31:27.432681+0000","last_active":"2024-05-10T03:31:27.437442+0000","last_peered":"2024-05-10T03:31:27.437442+0000","last_clean":"2024-05-10T03:31:27.437442+0000","last_became_active":"2024-05-10T03:31:27.432300+0000","last_became_peered":"2024-05-10T03:31:27.432300+0000","last_unstale":"2024-05-10T03:31:27.437442+0000","last_undegraded":"2024-05-10T03:31:27.437442+0000","last_fullsized":"2024-05-10T03:31:27.437442+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:43:51.232581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.430431+0000","last_change":"2024-05-10T03:31:27.416300+0000","last_active":"2024-05-10T03:31:27.430431+0000","last_peered":"2024-05-10T03:31:27.430431+0000","last_clean":"2024-05-10T03:31:27.430431+0000","last_became_active":"2024-05-10T03:31:27.415932+0000","last_became_peered":"2024-05-10T03:31:27.415932+0000","last_unstale":"2024-05-10T03:31:27.430431+0000","last_undegraded":"2024-05-10T03:31:27.430431+0000","last_fullsized":"2024-05-10T03:31:27.430431+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:04:01.205590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.449437+0000","last_change":"2024-05-10T03:31:27.446563+0000","last_active":"2024-05-10T03:31:27.449437+0000","last_peered":"2024-05-10T03:31:27.449437+0000","last_clean":"2024-05-10T03:31:27.449437+0000","last_became_active":"2024-05-10T03:31:27.446067+0000","last_became_peered":"2024-05-10T03:31:27.446067+0000","last_unstale":"2024-05-10T03:31:27.449437+0000","last_undegraded":"2024-05-10T03:31:27.449437+0000","last_fullsized":"2024-05-10T03:31:27.449437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.451808+0000","last_change":"2024-05-10T03:31:27.448272+0000","last_active":"2024-05-10T03:31:27.451808+0000","last_peered":"2024-05-10T03:31:27.451808+0000","last_clean":"2024-05-10T03:31:27.451808+0000","last_became_active":"2024-05-10T03:31:27.447978+0000","last_became_peered":"2024-05-10T03:31:27.447978+0000","last_unstale":"2024-05-10T03:31:27.451808+0000","last_undegraded":"2024-05-10T03:31:27.451808+0000","last_fullsized":"2024-05-10T03:31:27.451808+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:17.715674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.444705+0000","last_change":"2024-05-10T03:31:27.436677+0000","last_active":"2024-05-10T03:31:27.444705+0000","last_peered":"2024-05-10T03:31:27.444705+0000","last_clean":"2024-05-10T03:31:27.444705+0000","last_became_active":"2024-05-10T03:31:27.436331+0000","last_became_peered":"2024-05-10T03:31:27.436331+0000","last_unstale":"2024-05-10T03:31:27.444705+0000","last_undegraded":"2024-05-10T03:31:27.444705+0000","last_fullsized":"2024-05-10T03:31:27.444705+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.424679+0000","last_change":"2024-05-10T03:31:27.412738+0000","last_active":"2024-05-10T03:31:27.424679+0000","last_peered":"2024-05-10T03:31:27.424679+0000","last_clean":"2024-05-10T03:31:27.424679+0000","last_became_active":"2024-05-10T03:31:27.412371+0000","last_became_peered":"2024-05-10T03:31:27.412371+0000","last_unstale":"2024-05-10T03:31:27.424679+0000","last_undegraded":"2024-05-10T03:31:27.424679+0000","last_fullsized":"2024-05-10T03:31:27.424679+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:52.920514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.444639+0000","last_change":"2024-05-10T03:31:27.435854+0000","last_active":"2024-05-10T03:31:27.444639+0000","last_peered":"2024-05-10T03:31:27.444639+0000","last_clean":"2024-05-10T03:31:27.444639+0000","last_became_active":"2024-05-10T03:31:27.435408+0000","last_became_peered":"2024-05-10T03:31:27.435408+0000","last_unstale":"2024-05-10T03:31:27.444639+0000","last_undegraded":"2024-05-10T03:31:27.444639+0000","last_fullsized":"2024-05-10T03:31:27.444639+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.424341+0000","last_change":"2024-05-10T03:31:27.418283+0000","last_active":"2024-05-10T03:31:27.424341+0000","last_peered":"2024-05-10T03:31:27.424341+0000","last_clean":"2024-05-10T03:31:27.424341+0000","last_became_active":"2024-05-10T03:31:27.417963+0000","last_became_peered":"2024-05-10T03:31:27.417963+0000","last_unstale":"2024-05-10T03:31:27.424341+0000","last_undegraded":"2024-05-10T03:31:27.424341+0000","last_fullsized":"2024-05-10T03:31:27.424341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:20.956387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.424341+0000","last_change":"2024-05-10T03:31:27.413889+0000","last_active":"2024-05-10T03:31:27.424341+0000","last_peered":"2024-05-10T03:31:27.424341+0000","last_clean":"2024-05-10T03:31:27.424341+0000","last_became_active":"2024-05-10T03:31:27.413554+0000","last_became_peered":"2024-05-10T03:31:27.413554+0000","last_unstale":"2024-05-10T03:31:27.424341+0000","last_undegraded":"2024-05-10T03:31:27.424341+0000","last_fullsized":"2024-05-10T03:31:27.424341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:00:57.396408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.446054+0000","last_change":"2024-05-10T03:31:27.435013+0000","last_active":"2024-05-10T03:31:27.446054+0000","last_peered":"2024-05-10T03:31:27.446054+0000","last_clean":"2024-05-10T03:31:27.446054+0000","last_became_active":"2024-05-10T03:31:27.434779+0000","last_became_peered":"2024-05-10T03:31:27.434779+0000","last_unstale":"2024-05-10T03:31:27.446054+0000","last_undegraded":"2024-05-10T03:31:27.446054+0000","last_fullsized":"2024-05-10T03:31:27.446054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:24.950508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433655+0000","last_change":"2024-05-10T03:31:27.417597+0000","last_active":"2024-05-10T03:31:27.433655+0000","last_peered":"2024-05-10T03:31:27.433655+0000","last_clean":"2024-05-10T03:31:27.433655+0000","last_became_active":"2024-05-10T03:31:27.417250+0000","last_became_peered":"2024-05-10T03:31:27.417250+0000","last_unstale":"2024-05-10T03:31:27.433655+0000","last_undegraded":"2024-05-10T03:31:27.433655+0000","last_fullsized":"2024-05-10T03:31:27.433655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:51:36.759651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.444780+0000","last_change":"2024-05-10T03:31:27.437287+0000","last_active":"2024-05-10T03:31:27.444780+0000","last_peered":"2024-05-10T03:31:27.444780+0000","last_clean":"2024-05-10T03:31:27.444780+0000","last_became_active":"2024-05-10T03:31:27.436924+0000","last_became_peered":"2024-05-10T03:31:27.436924+0000","last_unstale":"2024-05-10T03:31:27.444780+0000","last_undegraded":"2024-05-10T03:31:27.444780+0000","last_fullsized":"2024-05-10T03:31:27.444780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433219+0000","last_change":"2024-05-10T03:31:27.426745+0000","last_active":"2024-05-10T03:31:27.433219+0000","last_peered":"2024-05-10T03:31:27.433219+0000","last_clean":"2024-05-10T03:31:27.433219+0000","last_became_active":"2024-05-10T03:31:27.425452+0000","last_became_peered":"2024-05-10T03:31:27.425452+0000","last_unstale":"2024-05-10T03:31:27.433219+0000","last_undegraded":"2024-05-10T03:31:27.433219+0000","last_fullsized":"2024-05-10T03:31:27.433219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:43.991795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.446889+0000","last_change":"2024-05-10T03:31:27.439468+0000","last_active":"2024-05-10T03:31:27.446889+0000","last_peered":"2024-05-10T03:31:27.446889+0000","last_clean":"2024-05-10T03:31:27.446889+0000","last_became_active":"2024-05-10T03:31:27.436954+0000","last_became_peered":"2024-05-10T03:31:27.436954+0000","last_unstale":"2024-05-10T03:31:27.446889+0000","last_undegraded":"2024-05-10T03:31:27.446889+0000","last_fullsized":"2024-05-10T03:31:27.446889+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:24.950508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.430113+0000","last_change":"2024-05-10T03:31:27.423990+0000","last_active":"2024-05-10T03:31:27.430113+0000","last_peered":"2024-05-10T03:31:27.430113+0000","last_clean":"2024-05-10T03:31:27.430113+0000","last_became_active":"2024-05-10T03:31:27.423522+0000","last_became_peered":"2024-05-10T03:31:27.423522+0000","last_unstale":"2024-05-10T03:31:27.430113+0000","last_undegraded":"2024-05-10T03:31:27.430113+0000","last_fullsized":"2024-05-10T03:31:27.430113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:00:57.396408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.446867+0000","last_change":"2024-05-10T03:31:27.439048+0000","last_active":"2024-05-10T03:31:27.446867+0000","last_peered":"2024-05-10T03:31:27.446867+0000","last_clean":"2024-05-10T03:31:27.446867+0000","last_became_active":"2024-05-10T03:31:27.432840+0000","last_became_peered":"2024-05-10T03:31:27.432840+0000","last_unstale":"2024-05-10T03:31:27.446867+0000","last_undegraded":"2024-05-10T03:31:27.446867+0000","last_fullsized":"2024-05-10T03:31:27.446867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:09.174922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.444702+0000","last_change":"2024-05-10T03:31:27.435782+0000","last_active":"2024-05-10T03:31:27.444702+0000","last_peered":"2024-05-10T03:31:27.444702+0000","last_clean":"2024-05-10T03:31:27.444702+0000","last_became_active":"2024-05-10T03:31:27.434438+0000","last_became_peered":"2024-05-10T03:31:27.434438+0000","last_unstale":"2024-05-10T03:31:27.444702+0000","last_undegraded":"2024-05-10T03:31:27.444702+0000","last_fullsized":"2024-05-10T03:31:27.444702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:34:45.541326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447284+0000","last_change":"2024-05-10T03:31:27.440019+0000","last_active":"2024-05-10T03:31:27.447284+0000","last_peered":"2024-05-10T03:31:27.447284+0000","last_clean":"2024-05-10T03:31:27.447284+0000","last_became_active":"2024-05-10T03:31:27.439643+0000","last_became_peered":"2024-05-10T03:31:27.439643+0000","last_unstale":"2024-05-10T03:31:27.447284+0000","last_undegraded":"2024-05-10T03:31:27.447284+0000","last_fullsized":"2024-05-10T03:31:27.447284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:09.174922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.415677+0000","last_change":"2024-05-10T03:31:27.405026+0000","last_active":"2024-05-10T03:31:27.415677+0000","last_peered":"2024-05-10T03:31:27.415677+0000","last_clean":"2024-05-10T03:31:27.415677+0000","last_became_active":"2024-05-10T03:31:27.404718+0000","last_became_peered":"2024-05-10T03:31:27.404718+0000","last_unstale":"2024-05-10T03:31:27.415677+0000","last_undegraded":"2024-05-10T03:31:27.415677+0000","last_fullsized":"2024-05-10T03:31:27.415677+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447581+0000","last_change":"2024-05-10T03:31:27.441274+0000","last_active":"2024-05-10T03:31:27.447581+0000","last_peered":"2024-05-10T03:31:27.447581+0000","last_clean":"2024-05-10T03:31:27.447581+0000","last_became_active":"2024-05-10T03:31:27.440965+0000","last_became_peered":"2024-05-10T03:31:27.440965+0000","last_unstale":"2024-05-10T03:31:27.447581+0000","last_undegraded":"2024-05-10T03:31:27.447581+0000","last_fullsized":"2024-05-10T03:31:27.447581+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:23.509798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.437167+0000","last_change":"2024-05-10T03:31:26.847390+0000","last_active":"2024-05-10T03:31:27.437167+0000","last_peered":"2024-05-10T03:31:27.437167+0000","last_clean":"2024-05-10T03:31:27.437167+0000","last_became_active":"2024-05-10T03:31:26.847046+0000","last_became_peered":"2024-05-10T03:31:26.847046+0000","last_unstale":"2024-05-10T03:31:27.437167+0000","last_undegraded":"2024-05-10T03:31:27.437167+0000","last_fullsized":"2024-05-10T03:31:27.437167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:18.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433107+0000","last_change":"2024-05-10T03:31:27.425677+0000","last_active":"2024-05-10T03:31:27.433107+0000","last_peered":"2024-05-10T03:31:27.433107+0000","last_clean":"2024-05-10T03:31:27.433107+0000","last_became_active":"2024-05-10T03:31:27.425241+0000","last_became_peered":"2024-05-10T03:31:27.425241+0000","last_unstale":"2024-05-10T03:31:27.433107+0000","last_undegraded":"2024-05-10T03:31:27.433107+0000","last_fullsized":"2024-05-10T03:31:27.433107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.431839+0000","last_change":"2024-05-10T03:31:26.843441+0000","last_active":"2024-05-10T03:31:27.431839+0000","last_peered":"2024-05-10T03:31:27.431839+0000","last_clean":"2024-05-10T03:31:27.431839+0000","last_became_active":"2024-05-10T03:31:26.843089+0000","last_became_peered":"2024-05-10T03:31:26.843089+0000","last_unstale":"2024-05-10T03:31:27.431839+0000","last_undegraded":"2024-05-10T03:31:27.431839+0000","last_fullsized":"2024-05-10T03:31:27.431839+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:03:32.235889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434966+0000","last_change":"2024-05-10T03:31:27.429056+0000","last_active":"2024-05-10T03:31:27.434966+0000","last_peered":"2024-05-10T03:31:27.434966+0000","last_clean":"2024-05-10T03:31:27.434966+0000","last_became_active":"2024-05-10T03:31:27.427959+0000","last_became_peered":"2024-05-10T03:31:27.427959+0000","last_unstale":"2024-05-10T03:31:27.434966+0000","last_undegraded":"2024-05-10T03:31:27.434966+0000","last_fullsized":"2024-05-10T03:31:27.434966+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:34.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.437254+0000","last_change":"2024-05-10T03:31:26.847776+0000","last_active":"2024-05-10T03:31:27.437254+0000","last_peered":"2024-05-10T03:31:27.437254+0000","last_clean":"2024-05-10T03:31:27.437254+0000","last_became_active":"2024-05-10T03:31:26.845100+0000","last_became_peered":"2024-05-10T03:31:26.845100+0000","last_unstale":"2024-05-10T03:31:27.437254+0000","last_undegraded":"2024-05-10T03:31:27.437254+0000","last_fullsized":"2024-05-10T03:31:27.437254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:06:21.809607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.435065+0000","last_change":"2024-05-10T03:31:27.432363+0000","last_active":"2024-05-10T03:31:27.435065+0000","last_peered":"2024-05-10T03:31:27.435065+0000","last_clean":"2024-05-10T03:31:27.435065+0000","last_became_active":"2024-05-10T03:31:27.432067+0000","last_became_peered":"2024-05-10T03:31:27.432067+0000","last_unstale":"2024-05-10T03:31:27.435065+0000","last_undegraded":"2024-05-10T03:31:27.435065+0000","last_fullsized":"2024-05-10T03:31:27.435065+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:58.044652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.437631+0000","last_change":"2024-05-10T03:31:26.849723+0000","last_active":"2024-05-10T03:31:27.437631+0000","last_peered":"2024-05-10T03:31:27.437631+0000","last_clean":"2024-05-10T03:31:27.437631+0000","last_became_active":"2024-05-10T03:31:26.848734+0000","last_became_peered":"2024-05-10T03:31:26.848734+0000","last_unstale":"2024-05-10T03:31:27.437631+0000","last_undegraded":"2024-05-10T03:31:27.437631+0000","last_fullsized":"2024-05-10T03:31:27.437631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:41:10.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.435201+0000","last_change":"2024-05-10T03:31:27.422572+0000","last_active":"2024-05-10T03:31:27.435201+0000","last_peered":"2024-05-10T03:31:27.435201+0000","last_clean":"2024-05-10T03:31:27.435201+0000","last_became_active":"2024-05-10T03:31:27.422260+0000","last_became_peered":"2024-05-10T03:31:27.422260+0000","last_unstale":"2024-05-10T03:31:27.435201+0000","last_undegraded":"2024-05-10T03:31:27.435201+0000","last_fullsized":"2024-05-10T03:31:27.435201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.440390+0000","last_change":"2024-05-10T03:31:26.844933+0000","last_active":"2024-05-10T03:31:27.440390+0000","last_peered":"2024-05-10T03:31:27.440390+0000","last_clean":"2024-05-10T03:31:27.440390+0000","last_became_active":"2024-05-10T03:31:26.844536+0000","last_became_peered":"2024-05-10T03:31:26.844536+0000","last_unstale":"2024-05-10T03:31:27.440390+0000","last_undegraded":"2024-05-10T03:31:27.440390+0000","last_fullsized":"2024-05-10T03:31:27.440390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:20.235961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.430451+0000","last_change":"2024-05-10T03:31:27.416939+0000","last_active":"2024-05-10T03:31:27.430451+0000","last_peered":"2024-05-10T03:31:27.430451+0000","last_clean":"2024-05-10T03:31:27.430451+0000","last_became_active":"2024-05-10T03:31:27.416640+0000","last_became_peered":"2024-05-10T03:31:27.416640+0000","last_unstale":"2024-05-10T03:31:27.430451+0000","last_undegraded":"2024-05-10T03:31:27.430451+0000","last_fullsized":"2024-05-10T03:31:27.430451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:08.633258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.440287+0000","last_change":"2024-05-10T03:31:26.848362+0000","last_active":"2024-05-10T03:31:27.440287+0000","last_peered":"2024-05-10T03:31:27.440287+0000","last_clean":"2024-05-10T03:31:27.440287+0000","last_became_active":"2024-05-10T03:31:26.848023+0000","last_became_peered":"2024-05-10T03:31:26.848023+0000","last_unstale":"2024-05-10T03:31:27.440287+0000","last_undegraded":"2024-05-10T03:31:27.440287+0000","last_fullsized":"2024-05-10T03:31:27.440287+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:51:35.745245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447337+0000","last_change":"2024-05-10T03:31:27.442970+0000","last_active":"2024-05-10T03:31:27.447337+0000","last_peered":"2024-05-10T03:31:27.447337+0000","last_clean":"2024-05-10T03:31:27.447337+0000","last_became_active":"2024-05-10T03:31:27.442616+0000","last_became_peered":"2024-05-10T03:31:27.442616+0000","last_unstale":"2024-05-10T03:31:27.447337+0000","last_undegraded":"2024-05-10T03:31:27.447337+0000","last_fullsized":"2024-05-10T03:31:27.447337+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:24:40.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.397235+0000","last_change":"2024-05-10T03:31:26.848438+0000","last_active":"2024-05-10T03:31:27.397235+0000","last_peered":"2024-05-10T03:31:27.397235+0000","last_clean":"2024-05-10T03:31:27.397235+0000","last_became_active":"2024-05-10T03:31:26.848160+0000","last_became_peered":"2024-05-10T03:31:26.848160+0000","last_unstale":"2024-05-10T03:31:27.397235+0000","last_undegraded":"2024-05-10T03:31:27.397235+0000","last_fullsized":"2024-05-10T03:31:27.397235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:20:50.010449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434496+0000","last_change":"2024-05-10T03:31:27.427680+0000","last_active":"2024-05-10T03:31:27.434496+0000","last_peered":"2024-05-10T03:31:27.434496+0000","last_clean":"2024-05-10T03:31:27.434496+0000","last_became_active":"2024-05-10T03:31:27.427367+0000","last_became_peered":"2024-05-10T03:31:27.427367+0000","last_unstale":"2024-05-10T03:31:27.434496+0000","last_undegraded":"2024-05-10T03:31:27.434496+0000","last_fullsized":"2024-05-10T03:31:27.434496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:58:56.137036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.396443+0000","last_change":"2024-05-10T03:31:26.850037+0000","last_active":"2024-05-10T03:31:27.396443+0000","last_peered":"2024-05-10T03:31:27.396443+0000","last_clean":"2024-05-10T03:31:27.396443+0000","last_became_active":"2024-05-10T03:31:26.849717+0000","last_became_peered":"2024-05-10T03:31:26.849717+0000","last_unstale":"2024-05-10T03:31:27.396443+0000","last_undegraded":"2024-05-10T03:31:27.396443+0000","last_fullsized":"2024-05-10T03:31:27.396443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:43.257447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433915+0000","last_change":"2024-05-10T03:31:27.417939+0000","last_active":"2024-05-10T03:31:27.433915+0000","last_peered":"2024-05-10T03:31:27.433915+0000","last_clean":"2024-05-10T03:31:27.433915+0000","last_became_active":"2024-05-10T03:31:27.417705+0000","last_became_peered":"2024-05-10T03:31:27.417705+0000","last_unstale":"2024-05-10T03:31:27.433915+0000","last_undegraded":"2024-05-10T03:31:27.433915+0000","last_fullsized":"2024-05-10T03:31:27.433915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:46:17.322835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.403279+0000","last_change":"2024-05-10T03:31:26.848053+0000","last_active":"2024-05-10T03:31:27.403279+0000","last_peered":"2024-05-10T03:31:27.403279+0000","last_clean":"2024-05-10T03:31:27.403279+0000","last_became_active":"2024-05-10T03:31:26.847746+0000","last_became_peered":"2024-05-10T03:31:26.847746+0000","last_unstale":"2024-05-10T03:31:27.403279+0000","last_undegraded":"2024-05-10T03:31:27.403279+0000","last_fullsized":"2024-05-10T03:31:27.403279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:33.667505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.445821+0000","last_change":"2024-05-10T03:31:27.434152+0000","last_active":"2024-05-10T03:31:27.445821+0000","last_peered":"2024-05-10T03:31:27.445821+0000","last_clean":"2024-05-10T03:31:27.445821+0000","last_became_active":"2024-05-10T03:31:27.432202+0000","last_became_peered":"2024-05-10T03:31:27.432202+0000","last_unstale":"2024-05-10T03:31:27.445821+0000","last_undegraded":"2024-05-10T03:31:27.445821+0000","last_fullsized":"2024-05-10T03:31:27.445821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:43.991795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.409868+0000","last_change":"2024-05-10T03:31:26.844807+0000","last_active":"2024-05-10T03:31:27.409868+0000","last_peered":"2024-05-10T03:31:27.409868+0000","last_clean":"2024-05-10T03:31:27.409868+0000","last_became_active":"2024-05-10T03:31:26.844488+0000","last_became_peered":"2024-05-10T03:31:26.844488+0000","last_unstale":"2024-05-10T03:31:27.409868+0000","last_undegraded":"2024-05-10T03:31:27.409868+0000","last_fullsized":"2024-05-10T03:31:27.409868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:03:32.235889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.435335+0000","last_change":"2024-05-10T03:31:27.428750+0000","last_active":"2024-05-10T03:31:27.435335+0000","last_peered":"2024-05-10T03:31:27.435335+0000","last_clean":"2024-05-10T03:31:27.435335+0000","last_became_active":"2024-05-10T03:31:27.428350+0000","last_became_peered":"2024-05-10T03:31:27.428350+0000","last_unstale":"2024-05-10T03:31:27.435335+0000","last_undegraded":"2024-05-10T03:31:27.435335+0000","last_fullsized":"2024-05-10T03:31:27.435335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:06.010217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447023+0000","last_change":"2024-05-10T03:31:26.844675+0000","last_active":"2024-05-10T03:31:27.447023+0000","last_peered":"2024-05-10T03:31:27.447023+0000","last_clean":"2024-05-10T03:31:27.447023+0000","last_became_active":"2024-05-10T03:31:26.844168+0000","last_became_peered":"2024-05-10T03:31:26.844168+0000","last_unstale":"2024-05-10T03:31:27.447023+0000","last_undegraded":"2024-05-10T03:31:27.447023+0000","last_fullsized":"2024-05-10T03:31:27.447023+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:08.160516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.425139+0000","last_change":"2024-05-10T03:31:27.418700+0000","last_active":"2024-05-10T03:31:27.425139+0000","last_peered":"2024-05-10T03:31:27.425139+0000","last_clean":"2024-05-10T03:31:27.425139+0000","last_became_active":"2024-05-10T03:31:27.418422+0000","last_became_peered":"2024-05-10T03:31:27.418422+0000","last_unstale":"2024-05-10T03:31:27.425139+0000","last_undegraded":"2024-05-10T03:31:27.425139+0000","last_fullsized":"2024-05-10T03:31:27.425139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:06.010217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.449800+0000","last_change":"2024-05-10T03:31:26.841092+0000","last_active":"2024-05-10T03:31:27.449800+0000","last_peered":"2024-05-10T03:31:27.449800+0000","last_clean":"2024-05-10T03:31:27.449800+0000","last_became_active":"2024-05-10T03:31:26.839870+0000","last_became_peered":"2024-05-10T03:31:26.839870+0000","last_unstale":"2024-05-10T03:31:27.449800+0000","last_undegraded":"2024-05-10T03:31:27.449800+0000","last_fullsized":"2024-05-10T03:31:27.449800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:53:42.977389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.442648+0000","last_change":"2024-05-10T03:31:27.440884+0000","last_active":"2024-05-10T03:31:27.442648+0000","last_peered":"2024-05-10T03:31:27.442648+0000","last_clean":"2024-05-10T03:31:27.442648+0000","last_became_active":"2024-05-10T03:31:27.440501+0000","last_became_peered":"2024-05-10T03:31:27.440501+0000","last_unstale":"2024-05-10T03:31:27.442648+0000","last_undegraded":"2024-05-10T03:31:27.442648+0000","last_fullsized":"2024-05-10T03:31:27.442648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:34.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.429270+0000","last_change":"2024-05-10T03:31:26.843203+0000","last_active":"2024-05-10T03:31:27.429270+0000","last_peered":"2024-05-10T03:31:27.429270+0000","last_clean":"2024-05-10T03:31:27.429270+0000","last_became_active":"2024-05-10T03:31:26.842873+0000","last_became_peered":"2024-05-10T03:31:26.842873+0000","last_unstale":"2024-05-10T03:31:27.429270+0000","last_undegraded":"2024-05-10T03:31:27.429270+0000","last_fullsized":"2024-05-10T03:31:27.429270+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:22.495392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.429225+0000","last_change":"2024-05-10T03:31:27.420286+0000","last_active":"2024-05-10T03:31:27.429225+0000","last_peered":"2024-05-10T03:31:27.429225+0000","last_clean":"2024-05-10T03:31:27.429225+0000","last_became_active":"2024-05-10T03:31:27.419527+0000","last_became_peered":"2024-05-10T03:31:27.419527+0000","last_unstale":"2024-05-10T03:31:27.429225+0000","last_undegraded":"2024-05-10T03:31:27.429225+0000","last_fullsized":"2024-05-10T03:31:27.429225+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:41.040971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.399351+0000","last_change":"2024-05-10T03:31:26.846098+0000","last_active":"2024-05-10T03:31:27.399351+0000","last_peered":"2024-05-10T03:31:27.399351+0000","last_clean":"2024-05-10T03:31:27.399351+0000","last_became_active":"2024-05-10T03:31:26.845408+0000","last_became_peered":"2024-05-10T03:31:26.845408+0000","last_unstale":"2024-05-10T03:31:27.399351+0000","last_undegraded":"2024-05-10T03:31:27.399351+0000","last_fullsized":"2024-05-10T03:31:27.399351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:25:59.627178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447614+0000","last_change":"2024-05-10T03:31:27.441604+0000","last_active":"2024-05-10T03:31:27.447614+0000","last_peered":"2024-05-10T03:31:27.447614+0000","last_clean":"2024-05-10T03:31:27.447614+0000","last_became_active":"2024-05-10T03:31:27.440033+0000","last_became_peered":"2024-05-10T03:31:27.440033+0000","last_unstale":"2024-05-10T03:31:27.447614+0000","last_undegraded":"2024-05-10T03:31:27.447614+0000","last_fullsized":"2024-05-10T03:31:27.447614+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.416950+0000","last_change":"2024-05-10T03:31:26.847776+0000","last_active":"2024-05-10T03:31:27.416950+0000","last_peered":"2024-05-10T03:31:27.416950+0000","last_clean":"2024-05-10T03:31:27.416950+0000","last_became_active":"2024-05-10T03:31:26.846754+0000","last_became_peered":"2024-05-10T03:31:26.846754+0000","last_unstale":"2024-05-10T03:31:27.416950+0000","last_undegraded":"2024-05-10T03:31:27.416950+0000","last_fullsized":"2024-05-10T03:31:27.416950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:25:59.627178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.429197+0000","last_change":"2024-05-10T03:31:27.418849+0000","last_active":"2024-05-10T03:31:27.429197+0000","last_peered":"2024-05-10T03:31:27.429197+0000","last_clean":"2024-05-10T03:31:27.429197+0000","last_became_active":"2024-05-10T03:31:27.416292+0000","last_became_peered":"2024-05-10T03:31:27.416292+0000","last_unstale":"2024-05-10T03:31:27.429197+0000","last_undegraded":"2024-05-10T03:31:27.429197+0000","last_fullsized":"2024-05-10T03:31:27.429197+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:03.580906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.444893+0000","last_change":"2024-05-10T03:31:26.845895+0000","last_active":"2024-05-10T03:31:27.444893+0000","last_peered":"2024-05-10T03:31:27.444893+0000","last_clean":"2024-05-10T03:31:27.444893+0000","last_became_active":"2024-05-10T03:31:26.845210+0000","last_became_peered":"2024-05-10T03:31:26.845210+0000","last_unstale":"2024-05-10T03:31:27.444893+0000","last_undegraded":"2024-05-10T03:31:27.444893+0000","last_fullsized":"2024-05-10T03:31:27.444893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.118579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.435867+0000","last_change":"2024-05-10T03:31:27.433839+0000","last_active":"2024-05-10T03:31:27.435867+0000","last_peered":"2024-05-10T03:31:27.435867+0000","last_clean":"2024-05-10T03:31:27.435867+0000","last_became_active":"2024-05-10T03:31:27.432117+0000","last_became_peered":"2024-05-10T03:31:27.432117+0000","last_unstale":"2024-05-10T03:31:27.435867+0000","last_undegraded":"2024-05-10T03:31:27.435867+0000","last_fullsized":"2024-05-10T03:31:27.435867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:08.633258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447760+0000","last_change":"2024-05-10T03:31:26.848467+0000","last_active":"2024-05-10T03:31:27.447760+0000","last_peered":"2024-05-10T03:31:27.447760+0000","last_clean":"2024-05-10T03:31:27.447760+0000","last_became_active":"2024-05-10T03:31:26.847896+0000","last_became_peered":"2024-05-10T03:31:26.847896+0000","last_unstale":"2024-05-10T03:31:27.447760+0000","last_undegraded":"2024-05-10T03:31:27.447760+0000","last_fullsized":"2024-05-10T03:31:27.447760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:41:10.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.439465+0000","last_change":"2024-05-10T03:31:27.423041+0000","last_active":"2024-05-10T03:31:27.439465+0000","last_peered":"2024-05-10T03:31:27.439465+0000","last_clean":"2024-05-10T03:31:27.439465+0000","last_became_active":"2024-05-10T03:31:27.422593+0000","last_became_peered":"2024-05-10T03:31:27.422593+0000","last_unstale":"2024-05-10T03:31:27.439465+0000","last_undegraded":"2024-05-10T03:31:27.439465+0000","last_fullsized":"2024-05-10T03:31:27.439465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:49.589365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.445449+0000","last_change":"2024-05-10T03:31:26.849184+0000","last_active":"2024-05-10T03:31:27.445449+0000","last_peered":"2024-05-10T03:31:27.445449+0000","last_clean":"2024-05-10T03:31:27.445449+0000","last_became_active":"2024-05-10T03:31:26.848854+0000","last_became_peered":"2024-05-10T03:31:26.848854+0000","last_unstale":"2024-05-10T03:31:27.445449+0000","last_undegraded":"2024-05-10T03:31:27.445449+0000","last_fullsized":"2024-05-10T03:31:27.445449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:08.160516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433156+0000","last_change":"2024-05-10T03:31:27.426093+0000","last_active":"2024-05-10T03:31:27.433156+0000","last_peered":"2024-05-10T03:31:27.433156+0000","last_clean":"2024-05-10T03:31:27.433156+0000","last_became_active":"2024-05-10T03:31:27.424400+0000","last_became_peered":"2024-05-10T03:31:27.424400+0000","last_unstale":"2024-05-10T03:31:27.433156+0000","last_undegraded":"2024-05-10T03:31:27.433156+0000","last_fullsized":"2024-05-10T03:31:27.433156+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:29:08.459272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.424801+0000","last_change":"2024-05-10T03:31:26.834653+0000","last_active":"2024-05-10T03:31:27.424801+0000","last_peered":"2024-05-10T03:31:27.424801+0000","last_clean":"2024-05-10T03:31:27.424801+0000","last_became_active":"2024-05-10T03:31:26.834272+0000","last_became_peered":"2024-05-10T03:31:26.834272+0000","last_unstale":"2024-05-10T03:31:27.424801+0000","last_undegraded":"2024-05-10T03:31:27.424801+0000","last_fullsized":"2024-05-10T03:31:27.424801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:40:22.652785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.423265+0000","last_change":"2024-05-10T03:31:27.410783+0000","last_active":"2024-05-10T03:31:27.423265+0000","last_peered":"2024-05-10T03:31:27.423265+0000","last_clean":"2024-05-10T03:31:27.423265+0000","last_became_active":"2024-05-10T03:31:27.410461+0000","last_became_peered":"2024-05-10T03:31:27.410461+0000","last_unstale":"2024-05-10T03:31:27.423265+0000","last_undegraded":"2024-05-10T03:31:27.423265+0000","last_fullsized":"2024-05-10T03:31:27.423265+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:45.345427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.401901+0000","last_change":"2024-05-10T03:31:26.845483+0000","last_active":"2024-05-10T03:31:27.401901+0000","last_peered":"2024-05-10T03:31:27.401901+0000","last_clean":"2024-05-10T03:31:27.401901+0000","last_became_active":"2024-05-10T03:31:26.845147+0000","last_became_peered":"2024-05-10T03:31:26.845147+0000","last_unstale":"2024-05-10T03:31:27.401901+0000","last_undegraded":"2024-05-10T03:31:27.401901+0000","last_fullsized":"2024-05-10T03:31:27.401901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:18.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447728+0000","last_change":"2024-05-10T03:31:27.442683+0000","last_active":"2024-05-10T03:31:27.447728+0000","last_peered":"2024-05-10T03:31:27.447728+0000","last_clean":"2024-05-10T03:31:27.447728+0000","last_became_active":"2024-05-10T03:31:27.441716+0000","last_became_peered":"2024-05-10T03:31:27.441716+0000","last_unstale":"2024-05-10T03:31:27.447728+0000","last_undegraded":"2024-05-10T03:31:27.447728+0000","last_fullsized":"2024-05-10T03:31:27.447728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:34:45.541326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.430196+0000","last_change":"2024-05-10T03:31:26.844097+0000","last_active":"2024-05-10T03:31:27.430196+0000","last_peered":"2024-05-10T03:31:27.430196+0000","last_clean":"2024-05-10T03:31:27.430196+0000","last_became_active":"2024-05-10T03:31:26.840267+0000","last_became_peered":"2024-05-10T03:31:26.840267+0000","last_unstale":"2024-05-10T03:31:27.430196+0000","last_undegraded":"2024-05-10T03:31:27.430196+0000","last_fullsized":"2024-05-10T03:31:27.430196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:18.634658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434192+0000","last_change":"2024-05-10T03:31:27.429024+0000","last_active":"2024-05-10T03:31:27.434192+0000","last_peered":"2024-05-10T03:31:27.434192+0000","last_clean":"2024-05-10T03:31:27.434192+0000","last_became_active":"2024-05-10T03:31:27.428416+0000","last_became_peered":"2024-05-10T03:31:27.428416+0000","last_unstale":"2024-05-10T03:31:27.434192+0000","last_undegraded":"2024-05-10T03:31:27.434192+0000","last_fullsized":"2024-05-10T03:31:27.434192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:23.509798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.396444+0000","last_change":"2024-05-10T03:31:26.848815+0000","last_active":"2024-05-10T03:31:27.396444+0000","last_peered":"2024-05-10T03:31:27.396444+0000","last_clean":"2024-05-10T03:31:27.396444+0000","last_became_active":"2024-05-10T03:31:26.845792+0000","last_became_peered":"2024-05-10T03:31:26.845792+0000","last_unstale":"2024-05-10T03:31:27.396444+0000","last_undegraded":"2024-05-10T03:31:27.396444+0000","last_fullsized":"2024-05-10T03:31:27.396444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:51.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.436210+0000","last_change":"2024-05-10T03:31:27.427611+0000","last_active":"2024-05-10T03:31:27.436210+0000","last_peered":"2024-05-10T03:31:27.436210+0000","last_clean":"2024-05-10T03:31:27.436210+0000","last_became_active":"2024-05-10T03:31:27.426835+0000","last_became_peered":"2024-05-10T03:31:27.426835+0000","last_unstale":"2024-05-10T03:31:27.436210+0000","last_undegraded":"2024-05-10T03:31:27.436210+0000","last_fullsized":"2024-05-10T03:31:27.436210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:04:30.420174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.418883+0000","last_change":"2024-05-10T03:31:26.841418+0000","last_active":"2024-05-10T03:31:27.418883+0000","last_peered":"2024-05-10T03:31:27.418883+0000","last_clean":"2024-05-10T03:31:27.418883+0000","last_became_active":"2024-05-10T03:31:26.840974+0000","last_became_peered":"2024-05-10T03:31:26.840974+0000","last_unstale":"2024-05-10T03:31:27.418883+0000","last_undegraded":"2024-05-10T03:31:27.418883+0000","last_fullsized":"2024-05-10T03:31:27.418883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:51.906108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.432375+0000","last_change":"2024-05-10T03:31:27.425943+0000","last_active":"2024-05-10T03:31:27.432375+0000","last_peered":"2024-05-10T03:31:27.432375+0000","last_clean":"2024-05-10T03:31:27.432375+0000","last_became_active":"2024-05-10T03:31:27.423719+0000","last_became_peered":"2024-05-10T03:31:27.423719+0000","last_unstale":"2024-05-10T03:31:27.432375+0000","last_undegraded":"2024-05-10T03:31:27.432375+0000","last_fullsized":"2024-05-10T03:31:27.432375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:55:32.871492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.397752+0000","last_change":"2024-05-10T03:31:26.835674+0000","last_active":"2024-05-10T03:31:27.397752+0000","last_peered":"2024-05-10T03:31:27.397752+0000","last_clean":"2024-05-10T03:31:27.397752+0000","last_became_active":"2024-05-10T03:31:26.832501+0000","last_became_peered":"2024-05-10T03:31:26.832501+0000","last_unstale":"2024-05-10T03:31:27.397752+0000","last_undegraded":"2024-05-10T03:31:27.397752+0000","last_fullsized":"2024-05-10T03:31:27.397752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:19.915231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.423884+0000","last_change":"2024-05-10T03:31:27.414268+0000","last_active":"2024-05-10T03:31:27.423884+0000","last_peered":"2024-05-10T03:31:27.423884+0000","last_clean":"2024-05-10T03:31:27.423884+0000","last_became_active":"2024-05-10T03:31:27.413802+0000","last_became_peered":"2024-05-10T03:31:27.413802+0000","last_unstale":"2024-05-10T03:31:27.423884+0000","last_undegraded":"2024-05-10T03:31:27.423884+0000","last_fullsized":"2024-05-10T03:31:27.423884+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:39.684321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.445823+0000","last_change":"2024-05-10T03:31:26.852059+0000","last_active":"2024-05-10T03:31:27.445823+0000","last_peered":"2024-05-10T03:31:27.445823+0000","last_clean":"2024-05-10T03:31:27.445823+0000","last_became_active":"2024-05-10T03:31:26.851696+0000","last_became_peered":"2024-05-10T03:31:26.851696+0000","last_unstale":"2024-05-10T03:31:27.445823+0000","last_undegraded":"2024-05-10T03:31:27.445823+0000","last_fullsized":"2024-05-10T03:31:27.445823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:43.257447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.445849+0000","last_change":"2024-05-10T03:31:27.434176+0000","last_active":"2024-05-10T03:31:27.445849+0000","last_peered":"2024-05-10T03:31:27.445849+0000","last_clean":"2024-05-10T03:31:27.445849+0000","last_became_active":"2024-05-10T03:31:27.432751+0000","last_became_peered":"2024-05-10T03:31:27.432751+0000","last_unstale":"2024-05-10T03:31:27.445849+0000","last_undegraded":"2024-05-10T03:31:27.445849+0000","last_fullsized":"2024-05-10T03:31:27.445849+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:53:51.129373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.396505+0000","last_change":"2024-05-10T03:31:26.846345+0000","last_active":"2024-05-10T03:31:27.396505+0000","last_peered":"2024-05-10T03:31:27.396505+0000","last_clean":"2024-05-10T03:31:27.396505+0000","last_became_active":"2024-05-10T03:31:26.845936+0000","last_became_peered":"2024-05-10T03:31:26.845936+0000","last_unstale":"2024-05-10T03:31:27.396505+0000","last_undegraded":"2024-05-10T03:31:27.396505+0000","last_fullsized":"2024-05-10T03:31:27.396505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:19:28.175997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.429670+0000","last_change":"2024-05-10T03:31:27.420919+0000","last_active":"2024-05-10T03:31:27.429670+0000","last_peered":"2024-05-10T03:31:27.429670+0000","last_clean":"2024-05-10T03:31:27.429670+0000","last_became_active":"2024-05-10T03:31:27.420597+0000","last_became_peered":"2024-05-10T03:31:27.420597+0000","last_unstale":"2024-05-10T03:31:27.429670+0000","last_undegraded":"2024-05-10T03:31:27.429670+0000","last_fullsized":"2024-05-10T03:31:27.429670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:09:56.820377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.412557+0000","last_change":"2024-05-10T03:31:26.840909+0000","last_active":"2024-05-10T03:31:27.412557+0000","last_peered":"2024-05-10T03:31:27.412557+0000","last_clean":"2024-05-10T03:31:27.412557+0000","last_became_active":"2024-05-10T03:31:26.840728+0000","last_became_peered":"2024-05-10T03:31:26.840728+0000","last_unstale":"2024-05-10T03:31:27.412557+0000","last_undegraded":"2024-05-10T03:31:27.412557+0000","last_fullsized":"2024-05-10T03:31:27.412557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:37:34.541792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.432269+0000","last_change":"2024-05-10T03:31:27.415486+0000","last_active":"2024-05-10T03:31:27.432269+0000","last_peered":"2024-05-10T03:31:27.432269+0000","last_clean":"2024-05-10T03:31:27.432269+0000","last_became_active":"2024-05-10T03:31:27.415137+0000","last_became_peered":"2024-05-10T03:31:27.415137+0000","last_unstale":"2024-05-10T03:31:27.432269+0000","last_undegraded":"2024-05-10T03:31:27.432269+0000","last_fullsized":"2024-05-10T03:31:27.432269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:24:40.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.411136+0000","last_change":"2024-05-10T03:31:26.841896+0000","last_active":"2024-05-10T03:31:27.411136+0000","last_peered":"2024-05-10T03:31:27.411136+0000","last_clean":"2024-05-10T03:31:27.411136+0000","last_became_active":"2024-05-10T03:31:26.841326+0000","last_became_peered":"2024-05-10T03:31:26.841326+0000","last_unstale":"2024-05-10T03:31:27.411136+0000","last_undegraded":"2024-05-10T03:31:27.411136+0000","last_fullsized":"2024-05-10T03:31:27.411136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:08:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.433078+0000","last_change":"2024-05-10T03:31:27.425965+0000","last_active":"2024-05-10T03:31:27.433078+0000","last_peered":"2024-05-10T03:31:27.433078+0000","last_clean":"2024-05-10T03:31:27.433078+0000","last_became_active":"2024-05-10T03:31:27.425642+0000","last_became_peered":"2024-05-10T03:31:27.425642+0000","last_unstale":"2024-05-10T03:31:27.433078+0000","last_undegraded":"2024-05-10T03:31:27.433078+0000","last_fullsized":"2024-05-10T03:31:27.433078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:13:44.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.423222+0000","last_change":"2024-05-10T03:31:26.842721+0000","last_active":"2024-05-10T03:31:27.423222+0000","last_peered":"2024-05-10T03:31:27.423222+0000","last_clean":"2024-05-10T03:31:27.423222+0000","last_became_active":"2024-05-10T03:31:26.841642+0000","last_became_peered":"2024-05-10T03:31:26.841642+0000","last_unstale":"2024-05-10T03:31:27.423222+0000","last_undegraded":"2024-05-10T03:31:27.423222+0000","last_fullsized":"2024-05-10T03:31:27.423222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:06:21.809607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.434513+0000","last_change":"2024-05-10T03:31:27.424509+0000","last_active":"2024-05-10T03:31:27.434513+0000","last_peered":"2024-05-10T03:31:27.434513+0000","last_clean":"2024-05-10T03:31:27.434513+0000","last_became_active":"2024-05-10T03:31:27.424118+0000","last_became_peered":"2024-05-10T03:31:27.424118+0000","last_unstale":"2024-05-10T03:31:27.434513+0000","last_undegraded":"2024-05-10T03:31:27.434513+0000","last_fullsized":"2024-05-10T03:31:27.434513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.397669+0000","last_change":"2024-05-10T03:31:26.852845+0000","last_active":"2024-05-10T03:31:27.397669+0000","last_peered":"2024-05-10T03:31:27.397669+0000","last_clean":"2024-05-10T03:31:27.397669+0000","last_became_active":"2024-05-10T03:31:26.852546+0000","last_became_peered":"2024-05-10T03:31:26.852546+0000","last_unstale":"2024-05-10T03:31:27.397669+0000","last_undegraded":"2024-05-10T03:31:27.397669+0000","last_fullsized":"2024-05-10T03:31:27.397669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:20.235961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.447473+0000","last_change":"2024-05-10T03:31:27.440123+0000","last_active":"2024-05-10T03:31:27.447473+0000","last_peered":"2024-05-10T03:31:27.447473+0000","last_clean":"2024-05-10T03:31:27.447473+0000","last_became_active":"2024-05-10T03:31:27.439792+0000","last_became_peered":"2024-05-10T03:31:27.439792+0000","last_unstale":"2024-05-10T03:31:27.447473+0000","last_undegraded":"2024-05-10T03:31:27.447473+0000","last_fullsized":"2024-05-10T03:31:27.447473+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:11.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.399667+0000","last_change":"2024-05-10T03:31:26.843177+0000","last_active":"2024-05-10T03:31:27.399667+0000","last_peered":"2024-05-10T03:31:27.399667+0000","last_clean":"2024-05-10T03:31:27.399667+0000","last_became_active":"2024-05-10T03:31:26.842849+0000","last_became_peered":"2024-05-10T03:31:26.842849+0000","last_unstale":"2024-05-10T03:31:27.399667+0000","last_undegraded":"2024-05-10T03:31:27.399667+0000","last_fullsized":"2024-05-10T03:31:27.399667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:39:16.701268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.435732+0000","last_change":"2024-05-10T03:31:27.419099+0000","last_active":"2024-05-10T03:31:27.435732+0000","last_peered":"2024-05-10T03:31:27.435732+0000","last_clean":"2024-05-10T03:31:27.435732+0000","last_became_active":"2024-05-10T03:31:27.418793+0000","last_became_peered":"2024-05-10T03:31:27.418793+0000","last_unstale":"2024-05-10T03:31:27.435732+0000","last_undegraded":"2024-05-10T03:31:27.435732+0000","last_fullsized":"2024-05-10T03:31:27.435732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:26.786622+0000","last_clean_scrub_stamp":"2024-05-10T03:31:26.786622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:04.286282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.417536+0000","last_change":"2024-05-10T03:31:26.844431+0000","last_active":"2024-05-10T03:31:27.417536+0000","last_peered":"2024-05-10T03:31:27.417536+0000","last_clean":"2024-05-10T03:31:27.417536+0000","last_became_active":"2024-05-10T03:31:26.843146+0000","last_became_peered":"2024-05-10T03:31:26.843146+0000","last_unstale":"2024-05-10T03:31:27.417536+0000","last_undegraded":"2024-05-10T03:31:27.417536+0000","last_fullsized":"2024-05-10T03:31:27.417536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:31:25.772216+0000","last_clean_scrub_stamp":"2024-05-10T03:31:25.772216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:44.331021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-10T03:31:27.422011+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:31:27.422011+0000","last_peered":"2024-05-10T03:31:27.422011+0000","last_clean":"2024-05-10T03:31:27.422011+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:31:27.422011+0000","last_undegraded":"2024-05-10T03:31:27.422011+0000","last_fullsized":"2024-05-10T03:31:27.422011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1073152,"data_stored":791128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.603},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.5489999999999999}]},{"osd":1,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.333},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.169}]},{"osd":2,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.425},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.4990000000000001}]},{"osd":3,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.712}]},{"osd":4,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.665},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.742}]},{"osd":5,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.7,"5min":1.7,"15min":1.7},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.792}]},{"osd":6,"last update":"Fri May 10 03:30:35 2024","interfaces":[{"interface":"back","average":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5590000000000002,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.9650000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.8859999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.1970000000000001}]},{"osd":2,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.359},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":1.425}]},{"osd":3,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.647,"15min":1.647},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.0869999999999997,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.321},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.1259999999999999}]},{"osd":4,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.8049999999999999}]},{"osd":5,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.238},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.6519999999999999}]},{"osd":7,"last update":"Fri May 10 03:30:34 2024","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"min":{"1min":1.339,"5min":1.339,"15min":1.339},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.518}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":199238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.042}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.375},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.244}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.9259999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":2.0289999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.468},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.952}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":2.4510000000000001},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":2.0880000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":199730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.345}]},{"osd":1,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.734},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.3859999999999999}]},{"osd":2,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.097},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.24}]},{"osd":3,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.7709999999999999,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.9530000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.516},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.8799999999999999}]},{"osd":6,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":2.1139999999999999},{"interface":"front","average":{"1min":1.804,"5min":1.804,"15min":1.804},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":5.3529999999999998,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.633}]},{"osd":7,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":2.069},{"interface":"front","average":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":2.282}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":479232,"data_stored":200724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.72499999999999998}]},{"osd":1,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":1.4370000000000001}]},{"osd":2,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.218}]},{"osd":4,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":2.0459999999999998}]},{"osd":5,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.304},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.341}]},{"osd":6,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":2.2559999999999998},{"interface":"front","average":{"1min":1.819,"5min":1.819,"15min":1.819},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.3759999999999999}]},{"osd":7,"last update":"Fri May 10 03:30:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":1.089,"5min":1.089,"15min":1.089},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.538},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":2.1899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":466944,"data_stored":199196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.431}]},{"osd":1,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.514}]},{"osd":3,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.353},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.9650000000000001}]},{"osd":4,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.6830000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":2.036},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.1190000000000002}]},{"osd":6,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.569}]},{"osd":7,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.7690000000000001,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.849}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":466944,"data_stored":199216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.2270000000000001}]},{"osd":2,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.0389999999999999}]},{"osd":4,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":3.8029999999999999,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.661,"5min":1.661,"15min":1.661},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.4239999999999999}]},{"osd":6,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":3.48,"5min":3.48,"15min":3.48},"last":1.47},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.3999999999999999}]},{"osd":7,"last update":"Fri May 10 03:30:31 2024","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.222,"5min":1.222,"15min":1.222},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.532},{"interface":"front","average":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.3640000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1060864,"data_stored":789528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9390000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.877}]},{"osd":3,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":3.1040000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.228},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.153},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"min":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.119}]},{"osd":7,"last update":"Fri May 10 03:30:32 2024","interfaces":[{"interface":"back","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.177,"5min":1.177,"15min":1.177},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.357}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:31:33.147 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:31:33.147 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:31:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.479+0000 7f71f8e00640 1 Processor -- start 2024-05-10T03:31:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.479+0000 7f71f8e00640 1 -- start start 2024-05-10T03:31:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.481+0000 7f71f8e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71f41615b0 0x7f71f40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.481+0000 7f71f8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40ce140 0x7f71f40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.481+0000 7f71f8e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40b3fd0 con 0x7f71f40b2450 2024-05-10T03:31:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f8e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f40cea90 con 0x7f71f41615b0 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40b33c0 con 0x7f71f40ce140 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71f41615b0 0x7f71f40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71f41615b0 0x7f71f40b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37042/0 (socket says 172.21.15.55:37042) 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 -- 172.21.15.55:0/2138235363 learned_addr learned my addr 172.21.15.55:0/2138235363 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 -- 172.21.15.55:0/2138235363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40ce140 msgr2=0x7f71f40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 --2- 172.21.15.55:0/2138235363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40ce140 0x7f71f40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 -- 172.21.15.55:0/2138235363 >> v1:172.21.15.162:6789/0 conn(0x7f71f40b2450 legacy=0x7f71f40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.482+0000 7f71f1e00640 1 -- 172.21.15.55:0/2138235363 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40b36a0 con 0x7f71f41615b0 2024-05-10T03:31:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.484+0000 7f71f1e00640 1 --2- 172.21.15.55:0/2138235363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f40b4460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009870 tx=0x7f71e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96260c865f6ad7f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.484+0000 7f71f0a00640 1 -- 172.21.15.55:0/2138235363 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032440 con 0x7f71f41615b0 2024-05-10T03:31:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.484+0000 7f71f0a00640 1 -- 172.21.15.55:0/2138235363 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e40325a0 con 0x7f71f41615b0 2024-05-10T03:31:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.484+0000 7f71f0a00640 1 -- 172.21.15.55:0/2138235363 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032930 con 0x7f71f41615b0 2024-05-10T03:31:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 -- 172.21.15.55:0/2138235363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 msgr2=0x7f71f40b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2138235363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f40b4460 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009870 tx=0x7f71e402a210 comp rx=0 tx=0).stop 2024-05-10T03:31:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 -- 172.21.15.55:0/2138235363 shutdown_connections 2024-05-10T03:31:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2138235363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40ce140 0x7f71f40ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2138235363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f40b4460 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 -- 172.21.15.55:0/2138235363 >> 172.21.15.55:0/2138235363 conn(0x7f71f40d08a0 msgr2=0x7f71f40d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 -- 172.21.15.55:0/2138235363 shutdown_connections 2024-05-10T03:31:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.485+0000 7f71f8e00640 1 -- 172.21.15.55:0/2138235363 wait complete. 2024-05-10T03:31:33.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.489+0000 7f71f8e00640 1 Processor -- start 2024-05-10T03:31:33.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.489+0000 7f71f8e00640 1 -- start start 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40b2450 0x7f71f4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71f40ce140 0x7f71f415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f41f4730 con 0x7f71f41615b0 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40cd460 con 0x7f71f40ce140 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.491+0000 7f71f8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c7eb0 con 0x7f71f40b2450 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37046/0 (socket says 172.21.15.55:37046) 2024-05-10T03:31:33.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 -- 172.21.15.55:0/2360495940 learned_addr learned my addr 172.21.15.55:0/2360495940 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f1e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40b2450 0x7f71f4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f1400640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71f40ce140 0x7f71f415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 -- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40b2450 msgr2=0x7f71f4150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40b2450 0x7f71f4150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 -- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71f40ce140 msgr2=0x7f71f415b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71f40ce140 0x7f71f415b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.492+0000 7f71f2800640 1 -- 172.21.15.55:0/2360495940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40094d0 con 0x7f71f41615b0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71f2800640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00b410 tx=0x7f71ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec011830 con 0x7f71f41615b0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec011990 con 0x7f71f41615b0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec011d40 con 0x7f71f41615b0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f40c9980 con 0x7f71f41615b0 2024-05-10T03:31:33.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.493+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f40cd990 con 0x7f71f41615b0 2024-05-10T03:31:33.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.495+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f406bf50 con 0x7f71f41615b0 2024-05-10T03:31:33.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.496+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f71ec01a400 con 0x7f71f41615b0 2024-05-10T03:31:33.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.497+0000 7f71ea000640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 0x7f71d006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:33.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.497+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f71ec089560 con 0x7f71f41615b0 2024-05-10T03:31:33.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.498+0000 7f71f1e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 0x7f71d006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:33.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.499+0000 7f71f1e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 0x7f71d006d140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034000 tx=0x7f71e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:33.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.502+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f71ec054560 con 0x7f71f41615b0 2024-05-10T03:31:33.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.768+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f71f40d1150 con 0x7f71f41615b0 2024-05-10T03:31:33.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.779+0000 7f71ea000640 1 -- 172.21.15.55:0/2360495940 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f71ec057b80 con 0x7f71f41615b0 2024-05-10T03:31:33.784 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:31:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 msgr2=0x7f71d006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 0x7f71d006d140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034000 tx=0x7f71e402e000 comp rx=0 tx=0).stop 2024-05-10T03:31:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 msgr2=0x7f71f415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:33.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00b410 tx=0x7f71ec00b8d0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 shutdown_connections 2024-05-10T03:31:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71f41615b0 0x7f71f415b970 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71d006ac30 0x7f71d006d140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71f40ce140 0x7f71f415b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 --2- 172.21.15.55:0/2360495940 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71f40b2450 0x7f71f4150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.782+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 >> 172.21.15.55:0/2360495940 conn(0x7f71f40d08a0 msgr2=0x7f71f415c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.783+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 shutdown_connections 2024-05-10T03:31:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:33.783+0000 7f71f8e00640 1 -- 172.21.15.55:0/2360495940 wait complete. 2024-05-10T03:31:33.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:34.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.088+0000 7f4c89c00640 1 Processor -- start 2024-05-10T03:31:34.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.088+0000 7f4c89c00640 1 -- start start 2024-05-10T03:31:34.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.090+0000 7f4c89c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c8415fe50 0x7f4c84141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.090+0000 7f4c89c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4c840b2140 0x7f4c84142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.090+0000 7f4c89c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2550 0x7f4c84142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c89c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8406ad20 con 0x7f4c8415fe50 2024-05-10T03:31:34.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c89c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c84141f50 con 0x7f4c840b2140 2024-05-10T03:31:34.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c89c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8406a110 con 0x7f4c840b2550 2024-05-10T03:31:34.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c82a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c8415fe50 0x7f4c84141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c82a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c8415fe50 0x7f4c84141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37048/0 (socket says 172.21.15.55:37048) 2024-05-10T03:31:34.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c82a00640 1 -- 172.21.15.55:0/3988192609 learned_addr learned my addr 172.21.15.55:0/3988192609 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c82000640 1 --2- 172.21.15.55:0/3988192609 >> v2:172.21.15.162:3300/0 conn(0x7f4c840b2140 0x7f4c84142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.091+0000 7f4c83400640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2550 0x7f4c84142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.092+0000 7f4c82a00640 1 -- 172.21.15.55:0/3988192609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2550 msgr2=0x7f4c84142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.092+0000 7f4c82a00640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2550 0x7f4c84142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.092+0000 7f4c82a00640 1 -- 172.21.15.55:0/3988192609 >> v2:172.21.15.162:3300/0 conn(0x7f4c840b2140 msgr2=0x7f4c84142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.092+0000 7f4c82a00640 1 --2- 172.21.15.55:0/3988192609 >> v2:172.21.15.162:3300/0 conn(0x7f4c840b2140 0x7f4c84142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.092+0000 7f4c82a00640 1 -- 172.21.15.55:0/3988192609 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8406a3f0 con 0x7f4c8415fe50 2024-05-10T03:31:34.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.093+0000 7f4c82a00640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c8415fe50 0x7f4c84141a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c841454b0 tx=0x7f4c7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26705bd61b79b896 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:34.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.093+0000 7f4c81600640 1 -- 172.21.15.55:0/3988192609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74032440 con 0x7f4c8415fe50 2024-05-10T03:31:34.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.093+0000 7f4c81600640 1 -- 172.21.15.55:0/3988192609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c740325a0 con 0x7f4c8415fe50 2024-05-10T03:31:34.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.093+0000 7f4c81600640 1 -- 172.21.15.55:0/3988192609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74032930 con 0x7f4c8415fe50 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 -- 172.21.15.55:0/3988192609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c8415fe50 msgr2=0x7f4c84141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c8415fe50 0x7f4c84141a10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c841454b0 tx=0x7f4c7402a270 comp rx=0 tx=0).stop 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 -- 172.21.15.55:0/3988192609 shutdown_connections 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2550 0x7f4c84142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 --2- 172.21.15.55:0/3988192609 >> v2:172.21.15.162:3300/0 conn(0x7f4c840b2140 0x7f4c84142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 --2- 172.21.15.55:0/3988192609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c8415fe50 0x7f4c84141a10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 -- 172.21.15.55:0/3988192609 >> 172.21.15.55:0/3988192609 conn(0x7f4c840cbb90 msgr2=0x7f4c840cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 -- 172.21.15.55:0/3988192609 shutdown_connections 2024-05-10T03:31:34.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.094+0000 7f4c89c00640 1 -- 172.21.15.55:0/3988192609 wait complete. 2024-05-10T03:31:34.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.097+0000 7f4c89c00640 1 Processor -- start 2024-05-10T03:31:34.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.097+0000 7f4c89c00640 1 -- start start 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2140 0x7f4c8415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 0x7f4c841e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c841e5ff0 con 0x7f4c840b2550 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c841e6c20 con 0x7f4c8415fe50 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.098+0000 7f4c89c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8415b920 con 0x7f4c840b2140 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37060/0 (socket says 172.21.15.55:37060) 2024-05-10T03:31:34.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 -- 172.21.15.55:0/450134100 learned_addr learned my addr 172.21.15.55:0/450134100 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:34.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82a00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2140 0x7f4c8415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c83400640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 0x7f4c841e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 -- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2140 msgr2=0x7f4c8415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2140 0x7f4c8415b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 -- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 msgr2=0x7f4c841e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 0x7f4c841e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c82000640 1 -- 172.21.15.55:0/450134100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740094d0 con 0x7f4c840b2550 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.099+0000 7f4c83400640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 0x7f4c841e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c82000640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000d620 tx=0x7f4c7000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000b620 con 0x7f4c840b2550 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7000b780 con 0x7f4c840b2550 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000bb30 con 0x7f4c840b2550 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c841e9b00 con 0x7f4c840b2550 2024-05-10T03:31:34.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.100+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c841e5500 con 0x7f4c840b2550 2024-05-10T03:31:34.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.101+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c84066f60 con 0x7f4c840b2550 2024-05-10T03:31:34.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.102+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4c7000bc90 con 0x7f4c840b2550 2024-05-10T03:31:34.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.104+0000 7f4c7aa00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 0x7f4c6406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:34.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.104+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4c700898b0 con 0x7f4c840b2550 2024-05-10T03:31:34.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.105+0000 7f4c82a00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 0x7f4c6406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:34.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.106+0000 7f4c82a00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 0x7f4c6406d160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c74034000 tx=0x7f4c7402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:34.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.109+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4c70054a30 con 0x7f4c840b2550 2024-05-10T03:31:34.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.338+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c84067220 con 0x7f4c840b2550 2024-05-10T03:31:34.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.339+0000 7f4c7aa00640 1 -- 172.21.15.55:0/450134100 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f4c70058050 con 0x7f4c840b2550 2024-05-10T03:31:34.343 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:34.343 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:34.343 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:34.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 msgr2=0x7f4c6406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:34.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 0x7f4c6406d160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c74034000 tx=0x7f4c7402e000 comp rx=0 tx=0).stop 2024-05-10T03:31:34.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 msgr2=0x7f4c841e61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:34.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000d620 tx=0x7f4c7000dae0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 shutdown_connections 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4c6406ac50 0x7f4c6406d160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8415fe50 0x7f4c841e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c840b2550 0x7f4c841e61a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 --2- 172.21.15.55:0/450134100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c840b2140 0x7f4c8415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 >> 172.21.15.55:0/450134100 conn(0x7f4c840cbb90 msgr2=0x7f4c84070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.342+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 shutdown_connections 2024-05-10T03:31:34.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:34.343+0000 7f4c89c00640 1 -- 172.21.15.55:0/450134100 wait complete. 2024-05-10T03:31:34.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:31:27.383309+0000', 'modified': '2024-05-10T03:31:30.836445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5256': {'gid': 5256, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.55:6835/455504925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 455504925}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 455504925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:31:34.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:31:34.775 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T03:31:35.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.065+0000 7fae1be00640 1 Processor -- start 2024-05-10T03:31:35.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.065+0000 7fae1be00640 1 -- start start 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.068+0000 7fae1be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae1c06b830 0x7fae1c166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.068+0000 7fae1be00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c15bde0 con 0x7fae1c15b9d0 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.068+0000 7fae1be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c13eae0 con 0x7fae1c13e390 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.068+0000 7fae1be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c13e7a0 con 0x7fae1c06b830 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae12000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fae1c15b9d0 0x7fae1c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37934/0 (socket says 172.21.15.55:37934) 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae12000640 1 -- 172.21.15.55:0/3586553436 learned_addr learned my addr 172.21.15.55:0/3586553436 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:35.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1b400640 1 --2- 172.21.15.55:0/3586553436 >> v2:172.21.15.55:3300/0 conn(0x7fae1c06b830 0x7fae1c166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1a000640 1 -- 172.21.15.55:0/3586553436 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936865681 0 0) 0x7fae04001290 con 0x7fae1c15b9d0 2024-05-10T03:31:35.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1a000640 1 -- 172.21.15.55:0/3586553436 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae00003520 con 0x7fae1c15b9d0 2024-05-10T03:31:35.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1b400640 1 -- 172.21.15.55:0/3586553436 >> v1:172.21.15.162:6789/0 conn(0x7fae1c13e390 legacy=0x7fae1c1414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1b400640 1 -- 172.21.15.55:0/3586553436 >> v1:172.21.15.55:6789/0 conn(0x7fae1c15b9d0 legacy=0x7fae1c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.069+0000 7fae1b400640 1 -- 172.21.15.55:0/3586553436 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c15c020 con 0x7fae1c06b830 2024-05-10T03:31:35.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.070+0000 7fae1b400640 1 --2- 172.21.15.55:0/3586553436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c06b830 0x7fae1c166090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae1402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d936e25fe18a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.071+0000 7fae1a000640 1 -- 172.21.15.55:0/3586553436 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1402ae50 con 0x7fae1c06b830 2024-05-10T03:31:35.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.071+0000 7fae1a000640 1 -- 172.21.15.55:0/3586553436 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14032440 con 0x7fae1c06b830 2024-05-10T03:31:35.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.071+0000 7fae1a000640 1 -- 172.21.15.55:0/3586553436 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140327d0 con 0x7fae1c06b830 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.071+0000 7fae1be00640 1 -- 172.21.15.55:0/3586553436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c06b830 msgr2=0x7fae1c166090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.071+0000 7fae1be00640 1 --2- 172.21.15.55:0/3586553436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c06b830 0x7fae1c166090 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae1402a140 comp rx=0 tx=0).stop 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.072+0000 7fae1be00640 1 -- 172.21.15.55:0/3586553436 shutdown_connections 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.072+0000 7fae1be00640 1 --2- 172.21.15.55:0/3586553436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c06b830 0x7fae1c166090 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.072+0000 7fae1be00640 1 -- 172.21.15.55:0/3586553436 >> 172.21.15.55:0/3586553436 conn(0x7fae1c06c730 msgr2=0x7fae1c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.072+0000 7fae1be00640 1 -- 172.21.15.55:0/3586553436 shutdown_connections 2024-05-10T03:31:35.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.072+0000 7fae1be00640 1 -- 172.21.15.55:0/3586553436 wait complete. 2024-05-10T03:31:35.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.074+0000 7fae1be00640 1 Processor -- start 2024-05-10T03:31:35.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.074+0000 7fae1be00640 1 -- start start 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1c15b9d0 0x7fae1c13b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1c0acc40 0x7fae1c0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ac2f0 con 0x7fae1c13e390 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0a8600 con 0x7fae1c0acc40 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0abd30 con 0x7fae1c15b9d0 2024-05-10T03:31:35.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37090/0 (socket says 172.21.15.55:37090) 2024-05-10T03:31:35.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 -- 172.21.15.55:0/2243801620 learned_addr learned my addr 172.21.15.55:0/2243801620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 -- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1c15b9d0 msgr2=0x7fae1c13b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1c15b9d0 0x7fae1c13b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 -- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1c0acc40 msgr2=0x7fae1c0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1c0acc40 0x7fae1c0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.076+0000 7fae1aa00640 1 -- 172.21.15.55:0/2243801620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009400 con 0x7fae1c13e390 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae1aa00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b9c0 tx=0x7fae0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae13400640 1 -- 172.21.15.55:0/2243801620 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00cba0 con 0x7fae1c13e390 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae13400640 1 -- 172.21.15.55:0/2243801620 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c00cd00 con 0x7fae1c13e390 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae13400640 1 -- 172.21.15.55:0/2243801620 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c003bb0 con 0x7fae1c13e390 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c1f1540 con 0x7fae1c13e390 2024-05-10T03:31:35.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.077+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c0a88a0 con 0x7fae1c13e390 2024-05-10T03:31:35.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.081+0000 7fae13400640 1 -- 172.21.15.55:0/2243801620 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae0c014440 con 0x7fae1c13e390 2024-05-10T03:31:35.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.082+0000 7fae13400640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 0x7fadf806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.082+0000 7fae13400640 1 -- 172.21.15.55:0/2243801620 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae0c089270 con 0x7fae1c13e390 2024-05-10T03:31:35.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.082+0000 7fae12000640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 0x7fadf806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.083+0000 7fae28627740 1 Processor -- start 2024-05-10T03:31:35.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.083+0000 7fae28627740 1 -- start start 2024-05-10T03:31:35.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae12000640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 0x7fadf806d140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae040014f0 tx=0x7fae04001270 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 0x55e3af93acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 0x55e3af955a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af879a70 con 0x55e3af550590 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af877200 con 0x55e3af5483b0 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.084+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3af872af0 con 0x55e3af571270 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37096/0 (socket says 172.21.15.55:37096) 2024-05-10T03:31:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 -- 172.21.15.55:0/1837862914 learned_addr learned my addr 172.21.15.55:0/1837862914 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae12000640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 0x55e3af955a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1aa00640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 0x55e3af93acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 -- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 msgr2=0x55e3af93acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 0x55e3af93acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 -- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 msgr2=0x55e3af955a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 0x55e3af955a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1b400640 1 -- 172.21.15.55:0/1837862914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3af83db90 con 0x55e3af550590 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae1aa00640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 0x55e3af93acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.085+0000 7fae12000640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 0x55e3af955a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.086+0000 7fae1b400640 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae140098c0 tx=0x7fae1402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24037d36b215bd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.086+0000 7fae0b000640 1 -- 172.21.15.55:0/1837862914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140328b0 con 0x55e3af550590 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.086+0000 7fae0b000640 1 -- 172.21.15.55:0/1837862914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14032a10 con 0x55e3af550590 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.086+0000 7fae0b000640 1 -- 172.21.15.55:0/1837862914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14032d80 con 0x55e3af550590 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 -- 172.21.15.55:0/1837862914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 msgr2=0x55e3af955f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae140098c0 tx=0x7fae1402c070 comp rx=0 tx=0).stop 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 -- 172.21.15.55:0/1837862914 shutdown_connections 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af955f40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af5483b0 0x55e3af955a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 --2- 172.21.15.55:0/1837862914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af571270 0x55e3af93acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 -- 172.21.15.55:0/1837862914 >> 172.21.15.55:0/1837862914 conn(0x55e3af5653c0 msgr2=0x55e3af910330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 -- 172.21.15.55:0/1837862914 shutdown_connections 2024-05-10T03:31:35.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.087+0000 7fae28627740 1 -- 172.21.15.55:0/1837862914 wait complete. 2024-05-10T03:31:35.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.088+0000 7fae28627740 1 -- start start 2024-05-10T03:31:35.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.088+0000 7fae28627740 1 Processor -- start 2024-05-10T03:31:35.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af93c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 0x55e3af910330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af814220 con 0x55e3af550590 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af813ec0 con 0x55e3af571270 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3af5d9f20 con 0x55e3af5483b0 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.090+0000 7fae28627740 10 client.0 _ll_register_callbacks cb 0x55e3af86c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37102/0 (socket says 172.21.15.55:37102) 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 -- 172.21.15.55:0/1513307004 learned_addr learned my addr 172.21.15.55:0/1513307004 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae1b400640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 0x55e3af910330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae1aa00640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af93c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 -- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 msgr2=0x55e3af93c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af93c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 -- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 msgr2=0x55e3af910330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 0x55e3af910330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 -- 172.21.15.55:0/1513307004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c15c020 con 0x55e3af550590 2024-05-10T03:31:35.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae1b400640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 0x55e3af910330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fae12000640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0aec80 tx=0x7fae04001050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643676c3d6860e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.091+0000 7fadff400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04014dd0 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae04022c10 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3af89cc90 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae04022dd0 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fae28627740 10 client.5421 fetch_fsmap learned FSMap version 16 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e3af871170 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.092+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3af89cc90 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fae04012440 con 0x55e3af550590 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 10 client.5421 fetch_fsmap finished waiting for FSMap version 16 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 10 client.5421 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 20 client.5421 populate_metadata read hostname 'smithi055' 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af52fe40 0x55e3af9685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 4 client.5421 mds_command: new command op to 5256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 --> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3af5663c0 con 0x55e3af52fe40 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.093+0000 7fae1aa00640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af52fe40 0x55e3af9685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.094+0000 7fae1aa00640 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af52fe40 0x55e3af9685e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60848fe53bf57271 server_cookie=438665414aad643 in_seq=0 out_seq=0 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.094+0000 7fadff400640 10 client.5421 ms_handle_connect on v2:172.21.15.55:6834/455504925 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.094+0000 7fadff400640 1 -- 172.21.15.55:0/1513307004 <== mds.0 v2:172.21.15.55:6834/455504925 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3af5663c0 con 0x55e3af52fe40 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.094+0000 7fadff400640 10 client.5421 handle_command_reply: tid=0 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 client.5421 shutdown 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 msgr2=0x55e3af936480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0aec80 tx=0x7fae04001050 comp rx=0 tx=0).stop 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 shutdown_connections 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af52fe40 0x55e3af9685e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af571270 0x55e3af910330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af550590 0x55e3af936480 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fadff400640 0 client.5421 ms_handle_reset on v2:172.21.15.55:6834/455504925 2024-05-10T03:31:35.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 --2- 172.21.15.55:0/1513307004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af93c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 >> 172.21.15.55:0/1513307004 conn(0x55e3af5653c0 msgr2=0x55e3af956420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 shutdown_connections 2024-05-10T03:31:35.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 1 -- 172.21.15.55:0/1513307004 wait complete. 2024-05-10T03:31:35.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.095+0000 7fae28627740 20 client.5421 trim_cache size 0 max 16384 2024-05-10T03:31:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.112+0000 7fae28627740 1 Processor -- start 2024-05-10T03:31:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.113+0000 7fae28627740 1 -- start start 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af950e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af94ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af93b820 con 0x55e3af571270 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af93b5d0 con 0x55e3af550590 2024-05-10T03:31:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3af93abd0 con 0x55e3af5483b0 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37116/0 (socket says 172.21.15.55:37116) 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.114+0000 7fae1b400640 1 -- 172.21.15.55:0/7173893 learned_addr learned my addr 172.21.15.55:0/7173893 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae12000640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af94ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1aa00640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af950e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 -- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 msgr2=0x55e3af950e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af950e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 -- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 msgr2=0x55e3af94ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af94ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 -- 172.21.15.55:0/7173893 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3af8a6130 con 0x55e3af571270 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1aa00640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af950e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.115+0000 7fae1b400640 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae1402a730 tx=0x7fae1402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648ac77514303632 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae0ba00640 1 -- 172.21.15.55:0/7173893 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14032d80 con 0x55e3af571270 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae0ba00640 1 -- 172.21.15.55:0/7173893 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14032830 con 0x55e3af571270 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae0ba00640 1 -- 172.21.15.55:0/7173893 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14004270 con 0x55e3af571270 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 -- 172.21.15.55:0/7173893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 msgr2=0x55e3af911930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae1402a730 tx=0x7fae1402e000 comp rx=0 tx=0).stop 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 -- 172.21.15.55:0/7173893 shutdown_connections 2024-05-10T03:31:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af911930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af94ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 --2- 172.21.15.55:0/7173893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af950e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 -- 172.21.15.55:0/7173893 >> 172.21.15.55:0/7173893 conn(0x55e3af5653c0 msgr2=0x55e3af910020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.116+0000 7fae28627740 1 -- 172.21.15.55:0/7173893 shutdown_connections 2024-05-10T03:31:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.117+0000 7fae28627740 1 -- 172.21.15.55:0/7173893 wait complete. 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.117+0000 7fae28627740 1 -- start start 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.118+0000 7fae28627740 1 Processor -- start 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af911930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af911e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af9bf650 con 0x55e3af571270 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3af936fa0 con 0x55e3af550590 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3af93df40 con 0x55e3af5483b0 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae28627740 10 client.0 _ll_register_callbacks cb 0x7fae04007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.119+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37118/0 (socket says 172.21.15.55:37118) 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 -- 172.21.15.55:0/4243540168 learned_addr learned my addr 172.21.15.55:0/4243540168 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae12000640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af911e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1aa00640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af911930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 -- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 msgr2=0x55e3af911930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af911930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 -- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 msgr2=0x55e3af911e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af911e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 -- 172.21.15.55:0/4243540168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3af83db90 con 0x55e3af571270 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1aa00640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af911930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae12000640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af911e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fae1b400640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f18565451389040 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.120+0000 7fadff400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003a20 con 0x55e3af571270 2024-05-10T03:31:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae14056070 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae14032a10 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae14003b80 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fae28627740 10 client.5427 fetch_fsmap learned FSMap version 16 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e3af813430 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14032a10 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fae14003ce0 con 0x55e3af571270 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fae28627740 10 client.5427 fetch_fsmap finished waiting for FSMap version 16 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.121+0000 7fae28627740 10 client.5427 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae28627740 20 client.5427 populate_metadata read hostname 'smithi055' 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af9c0b90 0x55e3af9beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae28627740 4 client.5427 mds_command: new command op to 5256 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 --> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e3af5776c0 con 0x55e3af9c0b90 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae1aa00640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af9c0b90 0x55e3af9beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fae1aa00640 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af9c0b90 0x55e3af9beb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2eaf99a95a98e96 server_cookie=52275eb264d811f7 in_seq=0 out_seq=0 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.122+0000 7fadff400640 10 client.5427 ms_handle_connect on v2:172.21.15.55:6834/455504925 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fadff400640 1 -- 172.21.15.55:0/4243540168 <== mds.0 v2:172.21.15.55:6834/455504925 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e3af5776c0 con 0x55e3af9c0b90 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fadff400640 10 client.5427 handle_command_reply: tid=0 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 client.5427 shutdown 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 msgr2=0x55e3af9be620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae14004730 comp rx=0 tx=0).stop 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 shutdown_connections 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:6834/455504925,v1:172.21.15.55:6835/455504925] conn(0x55e3af9c0b90 0x55e3af9beb60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3af571270 0x55e3af9be620 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e3af550590 0x55e3af911e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 --2- 172.21.15.55:0/4243540168 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e3af5483b0 0x55e3af911930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fadff400640 0 client.5427 ms_handle_reset on v2:172.21.15.55:6834/455504925 2024-05-10T03:31:35.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 >> 172.21.15.55:0/4243540168 conn(0x55e3af5653c0 msgr2=0x55e3af93e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 shutdown_connections 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 1 -- 172.21.15.55:0/4243540168 wait complete. 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.123+0000 7fae28627740 20 client.5427 trim_cache size 0 max 16384 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 5256, 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:31:35.131 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.7350492409999996, 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 16, 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 40, 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 40, 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 17.648604264999999 2024-05-10T03:31:35.132 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 msgr2=0x7fadf806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 0x7fadf806d140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae040014f0 tx=0x7fae04001270 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 msgr2=0x7fae1c0ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b9c0 tx=0x7fae0c00be80 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 shutdown_connections 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fadf806ac30 0x7fadf806d140 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1c0acc40 0x7fae1c0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1c15b9d0 0x7fae1c13b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 --2- 172.21.15.55:0/2243801620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1c13e390 0x7fae1c0ab690 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 >> 172.21.15.55:0/2243801620 conn(0x7fae1c06c730 msgr2=0x7fae1c0b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 shutdown_connections 2024-05-10T03:31:35.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.124+0000 7fae1be00640 1 -- 172.21.15.55:0/2243801620 wait complete. 2024-05-10T03:31:35.153 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:31:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.432+0000 7efd79c00640 1 Processor -- start 2024-05-10T03:31:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.432+0000 7efd79c00640 1 -- start start 2024-05-10T03:31:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd79c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efd74161e40 0x7efd7406c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd79c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efd7415df60 0x7efd7415e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd79c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7406bf70 con 0x7efd7406e740 2024-05-10T03:31:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd79c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7406b710 con 0x7efd7415df60 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd79c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7415e9b0 con 0x7efd74161e40 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd6a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efd7415df60 0x7efd7415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd6a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efd7415df60 0x7efd7415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37128/0 (socket says 172.21.15.55:37128) 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.435+0000 7efd6a000640 1 -- 172.21.15.55:0/3038584364 learned_addr learned my addr 172.21.15.55:0/3038584364 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.436+0000 7efd6a000640 1 -- 172.21.15.55:0/3038584364 >> v2:172.21.15.162:3300/0 conn(0x7efd74161e40 msgr2=0x7efd7406c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.436+0000 7efd6a000640 1 --2- 172.21.15.55:0/3038584364 >> v2:172.21.15.162:3300/0 conn(0x7efd74161e40 0x7efd7406c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.436+0000 7efd6a000640 1 -- 172.21.15.55:0/3038584364 >> v1:172.21.15.162:6789/0 conn(0x7efd7406e740 legacy=0x7efd7406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.436+0000 7efd6a000640 1 -- 172.21.15.55:0/3038584364 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7415ec90 con 0x7efd7415df60 2024-05-10T03:31:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.437+0000 7efd6a000640 1 --2- 172.21.15.55:0/3038584364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7415df60 0x7efd7415e370 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009870 tx=0x7efd5c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721b8f51164f9db4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.437+0000 7efd72000640 1 -- 172.21.15.55:0/3038584364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c02ae40 con 0x7efd7415df60 2024-05-10T03:31:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.437+0000 7efd72000640 1 -- 172.21.15.55:0/3038584364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c032440 con 0x7efd7415df60 2024-05-10T03:31:35.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.437+0000 7efd72000640 1 -- 172.21.15.55:0/3038584364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0327d0 con 0x7efd7415df60 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 -- 172.21.15.55:0/3038584364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7415df60 msgr2=0x7efd7415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 --2- 172.21.15.55:0/3038584364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7415df60 0x7efd7415e370 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009870 tx=0x7efd5c02a130 comp rx=0 tx=0).stop 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 -- 172.21.15.55:0/3038584364 shutdown_connections 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 --2- 172.21.15.55:0/3038584364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7415df60 0x7efd7415e370 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 --2- 172.21.15.55:0/3038584364 >> v2:172.21.15.162:3300/0 conn(0x7efd74161e40 0x7efd7406c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.438+0000 7efd79c00640 1 -- 172.21.15.55:0/3038584364 >> 172.21.15.55:0/3038584364 conn(0x7efd740cd2f0 msgr2=0x7efd740cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.439+0000 7efd79c00640 1 -- 172.21.15.55:0/3038584364 shutdown_connections 2024-05-10T03:31:35.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.439+0000 7efd79c00640 1 -- 172.21.15.55:0/3038584364 wait complete. 2024-05-10T03:31:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.441+0000 7efd79c00640 1 Processor -- start 2024-05-10T03:31:35.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.441+0000 7efd79c00640 1 -- start start 2024-05-10T03:31:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 0x7efd74155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 0x7efd740acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7415cf80 con 0x7efd7406e740 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740b0810 con 0x7efd7415df60 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd79c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd741594e0 con 0x7efd74161e40 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd72a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd72a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37132/0 (socket says 172.21.15.55:37132) 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.443+0000 7efd72a00640 1 -- 172.21.15.55:0/188490806 learned_addr learned my addr 172.21.15.55:0/188490806 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd6a000640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 0x7efd74155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd73400640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 0x7efd740acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 -- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 msgr2=0x7efd740acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 0x7efd740acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 -- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 msgr2=0x7efd74155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 0x7efd74155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 -- 172.21.15.55:0/188490806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0094d0 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd6a000640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 0x7efd74155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd73400640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 0x7efd740acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd72a00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b440 tx=0x7efd6400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64011890 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd640119f0 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64011da0 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.444+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd74156250 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.445+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd74159730 con 0x7efd7406e740 2024-05-10T03:31:35.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.446+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd740cca40 con 0x7efd7406e740 2024-05-10T03:31:35.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.447+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efd6401a400 con 0x7efd7406e740 2024-05-10T03:31:35.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.448+0000 7efd6b400640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 0x7efd5006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:35.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.448+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efd64089770 con 0x7efd7406e740 2024-05-10T03:31:35.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.448+0000 7efd6a000640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 0x7efd5006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:35.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.450+0000 7efd6a000640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 0x7efd5006d140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd5c034000 tx=0x7efd5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:35.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.453+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efd64054810 con 0x7efd7406e740 2024-05-10T03:31:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.688+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd74067220 con 0x7efd7406e740 2024-05-10T03:31:35.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.689+0000 7efd6b400640 1 -- 172.21.15.55:0/188490806 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd64057e30 con 0x7efd7406e740 2024-05-10T03:31:35.693 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:31:35.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 msgr2=0x7efd5006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 0x7efd5006d140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd5c034000 tx=0x7efd5c02e000 comp rx=0 tx=0).stop 2024-05-10T03:31:35.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 msgr2=0x7efd7415ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b440 tx=0x7efd6400b900 comp rx=0 tx=0).stop 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 shutdown_connections 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd74161e40 0x7efd740acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd5006ac30 0x7efd5006d140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd7415df60 0x7efd74155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 --2- 172.21.15.55:0/188490806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd7406e740 0x7efd7415ca40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 >> 172.21.15.55:0/188490806 conn(0x7efd740cd2f0 msgr2=0x7efd74070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 shutdown_connections 2024-05-10T03:31:35.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:35.692+0000 7efd79c00640 1 -- 172.21.15.55:0/188490806 wait complete. 2024-05-10T03:31:35.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:31:35.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:31:35.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:31:35.722 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:31:35.723 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:35.723 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:31:35.723 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:31:35.723 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:31:35.723 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:35.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:35.723 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:31:35.777 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42356sec preferred_lft 42356sec 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:31:35.778 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:31:35.779 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:35.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:35.780 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:31:35.862 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:31:35.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:35.867 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:31:35.925 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:31:35.925 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:31:35.925 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:31:35.925 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:31:35.926 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:31:35.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:31:35.926 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:31:36.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:36.083 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:31:36.139 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:31:36.139 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:31:36.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:36.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:36.139 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:31:36.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:36.229 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:36.287 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:36.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:36.288 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:36.342 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:31:36.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:36.342 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:36.550 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:36.551 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:36.551 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:36.578 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:36.578 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:36.578 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:36.578 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:36.579 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:31:36.626 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:31:36.629 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:31:36.629 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:31:36.629 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:31:36.629 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:31:36.629 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:31:36.634 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:31:36.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:36.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:36.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:36.723 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:31:36.805 INFO:teuthology.orchestra.run.smithi055.stdout:client5436 2024-05-10T03:31:36.805 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:36.805 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:36.868 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:31:37.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.152+0000 7f1ac6c00640 1 Processor -- start 2024-05-10T03:31:37.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.152+0000 7f1ac6c00640 1 -- start start 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac6c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ac0144710 0x7f1ac0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac6c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 0x7f1ac00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac6c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac006d350 con 0x7f1ac00b2fc0 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac6c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006c740 con 0x7f1ac0144710 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac6c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006ca20 con 0x7f1ac006d7e0 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac5800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ac0144710 0x7f1ac0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac5800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ac0144710 0x7f1ac0144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37138/0 (socket says 172.21.15.55:37138) 2024-05-10T03:31:37.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.155+0000 7f1ac5800640 1 -- 172.21.15.55:0/4055371608 learned_addr learned my addr 172.21.15.55:0/4055371608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac4e00640 1 --2- 172.21.15.55:0/4055371608 >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 0x7f1ac00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac5800640 1 -- 172.21.15.55:0/4055371608 >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 msgr2=0x7f1ac00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac5800640 1 --2- 172.21.15.55:0/4055371608 >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 0x7f1ac00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac5800640 1 -- 172.21.15.55:0/4055371608 >> v1:172.21.15.162:6789/0 conn(0x7f1ac00b2fc0 legacy=0x7f1ac015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac5800640 1 -- 172.21.15.55:0/4055371608 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac006cd00 con 0x7f1ac0144710 2024-05-10T03:31:37.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.156+0000 7f1ac4e00640 1 --2- 172.21.15.55:0/4055371608 >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 0x7f1ac00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:37.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.157+0000 7f1ac5800640 1 --2- 172.21.15.55:0/4055371608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0144710 0x7f1ac0144120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ae870 tx=0x7f1ab402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d446ae310794d765 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:37.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.158+0000 7f1abfe00640 1 -- 172.21.15.55:0/4055371608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4031600 con 0x7f1ac0144710 2024-05-10T03:31:37.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.158+0000 7f1abfe00640 1 -- 172.21.15.55:0/4055371608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab402d050 con 0x7f1ac0144710 2024-05-10T03:31:37.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.158+0000 7f1abfe00640 1 -- 172.21.15.55:0/4055371608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4031980 con 0x7f1ac0144710 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.158+0000 7f1ac6c00640 1 -- 172.21.15.55:0/4055371608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0144710 msgr2=0x7f1ac0144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/4055371608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0144710 0x7f1ac0144120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ae870 tx=0x7f1ab402a660 comp rx=0 tx=0).stop 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 -- 172.21.15.55:0/4055371608 shutdown_connections 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/4055371608 >> v2:172.21.15.162:3300/0 conn(0x7f1ac006d7e0 0x7f1ac00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/4055371608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0144710 0x7f1ac0144120 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 -- 172.21.15.55:0/4055371608 >> 172.21.15.55:0/4055371608 conn(0x7f1ac006a110 msgr2=0x7f1ac006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 -- 172.21.15.55:0/4055371608 shutdown_connections 2024-05-10T03:31:37.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.159+0000 7f1ac6c00640 1 -- 172.21.15.55:0/4055371608 wait complete. 2024-05-10T03:31:37.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.162+0000 7f1ac6c00640 1 Processor -- start 2024-05-10T03:31:37.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.162+0000 7f1ac6c00640 1 -- start start 2024-05-10T03:31:37.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac00b2fc0 0x7f1ac013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac0144710 0x7f1ac00af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac014f5c0 con 0x7f1ac006d7e0 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01404d0 con 0x7f1ac0144710 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01514d0 con 0x7f1ac00b2fc0 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.164+0000 7f1ac5800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37140/0 (socket says 172.21.15.55:37140) 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 -- 172.21.15.55:0/2993044244 learned_addr learned my addr 172.21.15.55:0/2993044244 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac4e00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac00b2fc0 0x7f1ac013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac6200640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac0144710 0x7f1ac00af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 -- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac00b2fc0 msgr2=0x7f1ac013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac00b2fc0 0x7f1ac013fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 -- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac0144710 msgr2=0x7f1ac00af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac0144710 0x7f1ac00af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.165+0000 7f1ac5800640 1 -- 172.21.15.55:0/2993044244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009b30 con 0x7f1ac006d7e0 2024-05-10T03:31:37.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1ac5800640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034040 tx=0x7f1ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:37.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab403d860 con 0x7f1ac006d7e0 2024-05-10T03:31:37.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab403d9c0 con 0x7f1ac006d7e0 2024-05-10T03:31:37.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab403dd70 con 0x7f1ac006d7e0 2024-05-10T03:31:37.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac015e7d0 con 0x7f1ac006d7e0 2024-05-10T03:31:37.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.166+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac0140a00 con 0x7f1ac006d7e0 2024-05-10T03:31:37.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.167+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac0066ce0 con 0x7f1ac006d7e0 2024-05-10T03:31:37.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.169+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ab404f420 con 0x7f1ac006d7e0 2024-05-10T03:31:37.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.170+0000 7f1abd600640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 0x7f1aa006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:37.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.170+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1ab402ae40 con 0x7f1ac006d7e0 2024-05-10T03:31:37.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.171+0000 7f1ac4e00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 0x7f1aa006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:37.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.172+0000 7f1ac4e00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 0x7f1aa006d140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01517e0 tx=0x7f1aac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:37.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.174+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1ab4075b50 con 0x7f1ac006d7e0 2024-05-10T03:31:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.417+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ac004e290 con 0x7f1ac006d7e0 2024-05-10T03:31:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.419+0000 7f1abd600640 1 -- 172.21.15.55:0/2993044244 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ab4009510 con 0x7f1ac006d7e0 2024-05-10T03:31:37.422 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.421+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 msgr2=0x7f1aa006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.421+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 0x7f1aa006d140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ac01517e0 tx=0x7f1aac008040 comp rx=0 tx=0).stop 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.421+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 msgr2=0x7f1ac0150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.421+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4034040 tx=0x7f1ab4004a40 comp rx=0 tx=0).stop 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 shutdown_connections 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1aa006ac30 0x7f1aa006d140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac0144710 0x7f1ac00af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac00b2fc0 0x7f1ac013fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 --2- 172.21.15.55:0/2993044244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac006d7e0 0x7f1ac0150f90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 >> 172.21.15.55:0/2993044244 conn(0x7f1ac006a110 msgr2=0x7f1ac00b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 shutdown_connections 2024-05-10T03:31:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:37.422+0000 7f1ac6c00640 1 -- 172.21.15.55:0/2993044244 wait complete. 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:31:37.452 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:31:37.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.453 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42379sec preferred_lft 42379sec 2024-05-10T03:31:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:31:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:31:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:37.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.517 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:31:37.596 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:31:37.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.601 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:31:37.651 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:31:37.652 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:31:37.652 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:31:37.652 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:31:37.652 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:31:37.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.652 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:31:37.652 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:31:37.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:31:37.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:31:37.653 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:31:37.653 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:31:37.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.793 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:31:37.840 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:31:37.840 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:31:37.840 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:31:37.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:37.841 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:31:37.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:37.915 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:31:37.961 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:31:37.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:37.962 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:31:38.055 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:31:38.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:38.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:31:38.251 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:31:38.252 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:31:38.252 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:31:38.278 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:31:38.278 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:31:38.278 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:31:38.278 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:38.278 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:31:38.313 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:31:38.315 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:31:38.315 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:31:38.315 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:31:38.315 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:31:38.315 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:31:38.320 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:31:38.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:38.320 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:31:38.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:38.396 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:31:38.452 INFO:teuthology.orchestra.run.smithi162.stdout:client5053 2024-05-10T03:31:38.452 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:38.453 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:38.501 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-05-10T03:31:38.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.797+0000 7f1105000640 1 Processor -- start 2024-05-10T03:31:38.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.798+0000 7f1105000640 1 -- start start 2024-05-10T03:31:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.799+0000 7f1105000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f11000cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.799+0000 7f1105000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f110006b120 0x7f110006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f1105000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000b2a30 con 0x7f11000710f0 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f1105000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110015a900 con 0x7f110006b120 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f1105000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110015eec0 con 0x7f110015e920 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10ffe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11000710f0 0x7f1100071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37946/0 (socket says 172.21.15.55:37946) 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10ffe00640 1 -- 172.21.15.55:0/1955699643 learned_addr learned my addr 172.21.15.55:0/1955699643 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490208771 0 0) 0x7f11000b2a30 con 0x7f11000710f0 2024-05-10T03:31:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4005730 con 0x7f11000710f0 2024-05-10T03:31:38.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10fea00640 1 --2- 172.21.15.55:0/1955699643 >> v2:172.21.15.162:3300/0 conn(0x7f110006b120 0x7f110006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.800+0000 7f10ff400640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f11000cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448854228 0 0) 0x7f10e4005730 con 0x7f11000710f0 2024-05-10T03:31:38.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 msgr2=0x7f11000cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:38.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f11000cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 >> v2:172.21.15.162:3300/0 conn(0x7f110006b120 msgr2=0x7f110006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 --2- 172.21.15.55:0/1955699643 >> v2:172.21.15.162:3300/0 conn(0x7f110006b120 0x7f110006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110006ba70 con 0x7f11000710f0 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f10f80019f0 con 0x7f11000710f0 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 >> v1:172.21.15.55:6789/0 conn(0x7f11000710f0 legacy=0x7f1100071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4005bf0 0x7f10e4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f10e40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f10e4007340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110015eec0 con 0x7f10e4006f10 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400dd20 con 0x7f10e4005bf0 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400de90 con 0x7f110015e920 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.801+0000 7f10ffe00640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f10e4007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ff400640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4005bf0 0x7f10e4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10fea00640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f10e40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ffe00640 1 -- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 msgr2=0x7f10e40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ffe00640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110015e920 0x7f10e40069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ffe00640 1 -- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4005bf0 msgr2=0x7f10e4006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ffe00640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e4005bf0 0x7f10e4006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.802+0000 7f10ffe00640 1 -- 172.21.15.55:0/1955699643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4005730 con 0x7f10e4006f10 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.803+0000 7f10ffe00640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f10e4007340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f10f800a830 tx=0x7f10e4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5692d772aab37fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.803+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f802ba50 con 0x7f10e4006f10 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.803+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f802bbb0 con 0x7f10e4006f10 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.803+0000 7f10fe000640 1 -- 172.21.15.55:0/1955699643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8034470 con 0x7f10e4006f10 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 -- 172.21.15.55:0/1955699643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 msgr2=0x7f10e4007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 --2- 172.21.15.55:0/1955699643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f10e4007340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f10f800a830 tx=0x7f10e4007a00 comp rx=0 tx=0).stop 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f10ffe00640 1 -- 172.21.15.55:0/1955699643 reap_dead start 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 -- 172.21.15.55:0/1955699643 shutdown_connections 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 -- 172.21.15.55:0/1955699643 >> 172.21.15.55:0/1955699643 conn(0x7f110006c750 msgr2=0x7f11000b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 -- 172.21.15.55:0/1955699643 shutdown_connections 2024-05-10T03:31:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.804+0000 7f1105000640 1 -- 172.21.15.55:0/1955699643 wait complete. 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.807+0000 7f1105000640 1 Processor -- start 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.807+0000 7f1105000640 1 -- start start 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110013d830 0x7f1100159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110013f390 0x7f110013f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001f37a0 con 0x7f10e4006f10 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001f3bd0 con 0x7f110013d830 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.808+0000 7f1105000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110013e0a0 con 0x7f110013f390 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37160/0 (socket says 172.21.15.55:37160) 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 -- 172.21.15.55:0/3289186085 learned_addr learned my addr 172.21.15.55:0/3289186085 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10fea00640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110013d830 0x7f1100159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ffe00640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110013f390 0x7f110013f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 -- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110013f390 msgr2=0x7f110013f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110013f390 0x7f110013f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 -- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110013d830 msgr2=0x7f1100159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110013d830 0x7f1100159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.809+0000 7f10ff400640 1 -- 172.21.15.55:0/3289186085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110013fce0 con 0x7f10e4006f10 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f10ff400640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f10f000f580 tx=0x7f10f000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:38.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0013810 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f0013970 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0013ce0 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110013e2f0 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.810+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110013e8b0 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.811+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110004e290 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.812+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10f001f460 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.813+0000 7f10f7400640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 0x7f10e006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.813+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f10f0089940 con 0x7f10e4006f10 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.814+0000 7f10fea00640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 0x7f10e006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:38.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.815+0000 7f10fea00640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 0x7f10e006d140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11000cc650 tx=0x7f10ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:38.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:38.819+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10f0054960 con 0x7f10e4006f10 2024-05-10T03:31:39.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.016+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) v1 -- 0x7f11000b2300 con 0x7f10e4006f10 2024-05-10T03:31:39.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.027+0000 7f10f7400640 1 -- 172.21.15.55:0/3289186085 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v11)=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7f10f0057f80 con 0x7f10e4006f10 2024-05-10T03:31:39.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.029+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 msgr2=0x7f10e006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.029+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 0x7f10e006d140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11000cc650 tx=0x7f10ec008040 comp rx=0 tx=0).stop 2024-05-10T03:31:39.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.029+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 msgr2=0x7f11001f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.029+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f10f000f580 tx=0x7f10f000fa40 comp rx=0 tx=0).stop 2024-05-10T03:31:39.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 shutdown_connections 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110013f390 0x7f110013f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f10e006ac30 0x7f10e006d140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110013d830 0x7f1100159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 --2- 172.21.15.55:0/3289186085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10e4006f10 0x7f11001f3260 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 >> 172.21.15.55:0/3289186085 conn(0x7f110006c750 msgr2=0x7f11000cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 shutdown_connections 2024-05-10T03:31:39.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.030+0000 7f1105000640 1 -- 172.21.15.55:0/3289186085 wait complete. 2024-05-10T03:31:39.065 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:39.065 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:39.117 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:39.117 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:39.217 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.499+0000 7f218fe00640 1 Processor -- start 2024-05-10T03:31:39.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.499+0000 7f218fe00640 1 -- start start 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f219015a5d0 0x7f21900cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f21900b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21900b4ac0 con 0x7f21900b46b0 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21900cdc90 con 0x7f219015a5d0 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219015ad30 con 0x7f21900cb730 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f219015a5d0 0x7f21900cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f219015a5d0 0x7f21900cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37170/0 (socket says 172.21.15.55:37170) 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.502+0000 7f218ea00640 1 -- 172.21.15.55:0/1315581138 learned_addr learned my addr 172.21.15.55:0/1315581138 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.503+0000 7f218ea00640 1 -- 172.21.15.55:0/1315581138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 msgr2=0x7f21900b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.503+0000 7f2187e00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f21900b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.503+0000 7f218ea00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f21900b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.503+0000 7f218ea00640 1 -- 172.21.15.55:0/1315581138 >> v1:172.21.15.162:6789/0 conn(0x7f21900b46b0 legacy=0x7f219015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.503+0000 7f218ea00640 1 -- 172.21.15.55:0/1315581138 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219015b010 con 0x7f219015a5d0 2024-05-10T03:31:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.504+0000 7f218ea00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21900cb1f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2180009870 tx=0x7f218002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e1c8fb71255a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.504+0000 7f218e000640 1 -- 172.21.15.55:0/1315581138 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180032440 con 0x7f219015a5d0 2024-05-10T03:31:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.504+0000 7f218e000640 1 -- 172.21.15.55:0/1315581138 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21800325a0 con 0x7f219015a5d0 2024-05-10T03:31:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.504+0000 7f218e000640 1 -- 172.21.15.55:0/1315581138 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180032930 con 0x7f219015a5d0 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 -- 172.21.15.55:0/1315581138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 msgr2=0x7f21900cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21900cb1f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2180009870 tx=0x7f218002a210 comp rx=0 tx=0).stop 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 -- 172.21.15.55:0/1315581138 shutdown_connections 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f21900b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 --2- 172.21.15.55:0/1315581138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21900cb1f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 -- 172.21.15.55:0/1315581138 >> 172.21.15.55:0/1315581138 conn(0x7f219006a110 msgr2=0x7f219006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 -- 172.21.15.55:0/1315581138 shutdown_connections 2024-05-10T03:31:39.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.505+0000 7f218fe00640 1 -- 172.21.15.55:0/1315581138 wait complete. 2024-05-10T03:31:39.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.508+0000 7f218fe00640 1 Processor -- start 2024-05-10T03:31:39.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.508+0000 7f218fe00640 1 -- start start 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.509+0000 7f218fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900b46b0 0x7f21901575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.509+0000 7f218fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f2190148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.509+0000 7f218fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21901f2580 con 0x7f219015a5d0 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190155550 con 0x7f21900b46b0 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2190147740 con 0x7f21900cb730 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37172/0 (socket says 172.21.15.55:37172) 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218f400640 1 -- 172.21.15.55:0/2878787582 learned_addr learned my addr 172.21.15.55:0/2878787582 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f218ea00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900b46b0 0x7f21901575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.510+0000 7f2187e00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f2190148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 -- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 msgr2=0x7f2190148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f2190148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 -- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900b46b0 msgr2=0x7f21901575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900b46b0 0x7f21901575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 -- 172.21.15.55:0/2878787582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800094d0 con 0x7f219015a5d0 2024-05-10T03:31:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218f400640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f218800be60 tx=0x7f218800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188014440 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21880145a0 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188014950 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2190147a20 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.511+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2190147fe0 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.512+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2190066ce0 con 0x7f219015a5d0 2024-05-10T03:31:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.514+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f218801d3b0 con 0x7f219015a5d0 2024-05-10T03:31:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.515+0000 7f2187400640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 0x7f216c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:39.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.515+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f21880896e0 con 0x7f219015a5d0 2024-05-10T03:31:39.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.515+0000 7f218ea00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 0x7f216c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:39.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.517+0000 7f218ea00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 0x7f216c06d140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:39.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.519+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2188054650 con 0x7f219015a5d0 2024-05-10T03:31:39.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.725+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f219004e290 con 0x7f219015a5d0 2024-05-10T03:31:39.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.727+0000 7f2187400640 1 -- 172.21.15.55:0/2878787582 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f2188057c70 con 0x7f219015a5d0 2024-05-10T03:31:39.731 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:39.731 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 msgr2=0x7f216c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 0x7f216c06d140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f218002e000 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 msgr2=0x7f21901f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f218800be60 tx=0x7f218800bf50 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 shutdown_connections 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f219015a5d0 0x7f21901f19c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21900cb730 0x7f2190148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f216c06ac30 0x7f216c06d140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 --2- 172.21.15.55:0/2878787582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21900b46b0 0x7f21901575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 >> 172.21.15.55:0/2878787582 conn(0x7f219006a110 msgr2=0x7f219014b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 shutdown_connections 2024-05-10T03:31:39.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:39.730+0000 7f218fe00640 1 -- 172.21.15.55:0/2878787582 wait complete. 2024-05-10T03:31:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:31:24.993809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:31:26.385874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:31:39.761 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:40.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.043+0000 7f8fa2400640 1 Processor -- start 2024-05-10T03:31:40.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.043+0000 7f8fa2400640 1 -- start start 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.045+0000 7f8fa2400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0b2110 0x7f8f9c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8fa2400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c0ceb00 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8fa2400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c0d2150 con 0x7f8f9c0ce260 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8fa2400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9c15dcd0 con 0x7f8f9c0b2110 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8f93e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8f9c0b1bc0 0x7f8f9c0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37958/0 (socket says 172.21.15.55:37958) 2024-05-10T03:31:40.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8f93e00640 1 -- 172.21.15.55:0/331222971 learned_addr learned my addr 172.21.15.55:0/331222971 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.046+0000 7f8fa1000640 1 --2- 172.21.15.55:0/331222971 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0b2110 0x7f8f9c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734018960 0 0) 0x7f8f9c0ceb00 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f80003520 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347972082 0 0) 0x7f8f80003520 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0b2110 msgr2=0x7f8f9c15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 --2- 172.21.15.55:0/331222971 >> v2:172.21.15.162:3300/0 conn(0x7f8f9c0b2110 0x7f8f9c15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 >> v1:172.21.15.162:6789/0 conn(0x7f8f9c0ce260 legacy=0x7f8f9c06a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0b2910 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8f84001670 con 0x7f8f9c0b1bc0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 >> v1:172.21.15.55:6789/0 conn(0x7f8f9c0b1bc0 legacy=0x7f8f9c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 0x7f8f80003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800047c0 0x7f8f80004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f80005310 0x7f8f8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c15dcd0 con 0x7f8f800039e0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8000c210 con 0x7f8f80005310 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.047+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8000c3d0 con 0x7f8f800047c0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 0x7f8f80003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1a00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f80005310 0x7f8f8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 -- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800047c0 msgr2=0x7f8f80004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8f93e00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800047c0 0x7f8f80004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800047c0 0x7f8f80004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 -- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f80005310 msgr2=0x7f8f8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f80005310 0x7f8f8000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1000640 1 -- 172.21.15.55:0/331222971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80003520 con 0x7f8f800039e0 2024-05-10T03:31:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1a00640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f80005310 0x7f8f8000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:40.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.048+0000 7f8fa1a00640 1 -- 172.21.15.55:0/331222971 reap_dead start 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.049+0000 7f8fa1000640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 0x7f8f80003e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009ac0 tx=0x7f8f800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4255f5e5a9aecf7b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031400 con 0x7f8f800039e0 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c02d050 con 0x7f8f800039e0 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8f9be00640 1 -- 172.21.15.55:0/331222971 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031780 con 0x7f8f800039e0 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8fa2400640 1 -- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 msgr2=0x7f8f80003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8fa2400640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 0x7f8f80003e50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009ac0 tx=0x7f8f800057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8fa2400640 1 -- 172.21.15.55:0/331222971 shutdown_connections 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8fa2400640 1 --2- 172.21.15.55:0/331222971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f800039e0 0x7f8f80003e50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.050+0000 7f8fa2400640 1 -- 172.21.15.55:0/331222971 >> 172.21.15.55:0/331222971 conn(0x7f8f9c0d30d0 msgr2=0x7f8f9c0d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.051+0000 7f8fa2400640 1 -- 172.21.15.55:0/331222971 shutdown_connections 2024-05-10T03:31:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.051+0000 7f8fa2400640 1 -- 172.21.15.55:0/331222971 wait complete. 2024-05-10T03:31:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.053+0000 7f8fa2400640 1 Processor -- start 2024-05-10T03:31:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.053+0000 7f8fa2400640 1 -- start start 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800039e0 0x7f8f9c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9c160140 0x7f8f9c160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c1f7d50 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c15f7f0 con 0x7f8f9c160140 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8fa2400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c15e020 con 0x7f8f800039e0 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37182/0 (socket says 172.21.15.55:37182) 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.055+0000 7f8f93e00640 1 -- 172.21.15.55:0/4075885234 learned_addr learned my addr 172.21.15.55:0/4075885234 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8fa1a00640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9c160140 0x7f8f9c160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8fa1000640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800039e0 0x7f8f9c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 -- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800039e0 msgr2=0x7f8f9c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800039e0 0x7f8f9c0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 -- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9c160140 msgr2=0x7f8f9c160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9c160140 0x7f8f9c160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 -- 172.21.15.55:0/4075885234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c160a90 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f93e00640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8f84001200 tx=0x7f8f8400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840074e0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f84007640 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84004a00 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.056+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c1f74b0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.057+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c1f7a70 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.058+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c0d34e0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.059+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8f84004b60 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.060+0000 7f8f99600640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 0x7f8f7806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.060+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8f840891b0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.060+0000 7f8fa1000640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 0x7f8f7806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.062+0000 7f8fa1000640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 0x7f8f7806d110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c033000 tx=0x7f8f8c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.065+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8f840542a0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.262+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f9c15e190 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.264+0000 7f8f99600640 1 -- 172.21.15.55:0/4075885234 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f840190a0 con 0x7f8f9c1f6d80 2024-05-10T03:31:40.268 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:40.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 msgr2=0x7f8f7806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 0x7f8f7806d110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c033000 tx=0x7f8f8c008030 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 msgr2=0x7f8f9c1517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8f84001200 tx=0x7f8f8400dc70 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 shutdown_connections 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f7806ac00 0x7f8f7806d110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9c160140 0x7f8f9c160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f9c1f6d80 0x7f8f9c1517d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 --2- 172.21.15.55:0/4075885234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f800039e0 0x7f8f9c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 >> 172.21.15.55:0/4075885234 conn(0x7f8f9c0d30d0 msgr2=0x7f8f9c15be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 shutdown_connections 2024-05-10T03:31:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.267+0000 7f8fa2400640 1 -- 172.21.15.55:0/4075885234 wait complete. 2024-05-10T03:31:40.298 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:40.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.577+0000 7fc554a00640 1 Processor -- start 2024-05-10T03:31:40.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.578+0000 7fc554a00640 1 -- start start 2024-05-10T03:31:40.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.580+0000 7fc554a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5500719f0 0x7fc55006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.580+0000 7fc554a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc550071e00 0x7fc550141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.580+0000 7fc554a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55015fdf0 con 0x7fc55006b840 2024-05-10T03:31:40.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.580+0000 7fc554a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55015f1e0 con 0x7fc5500719f0 2024-05-10T03:31:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.580+0000 7fc554a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55015f4c0 con 0x7fc550071e00 2024-05-10T03:31:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54de00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc55006b840 0x7fc5500700e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37970/0 (socket says 172.21.15.55:37970) 2024-05-10T03:31:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54de00640 1 -- 172.21.15.55:0/3096048402 learned_addr learned my addr 172.21.15.55:0/3096048402 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54e800640 1 --2- 172.21.15.55:0/3096048402 >> v2:172.21.15.162:3300/0 conn(0x7fc5500719f0 0x7fc55006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54f200640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc550071e00 0x7fc550141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799464002 0 0) 0x7fc55015fdf0 con 0x7fc55006b840 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.581+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc534003520 con 0x7fc55006b840 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.582+0000 7fc54e800640 1 -- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc550071e00 msgr2=0x7fc550141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.582+0000 7fc54e800640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc550071e00 0x7fc550141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.582+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2878863051 0 0) 0x7fc534003520 con 0x7fc55006b840 2024-05-10T03:31:40.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.582+0000 7fc54e800640 1 -- 172.21.15.55:0/3096048402 >> v1:172.21.15.55:6789/0 conn(0x7fc55006b840 legacy=0x7fc5500700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.582+0000 7fc54e800640 1 -- 172.21.15.55:0/3096048402 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55015f7a0 con 0x7fc5500719f0 2024-05-10T03:31:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.584+0000 7fc54e800640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5500719f0 0x7fc55006b300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd7d2d6c7485cfb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.584+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400019f0 con 0x7fc5500719f0 2024-05-10T03:31:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53802ab10 con 0x7fc5500719f0 2024-05-10T03:31:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc54d400640 1 -- 172.21.15.55:0/3096048402 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53802aea0 con 0x7fc5500719f0 2024-05-10T03:31:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 -- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5500719f0 msgr2=0x7fc55006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5500719f0 0x7fc55006b300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802a080 comp rx=0 tx=0).stop 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 -- 172.21.15.55:0/3096048402 shutdown_connections 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc550071e00 0x7fc550141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 --2- 172.21.15.55:0/3096048402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5500719f0 0x7fc55006b300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 -- 172.21.15.55:0/3096048402 >> 172.21.15.55:0/3096048402 conn(0x7fc55006c750 msgr2=0x7fc5500b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.585+0000 7fc554a00640 1 -- 172.21.15.55:0/3096048402 shutdown_connections 2024-05-10T03:31:40.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.586+0000 7fc554a00640 1 -- 172.21.15.55:0/3096048402 wait complete. 2024-05-10T03:31:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.589+0000 7fc554a00640 1 Processor -- start 2024-05-10T03:31:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.589+0000 7fc554a00640 1 -- start start 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc55006b840 0x7fc5500ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 0x7fc550130480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc550130060 con 0x7fc550071e00 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5500ae8b0 con 0x7fc55006b840 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.591+0000 7fc554a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc550155f20 con 0x7fc5500719f0 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37184/0 (socket says 172.21.15.55:37184) 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 -- 172.21.15.55:0/1602351905 learned_addr learned my addr 172.21.15.55:0/1602351905 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54e800640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc55006b840 0x7fc5500ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54de00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 0x7fc550130480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 -- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 msgr2=0x7fc550130480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 0x7fc550130480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 -- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc55006b840 msgr2=0x7fc5500ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc55006b840 0x7fc5500ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54f200640 1 -- 172.21.15.55:0/1602351905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538009400 con 0x7fc550071e00 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.592+0000 7fc54de00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 0x7fc550130480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc54f200640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e820 tx=0x7fc54800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548009510 con 0x7fc550071e00 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800ee70 con 0x7fc550071e00 2024-05-10T03:31:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800cb80 con 0x7fc550071e00 2024-05-10T03:31:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc550158a80 con 0x7fc550071e00 2024-05-10T03:31:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.593+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55012f470 con 0x7fc550071e00 2024-05-10T03:31:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.594+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55004e290 con 0x7fc550071e00 2024-05-10T03:31:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.596+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc54800cce0 con 0x7fc550071e00 2024-05-10T03:31:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.597+0000 7fc546a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 0x7fc52c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.597+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc548089550 con 0x7fc550071e00 2024-05-10T03:31:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.598+0000 7fc54e800640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 0x7fc52c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:40.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.599+0000 7fc54e800640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 0x7fc52c06d110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc538034000 tx=0x7fc53802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:40.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.601+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc5480545e0 con 0x7fc550071e00 2024-05-10T03:31:40.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.800+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5500b1390 con 0x7fc550071e00 2024-05-10T03:31:40.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.801+0000 7fc546a00640 1 -- 172.21.15.55:0/1602351905 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc548057c00 con 0x7fc550071e00 2024-05-10T03:31:40.805 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.803+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 msgr2=0x7fc52c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 0x7fc52c06d110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc538034000 tx=0x7fc53802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 msgr2=0x7fc5501309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e820 tx=0x7fc54800ec20 comp rx=0 tx=0).stop 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 shutdown_connections 2024-05-10T03:31:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc550071e00 0x7fc5501309c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5500719f0 0x7fc550130480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc52c06ac00 0x7fc52c06d110 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 --2- 172.21.15.55:0/1602351905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc55006b840 0x7fc5500ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 >> 172.21.15.55:0/1602351905 conn(0x7fc55006c750 msgr2=0x7fc5500cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 shutdown_connections 2024-05-10T03:31:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:40.804+0000 7fc554a00640 1 -- 172.21.15.55:0/1602351905 wait complete. 2024-05-10T03:31:40.834 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:41.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.105+0000 7f9669400640 1 Processor -- start 2024-05-10T03:31:41.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.105+0000 7f9669400640 1 -- start start 2024-05-10T03:31:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.107+0000 7f9669400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f9664158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.107+0000 7f9669400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640708b0 con 0x7f96640b3860 2024-05-10T03:31:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.107+0000 7f9669400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966415b940 con 0x7f9664071db0 2024-05-10T03:31:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.107+0000 7f9669400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640b2490 con 0x7f966406a4b0 2024-05-10T03:31:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.108+0000 7f9663200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f96640b3860 0x7f96640b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37980/0 (socket says 172.21.15.55:37980) 2024-05-10T03:31:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.108+0000 7f9663200640 1 -- 172.21.15.55:0/1564277415 learned_addr learned my addr 172.21.15.55:0/1564277415 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.108+0000 7f9659e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f9664158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.108+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550206335 0 0) 0x7f96640708b0 con 0x7f96640b3860 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.108+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9648003520 con 0x7f96640b3860 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096170713 0 0) 0x7f9648003520 con 0x7f96640b3860 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 msgr2=0x7f9664158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f9664158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 >> v1:172.21.15.162:6789/0 conn(0x7f9664071db0 legacy=0x7f96640ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664070260 con 0x7f96640b3860 2024-05-10T03:31:41.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f965c001670 con 0x7f96640b3860 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 >> v1:172.21.15.55:6789/0 conn(0x7f96640b3860 legacy=0x7f96640b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f96480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9648004d00 0x7f9648005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bb10 con 0x7f9648003f00 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bc80 con 0x7f9648004d00 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964800e050 con 0x7f966406a4b0 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9659e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.109+0000 7f9662800640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f96480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9663200640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9648004d00 0x7f9648005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9659e00640 1 -- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 msgr2=0x7f96480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9659e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966406a4b0 0x7f96480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9659e00640 1 -- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9648004d00 msgr2=0x7f9648005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9659e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9648004d00 0x7f9648005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.110+0000 7f9659e00640 1 -- 172.21.15.55:0/1564277415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648003520 con 0x7f9648003f00 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.111+0000 7f9659e00640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96480047c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f964c009ac0 tx=0x7f96480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da96c118d4670cef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.111+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c031400 con 0x7f9648003f00 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.111+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c02d050 con 0x7f9648003f00 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.111+0000 7f9661e00640 1 -- 172.21.15.55:0/1564277415 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c031780 con 0x7f9648003f00 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 -- 172.21.15.55:0/1564277415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 msgr2=0x7f96480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 --2- 172.21.15.55:0/1564277415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96480047c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f964c009ac0 tx=0x7f96480057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9663200640 1 -- 172.21.15.55:0/1564277415 reap_dead start 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 -- 172.21.15.55:0/1564277415 shutdown_connections 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 -- 172.21.15.55:0/1564277415 >> 172.21.15.55:0/1564277415 conn(0x7f96640cbb90 msgr2=0x7f96640cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 -- 172.21.15.55:0/1564277415 shutdown_connections 2024-05-10T03:31:41.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.112+0000 7f9669400640 1 -- 172.21.15.55:0/1564277415 wait complete. 2024-05-10T03:31:41.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.115+0000 7f9669400640 1 Processor -- start 2024-05-10T03:31:41.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.115+0000 7f9669400640 1 -- start start 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 0x7f966415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9664134a90 0x7f9664134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664135dd0 con 0x7f9648003f00 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664137230 con 0x7f9664134a90 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.116+0000 7f9669400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966415d650 con 0x7f966415cd00 2024-05-10T03:31:41.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37204/0 (socket says 172.21.15.55:37204) 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 -- 172.21.15.55:0/1829291341 learned_addr learned my addr 172.21.15.55:0/1829291341 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9659e00640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 0x7f966415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 -- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 msgr2=0x7f966415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 0x7f966415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 -- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9664134a90 msgr2=0x7f9664134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9664134a90 0x7f9664134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 -- 172.21.15.55:0/1829291341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640cdd50 con 0x7f9648003f00 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9659e00640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 0x7f966415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.117+0000 7f9662800640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9654007ca0 tx=0x7f9654002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.118+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654003b80 con 0x7f9648003f00 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.118+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654003ce0 con 0x7f9648003f00 2024-05-10T03:31:41.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.118+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400c990 con 0x7f9648003f00 2024-05-10T03:31:41.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.118+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640cd790 con 0x7f9648003f00 2024-05-10T03:31:41.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.118+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9664134290 con 0x7f9648003f00 2024-05-10T03:31:41.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.119+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664066f60 con 0x7f9648003f00 2024-05-10T03:31:41.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.120+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f965400caf0 con 0x7f9648003f00 2024-05-10T03:31:41.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.121+0000 7f965b400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 0x7f964006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.121+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9654089780 con 0x7f9648003f00 2024-05-10T03:31:41.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.122+0000 7f9659e00640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 0x7f964006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.123+0000 7f9659e00640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 0x7f964006d110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f964c033000 tx=0x7f964c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.126+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9654054780 con 0x7f9648003f00 2024-05-10T03:31:41.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.326+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9664067220 con 0x7f9648003f00 2024-05-10T03:31:41.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.327+0000 7f965b400640 1 -- 172.21.15.55:0/1829291341 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f965401a310 con 0x7f9648003f00 2024-05-10T03:31:41.331 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 msgr2=0x7f964006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 0x7f964006d110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f964c033000 tx=0x7f964c008030 comp rx=0 tx=0).stop 2024-05-10T03:31:41.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 msgr2=0x7f96641570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9654007ca0 tx=0x7f9654002090 comp rx=0 tx=0).stop 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 shutdown_connections 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f964006ac00 0x7f964006d110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9664134a90 0x7f9664134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f966415cd00 0x7f966415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 --2- 172.21.15.55:0/1829291341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9648003f00 0x7f96641570e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 >> 172.21.15.55:0/1829291341 conn(0x7f96640cbb90 msgr2=0x7f96640cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 shutdown_connections 2024-05-10T03:31:41.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.330+0000 7f9669400640 1 -- 172.21.15.55:0/1829291341 wait complete. 2024-05-10T03:31:41.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:31:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.647+0000 7f2ddf000640 1 Processor -- start 2024-05-10T03:31:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.647+0000 7f2ddf000640 1 -- start start 2024-05-10T03:31:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.649+0000 7f2ddf000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2dd8161c70 0x7f2dd80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.649+0000 7f2ddf000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80d0110 con 0x7f2dd80b5df0 2024-05-10T03:31:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.649+0000 7f2ddf000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80cf330 con 0x7f2dd80cf7c0 2024-05-10T03:31:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.649+0000 7f2ddf000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80b6200 con 0x7f2dd8161c70 2024-05-10T03:31:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.650+0000 7f2dd7e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2dd80b5df0 0x7f2dd80b3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37984/0 (socket says 172.21.15.55:37984) 2024-05-10T03:31:41.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.650+0000 7f2dd7e00640 1 -- 172.21.15.55:0/3582464632 learned_addr learned my addr 172.21.15.55:0/3582464632 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.650+0000 7f2ddd000640 1 --2- 172.21.15.55:0/3582464632 >> v2:172.21.15.162:3300/0 conn(0x7f2dd8161c70 0x7f2dd80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.650+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594729744 0 0) 0x7f2dc8001110 con 0x7f2dd80b5df0 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.650+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dbc003520 con 0x7f2dd80b5df0 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2076562785 0 0) 0x7f2dbc003520 con 0x7f2dd80b5df0 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 >> v2:172.21.15.162:3300/0 conn(0x7f2dd8161c70 msgr2=0x7f2dd80cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 --2- 172.21.15.55:0/3582464632 >> v2:172.21.15.162:3300/0 conn(0x7f2dd8161c70 0x7f2dd80cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 >> v1:172.21.15.162:6789/0 conn(0x7f2dd80cf7c0 legacy=0x7f2dd8161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd8143ab0 con 0x7f2dd80b5df0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2dc8001800 con 0x7f2dd80b5df0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 >> v1:172.21.15.55:6789/0 conn(0x7f2dd80b5df0 legacy=0x7f2dd80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc003970 0x7f2dbc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc0047c0 0x7f2dbc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 0x7f2dbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80b6200 con 0x7f2dbc005180 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc00c120 con 0x7f2dbc003970 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc00c2e0 con 0x7f2dbc0047c0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.651+0000 7f2ddd000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 0x7f2dbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2dd7e00640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc003970 0x7f2dbc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2dcfe00640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc0047c0 0x7f2dbc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2ddd000640 1 -- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc0047c0 msgr2=0x7f2dbc004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2ddd000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc0047c0 0x7f2dbc004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2ddd000640 1 -- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc003970 msgr2=0x7f2dbc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:41.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2ddd000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc003970 0x7f2dbc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.652+0000 7f2ddd000640 1 -- 172.21.15.55:0/3582464632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc003520 con 0x7f2dbc005180 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.653+0000 7f2ddd000640 1 -- 172.21.15.55:0/3582464632 reap_dead start 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.653+0000 7f2ddd000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 0x7f2dbc00b980 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8071970 tx=0x7f2dbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be7580a92ee8976 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.653+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd002e050 con 0x7f2dbc005180 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.653+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000f040 con 0x7f2dbc005180 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.653+0000 7f2dd7400640 1 -- 172.21.15.55:0/3582464632 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003dd0 con 0x7f2dbc005180 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 -- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 msgr2=0x7f2dbc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 0x7f2dbc00b980 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8071970 tx=0x7f2dbc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 -- 172.21.15.55:0/3582464632 shutdown_connections 2024-05-10T03:31:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 --2- 172.21.15.55:0/3582464632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dbc005180 0x7f2dbc00b980 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 -- 172.21.15.55:0/3582464632 >> 172.21.15.55:0/3582464632 conn(0x7f2dd80b2c10 msgr2=0x7f2dd80b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 -- 172.21.15.55:0/3582464632 shutdown_connections 2024-05-10T03:31:41.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.654+0000 7f2ddf000640 1 -- 172.21.15.55:0/3582464632 wait complete. 2024-05-10T03:31:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.657+0000 7f2ddf000640 1 Processor -- start 2024-05-10T03:31:41.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.657+0000 7f2ddf000640 1 -- start start 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc005180 0x7f2dd8146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dd813e420 0x7f2dd80cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8144280 con 0x7f2dd813ecf0 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd81f48f0 con 0x7f2dbc005180 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddf000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8144490 con 0x7f2dd813e420 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37222/0 (socket says 172.21.15.55:37222) 2024-05-10T03:31:41.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 -- 172.21.15.55:0/484115595 learned_addr learned my addr 172.21.15.55:0/484115595 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dd7e00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc005180 0x7f2dd8146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2ddd000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dd813e420 0x7f2dd80cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 -- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dd813e420 msgr2=0x7f2dd80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dd813e420 0x7f2dd80cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 -- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc005180 msgr2=0x7f2dd8146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc005180 0x7f2dd8146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.659+0000 7f2dcfe00640 1 -- 172.21.15.55:0/484115595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd8144770 con 0x7f2dd813ecf0 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2dcfe00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000e4b0 tx=0x7f2dc000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00052e0 con 0x7f2dd813ecf0 2024-05-10T03:31:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0005440 con 0x7f2dd813ecf0 2024-05-10T03:31:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0003830 con 0x7f2dd813ecf0 2024-05-10T03:31:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd80cc750 con 0x7f2dd813ecf0 2024-05-10T03:31:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.660+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd81f4630 con 0x7f2dd813ecf0 2024-05-10T03:31:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.662+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd8067220 con 0x7f2dd813ecf0 2024-05-10T03:31:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.663+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2dc0003990 con 0x7f2dd813ecf0 2024-05-10T03:31:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.663+0000 7f2dd4c00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 0x7f2db406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.664+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2dc0089250 con 0x7f2dd813ecf0 2024-05-10T03:31:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.664+0000 7f2dd7e00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 0x7f2db406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.665+0000 7f2dd7e00640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 0x7f2db406d140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8001050 tx=0x7f2dc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.669+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2dc00541c0 con 0x7f2dd813ecf0 2024-05-10T03:31:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.890+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2dd8067930 con 0x7f2dd813ecf0 2024-05-10T03:31:41.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.891+0000 7f2dd4c00640 1 -- 172.21.15.55:0/484115595 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2dc00577e0 con 0x7f2dd813ecf0 2024-05-10T03:31:41.896 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 msgr2=0x7f2db406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 0x7f2db406d140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8001050 tx=0x7f2dc8009040 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 msgr2=0x7f2dd80ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000e4b0 tx=0x7f2dc000e970 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 shutdown_connections 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dd813e420 0x7f2dd80cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2dd813ecf0 0x7f2dd80ccf50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2db406ac30 0x7f2db406d140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 --2- 172.21.15.55:0/484115595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2dbc005180 0x7f2dd8146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:41.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.894+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 >> 172.21.15.55:0/484115595 conn(0x7f2dd80b2c10 msgr2=0x7f2dd80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.895+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 shutdown_connections 2024-05-10T03:31:41.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:41.895+0000 7f2ddf000640 1 -- 172.21.15.55:0/484115595 wait complete. 2024-05-10T03:31:41.925 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:31:41.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:41.925 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:42.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:31:42.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:42.019 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:31:42.073 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:42.073 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:42.132 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:42.132 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:42.183 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:42.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.474+0000 7f98e6000640 1 Processor -- start 2024-05-10T03:31:42.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.474+0000 7f98e6000640 1 -- start start 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.476+0000 7f98e6000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f98e0144120 0x7f98e006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.476+0000 7f98e6000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e006d350 con 0x7f98e00b3190 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.477+0000 7f98e6000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e006c740 con 0x7f98e0144710 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.477+0000 7f98e6000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e0144530 con 0x7f98e0144120 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.477+0000 7f98e5600640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98e00b3190 0x7f98e00cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37992/0 (socket says 172.21.15.55:37992) 2024-05-10T03:31:42.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.477+0000 7f98e5600640 1 -- 172.21.15.55:0/319592017 learned_addr learned my addr 172.21.15.55:0/319592017 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:42.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.477+0000 7f98dfe00640 1 --2- 172.21.15.55:0/319592017 >> v2:172.21.15.162:3300/0 conn(0x7f98e0144120 0x7f98e006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165894295 0 0) 0x7f98e006d350 con 0x7f98e00b3190 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c8003520 con 0x7f98e00b3190 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814621196 0 0) 0x7f98c8003520 con 0x7f98e00b3190 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98dfe00640 1 -- 172.21.15.55:0/319592017 >> v1:172.21.15.162:6789/0 conn(0x7f98e0144710 legacy=0x7f98e00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98dfe00640 1 -- 172.21.15.55:0/319592017 >> v1:172.21.15.55:6789/0 conn(0x7f98e00b3190 legacy=0x7f98e00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.478+0000 7f98dfe00640 1 -- 172.21.15.55:0/319592017 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e006ca20 con 0x7f98e0144120 2024-05-10T03:31:42.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.480+0000 7f98dfe00640 1 --2- 172.21.15.55:0/319592017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e006d7e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097a0 tx=0x7f98cc02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad8ab25a236482d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.480+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80019f0 con 0x7f98e0144120 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.480+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc02ab30 con 0x7f98e0144120 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.480+0000 7f98df400640 1 -- 172.21.15.55:0/319592017 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc032440 con 0x7f98e0144120 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 -- 172.21.15.55:0/319592017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 msgr2=0x7f98e006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 --2- 172.21.15.55:0/319592017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e006d7e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097a0 tx=0x7f98cc02a0a0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 -- 172.21.15.55:0/319592017 shutdown_connections 2024-05-10T03:31:42.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 --2- 172.21.15.55:0/319592017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e006d7e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 -- 172.21.15.55:0/319592017 >> 172.21.15.55:0/319592017 conn(0x7f98e006a110 msgr2=0x7f98e006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:42.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 -- 172.21.15.55:0/319592017 shutdown_connections 2024-05-10T03:31:42.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.481+0000 7f98e6000640 1 -- 172.21.15.55:0/319592017 wait complete. 2024-05-10T03:31:42.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.483+0000 7f98e6000640 1 Processor -- start 2024-05-10T03:31:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.484+0000 7f98e6000640 1 -- start start 2024-05-10T03:31:42.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98e00b3190 0x7f98e01366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:42.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e0140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0134e00 con 0x7f98e0144710 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01412b0 con 0x7f98e0144120 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.485+0000 7f98e6000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0140ff0 con 0x7f98e00b3190 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37228/0 (socket says 172.21.15.55:37228) 2024-05-10T03:31:42.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 -- 172.21.15.55:0/2621304913 learned_addr learned my addr 172.21.15.55:0/2621304913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98dfe00640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e0140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e4c00640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98e00b3190 0x7f98e01366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 -- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98e00b3190 msgr2=0x7f98e01366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98e00b3190 0x7f98e01366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 -- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 msgr2=0x7f98e0140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e0140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 -- 172.21.15.55:0/2621304913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009400 con 0x7f98e0144710 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.486+0000 7f98e5600640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cc10 tx=0x7f98d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.487+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800ce20 con 0x7f98e0144710 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.487+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8012400 con 0x7f98e0144710 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.487+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80127f0 con 0x7f98e0144710 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.487+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e0141500 con 0x7f98e0144710 2024-05-10T03:31:42.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.487+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e0141ac0 con 0x7f98e0144710 2024-05-10T03:31:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.488+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0066ce0 con 0x7f98e0144710 2024-05-10T03:31:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.489+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f98d8007070 con 0x7f98e0144710 2024-05-10T03:31:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.490+0000 7f98dcc00640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 0x7f98c006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.490+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98d80896c0 con 0x7f98e0144710 2024-05-10T03:31:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.491+0000 7f98e4c00640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 0x7f98c006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.492+0000 7f98e4c00640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 0x7f98c006d110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f60 tx=0x7f98d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.495+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f98d80546c0 con 0x7f98e0144710 2024-05-10T03:31:42.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.698+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98e004e290 con 0x7f98e0144710 2024-05-10T03:31:42.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.701+0000 7f98dcc00640 1 -- 172.21.15.55:0/2621304913 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f98d8057ce0 con 0x7f98e0144710 2024-05-10T03:31:42.706 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:42.706 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.704+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 msgr2=0x7f98c006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.704+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 0x7f98c006d110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f60 tx=0x7f98d4009040 comp rx=0 tx=0).stop 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.704+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 msgr2=0x7f98e01425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.704+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cc10 tx=0x7f98d800a430 comp rx=0 tx=0).stop 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 shutdown_connections 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98e0144710 0x7f98e01425a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98c006ac00 0x7f98c006d110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98e0144120 0x7f98e0140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 --2- 172.21.15.55:0/2621304913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98e00b3190 0x7f98e01366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 >> 172.21.15.55:0/2621304913 conn(0x7f98e006a110 msgr2=0x7f98e00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 shutdown_connections 2024-05-10T03:31:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:42.705+0000 7f98e6000640 1 -- 172.21.15.55:0/2621304913 wait complete. 2024-05-10T03:31:42.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:31:24.993809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:31:26.385874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:31:42.736 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.017+0000 7fa301c00640 1 Processor -- start 2024-05-10T03:31:43.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.018+0000 7fa301c00640 1 -- start start 2024-05-10T03:31:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2fc0719f0 0x7fa2fc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc06b840 0x7fa2fc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc15fdf0 con 0x7fa2fc071e00 2024-05-10T03:31:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc15f1e0 con 0x7fa2fc0719f0 2024-05-10T03:31:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc15f4c0 con 0x7fa2fc06b840 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa2fbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa2fc0719f0 0x7fa2fc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa2fc071e00 0x7fa2fc070620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59788/0 (socket says 172.21.15.55:59788) 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.020+0000 7fa301200640 1 -- 172.21.15.55:0/1939664239 learned_addr learned my addr 172.21.15.55:0/1939664239 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2f3e00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc06b840 0x7fa2fc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2fb400640 1 -- 172.21.15.55:0/1939664239 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774237508 0 0) 0x7fa2fc15fdf0 con 0x7fa2fc071e00 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2fbe00640 1 -- 172.21.15.55:0/1939664239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc06b840 msgr2=0x7fa2fc0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:43.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2fbe00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc06b840 0x7fa2fc0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2fbe00640 1 -- 172.21.15.55:0/1939664239 >> v1:172.21.15.55:6789/0 conn(0x7fa2fc071e00 legacy=0x7fa2fc070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.021+0000 7fa2fbe00640 1 -- 172.21.15.55:0/1939664239 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc15f7a0 con 0x7fa2fc0719f0 2024-05-10T03:31:43.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.022+0000 7fa2fbe00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc06b300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf4d311a6ca6ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.023+0000 7fa2fb400640 1 -- 172.21.15.55:0/1939664239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec02ae50 con 0x7fa2fc0719f0 2024-05-10T03:31:43.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.023+0000 7fa2fb400640 1 -- 172.21.15.55:0/1939664239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec032440 con 0x7fa2fc0719f0 2024-05-10T03:31:43.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.023+0000 7fa2fb400640 1 -- 172.21.15.55:0/1939664239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0327d0 con 0x7fa2fc0719f0 2024-05-10T03:31:43.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.023+0000 7fa301c00640 1 -- 172.21.15.55:0/1939664239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 msgr2=0x7fa2fc06b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.023+0000 7fa301c00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc06b300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0097a0 tx=0x7fa2ec02a100 comp rx=0 tx=0).stop 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 -- 172.21.15.55:0/1939664239 shutdown_connections 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc06b840 0x7fa2fc0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 --2- 172.21.15.55:0/1939664239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc06b300 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 -- 172.21.15.55:0/1939664239 >> 172.21.15.55:0/1939664239 conn(0x7fa2fc06c750 msgr2=0x7fa2fc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 -- 172.21.15.55:0/1939664239 shutdown_connections 2024-05-10T03:31:43.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.024+0000 7fa301c00640 1 -- 172.21.15.55:0/1939664239 wait complete. 2024-05-10T03:31:43.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.026+0000 7fa301c00640 1 Processor -- start 2024-05-10T03:31:43.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.026+0000 7fa301c00640 1 -- start start 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2fc06b840 0x7fa2fc15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc071e00 0x7fa2fc0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1f1550 con 0x7fa2fc0719f0 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1f16b0 con 0x7fa2fc06b840 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa301c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1f2e40 con 0x7fa2fc071e00 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60460/0 (socket says 172.21.15.55:60460) 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.028+0000 7fa2f3e00640 1 -- 172.21.15.55:0/2036073035 learned_addr learned my addr 172.21.15.55:0/2036073035 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2fbe00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2fc06b840 0x7fa2fc15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa301200640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc071e00 0x7fa2fc0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 -- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc071e00 msgr2=0x7fa2fc0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc071e00 0x7fa2fc0af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 -- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2fc06b840 msgr2=0x7fa2fc15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2fc06b840 0x7fa2fc15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 -- 172.21.15.55:0/2036073035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009400 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f3e00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400d5f0 tx=0x7fa2e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.029+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40125f0 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.030+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e4012750 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.030+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4012b00 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.030+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc0ad980 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.030+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc0adbd0 con 0x7fa2fc0719f0 2024-05-10T03:31:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.031+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc04e290 con 0x7fa2fc0719f0 2024-05-10T03:31:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.032+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2e4012c60 con 0x7fa2fc0719f0 2024-05-10T03:31:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.033+0000 7fa2f8c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 0x7fa2d806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.033+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2e4089a20 con 0x7fa2fc0719f0 2024-05-10T03:31:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.034+0000 7fa2fbe00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 0x7fa2d806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.035+0000 7fa2fbe00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 0x7fa2d806d110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec034000 tx=0x7fa2ec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.038+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa2e4054a20 con 0x7fa2fc0719f0 2024-05-10T03:31:43.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.242+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2fc143070 con 0x7fa2fc0719f0 2024-05-10T03:31:43.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.243+0000 7fa2f8c00640 1 -- 172.21.15.55:0/2036073035 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2e4058040 con 0x7fa2fc0719f0 2024-05-10T03:31:43.247 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:43.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 msgr2=0x7fa2d806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 0x7fa2d806d110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec034000 tx=0x7fa2ec02e000 comp rx=0 tx=0).stop 2024-05-10T03:31:43.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 msgr2=0x7fa2fc0c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400d5f0 tx=0x7fa2e400dab0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 shutdown_connections 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2fc071e00 0x7fa2fc0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2fc0719f0 0x7fa2fc0c6120 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d806ac00 0x7fa2d806d110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 --2- 172.21.15.55:0/2036073035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2fc06b840 0x7fa2fc15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 >> 172.21.15.55:0/2036073035 conn(0x7fa2fc06c750 msgr2=0x7fa2fc06ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 shutdown_connections 2024-05-10T03:31:43.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.246+0000 7fa301c00640 1 -- 172.21.15.55:0/2036073035 wait complete. 2024-05-10T03:31:43.278 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:43.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.565+0000 7f7062400640 1 Processor -- start 2024-05-10T03:31:43.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.565+0000 7f7062400640 1 -- start start 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.567+0000 7f7062400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.567+0000 7f7062400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f705c071e00 0x7f705c140b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.567+0000 7f7062400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c15fdf0 con 0x7f705c0719f0 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f7062400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c15f1e0 con 0x7f705c071e00 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f7062400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c15f4c0 con 0x7f705c0710f0 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f7061000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f705c0719f0 0x7f705c06b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59804/0 (socket says 172.21.15.55:59804) 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f7061000640 1 -- 172.21.15.55:0/3493154900 learned_addr learned my addr 172.21.15.55:0/3493154900 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:43.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f7061a00640 1 --2- 172.21.15.55:0/3493154900 >> v2:172.21.15.55:3300/0 conn(0x7f705c071e00 0x7f705c140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.568+0000 7f705be00640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.569+0000 7f7061a00640 1 -- 172.21.15.55:0/3493154900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 msgr2=0x7f705c1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.569+0000 7f7061a00640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.569+0000 7f7061a00640 1 -- 172.21.15.55:0/3493154900 >> v1:172.21.15.55:6789/0 conn(0x7f705c0719f0 legacy=0x7f705c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.569+0000 7f7061a00640 1 -- 172.21.15.55:0/3493154900 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c15f7a0 con 0x7f705c071e00 2024-05-10T03:31:43.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.569+0000 7f705b400640 1 -- 172.21.15.55:0/3493154900 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301521521 0 0) 0x7f705c15fdf0 con 0x7f705c0719f0 2024-05-10T03:31:43.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.570+0000 7f7061a00640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c140b30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f705402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd7a4e0c5f0d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.570+0000 7f705b400640 1 -- 172.21.15.55:0/3493154900 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054031600 con 0x7f705c071e00 2024-05-10T03:31:43.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.570+0000 7f705b400640 1 -- 172.21.15.55:0/3493154900 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705402d050 con 0x7f705c071e00 2024-05-10T03:31:43.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.570+0000 7f705b400640 1 -- 172.21.15.55:0/3493154900 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054031980 con 0x7f705c071e00 2024-05-10T03:31:43.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 -- 172.21.15.55:0/3493154900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 msgr2=0x7f705c140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c140b30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f705402a5d0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 -- 172.21.15.55:0/3493154900 shutdown_connections 2024-05-10T03:31:43.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 --2- 172.21.15.55:0/3493154900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c140b30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 -- 172.21.15.55:0/3493154900 >> 172.21.15.55:0/3493154900 conn(0x7f705c06c750 msgr2=0x7f705c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:43.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 -- 172.21.15.55:0/3493154900 shutdown_connections 2024-05-10T03:31:43.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.571+0000 7f7062400640 1 -- 172.21.15.55:0/3493154900 wait complete. 2024-05-10T03:31:43.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.574+0000 7f7062400640 1 Processor -- start 2024-05-10T03:31:43.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.574+0000 7f7062400640 1 -- start start 2024-05-10T03:31:43.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 0x7f705c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c135530 con 0x7f705c071e00 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c15dc40 con 0x7f705c0719f0 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.575+0000 7f7062400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c135060 con 0x7f705c0710f0 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60484/0 (socket says 172.21.15.55:60484) 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 -- 172.21.15.55:0/2850742724 learned_addr learned my addr 172.21.15.55:0/2850742724 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061000640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f705be00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 0x7f705c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 -- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 msgr2=0x7f705c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 -- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 msgr2=0x7f705c1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 0x7f705c1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f7061a00640 1 -- 172.21.15.55:0/2850742724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009a70 con 0x7f705c071e00 2024-05-10T03:31:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.576+0000 7f705be00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 0x7f705c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7061a00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f7054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705403b820 con 0x7f705c071e00 2024-05-10T03:31:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705403b980 con 0x7f705c071e00 2024-05-10T03:31:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705403bcf0 con 0x7f705c071e00 2024-05-10T03:31:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c15de90 con 0x7f705c071e00 2024-05-10T03:31:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.577+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c157ca0 con 0x7f705c071e00 2024-05-10T03:31:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.578+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c06b9c0 con 0x7f705c071e00 2024-05-10T03:31:43.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.580+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f705403a3a0 con 0x7f705c071e00 2024-05-10T03:31:43.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.580+0000 7f7058c00640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 0x7f703c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:43.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.581+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f70540aa920 con 0x7f705c071e00 2024-05-10T03:31:43.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.581+0000 7f7061000640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 0x7f703c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:43.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.583+0000 7f7061000640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 0x7f703c06d110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f7050007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:43.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.585+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7054079320 con 0x7f705c071e00 2024-05-10T03:31:43.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.781+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f705c04e290 con 0x7f705c071e00 2024-05-10T03:31:43.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.782+0000 7f7058c00640 1 -- 172.21.15.55:0/2850742724 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70540480a0 con 0x7f705c071e00 2024-05-10T03:31:43.787 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:43.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 msgr2=0x7f703c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 0x7f703c06d110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7050000f60 tx=0x7f7050007c00 comp rx=0 tx=0).stop 2024-05-10T03:31:43.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 msgr2=0x7f705c0b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:43.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f7054004a40 comp rx=0 tx=0).stop 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 shutdown_connections 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f705c071e00 0x7f705c0b02f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f703c06ac00 0x7f703c06d110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f705c0719f0 0x7f705c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 --2- 172.21.15.55:0/2850742724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c0710f0 0x7f705c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 >> 172.21.15.55:0/2850742724 conn(0x7f705c06c750 msgr2=0x7f705c1403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 shutdown_connections 2024-05-10T03:31:43.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:43.785+0000 7f7062400640 1 -- 172.21.15.55:0/2850742724 wait complete. 2024-05-10T03:31:43.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.106+0000 7fc8aa200640 1 Processor -- start 2024-05-10T03:31:44.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.106+0000 7fc8aa200640 1 -- start start 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8aa200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc8a41435d0 0x7fc8a4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8aa200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 0x7fc8a406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8aa200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a40cdbe0 con 0x7fc8a40b2040 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8aa200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a415e8b0 con 0x7fc8a41435d0 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8aa200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40ce140 con 0x7fc8a406b120 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8a8e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc8a40b2040 0x7fc8a40ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59814/0 (socket says 172.21.15.55:59814) 2024-05-10T03:31:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8a8e00640 1 -- 172.21.15.55:0/2660894287 learned_addr learned my addr 172.21.15.55:0/2660894287 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.109+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/2660894287 >> v2:172.21.15.55:3300/0 conn(0x7fc8a41435d0 0x7fc8a4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a9800640 1 --2- 172.21.15.55:0/2660894287 >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 0x7fc8a406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3400640 1 -- 172.21.15.55:0/2660894287 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265685546 0 0) 0x7fc8a40cdbe0 con 0x7fc8a40b2040 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3400640 1 -- 172.21.15.55:0/2660894287 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc88c003520 con 0x7fc8a40b2040 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3e00640 1 -- 172.21.15.55:0/2660894287 >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 msgr2=0x7fc8a406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/2660894287 >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 0x7fc8a406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3e00640 1 -- 172.21.15.55:0/2660894287 >> v1:172.21.15.55:6789/0 conn(0x7fc8a40b2040 legacy=0x7fc8a40ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a3e00640 1 -- 172.21.15.55:0/2660894287 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a406ba70 con 0x7fc8a41435d0 2024-05-10T03:31:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.110+0000 7fc8a9800640 1 --2- 172.21.15.55:0/2660894287 >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 0x7fc8a406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:44.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.111+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/2660894287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a41435d0 0x7fc8a4143020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc894009790 tx=0x7fc89402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deaa02b35c8ef6a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.111+0000 7fc8a3400640 1 -- 172.21.15.55:0/2660894287 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89402ae50 con 0x7fc8a41435d0 2024-05-10T03:31:44.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8a3400640 1 -- 172.21.15.55:0/2660894287 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894032440 con 0x7fc8a41435d0 2024-05-10T03:31:44.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8a3400640 1 -- 172.21.15.55:0/2660894287 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8940327b0 con 0x7fc8a41435d0 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 -- 172.21.15.55:0/2660894287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a41435d0 msgr2=0x7fc8a4143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 --2- 172.21.15.55:0/2660894287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a41435d0 0x7fc8a4143020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc894009790 tx=0x7fc89402a140 comp rx=0 tx=0).stop 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 -- 172.21.15.55:0/2660894287 shutdown_connections 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 --2- 172.21.15.55:0/2660894287 >> v2:172.21.15.162:3300/0 conn(0x7fc8a406b120 0x7fc8a406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 --2- 172.21.15.55:0/2660894287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a41435d0 0x7fc8a4143020 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.112+0000 7fc8aa200640 1 -- 172.21.15.55:0/2660894287 >> 172.21.15.55:0/2660894287 conn(0x7fc8a406c730 msgr2=0x7fc8a40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.113+0000 7fc8aa200640 1 -- 172.21.15.55:0/2660894287 shutdown_connections 2024-05-10T03:31:44.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.113+0000 7fc8aa200640 1 -- 172.21.15.55:0/2660894287 wait complete. 2024-05-10T03:31:44.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.115+0000 7fc8aa200640 1 Processor -- start 2024-05-10T03:31:44.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.115+0000 7fc8aa200640 1 -- start start 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a406b120 0x7fc8a413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a41435d0 0x7fc8a413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40a89a0 con 0x7fc8a40b2040 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4165760 con 0x7fc8a406b120 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8aa200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40acf90 con 0x7fc8a41435d0 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60508/0 (socket says 172.21.15.55:60508) 2024-05-10T03:31:44.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 -- 172.21.15.55:0/4269829605 learned_addr learned my addr 172.21.15.55:0/4269829605 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a8e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a406b120 0x7fc8a413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a9800640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a41435d0 0x7fc8a413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 -- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a41435d0 msgr2=0x7fc8a413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a41435d0 0x7fc8a413f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 -- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a406b120 msgr2=0x7fc8a413f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a406b120 0x7fc8a413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.117+0000 7fc8a3e00640 1 -- 172.21.15.55:0/4269829605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8940093a0 con 0x7fc8a40b2040 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8a3e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc894000c00 tx=0x7fc8940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003e50 con 0x7fc8a40b2040 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8940044c0 con 0x7fc8a40b2040 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8940327f0 con 0x7fc8a40b2040 2024-05-10T03:31:44.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a41659b0 con 0x7fc8a40b2040 2024-05-10T03:31:44.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.118+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a40ac940 con 0x7fc8a40b2040 2024-05-10T03:31:44.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.120+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a40b1680 con 0x7fc8a40b2040 2024-05-10T03:31:44.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.121+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc894032950 con 0x7fc8a40b2040 2024-05-10T03:31:44.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.122+0000 7fc8a0c00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 0x7fc88806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.122+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc8940aa0a0 con 0x7fc8a40b2040 2024-05-10T03:31:44.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.123+0000 7fc8a8e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 0x7fc88806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.124+0000 7fc8a8e00640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 0x7fc88806d140 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc898000f60 tx=0x7fc898009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.127+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc8940750b0 con 0x7fc8a40b2040 2024-05-10T03:31:44.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.326+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8a404e290 con 0x7fc8a40b2040 2024-05-10T03:31:44.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.327+0000 7fc8a0c00640 1 -- 172.21.15.55:0/4269829605 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8940786d0 con 0x7fc8a40b2040 2024-05-10T03:31:44.331 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 msgr2=0x7fc88806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 0x7fc88806d140 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc898000f60 tx=0x7fc898009040 comp rx=0 tx=0).stop 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 msgr2=0x7fc8a413ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc894000c00 tx=0x7fc8940046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 shutdown_connections 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a41435d0 0x7fc8a413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a40b2040 0x7fc8a413ec60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc88806ac30 0x7fc88806d140 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 --2- 172.21.15.55:0/4269829605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a406b120 0x7fc8a413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.329+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 >> 172.21.15.55:0/4269829605 conn(0x7fc8a406c730 msgr2=0x7fc8a4070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:44.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.330+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 shutdown_connections 2024-05-10T03:31:44.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.330+0000 7fc8aa200640 1 -- 172.21.15.55:0/4269829605 wait complete. 2024-05-10T03:31:44.358 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:31:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.632+0000 7f2cc4400640 1 Processor -- start 2024-05-10T03:31:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.632+0000 7f2cc4400640 1 -- start start 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.634+0000 7f2cc4400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 0x7f2cbc141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc4400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cbc0b2140 0x7f2cbc142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc4400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc4400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc06ad20 con 0x7f2cbc0b2140 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc4400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc141f50 con 0x7f2cbc15fe50 2024-05-10T03:31:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc4400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06a110 con 0x7f2cbc0b2550 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cbc0b2140 0x7f2cbc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cbc0b2140 0x7f2cbc142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60512/0 (socket says 172.21.15.55:60512) 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 -- 172.21.15.55:0/3710465335 learned_addr learned my addr 172.21.15.55:0/3710465335 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1a00640 1 --2- 172.21.15.55:0/3710465335 >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 0x7f2cbc141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 -- 172.21.15.55:0/3710465335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 msgr2=0x7f2cbc142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc2400640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.635+0000 7f2cc1000640 1 -- 172.21.15.55:0/3710465335 >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 msgr2=0x7f2cbc141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.636+0000 7f2cc1000640 1 --2- 172.21.15.55:0/3710465335 >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 0x7f2cbc141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.636+0000 7f2cc1000640 1 -- 172.21.15.55:0/3710465335 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc06a3f0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.636+0000 7f2cc1a00640 1 --2- 172.21.15.55:0/3710465335 >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 0x7f2cbc141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:44.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.636+0000 7f2cc2400640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.637+0000 7f2cc1000640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc142440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06e520 tx=0x7f2ca802a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcde34e24d5a0d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.637+0000 7f2cbbe00640 1 -- 172.21.15.55:0/3710465335 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8032440 con 0x7f2cbc0b2140 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.637+0000 7f2cbbe00640 1 -- 172.21.15.55:0/3710465335 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80325a0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.637+0000 7f2cbbe00640 1 -- 172.21.15.55:0/3710465335 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8032930 con 0x7f2cbc0b2140 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 -- 172.21.15.55:0/3710465335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 msgr2=0x7f2cbc142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc142440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06e520 tx=0x7f2ca802a260 comp rx=0 tx=0).stop 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 -- 172.21.15.55:0/3710465335 shutdown_connections 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 --2- 172.21.15.55:0/3710465335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc142440 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 --2- 172.21.15.55:0/3710465335 >> v2:172.21.15.162:3300/0 conn(0x7f2cbc15fe50 0x7f2cbc141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 -- 172.21.15.55:0/3710465335 >> 172.21.15.55:0/3710465335 conn(0x7f2cbc0cbb90 msgr2=0x7f2cbc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 -- 172.21.15.55:0/3710465335 shutdown_connections 2024-05-10T03:31:44.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.638+0000 7f2cc4400640 1 -- 172.21.15.55:0/3710465335 wait complete. 2024-05-10T03:31:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.640+0000 7f2cc4400640 1 Processor -- start 2024-05-10T03:31:44.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.640+0000 7f2cc4400640 1 -- start start 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc1f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc15fe50 0x7f2cbc1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15e540 con 0x7f2cbc0b2140 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc1446f0 con 0x7f2cbc15fe50 2024-05-10T03:31:44.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc4400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15c4d0 con 0x7f2cbc0b2550 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60524/0 (socket says 172.21.15.55:60524) 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 -- 172.21.15.55:0/1362617796 learned_addr learned my addr 172.21.15.55:0/1362617796 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 -- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 msgr2=0x7f2cbc1f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc1f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 -- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc15fe50 msgr2=0x7f2cbc1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc15fe50 0x7f2cbc1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.642+0000 7f2cc1a00640 1 -- 172.21.15.55:0/1362617796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80094d0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cc1a00640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c7f0 tx=0x7f2cb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0007820 con 0x7f2cbc0b2140 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0007980 con 0x7f2cbc0b2140 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0007cf0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc1e58c0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.643+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15c720 con 0x7f2cbc0b2140 2024-05-10T03:31:44.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.644+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc066f60 con 0x7f2cbc0b2140 2024-05-10T03:31:44.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.645+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2cb00113f0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.646+0000 7f2cb9600640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 0x7f2c9c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:44.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.646+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2cb0089780 con 0x7f2cbc0b2140 2024-05-10T03:31:44.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.647+0000 7f2cc1000640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 0x7f2c9c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:44.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.648+0000 7f2cc1000640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 0x7f2c9c06d110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8034000 tx=0x7f2ca802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:44.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.651+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cb0054790 con 0x7f2cbc0b2140 2024-05-10T03:31:44.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.863+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2cbc067220 con 0x7f2cbc0b2140 2024-05-10T03:31:44.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.864+0000 7f2cb9600640 1 -- 172.21.15.55:0/1362617796 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2cb0057db0 con 0x7f2cbc0b2140 2024-05-10T03:31:44.868 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:31:44.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 msgr2=0x7f2c9c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 0x7f2c9c06d110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8034000 tx=0x7f2ca802c000 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 msgr2=0x7f2cbc1e5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c7f0 tx=0x7f2cb000ccb0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 shutdown_connections 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac00 0x7f2c9c06d110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc15fe50 0x7f2cbc1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc0b2550 0x7f2cbc1f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 --2- 172.21.15.55:0/1362617796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2140 0x7f2cbc1e5380 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 >> 172.21.15.55:0/1362617796 conn(0x7f2cbc0cbb90 msgr2=0x7f2cbc070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 shutdown_connections 2024-05-10T03:31:44.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:44.867+0000 7f2cc4400640 1 -- 172.21.15.55:0/1362617796 wait complete. 2024-05-10T03:31:44.898 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:31:44.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:44.898 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:31:44.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:31:44.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:44.999 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:31:45.048 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:45.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.326+0000 7f33b0e00640 1 Processor -- start 2024-05-10T03:31:45.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.326+0000 7f33b0e00640 1 -- start start 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac0d0660 0x7f33ac0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33b0e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac143f30 con 0x7f33ac0b45a0 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33b0e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0b4db0 con 0x7f33ac143b20 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33b0e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0b5090 con 0x7f33ac0d0660 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33abe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f33ac0b45a0 0x7f33ac15f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59828/0 (socket says 172.21.15.55:59828) 2024-05-10T03:31:45.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33abe00640 1 -- 172.21.15.55:0/367523418 learned_addr learned my addr 172.21.15.55:0/367523418 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33aaa00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac0d0660 0x7f33ac0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.329+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887095380 0 0) 0x7f33a4001110 con 0x7f33ac0b45a0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3390003520 con 0x7f33ac0b45a0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 341724781 0 0) 0x7f3390003520 con 0x7f33ac0b45a0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac0d0660 msgr2=0x7f33ac0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac0d0660 0x7f33ac0b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 >> v1:172.21.15.162:6789/0 conn(0x7f33ac143b20 legacy=0x7f33ac0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0b5370 con 0x7f33ac0b45a0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f33a4001800 con 0x7f33ac0b45a0 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 >> v1:172.21.15.55:6789/0 conn(0x7f33ac0b45a0 legacy=0x7f33ac15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3390003a10 0x7f3390003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f3390004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3390005110 0x7f339000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0b5090 con 0x7f33900047c0 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339000bfa0 con 0x7f3390005110 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339000c160 con 0x7f3390003a10 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.330+0000 7f33aaa00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f3390004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33abe00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3390005110 0x7f339000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33aaa00640 1 -- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3390003a10 msgr2=0x7f3390003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33ab400640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3390003a10 0x7f3390003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33aaa00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3390003a10 0x7f3390003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33aaa00640 1 -- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3390005110 msgr2=0x7f339000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33aaa00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3390005110 0x7f339000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33aaa00640 1 -- 172.21.15.55:0/367523418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390003520 con 0x7f33900047c0 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33abe00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3390005110 0x7f339000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.331+0000 7f33ab400640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3390003a10 0x7f3390003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:45.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.332+0000 7f33aaa00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f3390004bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3394009ac0 tx=0x7f33900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fa9d8fef12e4db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:45.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.332+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394031400 con 0x7f33900047c0 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.332+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339402d050 con 0x7f33900047c0 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.332+0000 7f33aa000640 1 -- 172.21.15.55:0/367523418 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394031780 con 0x7f33900047c0 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 -- 172.21.15.55:0/367523418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 msgr2=0x7f3390004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 --2- 172.21.15.55:0/367523418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f3390004bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3394009ac0 tx=0x7f33900057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33abe00640 1 -- 172.21.15.55:0/367523418 reap_dead start 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 -- 172.21.15.55:0/367523418 shutdown_connections 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 -- 172.21.15.55:0/367523418 >> 172.21.15.55:0/367523418 conn(0x7f33ac06a2f0 msgr2=0x7f33ac0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 -- 172.21.15.55:0/367523418 shutdown_connections 2024-05-10T03:31:45.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.333+0000 7f33b0e00640 1 -- 172.21.15.55:0/367523418 wait complete. 2024-05-10T03:31:45.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.336+0000 7f33b0e00640 1 Processor -- start 2024-05-10T03:31:45.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.336+0000 7f33b0e00640 1 -- start start 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.337+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac140860 0x7f33ac153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33b0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 0x7f33ac1f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33b0e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac136eb0 con 0x7f33900047c0 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33b0e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1f4290 con 0x7f33ac1f3750 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33b0e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac152830 con 0x7f33ac140860 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60548/0 (socket says 172.21.15.55:60548) 2024-05-10T03:31:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 -- 172.21.15.55:0/249229765 learned_addr learned my addr 172.21.15.55:0/249229765 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 -- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac140860 msgr2=0x7f33ac153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33aaa00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac140860 0x7f33ac153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33abe00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 0x7f33ac1f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac140860 0x7f33ac153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 -- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 msgr2=0x7f33ac1f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 0x7f33ac1f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33ab400640 1 -- 172.21.15.55:0/249229765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac152a70 con 0x7f33900047c0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.338+0000 7f33abe00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 0x7f33ac1f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33ab400640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f339c00db40 tx=0x7f339c0020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c017070 con 0x7f33900047c0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c003ba0 con 0x7f33900047c0 2024-05-10T03:31:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c006b50 con 0x7f33900047c0 2024-05-10T03:31:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac1538d0 con 0x7f33900047c0 2024-05-10T03:31:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.339+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac1f40a0 con 0x7f33900047c0 2024-05-10T03:31:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.341+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac067740 con 0x7f33900047c0 2024-05-10T03:31:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.342+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f339c006cb0 con 0x7f33900047c0 2024-05-10T03:31:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.342+0000 7f33a3400640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 0x7f338806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.343+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f339c089060 con 0x7f33900047c0 2024-05-10T03:31:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.343+0000 7f33aaa00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 0x7f338806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:45.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.344+0000 7f33aaa00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 0x7f338806d110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3394033000 tx=0x7f3394008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.347+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f339c054190 con 0x7f33900047c0 2024-05-10T03:31:45.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.584+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33ac04e290 con 0x7f33900047c0 2024-05-10T03:31:45.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.586+0000 7f33a3400640 1 -- 172.21.15.55:0/249229765 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f339c0577b0 con 0x7f33900047c0 2024-05-10T03:31:45.590 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:45.590 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:45.590 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 msgr2=0x7f338806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 0x7f338806d110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3394033000 tx=0x7f3394008030 comp rx=0 tx=0).stop 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 msgr2=0x7f33ac136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f339c00db40 tx=0x7f339c0020a0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 shutdown_connections 2024-05-10T03:31:45.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f338806ac00 0x7f338806d110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33ac1f3750 0x7f33ac1f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33ac140860 0x7f33ac153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 --2- 172.21.15.55:0/249229765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33900047c0 0x7f33ac136970 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 >> 172.21.15.55:0/249229765 conn(0x7f33ac06a2f0 msgr2=0x7f33ac15ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 shutdown_connections 2024-05-10T03:31:45.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:45.589+0000 7f33b0e00640 1 -- 172.21.15.55:0/249229765 wait complete. 2024-05-10T03:31:45.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-10T03:31:45.923 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":43,"msgr_send_messages":36,"msgr_recv_bytes":12394,"msgr_send_bytes":22006,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.017679495,"msgr_running_send_time":0.002036595,"msgr_running_recv_time":0.009430825,"msgr_running_fast_dispatch_time":0.001594945,"msgr_send_messages_queue_lat":{"avgcount":35,"sum":0.168525444,"avgtime":0.004815012},"msgr_handle_ack_lat":{"avgcount":4,"sum":0.000004830,"avgtime":0.000001207},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":12,"msgr_send_messages":20,"msgr_recv_bytes":2979,"msgr_send_bytes":99608,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.018857509,"msgr_running_send_time":0.001937654,"msgr_running_recv_time":0.015964923,"msgr_running_fast_dispatch_time":0.000562173,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.241694241,"avgtime":0.012720749},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94752},"AsyncMessenger::Worker-2":{"msgr_recv_messages":97,"msgr_send_messages":85,"msgr_recv_bytes":123443,"msgr_send_bytes":47395,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.023975410,"msgr_running_send_time":0.007991680,"msgr_running_recv_time":0.021800135,"msgr_running_fast_dispatch_time":0.001497277,"msgr_send_messages_queue_lat":{"avgcount":85,"sum":0.139894383,"avgtime":0.001645816},"msgr_handle_ack_lat":{"avgcount":27,"sum":0.000020722,"avgtime":0.000000767},"msgr_recv_encrypted_bytes":111312,"msgr_send_encrypted_bytes":11648},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.003605911,"avgtime":0.000069344}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000019233,"avgtime":0.000009616}},"mds":{"request":26,"reply":26,"reply_latency":{"avgcount":26,"sum":0.015911099,"avgtime":0.000611965},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":51,"traverse_hit":51,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196981,"rdpos":0,"jlat":{"avgcount":6,"sum":0.030614445,"avgtime":0.005102407},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":107776,"heap":207224},"mds_server":{"dispatch_client_request":26,"dispatch_server_request":0,"handle_client_request":26,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":25,"sum":0.015277837,"avgtime":0.000611113},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000633262,"avgtime":0.000633262},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":24,"average_load":12,"avg_session_uptime":4,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":225239,"buffer_anon_items":82,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51880,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":45,"op_send_bytes":13887,"op_resend":31,"op_reply":45,"op_latency":{"avgcount":45,"sum":1.519969465,"avgtime":0.033777099},"op_inflight":0,"oplen_avg":{"avgcount":45,"sum":69},"op":45,"op_r":0,"op_w":45,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":40,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":144,"get_sum":121474,"get_or_fail_fail":0,"get_or_fail_success":144,"take":0,"take_sum":0,"put":144,"put_sum":121474,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":13887,"put":45,"put_sum":13887,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":45,"put":45,"put_sum":45,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5600617f4410":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2677,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":6,"put_sum":2677,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-10T03:31:45.924 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 4, "avgtime": 1.207e-06, "sum": 4.83e-06 }, "msgr_recv_bytes": 12394, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 43, "msgr_running_fast_dispatch_time": 0.001594945, "msgr_running_recv_time": 0.009430825, "msgr_running_send_time": 0.002036595, "msgr_running_total_time": 0.017679495, "msgr_send_bytes": 22006, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 36, "msgr_send_messages_queue_lat": { "avgcount": 35, "avgtime": 0.004815012, "sum": 0.168525444 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2979, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 12, "msgr_running_fast_dispatch_time": 0.000562173, "msgr_running_recv_time": 0.015964923, "msgr_running_send_time": 0.001937654, "msgr_running_total_time": 0.018857509, "msgr_send_bytes": 99608, "msgr_send_encrypted_bytes": 94752, "msgr_send_messages": 20, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.012720749, "sum": 0.241694241 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 27, "avgtime": 7.67e-07, "sum": 2.0722e-05 }, "msgr_recv_bytes": 123443, "msgr_recv_encrypted_bytes": 111312, "msgr_recv_messages": 97, "msgr_running_fast_dispatch_time": 0.001497277, "msgr_running_recv_time": 0.021800135, "msgr_running_send_time": 0.00799168, "msgr_running_total_time": 0.02397541, "msgr_send_bytes": 47395, "msgr_send_encrypted_bytes": 11648, "msgr_send_messages": 85, "msgr_send_messages_queue_lat": { "avgcount": 85, "avgtime": 0.001645816, "sum": 0.139894383 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 6.9344e-05, "sum": 0.003605911 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.616e-06, "sum": 1.9233e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 26, "reply_latency": { "avgcount": 26, "avgtime": 0.000611965, "sum": 0.015911099 }, "request": 26, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 51, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 51, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.005102407, "sum": 0.030614445 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4196981 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207224, "ino": 13, "ino+": 13, "ino-": 0, "rss": 107776 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 26, "dispatch_server_request": 0, "handle_client_request": 26, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 25, "avgtime": 0.000611113, "sum": 0.015277837 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000633262, "sum": 0.000633262 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 12, "avg_session_uptime": 4, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 24 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 225239, "buffer_anon_items": 82, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51880, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 40, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 45, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 45, "avgtime": 0.033777099, "sum": 1.519969465 }, "op_pg": 0, "op_r": 0, "op_reply": 45, "op_resend": 31, "op_rmw": 0, "op_send": 45, "op_send_bytes": 13887, "op_w": 45, "oplen_avg": { "avgcount": 45, "sum": 69 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 144, "get_or_fail_fail": 0, "get_or_fail_success": 144, "get_started": 0, "get_sum": 121474, "max": 104857600, "put": 144, "put_sum": 121474, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 45, "put_sum": 13887, "take": 45, "take_sum": 13887, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 45, "put_sum": 45, "take": 45, "take_sum": 45, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5600617f4410": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2677, "max": 3758096384, "put": 6, "put_sum": 2677, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-10T03:31:45.925 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:31:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.205+0000 7f70dee00640 1 Processor -- start 2024-05-10T03:31:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.205+0000 7f70dee00640 1 -- start start 2024-05-10T03:31:46.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.207+0000 7f70dee00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70d8161c50 0x7f70d80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.207+0000 7f70dee00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d80d2410 con 0x7f70d80d28a0 2024-05-10T03:31:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.207+0000 7f70dee00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d8162060 con 0x7f70d8165e60 2024-05-10T03:31:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.207+0000 7f70dee00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d81665f0 con 0x7f70d8161c50 2024-05-10T03:31:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.208+0000 7f70dd000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f70d80d28a0 0x7f70d8165750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59830/0 (socket says 172.21.15.55:59830) 2024-05-10T03:31:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.208+0000 7f70dd000640 1 -- 172.21.15.55:0/1272876447 learned_addr learned my addr 172.21.15.55:0/1272876447 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.208+0000 7f70dda00640 1 --2- 172.21.15.55:0/1272876447 >> v2:172.21.15.162:3300/0 conn(0x7f70d8161c50 0x7f70d80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.208+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633903859 0 0) 0x7f70d80d2410 con 0x7f70d80d28a0 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.208+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70bc003520 con 0x7f70d80d28a0 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794407305 0 0) 0x7f70bc003520 con 0x7f70d80d28a0 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 >> v2:172.21.15.162:3300/0 conn(0x7f70d8161c50 msgr2=0x7f70d80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 --2- 172.21.15.55:0/1272876447 >> v2:172.21.15.162:3300/0 conn(0x7f70d8161c50 0x7f70d80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 >> v1:172.21.15.162:6789/0 conn(0x7f70d8165e60 legacy=0x7f70d80b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80d1800 con 0x7f70d80d28a0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f70c4001670 con 0x7f70d80d28a0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 >> v1:172.21.15.55:6789/0 conn(0x7f70d80d28a0 legacy=0x7f70d8165750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70bc0039e0 0x7f70bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 0x7f70bc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc005360 0x7f70bc00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81665f0 con 0x7f70bc0047c0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c2c0 con 0x7f70bc005360 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c480 con 0x7f70bc0039e0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.209+0000 7f70dd000640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 0x7f70bc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dda00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70bc0039e0 0x7f70bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70de400640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc005360 0x7f70bc00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dd000640 1 -- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70bc0039e0 msgr2=0x7f70bc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dd000640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70bc0039e0 0x7f70bc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dd000640 1 -- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc005360 msgr2=0x7f70bc00b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dd000640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc005360 0x7f70bc00b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70dd000640 1 -- 172.21.15.55:0/1272876447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc003520 con 0x7f70bc0047c0 2024-05-10T03:31:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70de400640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc005360 0x7f70bc00b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.210+0000 7f70de400640 1 -- 172.21.15.55:0/1272876447 reap_dead start 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.211+0000 7f70dd000640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 0x7f70bc004bd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f70c400c240 tx=0x7f70bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e0b96ea9dc8d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.211+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c402e070 con 0x7f70bc0047c0 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.211+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c40058d0 con 0x7f70bc0047c0 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.211+0000 7f70d7e00640 1 -- 172.21.15.55:0/1272876447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4005d00 con 0x7f70bc0047c0 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 -- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 msgr2=0x7f70bc004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 0x7f70bc004bd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f70c400c240 tx=0x7f70bc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 -- 172.21.15.55:0/1272876447 shutdown_connections 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 --2- 172.21.15.55:0/1272876447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70bc0047c0 0x7f70bc004bd0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 -- 172.21.15.55:0/1272876447 >> 172.21.15.55:0/1272876447 conn(0x7f70d80d31a0 msgr2=0x7f70d80d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:46.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 -- 172.21.15.55:0/1272876447 shutdown_connections 2024-05-10T03:31:46.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.212+0000 7f70dee00640 1 -- 172.21.15.55:0/1272876447 wait complete. 2024-05-10T03:31:46.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.215+0000 7f70dee00640 1 Processor -- start 2024-05-10T03:31:46.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.215+0000 7f70dee00640 1 -- start start 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc0047c0 0x7f70d8158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70d81fb140 0x7f70d8144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80b81c0 con 0x7f70d80cd660 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81587e0 con 0x7f70bc0047c0 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.216+0000 7f70dee00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81f5080 con 0x7f70d81fb140 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60570/0 (socket says 172.21.15.55:60570) 2024-05-10T03:31:46.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 -- 172.21.15.55:0/3351506609 learned_addr learned my addr 172.21.15.55:0/3351506609 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dda00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc0047c0 0x7f70d8158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70de400640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70d81fb140 0x7f70d8144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 -- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70d81fb140 msgr2=0x7f70d8144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70d81fb140 0x7f70d8144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 -- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc0047c0 msgr2=0x7f70d8158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc0047c0 0x7f70d8158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.217+0000 7f70dd000640 1 -- 172.21.15.55:0/3351506609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d81fb550 con 0x7f70d80cd660 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70dd000640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001930 tx=0x7f70c4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40049e0 con 0x7f70d80cd660 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c4004b40 con 0x7f70d80cd660 2024-05-10T03:31:46.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4005040 con 0x7f70d80cd660 2024-05-10T03:31:46.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d8158a30 con 0x7f70d80cd660 2024-05-10T03:31:46.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.218+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d8158ff0 con 0x7f70d80cd660 2024-05-10T03:31:46.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.220+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d804e290 con 0x7f70d80cd660 2024-05-10T03:31:46.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.221+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f70c40051a0 con 0x7f70d80cd660 2024-05-10T03:31:46.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.222+0000 7f70d5600640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 0x7f70b806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:46.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.222+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f70c40a9670 con 0x7f70d80cd660 2024-05-10T03:31:46.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.222+0000 7f70dda00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 0x7f70b806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:46.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.224+0000 7f70dda00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 0x7f70b806d110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007380 tx=0x7f70c800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:46.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.226+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f70c4074820 con 0x7f70d80cd660 2024-05-10T03:31:46.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.468+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70d804e290 con 0x7f70d80cd660 2024-05-10T03:31:46.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.469+0000 7f70d5600640 1 -- 172.21.15.55:0/3351506609 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70c4032e60 con 0x7f70d80cd660 2024-05-10T03:31:46.474 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 msgr2=0x7f70b806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 0x7f70b806d110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007380 tx=0x7f70c800b040 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 msgr2=0x7f70d81fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001930 tx=0x7f70c4001a00 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 shutdown_connections 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.472+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70d81fb140 0x7f70d8144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d80cd660 0x7f70d81fac00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f70b806ac00 0x7f70b806d110 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 --2- 172.21.15.55:0/3351506609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70bc0047c0 0x7f70d8158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 >> 172.21.15.55:0/3351506609 conn(0x7f70d80d31a0 msgr2=0x7f70d8162a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 shutdown_connections 2024-05-10T03:31:46.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:46.473+0000 7f70dee00640 1 -- 172.21.15.55:0/3351506609 wait complete. 2024-05-10T03:31:46.504 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:31:46.505 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:46.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.506 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:31:46.559 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:31:46.559 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:31:46.559 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:46.560 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42345sec preferred_lft 42345sec 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.561 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:31:46.562 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:31:46.563 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:31:46.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.564 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:31:46.647 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:31:46.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.652 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:31:46.707 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:31:46.707 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:31:46.708 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:31:46.708 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:31:46.708 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:31:46.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:31:46.709 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:31:46.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.871 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:31:46.908 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:31:46.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:31:46.926 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:31:46.926 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:31:46.926 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:46.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:46.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:31:47.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:47.047 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:47.145 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:31:47.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:31:47.145 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:47.192 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:31:47.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:47.193 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:47.393 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:47.393 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:47.393 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:47.421 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:47.421 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:47.421 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:47.421 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:47.422 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:31:47.459 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:31:47.460 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:31:47.461 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:31:47.461 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:31:47.461 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:31:47.461 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:31:47.461 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:31:47.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:31:47.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:47.466 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:47.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:47.540 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:31:47.596 INFO:teuthology.orchestra.run.smithi055.stdout:client5113 2024-05-10T03:31:47.597 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:47.597 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:47.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-10T03:31:48.004 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":44,"msgr_send_messages":37,"msgr_recv_bytes":12640,"msgr_send_bytes":22575,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.018087665,"msgr_running_send_time":0.002104140,"msgr_running_recv_time":0.009542163,"msgr_running_fast_dispatch_time":0.001654981,"msgr_send_messages_queue_lat":{"avgcount":36,"sum":0.168579343,"avgtime":0.004682759},"msgr_handle_ack_lat":{"avgcount":4,"sum":0.000004830,"avgtime":0.000001207},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":12,"msgr_send_messages":20,"msgr_recv_bytes":2979,"msgr_send_bytes":99608,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.018857509,"msgr_running_send_time":0.001937654,"msgr_running_recv_time":0.015964923,"msgr_running_fast_dispatch_time":0.000562173,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.241694241,"avgtime":0.012720749},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94752},"AsyncMessenger::Worker-2":{"msgr_recv_messages":102,"msgr_send_messages":90,"msgr_recv_bytes":124766,"msgr_send_bytes":49619,"msgr_created_connections":8,"msgr_active_connections":4,"msgr_running_total_time":0.025396581,"msgr_running_send_time":0.008551292,"msgr_running_recv_time":0.022720201,"msgr_running_fast_dispatch_time":0.001522765,"msgr_send_messages_queue_lat":{"avgcount":90,"sum":0.140151616,"avgtime":0.001557240},"msgr_handle_ack_lat":{"avgcount":31,"sum":0.000025613,"avgtime":0.000000826},"msgr_recv_encrypted_bytes":111584,"msgr_send_encrypted_bytes":12240},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003679704,"avgtime":0.000068142}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000019233,"avgtime":0.000009616}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.016735402,"avgtime":0.000577082},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":57,"traverse_hit":57,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197239,"rdpos":0,"jlat":{"avgcount":7,"sum":0.034574878,"avgtime":0.004939268},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":107904,"heap":207224},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":28,"sum":0.016102140,"avgtime":0.000575076},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000633262,"avgtime":0.000633262},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":26,"average_load":26,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":241447,"buffer_anon_items":87,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51880,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":14145,"op_resend":31,"op_reply":46,"op_latency":{"avgcount":46,"sum":1.523969385,"avgtime":0.033129769},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":70},"op":46,"op_r":0,"op_w":46,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":40,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":150,"get_sum":122515,"get_or_fail_fail":0,"get_or_fail_success":150,"take":0,"take_sum":0,"put":150,"put_sum":122515,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":14145,"put":46,"put_sum":14145,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5600617f4410":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":2935,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":7,"put_sum":2935,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-10T03:31:48.006 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 4, "avgtime": 1.207e-06, "sum": 4.83e-06 }, "msgr_recv_bytes": 12640, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 44, "msgr_running_fast_dispatch_time": 0.001654981, "msgr_running_recv_time": 0.009542163, "msgr_running_send_time": 0.00210414, "msgr_running_total_time": 0.018087665, "msgr_send_bytes": 22575, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 36, "avgtime": 0.004682759, "sum": 0.168579343 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2979, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 12, "msgr_running_fast_dispatch_time": 0.000562173, "msgr_running_recv_time": 0.015964923, "msgr_running_send_time": 0.001937654, "msgr_running_total_time": 0.018857509, "msgr_send_bytes": 99608, "msgr_send_encrypted_bytes": 94752, "msgr_send_messages": 20, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.012720749, "sum": 0.241694241 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 31, "avgtime": 8.26e-07, "sum": 2.5613e-05 }, "msgr_recv_bytes": 124766, "msgr_recv_encrypted_bytes": 111584, "msgr_recv_messages": 102, "msgr_running_fast_dispatch_time": 0.001522765, "msgr_running_recv_time": 0.022720201, "msgr_running_send_time": 0.008551292, "msgr_running_total_time": 0.025396581, "msgr_send_bytes": 49619, "msgr_send_encrypted_bytes": 12240, "msgr_send_messages": 90, "msgr_send_messages_queue_lat": { "avgcount": 90, "avgtime": 0.00155724, "sum": 0.140151616 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.8142e-05, "sum": 0.003679704 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.616e-06, "sum": 1.9233e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000577082, "sum": 0.016735402 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 57, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 57, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.004939268, "sum": 0.034574878 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4197239 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207224, "ino": 13, "ino+": 13, "ino-": 0, "rss": 107904 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 28, "avgtime": 0.000575076, "sum": 0.01610214 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000633262, "sum": 0.000633262 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 26, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 26 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 241447, "buffer_anon_items": 87, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51880, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 40, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.033129769, "sum": 1.523969385 }, "op_pg": 0, "op_r": 0, "op_reply": 46, "op_resend": 31, "op_rmw": 0, "op_send": 46, "op_send_bytes": 14145, "op_w": 46, "oplen_avg": { "avgcount": 46, "sum": 70 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 150, "get_or_fail_fail": 0, "get_or_fail_success": 150, "get_started": 0, "get_sum": 122515, "max": 104857600, "put": 150, "put_sum": 122515, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 14145, "take": 46, "take_sum": 14145, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5600617f4410": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 2935, "max": 3758096384, "put": 7, "put_sum": 2935, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-10T03:31:48.007 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:48.008 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:48.063 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:48.063 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:48.121 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:48.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.402+0000 7f9de3e00640 1 Processor -- start 2024-05-10T03:31:48.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.402+0000 7f9de3e00640 1 -- start start 2024-05-10T03:31:48.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9de40b2d00 0x7f9de41471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de4162c50 con 0x7f9de4162130 2024-05-10T03:31:48.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de40b1f60 con 0x7f9de40b2d00 2024-05-10T03:31:48.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de4162ed0 con 0x7f9de4144b60 2024-05-10T03:31:48.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9de4162130 0x7f9de4162540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59834/0 (socket says 172.21.15.55:59834) 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de3400640 1 -- 172.21.15.55:0/3618136433 learned_addr learned my addr 172.21.15.55:0/3618136433 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.405+0000 7f9de2a00640 1 --2- 172.21.15.55:0/3618136433 >> v2:172.21.15.55:3300/0 conn(0x7f9de40b2d00 0x7f9de41471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2000640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de1600640 1 -- 172.21.15.55:0/3618136433 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790175569 0 0) 0x7f9de4162c50 con 0x7f9de4162130 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de1600640 1 -- 172.21.15.55:0/3618136433 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc8003520 con 0x7f9de4162130 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2a00640 1 -- 172.21.15.55:0/3618136433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 msgr2=0x7f9de4147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2a00640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2a00640 1 -- 172.21.15.55:0/3618136433 >> v1:172.21.15.55:6789/0 conn(0x7f9de4162130 legacy=0x7f9de4162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2a00640 1 -- 172.21.15.55:0/3618136433 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40b2220 con 0x7f9de40b2d00 2024-05-10T03:31:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.406+0000 7f9de2000640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.408+0000 7f9de2a00640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de40b2d00 0x7f9de41471f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009d90 tx=0x7f9dd402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d4076a85190f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.408+0000 7f9de1600640 1 -- 172.21.15.55:0/3618136433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4031600 con 0x7f9de40b2d00 2024-05-10T03:31:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.408+0000 7f9de1600640 1 -- 172.21.15.55:0/3618136433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd402d050 con 0x7f9de40b2d00 2024-05-10T03:31:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.408+0000 7f9de1600640 1 -- 172.21.15.55:0/3618136433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4031980 con 0x7f9de40b2d00 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 -- 172.21.15.55:0/3618136433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de40b2d00 msgr2=0x7f9de41471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de40b2d00 0x7f9de41471f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009d90 tx=0x7f9dd402a5d0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 -- 172.21.15.55:0/3618136433 shutdown_connections 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 --2- 172.21.15.55:0/3618136433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de40b2d00 0x7f9de41471f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 -- 172.21.15.55:0/3618136433 >> 172.21.15.55:0/3618136433 conn(0x7f9de406a110 msgr2=0x7f9de406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 -- 172.21.15.55:0/3618136433 shutdown_connections 2024-05-10T03:31:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.409+0000 7f9de3e00640 1 -- 172.21.15.55:0/3618136433 wait complete. 2024-05-10T03:31:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.411+0000 7f9de3e00640 1 Processor -- start 2024-05-10T03:31:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.412+0000 7f9de3e00640 1 -- start start 2024-05-10T03:31:48.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de40b2d00 0x7f9de413b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de413bd60 con 0x7f9de4162130 2024-05-10T03:31:48.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de413bb50 con 0x7f9de40b2d00 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.413+0000 7f9de3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de413e450 con 0x7f9de4144b60 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60588/0 (socket says 172.21.15.55:60588) 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 -- 172.21.15.55:0/1388348250 learned_addr learned my addr 172.21.15.55:0/1388348250 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de2a00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de40b2d00 0x7f9de413b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de2000640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 -- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 msgr2=0x7f9de4148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 -- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de40b2d00 msgr2=0x7f9de413b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de40b2d00 0x7f9de413b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.414+0000 7f9de3400640 1 -- 172.21.15.55:0/1388348250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4009a70 con 0x7f9de4162130 2024-05-10T03:31:48.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9de3400640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc001eb0 tx=0x7f9ddc00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc005ce0 con 0x7f9de4162130 2024-05-10T03:31:48.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc005e40 con 0x7f9de4162130 2024-05-10T03:31:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0072d0 con 0x7f9de4162130 2024-05-10T03:31:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de41493a0 con 0x7f9de4162130 2024-05-10T03:31:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.415+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de41453f0 con 0x7f9de4162130 2024-05-10T03:31:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.416+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de4066ce0 con 0x7f9de4162130 2024-05-10T03:31:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.417+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9ddc0074f0 con 0x7f9de4162130 2024-05-10T03:31:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.419+0000 7f9ddb400640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 0x7f9dc006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.419+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ddc088aa0 con 0x7f9de4162130 2024-05-10T03:31:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.420+0000 7f9de2a00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 0x7f9dc006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.421+0000 7f9de2a00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 0x7f9dc006d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4034000 tx=0x7f9dd4033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.424+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9ddc053ac0 con 0x7f9de4162130 2024-05-10T03:31:48.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.625+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9de404e290 con 0x7f9de4162130 2024-05-10T03:31:48.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.628+0000 7f9ddb400640 1 -- 172.21.15.55:0/1388348250 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f9ddc017020 con 0x7f9de4162130 2024-05-10T03:31:48.632 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:48.632 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.630+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 msgr2=0x7f9dc006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.630+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 0x7f9dc006d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4034000 tx=0x7f9dd4033040 comp rx=0 tx=0).stop 2024-05-10T03:31:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.630+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 msgr2=0x7f9de4148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.630+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc001eb0 tx=0x7f9ddc00c820 comp rx=0 tx=0).stop 2024-05-10T03:31:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 shutdown_connections 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de4162130 0x7f9de4148e60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de4144b60 0x7f9de4148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9dc006ac00 0x7f9dc006d110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 --2- 172.21.15.55:0/1388348250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de40b2d00 0x7f9de413b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 >> 172.21.15.55:0/1388348250 conn(0x7f9de406a110 msgr2=0x7f9de4158910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 shutdown_connections 2024-05-10T03:31:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.631+0000 7f9de3e00640 1 -- 172.21.15.55:0/1388348250 wait complete. 2024-05-10T03:31:48.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:31:24.993809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:31:26.385874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:31:48.662 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:48.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.946+0000 7f31da200640 1 Processor -- start 2024-05-10T03:31:48.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.946+0000 7f31da200640 1 -- start start 2024-05-10T03:31:48.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31da200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31d4141a30 0x7f31d40cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31da200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d415bce0 0x7f31d415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31da200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d4071ce0 con 0x7f31d4070050 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31da200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d4070f40 con 0x7f31d4141a30 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31da200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d415f280 con 0x7f31d415bce0 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31d2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31d4141a30 0x7f31d40cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31d2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31d4141a30 0x7f31d40cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60600/0 (socket says 172.21.15.55:60600) 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.948+0000 7f31d2a00640 1 -- 172.21.15.55:0/4196791991 learned_addr learned my addr 172.21.15.55:0/4196791991 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.949+0000 7f31d3e00640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d415bce0 0x7f31d415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.949+0000 7f31d2a00640 1 -- 172.21.15.55:0/4196791991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d415bce0 msgr2=0x7f31d415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.949+0000 7f31d2a00640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d415bce0 0x7f31d415c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.949+0000 7f31d2a00640 1 -- 172.21.15.55:0/4196791991 >> v1:172.21.15.162:6789/0 conn(0x7f31d4070050 legacy=0x7f31d415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:48.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.949+0000 7f31d2a00640 1 -- 172.21.15.55:0/4196791991 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d4070460 con 0x7f31d4141a30 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.950+0000 7f31d2a00640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d4141a30 0x7f31d40cd750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009870 tx=0x7f31c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355e51e5312cd6e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.950+0000 7f31d2000640 1 -- 172.21.15.55:0/4196791991 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0032440 con 0x7f31d4141a30 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.950+0000 7f31d2000640 1 -- 172.21.15.55:0/4196791991 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c00325a0 con 0x7f31d4141a30 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.950+0000 7f31d2000640 1 -- 172.21.15.55:0/4196791991 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0032930 con 0x7f31d4141a30 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 -- 172.21.15.55:0/4196791991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d4141a30 msgr2=0x7f31d40cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d4141a30 0x7f31d40cd750 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009870 tx=0x7f31c002a210 comp rx=0 tx=0).stop 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 -- 172.21.15.55:0/4196791991 shutdown_connections 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d415bce0 0x7f31d415c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 --2- 172.21.15.55:0/4196791991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d4141a30 0x7f31d40cd750 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 -- 172.21.15.55:0/4196791991 >> 172.21.15.55:0/4196791991 conn(0x7f31d406c120 msgr2=0x7f31d406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 -- 172.21.15.55:0/4196791991 shutdown_connections 2024-05-10T03:31:48.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.951+0000 7f31da200640 1 -- 172.21.15.55:0/4196791991 wait complete. 2024-05-10T03:31:48.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.954+0000 7f31da200640 1 Processor -- start 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.954+0000 7f31da200640 1 -- start start 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d4070050 0x7f31d4139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 0x7f31d4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4158930 con 0x7f31d415bce0 2024-05-10T03:31:48.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4134ec0 con 0x7f31d4141a30 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31da200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d4158dd0 con 0x7f31d4070050 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31d3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60608/0 (socket says 172.21.15.55:60608) 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31d3e00640 1 -- 172.21.15.55:0/1074273139 learned_addr learned my addr 172.21.15.55:0/1074273139 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31d2a00640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 0x7f31d4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.956+0000 7f31d3400640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d4070050 0x7f31d4139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 -- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d4070050 msgr2=0x7f31d4139820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d4070050 0x7f31d4139820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 -- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 msgr2=0x7f31d4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 0x7f31d4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 -- 172.21.15.55:0/1074273139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c00094d0 con 0x7f31d415bce0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d2a00640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 0x7f31d4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31d3e00640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007e90 tx=0x7f31cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc014420 con 0x7f31d415bce0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc014580 con 0x7f31d415bce0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc014930 con 0x7f31d415bce0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.957+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d413f510 con 0x7f31d415bce0 2024-05-10T03:31:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.958+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d413fad0 con 0x7f31d415bce0 2024-05-10T03:31:48.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.959+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d4138860 con 0x7f31d415bce0 2024-05-10T03:31:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.960+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f31cc014a90 con 0x7f31d415bce0 2024-05-10T03:31:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.961+0000 7f31cb400640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 0x7f31b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.961+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f31cc089e50 con 0x7f31d415bce0 2024-05-10T03:31:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.962+0000 7f31d3400640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 0x7f31b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.963+0000 7f31d3400640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 0x7f31b406d110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:48.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:48.966+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f31cc054e50 con 0x7f31d415bce0 2024-05-10T03:31:49.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.173+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31d4066f60 con 0x7f31d415bce0 2024-05-10T03:31:49.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.174+0000 7f31cb400640 1 -- 172.21.15.55:0/1074273139 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31cc058470 con 0x7f31d415bce0 2024-05-10T03:31:49.179 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:49.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 msgr2=0x7f31b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 0x7f31b406d110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4008040 comp rx=0 tx=0).stop 2024-05-10T03:31:49.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 msgr2=0x7f31d4157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007e90 tx=0x7f31cc00e4b0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 shutdown_connections 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31d415bce0 0x7f31d4157a90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31b406ac00 0x7f31b406d110 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31d4141a30 0x7f31d4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 --2- 172.21.15.55:0/1074273139 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31d4070050 0x7f31d4139820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 >> 172.21.15.55:0/1074273139 conn(0x7f31d406c120 msgr2=0x7f31d40b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 shutdown_connections 2024-05-10T03:31:49.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.177+0000 7f31da200640 1 -- 172.21.15.55:0/1074273139 wait complete. 2024-05-10T03:31:49.210 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.496+0000 7f00dd800640 1 Processor -- start 2024-05-10T03:31:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.496+0000 7f00dd800640 1 -- start start 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00dd800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f00d8141a30 0x7f00d80cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00dd800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d8071ce0 con 0x7f00d8070050 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00dd800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d8070460 con 0x7f00d815bce0 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00dd800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d8070f40 con 0x7f00d8141a30 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00d6a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f00d8070050 0x7f00d815f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59846/0 (socket says 172.21.15.55:59846) 2024-05-10T03:31:49.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00d6a00640 1 -- 172.21.15.55:0/2791575215 learned_addr learned my addr 172.21.15.55:0/2791575215 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:49.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.499+0000 7f00ce000640 1 --2- 172.21.15.55:0/2791575215 >> v2:172.21.15.162:3300/0 conn(0x7f00d8141a30 0x7f00d80cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296747435 0 0) 0x7f00c8001110 con 0x7f00d8070050 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003520 con 0x7f00d8070050 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799300514 0 0) 0x7f00bc003520 con 0x7f00d8070050 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 >> v2:172.21.15.162:3300/0 conn(0x7f00d8141a30 msgr2=0x7f00d80cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 --2- 172.21.15.55:0/2791575215 >> v2:172.21.15.162:3300/0 conn(0x7f00d8141a30 0x7f00d80cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 >> v1:172.21.15.162:6789/0 conn(0x7f00d815bce0 legacy=0x7f00d806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8141e40 con 0x7f00d8070050 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f00c8001800 con 0x7f00d8070050 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 >> v1:172.21.15.55:6789/0 conn(0x7f00d8070050 legacy=0x7f00d815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.500+0000 7f00d6000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00bc003970 0x7f00bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00bc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00bc005180 0x7f00bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8070f40 con 0x7f00bc0047c0 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc00c120 con 0x7f00bc003970 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc00c2e0 con 0x7f00bc005180 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00bc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 -- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00bc005180 msgr2=0x7f00bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00bc005180 0x7f00bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 -- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00bc003970 msgr2=0x7f00bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d6a00640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00bc003970 0x7f00bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00bc003970 0x7f00bc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00ce000640 1 -- 172.21.15.55:0/2791575215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc003520 con 0x7f00bc0047c0 2024-05-10T03:31:49.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.501+0000 7f00d7400640 1 -- 172.21.15.55:0/2791575215 reap_dead start 2024-05-10T03:31:49.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.502+0000 7f00ce000640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00bc004bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009ac0 tx=0x7f00bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fd24aae2f3db8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:49.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.503+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0031400 con 0x7f00bc0047c0 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.503+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c002d050 con 0x7f00bc0047c0 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.503+0000 7f00d6000640 1 -- 172.21.15.55:0/2791575215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0031780 con 0x7f00bc0047c0 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.503+0000 7f00dd800640 1 -- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 msgr2=0x7f00bc004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.503+0000 7f00dd800640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00bc004bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009ac0 tx=0x7f00bc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.504+0000 7f00dd800640 1 -- 172.21.15.55:0/2791575215 shutdown_connections 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.504+0000 7f00dd800640 1 --2- 172.21.15.55:0/2791575215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00bc004bd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.504+0000 7f00dd800640 1 -- 172.21.15.55:0/2791575215 >> 172.21.15.55:0/2791575215 conn(0x7f00d806c120 msgr2=0x7f00d806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.504+0000 7f00dd800640 1 -- 172.21.15.55:0/2791575215 shutdown_connections 2024-05-10T03:31:49.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.504+0000 7f00dd800640 1 -- 172.21.15.55:0/2791575215 wait complete. 2024-05-10T03:31:49.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.506+0000 7f00dd800640 1 Processor -- start 2024-05-10T03:31:49.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.506+0000 7f00dd800640 1 -- start start 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d80ab490 0x7f00d8140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d8159200 0x7f00d8140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d813d100 con 0x7f00bc0047c0 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80abc30 con 0x7f00d80ab490 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00dd800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d81447a0 con 0x7f00d8159200 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00d6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00d6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60614/0 (socket says 172.21.15.55:60614) 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.508+0000 7f00d6a00640 1 -- 172.21.15.55:0/1998503028 learned_addr learned my addr 172.21.15.55:0/1998503028 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00ce000640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d80ab490 0x7f00d8140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 -- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d8159200 msgr2=0x7f00d8140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d8159200 0x7f00d8140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 -- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d80ab490 msgr2=0x7f00d8140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d80ab490 0x7f00d8140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 -- 172.21.15.55:0/1998503028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8144030 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00d6a00640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d7d0 tx=0x7f00c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.509+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c80074e0 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.510+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c8007640 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.510+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8004a20 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.510+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d81449f0 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.510+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d8144fb0 con 0x7f00bc0047c0 2024-05-10T03:31:49.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.511+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d8066f60 con 0x7f00bc0047c0 2024-05-10T03:31:49.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.512+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f00c8004b80 con 0x7f00bc0047c0 2024-05-10T03:31:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.513+0000 7f00cf400640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 0x7f00b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.513+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f00c8088ec0 con 0x7f00bc0047c0 2024-05-10T03:31:49.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.514+0000 7f00ce000640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 0x7f00b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:49.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.516+0000 7f00ce000640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 0x7f00b406d110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00c0033000 tx=0x7f00c0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:49.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.518+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f00c8053f70 con 0x7f00bc0047c0 2024-05-10T03:31:49.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.718+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00d8067220 con 0x7f00bc0047c0 2024-05-10T03:31:49.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.720+0000 7f00cf400640 1 -- 172.21.15.55:0/1998503028 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00c8019020 con 0x7f00bc0047c0 2024-05-10T03:31:49.724 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.722+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 msgr2=0x7f00b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.722+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 0x7f00b406d110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00c0033000 tx=0x7f00c0008030 comp rx=0 tx=0).stop 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.722+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 msgr2=0x7f00d8143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.722+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d7d0 tx=0x7f00c800dc90 comp rx=0 tx=0).stop 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 shutdown_connections 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d8159200 0x7f00d8140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b406ac00 0x7f00b406d110 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d80ab490 0x7f00d8140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 --2- 172.21.15.55:0/1998503028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00bc0047c0 0x7f00d8143af0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:49.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 >> 172.21.15.55:0/1998503028 conn(0x7f00d806c120 msgr2=0x7f00d815e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:49.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 shutdown_connections 2024-05-10T03:31:49.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:49.723+0000 7f00dd800640 1 -- 172.21.15.55:0/1998503028 wait complete. 2024-05-10T03:31:49.755 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:50.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.032+0000 7fc9eac00640 1 Processor -- start 2024-05-10T03:31:50.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.032+0000 7fc9eac00640 1 -- start start 2024-05-10T03:31:50.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.034+0000 7fc9eac00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e40b2040 0x7fc9e4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.034+0000 7fc9eac00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e41435d0 0x7fc9e40ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.034+0000 7fc9eac00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40cdbe0 con 0x7fc9e406b120 2024-05-10T03:31:50.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.034+0000 7fc9eac00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e415e8b0 con 0x7fc9e40b2040 2024-05-10T03:31:50.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.034+0000 7fc9eac00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40ce140 con 0x7fc9e41435d0 2024-05-10T03:31:50.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e40b2040 0x7fc9e4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e40b2040 0x7fc9e4143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60618/0 (socket says 172.21.15.55:60618) 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 -- 172.21.15.55:0/3501469387 learned_addr learned my addr 172.21.15.55:0/3501469387 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e8e00640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e41435d0 0x7fc9e40ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 -- 172.21.15.55:0/3501469387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e41435d0 msgr2=0x7fc9e40ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e41435d0 0x7fc9e40ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 -- 172.21.15.55:0/3501469387 >> v1:172.21.15.162:6789/0 conn(0x7fc9e406b120 legacy=0x7fc9e406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:50.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.035+0000 7fc9e9800640 1 -- 172.21.15.55:0/3501469387 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e415ede0 con 0x7fc9e40b2040 2024-05-10T03:31:50.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.036+0000 7fc9e9800640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e40b2040 0x7fc9e4143020 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400d180 tx=0x7fc9d402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a521279732da6d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9e3e00640 1 -- 172.21.15.55:0/3501469387 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400b2d0 con 0x7fc9e40b2040 2024-05-10T03:31:50.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9e3e00640 1 -- 172.21.15.55:0/3501469387 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400b430 con 0x7fc9e40b2040 2024-05-10T03:31:50.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9e3e00640 1 -- 172.21.15.55:0/3501469387 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4009820 con 0x7fc9e40b2040 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 -- 172.21.15.55:0/3501469387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e40b2040 msgr2=0x7fc9e4143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e40b2040 0x7fc9e4143020 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400d180 tx=0x7fc9d402c870 comp rx=0 tx=0).stop 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 -- 172.21.15.55:0/3501469387 shutdown_connections 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e41435d0 0x7fc9e40ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 --2- 172.21.15.55:0/3501469387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e40b2040 0x7fc9e4143020 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.037+0000 7fc9eac00640 1 -- 172.21.15.55:0/3501469387 >> 172.21.15.55:0/3501469387 conn(0x7fc9e406c730 msgr2=0x7fc9e40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.038+0000 7fc9eac00640 1 -- 172.21.15.55:0/3501469387 shutdown_connections 2024-05-10T03:31:50.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.038+0000 7fc9eac00640 1 -- 172.21.15.55:0/3501469387 wait complete. 2024-05-10T03:31:50.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.040+0000 7fc9eac00640 1 Processor -- start 2024-05-10T03:31:50.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.040+0000 7fc9eac00640 1 -- start start 2024-05-10T03:31:50.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e406b120 0x7fc9e415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9e40b2040 0x7fc9e4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415d0a0 con 0x7fc9e41435d0 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415df00 con 0x7fc9e40b2040 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.041+0000 7fc9eac00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415a1b0 con 0x7fc9e406b120 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60630/0 (socket says 172.21.15.55:60630) 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 -- 172.21.15.55:0/114599447 learned_addr learned my addr 172.21.15.55:0/114599447 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9e9800640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e406b120 0x7fc9e415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9e8e00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9e40b2040 0x7fc9e4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 -- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e406b120 msgr2=0x7fc9e415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e406b120 0x7fc9e415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 -- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9e40b2040 msgr2=0x7fc9e4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9e40b2040 0x7fc9e4159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.042+0000 7fc9ea200640 1 -- 172.21.15.55:0/114599447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d400bd40 con 0x7fc9e41435d0 2024-05-10T03:31:50.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9ea200640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b3e0 tx=0x7fc9dc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc004020 con 0x7fc9e41435d0 2024-05-10T03:31:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc004180 con 0x7fc9e41435d0 2024-05-10T03:31:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e415d2f0 con 0x7fc9e41435d0 2024-05-10T03:31:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc011630 con 0x7fc9e41435d0 2024-05-10T03:31:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.043+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e415d8b0 con 0x7fc9e41435d0 2024-05-10T03:31:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.044+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40b1680 con 0x7fc9e41435d0 2024-05-10T03:31:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.045+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc9dc011790 con 0x7fc9e41435d0 2024-05-10T03:31:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.046+0000 7fc9e1600640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 0x7fc9c406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.046+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc9dc0891b0 con 0x7fc9e41435d0 2024-05-10T03:31:50.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.047+0000 7fc9e9800640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 0x7fc9c406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.048+0000 7fc9e9800640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 0x7fc9c406d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4036000 tx=0x7fc9d400b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.051+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc9dc0541c0 con 0x7fc9e41435d0 2024-05-10T03:31:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.245+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc9e415e070 con 0x7fc9e41435d0 2024-05-10T03:31:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.246+0000 7fc9e1600640 1 -- 172.21.15.55:0/114599447 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9dc0577e0 con 0x7fc9e41435d0 2024-05-10T03:31:50.250 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 msgr2=0x7fc9c406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 0x7fc9c406d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4036000 tx=0x7fc9d400b000 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 msgr2=0x7fc9e41e4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b3e0 tx=0x7fc9dc00b8a0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 shutdown_connections 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e41435d0 0x7fc9e41e4610 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc9c406ac00 0x7fc9c406d110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9e40b2040 0x7fc9e4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 --2- 172.21.15.55:0/114599447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9e406b120 0x7fc9e415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 >> 172.21.15.55:0/114599447 conn(0x7fc9e406c730 msgr2=0x7fc9e40b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 shutdown_connections 2024-05-10T03:31:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.249+0000 7fc9eac00640 1 -- 172.21.15.55:0/114599447 wait complete. 2024-05-10T03:31:50.280 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:31:50.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.555+0000 7f5652e00640 1 Processor -- start 2024-05-10T03:31:50.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.555+0000 7f5652e00640 1 -- start start 2024-05-10T03:31:50.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f5652e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f564c15edb0 0x7f564c0b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f5652e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f564c06ce00 0x7f564c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f5652e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c0b63e0 con 0x7f564c15b830 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f5652e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c15e080 con 0x7f564c15edb0 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f5652e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c15e920 con 0x7f564c06ce00 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f564be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f564c15edb0 0x7f564c0b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f564be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f564c15edb0 0x7f564c0b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60638/0 (socket says 172.21.15.55:60638) 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.558+0000 7f564be00640 1 -- 172.21.15.55:0/2397107227 learned_addr learned my addr 172.21.15.55:0/2397107227 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:50.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.559+0000 7f5650e00640 1 --2- 172.21.15.55:0/2397107227 >> v2:172.21.15.162:3300/0 conn(0x7f564c06ce00 0x7f564c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.559+0000 7f564be00640 1 -- 172.21.15.55:0/2397107227 >> v2:172.21.15.162:3300/0 conn(0x7f564c06ce00 msgr2=0x7f564c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.559+0000 7f564be00640 1 --2- 172.21.15.55:0/2397107227 >> v2:172.21.15.162:3300/0 conn(0x7f564c06ce00 0x7f564c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.559+0000 7f564be00640 1 -- 172.21.15.55:0/2397107227 >> v1:172.21.15.162:6789/0 conn(0x7f564c15b830 legacy=0x7f564c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.559+0000 7f564be00640 1 -- 172.21.15.55:0/2397107227 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c1623b0 con 0x7f564c15edb0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.560+0000 7f564be00640 1 --2- 172.21.15.55:0/2397107227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15edb0 0x7f564c0b5ea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f564c0b5c40 tx=0x7f563c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a6c8827b126a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.561+0000 7f564aa00640 1 -- 172.21.15.55:0/2397107227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c031600 con 0x7f564c15edb0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.561+0000 7f564aa00640 1 -- 172.21.15.55:0/2397107227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c02d050 con 0x7f564c15edb0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.561+0000 7f564aa00640 1 -- 172.21.15.55:0/2397107227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c031980 con 0x7f564c15edb0 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.561+0000 7f5652e00640 1 -- 172.21.15.55:0/2397107227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15edb0 msgr2=0x7f564c0b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.561+0000 7f5652e00640 1 --2- 172.21.15.55:0/2397107227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15edb0 0x7f564c0b5ea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f564c0b5c40 tx=0x7f563c02a6f0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 -- 172.21.15.55:0/2397107227 shutdown_connections 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 --2- 172.21.15.55:0/2397107227 >> v2:172.21.15.162:3300/0 conn(0x7f564c06ce00 0x7f564c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 --2- 172.21.15.55:0/2397107227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15edb0 0x7f564c0b5ea0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 -- 172.21.15.55:0/2397107227 >> 172.21.15.55:0/2397107227 conn(0x7f564c0d1160 msgr2=0x7f564c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:50.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 -- 172.21.15.55:0/2397107227 shutdown_connections 2024-05-10T03:31:50.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.562+0000 7f5652e00640 1 -- 172.21.15.55:0/2397107227 wait complete. 2024-05-10T03:31:50.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.564+0000 7f5652e00640 1 Processor -- start 2024-05-10T03:31:50.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.564+0000 7f5652e00640 1 -- start start 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f564c06ce00 0x7f564c142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 0x7f564c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c143150 con 0x7f564c15b830 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c143e00 con 0x7f564c15edb0 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5652e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c15ca30 con 0x7f564c06ce00 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f564b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f564b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60648/0 (socket says 172.21.15.55:60648) 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f564b400640 1 -- 172.21.15.55:0/1960506963 learned_addr learned my addr 172.21.15.55:0/1960506963 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.566+0000 7f5650e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 0x7f564c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 -- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f564c06ce00 msgr2=0x7f564c142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564be00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f564c06ce00 0x7f564c142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f564c06ce00 0x7f564c142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 -- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 msgr2=0x7f564c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 0x7f564c15cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 -- 172.21.15.55:0/1960506963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c009b30 con 0x7f564c15b830 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f5650e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 0x7f564c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:50.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f564b400640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f563400b3f0 tx=0x7f563400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011830 con 0x7f564c15b830 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634011990 con 0x7f564c15b830 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c143900 con 0x7f564c15b830 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.567+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011d40 con 0x7f564c15b830 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.568+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c15c100 con 0x7f564c15b830 2024-05-10T03:31:50.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.569+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c067930 con 0x7f564c15b830 2024-05-10T03:31:50.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.570+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f563401a400 con 0x7f564c15b830 2024-05-10T03:31:50.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.571+0000 7f5643e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 0x7f562806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:50.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.571+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5634089610 con 0x7f564c15b830 2024-05-10T03:31:50.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.572+0000 7f564be00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 0x7f562806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:50.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.573+0000 7f564be00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 0x7f562806d110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f563c033000 tx=0x7f563c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:50.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.576+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5634054610 con 0x7f564c15b830 2024-05-10T03:31:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.799+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f564c15cb70 con 0x7f564c15b830 2024-05-10T03:31:50.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.800+0000 7f5643e00640 1 -- 172.21.15.55:0/1960506963 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5634057c30 con 0x7f564c15b830 2024-05-10T03:31:50.804 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:31:50.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 msgr2=0x7f562806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 0x7f562806d110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f563c033000 tx=0x7f563c009000 comp rx=0 tx=0).stop 2024-05-10T03:31:50.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 msgr2=0x7f564c1433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:50.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f563400b3f0 tx=0x7f563400b8b0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 shutdown_connections 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.802+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f562806ac00 0x7f562806d110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f564c15edb0 0x7f564c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f564c15b830 0x7f564c1433c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 --2- 172.21.15.55:0/1960506963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f564c06ce00 0x7f564c142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 >> 172.21.15.55:0/1960506963 conn(0x7f564c0d1160 msgr2=0x7f564c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 shutdown_connections 2024-05-10T03:31:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:50.803+0000 7f5652e00640 1 -- 172.21.15.55:0/1960506963 wait complete. 2024-05-10T03:31:50.838 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:31:50.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:50.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:31:50.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:31:50.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:31:50.938 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:31:50.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump 2024-05-10T03:31:51.299 INFO:teuthology.orchestra.run.smithi055.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":45,"msgr_send_messages":38,"msgr_recv_bytes":12886,"msgr_send_bytes":23027,"msgr_created_connections":8,"msgr_active_connections":2,"msgr_running_total_time":0.018401489,"msgr_running_send_time":0.002211708,"msgr_running_recv_time":0.009596718,"msgr_running_fast_dispatch_time":0.001715403,"msgr_send_messages_queue_lat":{"avgcount":37,"sum":0.168659471,"avgtime":0.004558364},"msgr_handle_ack_lat":{"avgcount":4,"sum":0.000004830,"avgtime":0.000001207},"msgr_recv_encrypted_bytes":1568,"msgr_send_encrypted_bytes":448},"AsyncMessenger::Worker-1":{"msgr_recv_messages":12,"msgr_send_messages":21,"msgr_recv_bytes":2979,"msgr_send_bytes":101384,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":0.018930886,"msgr_running_send_time":0.001998693,"msgr_running_recv_time":0.015964923,"msgr_running_fast_dispatch_time":0.000562173,"msgr_send_messages_queue_lat":{"avgcount":20,"sum":0.241773192,"avgtime":0.012088659},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96528},"AsyncMessenger::Worker-2":{"msgr_recv_messages":107,"msgr_send_messages":91,"msgr_recv_bytes":126176,"msgr_send_bytes":49722,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.027053587,"msgr_running_send_time":0.009201731,"msgr_running_recv_time":0.023251935,"msgr_running_fast_dispatch_time":0.001749877,"msgr_send_messages_queue_lat":{"avgcount":91,"sum":0.140209099,"avgtime":0.001540759},"msgr_handle_ack_lat":{"avgcount":31,"sum":0.000025613,"avgtime":0.000000826},"msgr_recv_encrypted_bytes":111584,"msgr_send_encrypted_bytes":12240},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":56,"sum":0.003837941,"avgtime":0.000068534}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000019233,"avgtime":0.000009616}},"mds":{"request":29,"reply":29,"reply_latency":{"avgcount":29,"sum":0.016735402,"avgtime":0.000577082},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":57,"traverse_hit":57,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197380,"rdpos":0,"jlat":{"avgcount":8,"sum":0.039258685,"avgtime":0.004907335},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":108160,"heap":207224},"mds_server":{"dispatch_client_request":29,"dispatch_server_request":0,"handle_client_request":29,"handle_client_session":9,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":28,"sum":0.016102140,"avgtime":0.000575076},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000633262,"avgtime":0.000633262},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":26,"average_load":26,"avg_session_uptime":0,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":241447,"buffer_anon_items":87,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51880,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":47,"op_send_bytes":14286,"op_resend":31,"op_reply":47,"op_latency":{"avgcount":47,"sum":1.528969285,"avgtime":0.032531261},"op_inflight":0,"oplen_avg":{"avgcount":47,"sum":71},"op":47,"op_r":0,"op_w":47,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":8,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":40,"map_full":0,"map_inc":1,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":156,"get_sum":123711,"get_or_fail_fail":0,"get_or_fail_success":156,"take":0,"take_sum":0,"put":156,"put_sum":123711,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":14286,"put":47,"put_sum":14286,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":47,"put":47,"put_sum":47,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5600617f4410":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":3076,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":8,"put_sum":3076,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-10T03:31:51.300 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 4, "avgtime": 1.207e-06, "sum": 4.83e-06 }, "msgr_recv_bytes": 12886, "msgr_recv_encrypted_bytes": 1568, "msgr_recv_messages": 45, "msgr_running_fast_dispatch_time": 0.001715403, "msgr_running_recv_time": 0.009596718, "msgr_running_send_time": 0.002211708, "msgr_running_total_time": 0.018401489, "msgr_send_bytes": 23027, "msgr_send_encrypted_bytes": 448, "msgr_send_messages": 38, "msgr_send_messages_queue_lat": { "avgcount": 37, "avgtime": 0.004558364, "sum": 0.168659471 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2979, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 12, "msgr_running_fast_dispatch_time": 0.000562173, "msgr_running_recv_time": 0.015964923, "msgr_running_send_time": 0.001998693, "msgr_running_total_time": 0.018930886, "msgr_send_bytes": 101384, "msgr_send_encrypted_bytes": 96528, "msgr_send_messages": 21, "msgr_send_messages_queue_lat": { "avgcount": 20, "avgtime": 0.012088659, "sum": 0.241773192 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 31, "avgtime": 8.26e-07, "sum": 2.5613e-05 }, "msgr_recv_bytes": 126176, "msgr_recv_encrypted_bytes": 111584, "msgr_recv_messages": 107, "msgr_running_fast_dispatch_time": 0.001749877, "msgr_running_recv_time": 0.023251935, "msgr_running_send_time": 0.009201731, "msgr_running_total_time": 0.027053587, "msgr_send_bytes": 49722, "msgr_send_encrypted_bytes": 12240, "msgr_send_messages": 91, "msgr_send_messages_queue_lat": { "avgcount": 91, "avgtime": 0.001540759, "sum": 0.140209099 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 56, "avgtime": 6.8534e-05, "sum": 0.003837941 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.616e-06, "sum": 1.9233e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 29, "reply_latency": { "avgcount": 29, "avgtime": 0.000577082, "sum": 0.016735402 }, "request": 29, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 57, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 57, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 8, "avgtime": 0.004907335, "sum": 0.039258685 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4197380 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207224, "ino": 13, "ino+": 13, "ino-": 0, "rss": 108160 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 29, "dispatch_server_request": 0, "handle_client_request": 29, "handle_client_session": 9, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 28, "avgtime": 0.000575076, "sum": 0.01610214 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000633262, "sum": 0.000633262 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 26, "avg_session_uptime": 0, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 26 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 241447, "buffer_anon_items": 87, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51880, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 40, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 47, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 47, "avgtime": 0.032531261, "sum": 1.528969285 }, "op_pg": 0, "op_r": 0, "op_reply": 47, "op_resend": 31, "op_rmw": 0, "op_send": 47, "op_send_bytes": 14286, "op_w": 47, "oplen_avg": { "avgcount": 47, "sum": 71 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 8, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 156, "get_or_fail_fail": 0, "get_or_fail_success": 156, "get_started": 0, "get_sum": 123711, "max": 104857600, "put": 156, "put_sum": 123711, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 47, "put_sum": 14286, "take": 47, "take_sum": 14286, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 47, "put_sum": 47, "take": 47, "take_sum": 47, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5600617f4410": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 3076, "max": 3758096384, "put": 8, "put_sum": 3076, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-10T03:31:51.301 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T03:31:51.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:31:51.301 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:31:51.411 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:31:51.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:31:51.520 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:51.520 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:51.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:31:51.583 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:51.583 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:51.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:31:51.639 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:31:51.639 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:51.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:31:51.695 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:31:51.695 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:31:51.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:31:51.753 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.041+0000 7fa67a600640 1 Processor -- start 2024-05-10T03:31:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.042+0000 7fa67a600640 1 -- start start 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.044+0000 7fa67a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa674161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.044+0000 7fa67a600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6740720f0 0x7fa6741541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.044+0000 7fa67a600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6741623a0 con 0x7fa674161c70 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.044+0000 7fa67a600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6740cdd60 con 0x7fa6740720f0 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.044+0000 7fa67a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740ce040 con 0x7fa6740ad7b0 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6740720f0 0x7fa6741541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6740720f0 0x7fa6741541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60658/0 (socket says 172.21.15.55:60658) 2024-05-10T03:31:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 -- 172.21.15.55:0/2542733154 learned_addr learned my addr 172.21.15.55:0/2542733154 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673400640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa674161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 -- 172.21.15.55:0/2542733154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 msgr2=0x7fa674161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa674161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 -- 172.21.15.55:0/2542733154 >> v1:172.21.15.162:6789/0 conn(0x7fa674161c70 legacy=0x7fa6740719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.045+0000 7fa673e00640 1 -- 172.21.15.55:0/2542733154 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740ce320 con 0x7fa6740720f0 2024-05-10T03:31:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.046+0000 7fa673e00640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6740720f0 0x7fa6741541e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009e50 tx=0x7fa66c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4e7f6f99eafb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.047+0000 7fa672a00640 1 -- 172.21.15.55:0/2542733154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c031600 con 0x7fa6740720f0 2024-05-10T03:31:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.047+0000 7fa672a00640 1 -- 172.21.15.55:0/2542733154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c02d050 con 0x7fa6740720f0 2024-05-10T03:31:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.047+0000 7fa672a00640 1 -- 172.21.15.55:0/2542733154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c031980 con 0x7fa6740720f0 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 -- 172.21.15.55:0/2542733154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6740720f0 msgr2=0x7fa6741541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6740720f0 0x7fa6741541e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009e50 tx=0x7fa66c02a640 comp rx=0 tx=0).stop 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 -- 172.21.15.55:0/2542733154 shutdown_connections 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6740720f0 0x7fa6741541e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 --2- 172.21.15.55:0/2542733154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa674161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 -- 172.21.15.55:0/2542733154 >> 172.21.15.55:0/2542733154 conn(0x7fa6740cbb90 msgr2=0x7fa6740cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 -- 172.21.15.55:0/2542733154 shutdown_connections 2024-05-10T03:31:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.048+0000 7fa67a600640 1 -- 172.21.15.55:0/2542733154 wait complete. 2024-05-10T03:31:52.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.050+0000 7fa67a600640 1 Processor -- start 2024-05-10T03:31:52.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.051+0000 7fa67a600640 1 -- start start 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6740720f0 0x7fa6741f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa67415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6741f7150 con 0x7fa674161c70 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674157210 con 0x7fa6740720f0 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.052+0000 7fa67a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa674147510 con 0x7fa6740ad7b0 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60666/0 (socket says 172.21.15.55:60666) 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 -- 172.21.15.55:0/2135625011 learned_addr learned my addr 172.21.15.55:0/2135625011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673400640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6740720f0 0x7fa6741f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa66be00640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa67415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 -- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 msgr2=0x7fa67415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa67415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 -- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6740720f0 msgr2=0x7fa6741f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6740720f0 0x7fa6741f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.053+0000 7fa673e00640 1 -- 172.21.15.55:0/2135625011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c009b30 con 0x7fa674161c70 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa673e00640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f70 tx=0x7fa66c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c03b820 con 0x7fa674161c70 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c03b980 con 0x7fa674161c70 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c03bcf0 con 0x7fa674161c70 2024-05-10T03:31:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6741491c0 con 0x7fa674161c70 2024-05-10T03:31:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.054+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6741496f0 con 0x7fa674161c70 2024-05-10T03:31:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.055+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6741f72c0 con 0x7fa674161c70 2024-05-10T03:31:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.056+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa66c038050 con 0x7fa674161c70 2024-05-10T03:31:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.058+0000 7fa66b400640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 0x7fa65006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.058+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa66c0aa0b0 con 0x7fa674161c70 2024-05-10T03:31:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.058+0000 7fa673400640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 0x7fa65006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.060+0000 7fa673400640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 0x7fa65006d140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa664006fd0 tx=0x7fa664008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.062+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa6741f72c0 con 0x7fa674161c70 2024-05-10T03:31:52.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.305+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa674066f60 con 0x7fa674161c70 2024-05-10T03:31:52.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.306+0000 7fa66b400640 1 -- 172.21.15.55:0/2135625011 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fa66c04b090 con 0x7fa674161c70 2024-05-10T03:31:52.310 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:52.310 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.308+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 msgr2=0x7fa65006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 0x7fa65006d140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa664006fd0 tx=0x7fa664008040 comp rx=0 tx=0).stop 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 msgr2=0x7fa674146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009f70 tx=0x7fa66c004a40 comp rx=0 tx=0).stop 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 shutdown_connections 2024-05-10T03:31:52.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa674161c70 0x7fa674146100 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6740ad7b0 0x7fa67415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa65006ac30 0x7fa65006d140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 --2- 172.21.15.55:0/2135625011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6740720f0 0x7fa6741f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 >> 172.21.15.55:0/2135625011 conn(0x7fa6740cbb90 msgr2=0x7fa67406a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 shutdown_connections 2024-05-10T03:31:52.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.309+0000 7fa67a600640 1 -- 172.21.15.55:0/2135625011 wait complete. 2024-05-10T03:31:52.340 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:52.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.626+0000 7fb90de00640 1 Processor -- start 2024-05-10T03:31:52.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.626+0000 7fb90de00640 1 -- start start 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.628+0000 7fb90de00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb90806d300 0x7fb9080b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.628+0000 7fb90de00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90815d740 con 0x7fb9080b1f60 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.628+0000 7fb90de00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9080b2a20 con 0x7fb90815e190 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.628+0000 7fb90de00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9080b2d00 con 0x7fb90806d300 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.629+0000 7fb907e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb9080b1f60 0x7fb90806cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59858/0 (socket says 172.21.15.55:59858) 2024-05-10T03:31:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.629+0000 7fb907e00640 1 -- 172.21.15.55:0/3531954949 learned_addr learned my addr 172.21.15.55:0/3531954949 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.629+0000 7fb90d400640 1 --2- 172.21.15.55:0/3531954949 >> v2:172.21.15.162:3300/0 conn(0x7fb90806d300 0x7fb9080b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.629+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690303746 0 0) 0x7fb90815d740 con 0x7fb9080b1f60 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.629+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec003520 con 0x7fb9080b1f60 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901793510 0 0) 0x7fb8ec003520 con 0x7fb9080b1f60 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 >> v2:172.21.15.162:3300/0 conn(0x7fb90806d300 msgr2=0x7fb9080b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 --2- 172.21.15.55:0/3531954949 >> v2:172.21.15.162:3300/0 conn(0x7fb90806d300 0x7fb9080b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 >> v1:172.21.15.162:6789/0 conn(0x7fb90815e190 legacy=0x7fb9080ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90815d990 con 0x7fb9080b1f60 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb8f40019f0 con 0x7fb9080b1f60 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 >> v1:172.21.15.55:6789/0 conn(0x7fb9080b1f60 legacy=0x7fb90806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec0039e0 0x7fb8ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8ec0047c0 0x7fb8ec004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 0x7fb8ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080b2d00 con 0x7fb8ec005310 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec00c210 con 0x7fb8ec0039e0 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec00c3d0 con 0x7fb8ec0047c0 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.630+0000 7fb90d400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 0x7fb8ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb907e00640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8ec0047c0 0x7fb8ec004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90ca00640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec0039e0 0x7fb8ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90d400640 1 -- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8ec0047c0 msgr2=0x7fb8ec004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:52.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90d400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8ec0047c0 0x7fb8ec004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90d400640 1 -- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec0039e0 msgr2=0x7fb8ec003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:52.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90d400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec0039e0 0x7fb8ec003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90d400640 1 -- 172.21.15.55:0/3531954949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec003520 con 0x7fb8ec005310 2024-05-10T03:31:52.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.631+0000 7fb90ca00640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec0039e0 0x7fb8ec003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.632+0000 7fb90d400640 1 -- 172.21.15.55:0/3531954949 reap_dead start 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.632+0000 7fb90d400640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 0x7fb8ec00b980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb90000aa90 tx=0x7fb8ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4df9609f2881d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.632+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900004220 con 0x7fb8ec005310 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.632+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb900004380 con 0x7fb8ec005310 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.632+0000 7fb907400640 1 -- 172.21.15.55:0/3531954949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900003850 con 0x7fb8ec005310 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 -- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 msgr2=0x7fb8ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 0x7fb8ec00b980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb90000aa90 tx=0x7fb8ec0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 -- 172.21.15.55:0/3531954949 shutdown_connections 2024-05-10T03:31:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 --2- 172.21.15.55:0/3531954949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8ec005310 0x7fb8ec00b980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 -- 172.21.15.55:0/3531954949 >> 172.21.15.55:0/3531954949 conn(0x7fb90806a110 msgr2=0x7fb90806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 -- 172.21.15.55:0/3531954949 shutdown_connections 2024-05-10T03:31:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.633+0000 7fb90de00640 1 -- 172.21.15.55:0/3531954949 wait complete. 2024-05-10T03:31:52.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.636+0000 7fb90de00640 1 Processor -- start 2024-05-10T03:31:52.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.636+0000 7fb90de00640 1 -- start start 2024-05-10T03:31:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.637+0000 7fb90de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 0x7fb9081521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9081f3160 0x7fb9081f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90de00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908150850 con 0x7fb90814f790 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90de00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9081509f0 con 0x7fb8ec005310 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9081f3c40 con 0x7fb9081f3160 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60694/0 (socket says 172.21.15.55:60694) 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 -- 172.21.15.55:0/4092735773 learned_addr learned my addr 172.21.15.55:0/4092735773 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90ca00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 0x7fb9081521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb907e00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9081f3160 0x7fb9081f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 -- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9081f3160 msgr2=0x7fb9081f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9081f3160 0x7fb9081f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 -- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 msgr2=0x7fb9081521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 0x7fb9081521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.638+0000 7fb90d400640 1 -- 172.21.15.55:0/4092735773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9081f4170 con 0x7fb90814f790 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb90ca00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 0x7fb9081521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:52.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb90d400640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb90000ab80 tx=0x7fb900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900039070 con 0x7fb90814f790 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9000044f0 con 0x7fb90814f790 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9080cc950 con 0x7fb90814f790 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.639+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90002c760 con 0x7fb90814f790 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.640+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9081f3e90 con 0x7fb90814f790 2024-05-10T03:31:52.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.641+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9080ccae0 con 0x7fb90814f790 2024-05-10T03:31:52.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.642+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb90002c8c0 con 0x7fb90814f790 2024-05-10T03:31:52.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.643+0000 7fb904c00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 0x7fb8e806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:52.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.643+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9000a8ec0 con 0x7fb90814f790 2024-05-10T03:31:52.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.644+0000 7fb90ca00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 0x7fb8e806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:52.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.645+0000 7fb90ca00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 0x7fb8e806d160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001690 tx=0x7fb8f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:52.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.648+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb900073eb0 con 0x7fb90814f790 2024-05-10T03:31:52.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.854+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb908066ce0 con 0x7fb90814f790 2024-05-10T03:31:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.856+0000 7fb904c00640 1 -- 172.21.15.55:0/4092735773 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7fb900043070 con 0x7fb90814f790 2024-05-10T03:31:52.861 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:52.861 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 msgr2=0x7fb8e806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 0x7fb8e806d160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8001690 tx=0x7fb8f8009040 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 msgr2=0x7fb90814fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb90000ab80 tx=0x7fb900004730 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 shutdown_connections 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb90814f790 0x7fb90814fba0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9081f3160 0x7fb9081f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8e806ac50 0x7fb8e806d160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 --2- 172.21.15.55:0/4092735773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec005310 0x7fb9081521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 >> 172.21.15.55:0/4092735773 conn(0x7fb90806a110 msgr2=0x7fb908153e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.859+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 shutdown_connections 2024-05-10T03:31:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:52.860+0000 7fb90de00640 1 -- 172.21.15.55:0/4092735773 wait complete. 2024-05-10T03:31:52.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:31:24.993809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:31:26.385874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:31:52.892 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:31:53.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.176+0000 7f7d9e000640 1 Processor -- start 2024-05-10T03:31:53.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.176+0000 7f7d9e000640 1 -- start start 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d980b2110 0x7f7d9815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b1bc0 0x7f7d980cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d980ce090 0x7f7d98161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9815dcd0 con 0x7f7d980ce090 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980ceb00 con 0x7f7d980b2110 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980d2150 con 0x7f7d980b1bc0 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9d600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d980ce090 0x7f7d98161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9d600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d980ce090 0x7f7d98161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42512/0 (socket says 172.21.15.55:42512) 2024-05-10T03:31:53.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9d600640 1 -- 172.21.15.55:0/1593728566 learned_addr learned my addr 172.21.15.55:0/1593728566 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:53.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.179+0000 7f7d9cc00640 1 --2- 172.21.15.55:0/1593728566 >> v2:172.21.15.162:3300/0 conn(0x7f7d980b2110 0x7f7d9815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d97e00640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b1bc0 0x7f7d980cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d9d600640 1 -- 172.21.15.55:0/1593728566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b1bc0 msgr2=0x7f7d980cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d9d600640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b1bc0 0x7f7d980cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d9d600640 1 -- 172.21.15.55:0/1593728566 >> v2:172.21.15.162:3300/0 conn(0x7f7d980b2110 msgr2=0x7f7d9815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d9d600640 1 --2- 172.21.15.55:0/1593728566 >> v2:172.21.15.162:3300/0 conn(0x7f7d980b2110 0x7f7d9815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.180+0000 7f7d9d600640 1 -- 172.21.15.55:0/1593728566 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d980b2910 con 0x7f7d980ce090 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.181+0000 7f7d9d600640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980ce090 0x7f7d98161310 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806c910 tx=0x7f7d9002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc2a0465e4a3c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.181+0000 7f7d97400640 1 -- 172.21.15.55:0/1593728566 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90031600 con 0x7f7d980ce090 2024-05-10T03:31:53.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.181+0000 7f7d97400640 1 -- 172.21.15.55:0/1593728566 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9002d050 con 0x7f7d980ce090 2024-05-10T03:31:53.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.181+0000 7f7d97400640 1 -- 172.21.15.55:0/1593728566 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90031980 con 0x7f7d980ce090 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.182+0000 7f7d9e000640 1 -- 172.21.15.55:0/1593728566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980ce090 msgr2=0x7f7d98161310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.182+0000 7f7d9e000640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980ce090 0x7f7d98161310 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806c910 tx=0x7f7d9002a610 comp rx=0 tx=0).stop 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 -- 172.21.15.55:0/1593728566 shutdown_connections 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980ce090 0x7f7d98161310 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 --2- 172.21.15.55:0/1593728566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b1bc0 0x7f7d980cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 --2- 172.21.15.55:0/1593728566 >> v2:172.21.15.162:3300/0 conn(0x7f7d980b2110 0x7f7d9815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 -- 172.21.15.55:0/1593728566 >> 172.21.15.55:0/1593728566 conn(0x7f7d980d30d0 msgr2=0x7f7d980d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 -- 172.21.15.55:0/1593728566 shutdown_connections 2024-05-10T03:31:53.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.183+0000 7f7d9e000640 1 -- 172.21.15.55:0/1593728566 wait complete. 2024-05-10T03:31:53.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.185+0000 7f7d9e000640 1 Processor -- start 2024-05-10T03:31:53.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.185+0000 7f7d9e000640 1 -- start start 2024-05-10T03:31:53.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 0x7f7d980b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d980ce090 0x7f7d981f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d981f6bf0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980cb800 con 0x7f7d980ce090 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980cc6a0 con 0x7f7d980b2110 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42518/0 (socket says 172.21.15.55:42518) 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.187+0000 7f7d9cc00640 1 -- 172.21.15.55:0/2346442939 learned_addr learned my addr 172.21.15.55:0/2346442939 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9d600640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d980ce090 0x7f7d981f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d97e00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 0x7f7d980b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 -- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 msgr2=0x7f7d980b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 0x7f7d980b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 -- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d980ce090 msgr2=0x7f7d981f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d980ce090 0x7f7d981f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 -- 172.21.15.55:0/2346442939 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009b30 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d97e00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 0x7f7d980b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.188+0000 7f7d9cc00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d5f0 tx=0x7f7d8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.189+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0125f0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.189+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c012750 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.189+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d981525f0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.189+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c012b00 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.189+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d980ccbd0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.190+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9804e290 con 0x7f7d980b1bc0 2024-05-10T03:31:53.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.191+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d8c012c60 con 0x7f7d980b1bc0 2024-05-10T03:31:53.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.192+0000 7f7d94c00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 0x7f7d7806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.192+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d8c089990 con 0x7f7d980b1bc0 2024-05-10T03:31:53.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.193+0000 7f7d97e00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 0x7f7d7806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.194+0000 7f7d97e00640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 0x7f7d7806d110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d981622c0 tx=0x7f7d84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.197+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d8c0549a0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.402+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d980cd490 con 0x7f7d980b1bc0 2024-05-10T03:31:53.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.403+0000 7f7d94c00640 1 -- 172.21.15.55:0/2346442939 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d8c057fc0 con 0x7f7d980b1bc0 2024-05-10T03:31:53.407 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.405+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 msgr2=0x7f7d7806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.405+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 0x7f7d7806d110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d981622c0 tx=0x7f7d84008040 comp rx=0 tx=0).stop 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.405+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 msgr2=0x7f7d981520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.405+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d5f0 tx=0x7f7d8c00dab0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 shutdown_connections 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d7806ac00 0x7f7d7806d110 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d980ce090 0x7f7d981f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d980b2110 0x7f7d980b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 --2- 172.21.15.55:0/2346442939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d980b1bc0 0x7f7d981520b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 >> 172.21.15.55:0/2346442939 conn(0x7f7d980d30d0 msgr2=0x7f7d9806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 shutdown_connections 2024-05-10T03:31:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.406+0000 7f7d9e000640 1 -- 172.21.15.55:0/2346442939 wait complete. 2024-05-10T03:31:53.435 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:31:53.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.712+0000 7fbe37a00640 1 Processor -- start 2024-05-10T03:31:53.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.712+0000 7fbe37a00640 1 -- start start 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.714+0000 7fbe37a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbe300cc3a0 0x7fbe300cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe37a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3015b5e0 con 0x7fbe30140900 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe37a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300b4380 con 0x7fbe3006b830 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe37a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3015acf0 con 0x7fbe300cc3a0 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe2fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbe30140900 0x7fbe3006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54144/0 (socket says 172.21.15.55:54144) 2024-05-10T03:31:53.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe2fe00640 1 -- 172.21.15.55:0/2206049864 learned_addr learned my addr 172.21.15.55:0/2206049864 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe35000640 1 --2- 172.21.15.55:0/2206049864 >> v2:172.21.15.162:3300/0 conn(0x7fbe300cc3a0 0x7fbe300cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859254805 0 0) 0x7fbe3015b5e0 con 0x7fbe30140900 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.715+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe14003520 con 0x7fbe30140900 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355095554 0 0) 0x7fbe14003520 con 0x7fbe30140900 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 >> v2:172.21.15.162:3300/0 conn(0x7fbe300cc3a0 msgr2=0x7fbe300cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 --2- 172.21.15.55:0/2206049864 >> v2:172.21.15.162:3300/0 conn(0x7fbe300cc3a0 0x7fbe300cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 >> v1:172.21.15.162:6789/0 conn(0x7fbe3006b830 legacy=0x7fbe3015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3015af90 con 0x7fbe30140900 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbe1c001670 con 0x7fbe30140900 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 >> v1:172.21.15.55:6789/0 conn(0x7fbe30140900 legacy=0x7fbe3006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe14003970 0x7fbe14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 0x7fbe14004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe14005180 0x7fbe1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3015acf0 con 0x7fbe140047c0 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400c120 con 0x7fbe14003970 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400c2e0 con 0x7fbe14005180 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.716+0000 7fbe2fe00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 0x7fbe14004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe35000640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe14003970 0x7fbe14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe35a00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe14005180 0x7fbe1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe2fe00640 1 -- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe14005180 msgr2=0x7fbe1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe2fe00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe14005180 0x7fbe1400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe2fe00640 1 -- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe14003970 msgr2=0x7fbe14003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe2fe00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe14003970 0x7fbe14003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe2fe00640 1 -- 172.21.15.55:0/2206049864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14003520 con 0x7fbe140047c0 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe35000640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe14003970 0x7fbe14003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.717+0000 7fbe35a00640 1 -- 172.21.15.55:0/2206049864 reap_dead start 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.718+0000 7fbe2fe00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 0x7fbe14004bd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00c240 tx=0x7fbe140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e932de9e70a336a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.718+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c02e070 con 0x7fbe140047c0 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.718+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c0058d0 con 0x7fbe140047c0 2024-05-10T03:31:53.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.718+0000 7fbe2f400640 1 -- 172.21.15.55:0/2206049864 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c005d00 con 0x7fbe140047c0 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 -- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 msgr2=0x7fbe14004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 0x7fbe14004bd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00c240 tx=0x7fbe140057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 -- 172.21.15.55:0/2206049864 shutdown_connections 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 --2- 172.21.15.55:0/2206049864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe140047c0 0x7fbe14004bd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 -- 172.21.15.55:0/2206049864 >> 172.21.15.55:0/2206049864 conn(0x7fbe3006c120 msgr2=0x7fbe3006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 -- 172.21.15.55:0/2206049864 shutdown_connections 2024-05-10T03:31:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.719+0000 7fbe37a00640 1 -- 172.21.15.55:0/2206049864 wait complete. 2024-05-10T03:31:53.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.721+0000 7fbe37a00640 1 Processor -- start 2024-05-10T03:31:53.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.722+0000 7fbe37a00640 1 -- start start 2024-05-10T03:31:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe140047c0 0x7fbe30138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe300c8450 0x7fbe300a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301f47f0 con 0x7fbe300c8860 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300c8d70 con 0x7fbe300c8450 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.723+0000 7fbe37a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30138cc0 con 0x7fbe140047c0 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42536/0 (socket says 172.21.15.55:42536) 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 -- 172.21.15.55:0/3768444224 learned_addr learned my addr 172.21.15.55:0/3768444224 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35000640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe140047c0 0x7fbe30138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe2fe00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe300c8450 0x7fbe300a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 -- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe140047c0 msgr2=0x7fbe30138780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe140047c0 0x7fbe30138780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 -- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe300c8450 msgr2=0x7fbe300a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe300c8450 0x7fbe300a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 -- 172.21.15.55:0/3768444224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300c8fc0 con 0x7fbe300c8860 2024-05-10T03:31:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.724+0000 7fbe35a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe3006d050 tx=0x7fbe2800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.725+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003d80 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.725+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28005070 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.725+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280053e0 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.725+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe300c92a0 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.725+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe300c9860 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.726+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30066f60 con 0x7fbe300c8860 2024-05-10T03:31:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.727+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe28005540 con 0x7fbe300c8860 2024-05-10T03:31:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.729+0000 7fbe2cc00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 0x7fbe1006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.729+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbe28088c80 con 0x7fbe300c8860 2024-05-10T03:31:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.729+0000 7fbe35000640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 0x7fbe1006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:53.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.731+0000 7fbe35000640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 0x7fbe1006d0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009b20 tx=0x7fbe20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.734+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe28053d70 con 0x7fbe300c8860 2024-05-10T03:31:53.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.933+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe30067220 con 0x7fbe300c8860 2024-05-10T03:31:53.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.935+0000 7fbe2cc00640 1 -- 172.21.15.55:0/3768444224 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe28018070 con 0x7fbe300c8860 2024-05-10T03:31:53.939 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:31:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 msgr2=0x7fbe1006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 0x7fbe1006d0f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009b20 tx=0x7fbe20008040 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 msgr2=0x7fbe300aa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe3006d050 tx=0x7fbe2800d9d0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 shutdown_connections 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe300c8860 0x7fbe300aa110 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbe1006abe0 0x7fbe1006d0f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe300c8450 0x7fbe300a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 --2- 172.21.15.55:0/3768444224 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe140047c0 0x7fbe30138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 >> 172.21.15.55:0/3768444224 conn(0x7fbe3006c120 msgr2=0x7fbe30159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 shutdown_connections 2024-05-10T03:31:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:53.938+0000 7fbe37a00640 1 -- 172.21.15.55:0/3768444224 wait complete. 2024-05-10T03:31:53.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:31:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.253+0000 7eff5d400640 1 Processor -- start 2024-05-10T03:31:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.253+0000 7eff5d400640 1 -- start start 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5d400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff580b2040 0x7eff58143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5d400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff581435d0 0x7eff580ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5d400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff580cdbe0 con 0x7eff5806b120 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5d400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5815e8b0 con 0x7eff581435d0 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5d400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff580ce140 con 0x7eff580b2040 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5ca00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7eff5806b120 0x7eff5806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54148/0 (socket says 172.21.15.55:54148) 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff57400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff581435d0 0x7eff580ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.255+0000 7eff5ca00640 1 -- 172.21.15.55:0/863990604 learned_addr learned my addr 172.21.15.55:0/863990604 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.256+0000 7eff57e00640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff580b2040 0x7eff58143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.256+0000 7eff57400640 1 -- 172.21.15.55:0/863990604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff580b2040 msgr2=0x7eff58143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.256+0000 7eff57400640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff580b2040 0x7eff58143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.256+0000 7eff57400640 1 -- 172.21.15.55:0/863990604 >> v1:172.21.15.55:6789/0 conn(0x7eff5806b120 legacy=0x7eff5806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.256+0000 7eff57400640 1 -- 172.21.15.55:0/863990604 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5815ede0 con 0x7eff581435d0 2024-05-10T03:31:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.257+0000 7eff57400640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff581435d0 0x7eff580ce5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7eff40009790 tx=0x7eff4002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1af575ef6685672 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.257+0000 7eff56a00640 1 -- 172.21.15.55:0/863990604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4002ae40 con 0x7eff581435d0 2024-05-10T03:31:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.257+0000 7eff56a00640 1 -- 172.21.15.55:0/863990604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40032440 con 0x7eff581435d0 2024-05-10T03:31:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.257+0000 7eff56a00640 1 -- 172.21.15.55:0/863990604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400327d0 con 0x7eff581435d0 2024-05-10T03:31:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 -- 172.21.15.55:0/863990604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff581435d0 msgr2=0x7eff580ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff581435d0 0x7eff580ce5d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7eff40009790 tx=0x7eff4002a0f0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 -- 172.21.15.55:0/863990604 shutdown_connections 2024-05-10T03:31:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff581435d0 0x7eff580ce5d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 --2- 172.21.15.55:0/863990604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff580b2040 0x7eff58143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 -- 172.21.15.55:0/863990604 >> 172.21.15.55:0/863990604 conn(0x7eff5806c730 msgr2=0x7eff580b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 -- 172.21.15.55:0/863990604 shutdown_connections 2024-05-10T03:31:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.258+0000 7eff5d400640 1 -- 172.21.15.55:0/863990604 wait complete. 2024-05-10T03:31:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.261+0000 7eff5d400640 1 Processor -- start 2024-05-10T03:31:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.261+0000 7eff5d400640 1 -- start start 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 0x7eff5815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 0x7eff581595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff581e8260 con 0x7eff580b2040 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff581e4800 con 0x7eff581435d0 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.262+0000 7eff5d400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff581e4a50 con 0x7eff5806b120 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42544/0 (socket says 172.21.15.55:42544) 2024-05-10T03:31:54.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 -- 172.21.15.55:0/4098372608 learned_addr learned my addr 172.21.15.55:0/4098372608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff5ca00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 0x7eff581595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 -- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 msgr2=0x7eff5815cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57e00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 0x7eff5815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 0x7eff5815cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 -- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 msgr2=0x7eff581595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 0x7eff581595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57400640 1 -- 172.21.15.55:0/4098372608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400093a0 con 0x7eff580b2040 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff57e00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 0x7eff5815cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.263+0000 7eff5ca00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 0x7eff581595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff57400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7eff400098b0 tx=0x7eff400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4003e920 con 0x7eff580b2040 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4003eaa0 con 0x7eff580b2040 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4003ee50 con 0x7eff580b2040 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff581e4d30 con 0x7eff580b2040 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.264+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff581e52f0 con 0x7eff580b2040 2024-05-10T03:31:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.266+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff580b1680 con 0x7eff580b2040 2024-05-10T03:31:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.267+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7eff40030440 con 0x7eff580b2040 2024-05-10T03:31:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.267+0000 7eff4fe00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 0x7eff3406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.268+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7eff400aa550 con 0x7eff580b2040 2024-05-10T03:31:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.268+0000 7eff57e00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 0x7eff3406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.270+0000 7eff57e00640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 0x7eff3406d160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.272+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7eff40075540 con 0x7eff580b2040 2024-05-10T03:31:54.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.471+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff580ceba0 con 0x7eff580b2040 2024-05-10T03:31:54.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.473+0000 7eff4fe00640 1 -- 172.21.15.55:0/4098372608 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff40078b60 con 0x7eff580b2040 2024-05-10T03:31:54.477 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:31:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.475+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 msgr2=0x7eff3406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.475+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 0x7eff3406d160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7eff48006fd0 tx=0x7eff48008040 comp rx=0 tx=0).stop 2024-05-10T03:31:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 msgr2=0x7eff581e3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7eff400098b0 tx=0x7eff400046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 shutdown_connections 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff3406ac50 0x7eff3406d160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff581435d0 0x7eff581595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff580b2040 0x7eff581e3ce0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 --2- 172.21.15.55:0/4098372608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff5806b120 0x7eff5815cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 >> 172.21.15.55:0/4098372608 conn(0x7eff5806c730 msgr2=0x7eff580ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 shutdown_connections 2024-05-10T03:31:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.476+0000 7eff5d400640 1 -- 172.21.15.55:0/4098372608 wait complete. 2024-05-10T03:31:54.506 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:54.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.788+0000 7f2b51c00640 1 Processor -- start 2024-05-10T03:31:54.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.788+0000 7f2b51c00640 1 -- start start 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.790+0000 7f2b51c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b4c158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.790+0000 7f2b51c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4c0708b0 con 0x7f2b4c071db0 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.790+0000 7f2b51c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4c15b940 con 0x7f2b4c06a4b0 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.790+0000 7f2b51c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0b2490 con 0x7f2b4c0b3f70 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.791+0000 7f2b4aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2b4c071db0 0x7f2b4c0ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54160/0 (socket says 172.21.15.55:54160) 2024-05-10T03:31:54.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.791+0000 7f2b4aa00640 1 -- 172.21.15.55:0/352490739 learned_addr learned my addr 172.21.15.55:0/352490739 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.791+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122222052 0 0) 0x7f2b4c0708b0 con 0x7f2b4c071db0 2024-05-10T03:31:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.791+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2c003520 con 0x7f2b4c071db0 2024-05-10T03:31:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.791+0000 7f2b4b400640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b4c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146930968 0 0) 0x7f2b2c003520 con 0x7f2b4c071db0 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 msgr2=0x7f2b4c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b4c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 >> v1:172.21.15.162:6789/0 conn(0x7f2b4c06a4b0 legacy=0x7f2b4c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c070260 con 0x7f2b4c071db0 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2b3c001670 con 0x7f2b4c071db0 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 >> v1:172.21.15.55:6789/0 conn(0x7f2b4c071db0 legacy=0x7f2b4c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b2c0039a0 0x7f2b2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c004d00 0x7f2b2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00de30 con 0x7f2b2c004d00 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00dfa0 con 0x7f2b2c0039a0 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c00e160 con 0x7f2b4c0b3f70 2024-05-10T03:31:54.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.792+0000 7f2b4b400640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c004d00 0x7f2b2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4aa00640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b2c0039a0 0x7f2b2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4a000640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4b400640 1 -- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 msgr2=0x7f2b2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4b400640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c0b3f70 0x7f2b2c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4b400640 1 -- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b2c0039a0 msgr2=0x7f2b2c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4b400640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b2c0039a0 0x7f2b2c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4b400640 1 -- 172.21.15.55:0/352490739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c003520 con 0x7f2b2c004d00 2024-05-10T03:31:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.793+0000 7f2b4aa00640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b2c0039a0 0x7f2b2c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.794+0000 7f2b4b400640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c004d00 0x7f2b2c0051d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400ba10 tx=0x7f2b2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49847fd70db94b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402ade0 con 0x7f2b2c004d00 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4402e070 con 0x7f2b2c004d00 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b49600640 1 -- 172.21.15.55:0/352490739 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44034620 con 0x7f2b2c004d00 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b51c00640 1 -- 172.21.15.55:0/352490739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c004d00 msgr2=0x7f2b2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b51c00640 1 --2- 172.21.15.55:0/352490739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c004d00 0x7f2b2c0051d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400ba10 tx=0x7f2b2c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b4b400640 1 -- 172.21.15.55:0/352490739 reap_dead start 2024-05-10T03:31:54.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b51c00640 1 -- 172.21.15.55:0/352490739 shutdown_connections 2024-05-10T03:31:54.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.795+0000 7f2b51c00640 1 -- 172.21.15.55:0/352490739 >> 172.21.15.55:0/352490739 conn(0x7f2b4c0cbb90 msgr2=0x7f2b4c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:54.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.796+0000 7f2b51c00640 1 -- 172.21.15.55:0/352490739 shutdown_connections 2024-05-10T03:31:54.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.796+0000 7f2b51c00640 1 -- 172.21.15.55:0/352490739 wait complete. 2024-05-10T03:31:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.798+0000 7f2b51c00640 1 Processor -- start 2024-05-10T03:31:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.798+0000 7f2b51c00640 1 -- start start 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b4c14d050 0x7f2b4c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c141820 0x7f2b4c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0ad660 con 0x7f2b4c156ef0 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0ad940 con 0x7f2b4c14d050 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b51c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c0adc20 con 0x7f2b4c141820 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42562/0 (socket says 172.21.15.55:42562) 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.800+0000 7f2b4b400640 1 -- 172.21.15.55:0/1791829990 learned_addr learned my addr 172.21.15.55:0/1791829990 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4a000640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c141820 0x7f2b4c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4aa00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b4c14d050 0x7f2b4c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 -- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c141820 msgr2=0x7f2b4c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c141820 0x7f2b4c0afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 -- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b4c14d050 msgr2=0x7f2b4c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b4c14d050 0x7f2b4c0af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 -- 172.21.15.55:0/1791829990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c0adf00 con 0x7f2b4c156ef0 2024-05-10T03:31:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b4b400640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bb30 tx=0x7f2b44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.801+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44035d20 con 0x7f2b4c156ef0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.802+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b44035e80 con 0x7f2b4c156ef0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.802+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c0ad2c0 con 0x7f2b4c156ef0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.802+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4403f6b0 con 0x7f2b4c156ef0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.802+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c142d10 con 0x7f2b4c156ef0 2024-05-10T03:31:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.803+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c0ae420 con 0x7f2b4c156ef0 2024-05-10T03:31:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.804+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2b4403f810 con 0x7f2b4c156ef0 2024-05-10T03:31:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.805+0000 7f2b42a00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 0x7f2b2806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.805+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b440a9c40 con 0x7f2b4c156ef0 2024-05-10T03:31:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.806+0000 7f2b4aa00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 0x7f2b2806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:54.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.808+0000 7f2b4aa00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 0x7f2b2806d110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001050 tx=0x7f2b3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:54.809+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2b4c0ae420 con 0x7f2b4c156ef0 2024-05-10T03:31:55.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.050+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b4c0ae420 con 0x7f2b4c156ef0 2024-05-10T03:31:55.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.051+0000 7f2b42a00640 1 -- 172.21.15.55:0/1791829990 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f2b4c0ae420 con 0x7f2b4c156ef0 2024-05-10T03:31:55.056 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:55.056 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:55.056 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.054+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 msgr2=0x7f2b2806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.054+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 0x7f2b2806d110 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001050 tx=0x7f2b3c009040 comp rx=0 tx=0).stop 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.054+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 msgr2=0x7f2b4c1425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.054+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400bb30 tx=0x7f2b44004730 comp rx=0 tx=0).stop 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 shutdown_connections 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b4c156ef0 0x7f2b4c1425f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b4c141820 0x7f2b4c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2b2806ac00 0x7f2b2806d110 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 --2- 172.21.15.55:0/1791829990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b4c14d050 0x7f2b4c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 >> 172.21.15.55:0/1791829990 conn(0x7f2b4c0cbb90 msgr2=0x7f2b4c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 shutdown_connections 2024-05-10T03:31:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.055+0000 7f2b51c00640 1 -- 172.21.15.55:0/1791829990 wait complete. 2024-05-10T03:31:55.086 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:55.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.365+0000 7f922ec00640 1 Processor -- start 2024-05-10T03:31:55.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.365+0000 7f922ec00640 1 -- start start 2024-05-10T03:31:55.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.367+0000 7f922ec00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9228144710 0x7f9228144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.367+0000 7f922ec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 0x7f92280cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922ec00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922806d350 con 0x7f922806d7e0 2024-05-10T03:31:55.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922ec00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922806c740 con 0x7f9228144710 2024-05-10T03:31:55.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922ec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922806ca20 con 0x7f92280b3190 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922ce00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f922806d7e0 0x7f92280b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54162/0 (socket says 172.21.15.55:54162) 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922ce00640 1 -- 172.21.15.55:0/1274089053 learned_addr learned my addr 172.21.15.55:0/1274089053 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922e200640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 0x7f92280cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f922d800640 1 --2- 172.21.15.55:0/1274089053 >> v2:172.21.15.162:3300/0 conn(0x7f9228144710 0x7f9228144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.368+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224553066 0 0) 0x7f922806d350 con 0x7f922806d7e0 2024-05-10T03:31:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9210003520 con 0x7f922806d7e0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 868228475 0 0) 0x7f9210003520 con 0x7f922806d7e0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 msgr2=0x7f92280cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 0x7f92280cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 >> v2:172.21.15.162:3300/0 conn(0x7f9228144710 msgr2=0x7f9228144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 --2- 172.21.15.55:0/1274089053 >> v2:172.21.15.162:3300/0 conn(0x7f9228144710 0x7f9228144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922806cd00 con 0x7f922806d7e0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f9214001670 con 0x7f922806d7e0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 >> v1:172.21.15.55:6789/0 conn(0x7f922806d7e0 legacy=0x7f92280b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f9210003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92100047c0 0x7f9210004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 0x7f92100051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922806c740 con 0x7f92100039e0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922806ca20 con 0x7f92100047c0 2024-05-10T03:31:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921000bb10 con 0x7f92280b3190 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.369+0000 7f922d800640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f9210003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922ce00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92100047c0 0x7f9210004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922d800640 1 -- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 msgr2=0x7f92100051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922d800640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92280b3190 0x7f92100051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922d800640 1 -- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92100047c0 msgr2=0x7f9210004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922d800640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92100047c0 0x7f9210004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922d800640 1 -- 172.21.15.55:0/1274089053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210003520 con 0x7f92100039e0 2024-05-10T03:31:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.370+0000 7f922ce00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92100047c0 0x7f9210004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.371+0000 7f922d800640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f9210003e50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f921c009a90 tx=0x7f92100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dc9467bfb67e33 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.371+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c031400 con 0x7f92100039e0 2024-05-10T03:31:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.371+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c02d050 con 0x7f92100039e0 2024-05-10T03:31:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.371+0000 7f9227e00640 1 -- 172.21.15.55:0/1274089053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c0317a0 con 0x7f92100039e0 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 -- 172.21.15.55:0/1274089053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 msgr2=0x7f9210003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 --2- 172.21.15.55:0/1274089053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f9210003e50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f921c009a90 tx=0x7f92100057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922e200640 1 -- 172.21.15.55:0/1274089053 reap_dead start 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 -- 172.21.15.55:0/1274089053 shutdown_connections 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 -- 172.21.15.55:0/1274089053 >> 172.21.15.55:0/1274089053 conn(0x7f922806a110 msgr2=0x7f922806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 -- 172.21.15.55:0/1274089053 shutdown_connections 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.372+0000 7f922ec00640 1 -- 172.21.15.55:0/1274089053 wait complete. 2024-05-10T03:31:55.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.375+0000 7f922ec00640 1 Processor -- start 2024-05-10T03:31:55.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.375+0000 7f922ec00640 1 -- start start 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f922815d9e0 0x7f922815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92280711b0 0x7f92280715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922813b250 con 0x7f92100039e0 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92281f4150 con 0x7f92280711b0 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.376+0000 7f922ec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922815e0e0 con 0x7f922815d9e0 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42586/0 (socket says 172.21.15.55:42586) 2024-05-10T03:31:55.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 -- 172.21.15.55:0/1806412878 learned_addr learned my addr 172.21.15.55:0/1806412878 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:55.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922e200640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92280711b0 0x7f92280715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 -- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f922815d9e0 msgr2=0x7f922815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f922815d9e0 0x7f922815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 -- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92280711b0 msgr2=0x7f92280715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92280711b0 0x7f92280715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.377+0000 7f922d800640 1 -- 172.21.15.55:0/1806412878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922813a640 con 0x7f92100039e0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f922d800640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f921c009a90 tx=0x7f921c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c031ea0 con 0x7f92100039e0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c03a810 con 0x7f92100039e0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c03abc0 con 0x7f92100039e0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9228071b00 con 0x7f92100039e0 2024-05-10T03:31:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.378+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922813a8b0 con 0x7f92100039e0 2024-05-10T03:31:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.380+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228066ce0 con 0x7f92100039e0 2024-05-10T03:31:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.381+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f921c038040 con 0x7f92100039e0 2024-05-10T03:31:55.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.382+0000 7f9225600640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 0x7f920806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.382+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f921c0a95d0 con 0x7f92100039e0 2024-05-10T03:31:55.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.383+0000 7f922ce00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 0x7f920806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.384+0000 7f922ce00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 0x7f920806d110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92281f67b0 tx=0x7f9214009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.388+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f921c074720 con 0x7f92100039e0 2024-05-10T03:31:55.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.592+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92280b45c0 con 0x7f92100039e0 2024-05-10T03:31:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.595+0000 7f9225600640 1 -- 172.21.15.55:0/1806412878 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f921c009690 con 0x7f92100039e0 2024-05-10T03:31:55.600 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:55.600 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:55.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 msgr2=0x7f920806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 0x7f920806d110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92281f67b0 tx=0x7f9214009040 comp rx=0 tx=0).stop 2024-05-10T03:31:55.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 msgr2=0x7f922813b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f921c009a90 tx=0x7f921c004730 comp rx=0 tx=0).stop 2024-05-10T03:31:55.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 shutdown_connections 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f920806ac00 0x7f920806d110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92280711b0 0x7f92280715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f922815d9e0 0x7f922815a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 --2- 172.21.15.55:0/1806412878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92100039e0 0x7f922813b670 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 >> 172.21.15.55:0/1806412878 conn(0x7f922806a110 msgr2=0x7f92280ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 shutdown_connections 2024-05-10T03:31:55.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.598+0000 7f922ec00640 1 -- 172.21.15.55:0/1806412878 wait complete. 2024-05-10T03:31:55.629 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:31:55.629 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:55.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.907+0000 7fdb3b200640 1 Processor -- start 2024-05-10T03:31:55.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.907+0000 7fdb3b200640 1 -- start start 2024-05-10T03:31:55.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.909+0000 7fdb3b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb3415ddd0 0x7fdb34157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.909+0000 7fdb3b200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdb340ce5f0 0x7fdb340b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.909+0000 7fdb3b200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3415edc0 con 0x7fdb341613c0 2024-05-10T03:31:55.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.909+0000 7fdb3b200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb341623d0 con 0x7fdb3415ddd0 2024-05-10T03:31:55.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.909+0000 7fdb3b200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3415e1e0 con 0x7fdb340ce5f0 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb3415ddd0 0x7fdb34157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb3415ddd0 0x7fdb34157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42588/0 (socket says 172.21.15.55:42588) 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 -- 172.21.15.55:0/3045295978 learned_addr learned my addr 172.21.15.55:0/3045295978 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 -- 172.21.15.55:0/3045295978 >> v2:172.21.15.162:3300/0 conn(0x7fdb340ce5f0 msgr2=0x7fdb340b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 --2- 172.21.15.55:0/3045295978 >> v2:172.21.15.162:3300/0 conn(0x7fdb340ce5f0 0x7fdb340b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 -- 172.21.15.55:0/3045295978 >> v1:172.21.15.162:6789/0 conn(0x7fdb341613c0 legacy=0x7fdb340cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.910+0000 7fdb39e00640 1 -- 172.21.15.55:0/3045295978 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340cecd0 con 0x7fdb3415ddd0 2024-05-10T03:31:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.912+0000 7fdb39e00640 1 --2- 172.21.15.55:0/3045295978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb3415ddd0 0x7fdb34157e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdb340cf690 tx=0x7fdb2c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9541e389dcab2c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.912+0000 7fdb38a00640 1 -- 172.21.15.55:0/3045295978 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c031600 con 0x7fdb3415ddd0 2024-05-10T03:31:55.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.912+0000 7fdb38a00640 1 -- 172.21.15.55:0/3045295978 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c02d050 con 0x7fdb3415ddd0 2024-05-10T03:31:55.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.912+0000 7fdb38a00640 1 -- 172.21.15.55:0/3045295978 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c031980 con 0x7fdb3415ddd0 2024-05-10T03:31:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 -- 172.21.15.55:0/3045295978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb3415ddd0 msgr2=0x7fdb34157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 --2- 172.21.15.55:0/3045295978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb3415ddd0 0x7fdb34157e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdb340cf690 tx=0x7fdb2c02a6f0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 -- 172.21.15.55:0/3045295978 shutdown_connections 2024-05-10T03:31:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 --2- 172.21.15.55:0/3045295978 >> v2:172.21.15.162:3300/0 conn(0x7fdb340ce5f0 0x7fdb340b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 --2- 172.21.15.55:0/3045295978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb3415ddd0 0x7fdb34157e80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 -- 172.21.15.55:0/3045295978 >> 172.21.15.55:0/3045295978 conn(0x7fdb340d0370 msgr2=0x7fdb340d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:55.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 -- 172.21.15.55:0/3045295978 shutdown_connections 2024-05-10T03:31:55.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.913+0000 7fdb3b200640 1 -- 172.21.15.55:0/3045295978 wait complete. 2024-05-10T03:31:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.915+0000 7fdb3b200640 1 Processor -- start 2024-05-10T03:31:55.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.916+0000 7fdb3b200640 1 -- start start 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415ddd0 0x7fdb3414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb341613c0 0x7fdb340b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341f49e0 con 0x7fdb340b53d0 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34154640 con 0x7fdb3415ddd0 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.917+0000 7fdb3b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34159cc0 con 0x7fdb341613c0 2024-05-10T03:31:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42604/0 (socket says 172.21.15.55:42604) 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 -- 172.21.15.55:0/2891998116 learned_addr learned my addr 172.21.15.55:0/2891998116 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb39e00640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415ddd0 0x7fdb3414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 -- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb341613c0 msgr2=0x7fdb340b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb341613c0 0x7fdb340b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 -- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415ddd0 msgr2=0x7fdb3414f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415ddd0 0x7fdb3414f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:55.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 -- 172.21.15.55:0/2891998116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c009b30 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.918+0000 7fdb3a800640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000b440 tx=0x7fdb3000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.919+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30011830 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.919+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb30011990 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.919+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30011d40 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.919+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb34158f40 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.919+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb341594c0 con 0x7fdb340b53d0 2024-05-10T03:31:55.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.920+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3404e290 con 0x7fdb340b53d0 2024-05-10T03:31:55.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.921+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdb3001a400 con 0x7fdb340b53d0 2024-05-10T03:31:55.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.922+0000 7fdb2a000640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 0x7fdb1806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:55.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.922+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdb300895a0 con 0x7fdb340b53d0 2024-05-10T03:31:55.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.923+0000 7fdb39e00640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 0x7fdb1806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.924+0000 7fdb39e00640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 0x7fdb1806d140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c033000 tx=0x7fdb2c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:55.927+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdb30054640 con 0x7fdb340b53d0 2024-05-10T03:31:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.161+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb340b27e0 con 0x7fdb340b53d0 2024-05-10T03:31:56.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.162+0000 7fdb2a000640 1 -- 172.21.15.55:0/2891998116 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fdb30057c60 con 0x7fdb340b53d0 2024-05-10T03:31:56.167 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:56.167 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:56.167 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.165+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 msgr2=0x7fdb1806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.165+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 0x7fdb1806d140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c033000 tx=0x7fdb2c009000 comp rx=0 tx=0).stop 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.165+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 msgr2=0x7fdb340b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.165+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000b440 tx=0x7fdb3000b900 comp rx=0 tx=0).stop 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 shutdown_connections 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb340b53d0 0x7fdb340b7cb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb341613c0 0x7fdb340b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb1806ac30 0x7fdb1806d140 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 --2- 172.21.15.55:0/2891998116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415ddd0 0x7fdb3414f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 >> 172.21.15.55:0/2891998116 conn(0x7fdb340d0370 msgr2=0x7fdb341601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:56.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 shutdown_connections 2024-05-10T03:31:56.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.166+0000 7fdb3b200640 1 -- 172.21.15.55:0/2891998116 wait complete. 2024-05-10T03:31:56.195 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:31:56.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.457+0000 7f6ce0c00640 1 Processor -- start 2024-05-10T03:31:56.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.457+0000 7f6ce0c00640 1 -- start start 2024-05-10T03:31:56.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 0x7f6cdc06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cdc06da50 0x7f6cdc06de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc162de0 con 0x7f6cdc06da50 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc158f00 con 0x7f6cdc06c230 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6ce0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b3ae0 con 0x7f6cdc0b36d0 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.460+0000 7f6cdbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cdc06da50 0x7f6cdc06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cdc06da50 0x7f6cdc06de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42612/0 (socket says 172.21.15.55:42612) 2024-05-10T03:31:56.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 -- 172.21.15.55:0/937145441 learned_addr learned my addr 172.21.15.55:0/937145441 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdb400640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cd3e00640 1 --2- 172.21.15.55:0/937145441 >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 0x7f6cdc06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 -- 172.21.15.55:0/937145441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 msgr2=0x7f6cdc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 -- 172.21.15.55:0/937145441 >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 msgr2=0x7f6cdc06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 --2- 172.21.15.55:0/937145441 >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 0x7f6cdc06c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cdbe00640 1 -- 172.21.15.55:0/937145441 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc06cb80 con 0x7f6cdc06da50 2024-05-10T03:31:56.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.461+0000 7f6cd3e00640 1 --2- 172.21.15.55:0/937145441 >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 0x7f6cdc06c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:56.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.462+0000 7f6cdbe00640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06da50 0x7f6cdc06de60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc06b180 tx=0x7f6cd402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d503e65e92297fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:56.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.462+0000 7f6cdaa00640 1 -- 172.21.15.55:0/937145441 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4031600 con 0x7f6cdc06da50 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.462+0000 7f6cdaa00640 1 -- 172.21.15.55:0/937145441 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd402d050 con 0x7f6cdc06da50 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.462+0000 7f6cdaa00640 1 -- 172.21.15.55:0/937145441 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4031980 con 0x7f6cdc06da50 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 -- 172.21.15.55:0/937145441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06da50 msgr2=0x7f6cdc06de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06da50 0x7f6cdc06de60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc06b180 tx=0x7f6cd402a610 comp rx=0 tx=0).stop 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 -- 172.21.15.55:0/937145441 shutdown_connections 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06da50 0x7f6cdc06de60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/937145441 >> v2:172.21.15.162:3300/0 conn(0x7f6cdc06c230 0x7f6cdc06c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/937145441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.463+0000 7f6ce0c00640 1 -- 172.21.15.55:0/937145441 >> 172.21.15.55:0/937145441 conn(0x7f6cdc070eb0 msgr2=0x7f6cdc0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.464+0000 7f6ce0c00640 1 -- 172.21.15.55:0/937145441 shutdown_connections 2024-05-10T03:31:56.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.464+0000 7f6ce0c00640 1 -- 172.21.15.55:0/937145441 wait complete. 2024-05-10T03:31:56.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.466+0000 7f6ce0c00640 1 Processor -- start 2024-05-10T03:31:56.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.466+0000 7f6ce0c00640 1 -- start start 2024-05-10T03:31:56.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cdc06da50 0x7f6cdc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc0ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0abd20 con 0x7f6cdc06c230 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc14c550 con 0x7f6cdc06da50 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.467+0000 7f6ce0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc14dd80 con 0x7f6cdc0b36d0 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42628/0 (socket says 172.21.15.55:42628) 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 -- 172.21.15.55:0/2865512587 learned_addr learned my addr 172.21.15.55:0/2865512587 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cd3e00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cdc06da50 0x7f6cdc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdbe00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc0ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 -- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 msgr2=0x7f6cdc0ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc0ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 -- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cdc06da50 msgr2=0x7f6cdc0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cdc06da50 0x7f6cdc0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdb400640 1 -- 172.21.15.55:0/2865512587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009b30 con 0x7f6cdc06c230 2024-05-10T03:31:56.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.468+0000 7f6cdbe00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc0ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:56.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6cdb400640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00d5f0 tx=0x7f6ccc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:56.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0125f0 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc012750 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc14c7a0 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc012b00 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.469+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc14cd60 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.470+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc04e290 con 0x7f6cdc06c230 2024-05-10T03:31:56.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.471+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6ccc012c60 con 0x7f6cdc06c230 2024-05-10T03:31:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.472+0000 7f6cd3400640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.472+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ccc0899f0 con 0x7f6cdc06c230 2024-05-10T03:31:56.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.473+0000 7f6cd3e00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:56.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.474+0000 7f6cd3e00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0716f0 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:56.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.477+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6ccc0549f0 con 0x7f6cdc06c230 2024-05-10T03:31:56.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.710+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cdc0b3360 con 0x7f6cdc06c230 2024-05-10T03:31:56.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.711+0000 7f6cd3400640 1 -- 172.21.15.55:0/2865512587 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f6ccc058010 con 0x7f6cdc06c230 2024-05-10T03:31:56.716 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:56.716 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:31:27.383309+0000","modified":"2024-05-10T03:31:30.836445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/455504925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":455504925},{"type":"v1","addr":"172.21.15.55:6835","nonce":455504925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-10T03:31:56.716 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 16 2024-05-10T03:31:56.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 msgr2=0x7f6cb806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0716f0 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 msgr2=0x7f6cdc14d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00d5f0 tx=0x7f6ccc00dab0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 shutdown_connections 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cdc0b36d0 0x7f6cdc0ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cdc06da50 0x7f6cdc0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 --2- 172.21.15.55:0/2865512587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cdc06c230 0x7f6cdc14d840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 >> 172.21.15.55:0/2865512587 conn(0x7f6cdc070eb0 msgr2=0x7f6cdc163d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 shutdown_connections 2024-05-10T03:31:56.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:56.714+0000 7f6ce0c00640 1 -- 172.21.15.55:0/2865512587 wait complete. 2024-05-10T03:31:56.743 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:31:57.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.007+0000 7f8dee400640 1 Processor -- start 2024-05-10T03:31:57.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.007+0000 7f8dee400640 1 -- start start 2024-05-10T03:31:57.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.010+0000 7f8dee400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.010+0000 7f8dee400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8de815dec0 0x7f8de806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.010+0000 7f8dee400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de80cf580 con 0x7f8de806b660 2024-05-10T03:31:57.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.010+0000 7f8dee400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de815ee60 con 0x7f8de815dec0 2024-05-10T03:31:57.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.010+0000 7f8dee400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de806ba70 con 0x7f8de81614b0 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de6a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8de815dec0 0x7f8de806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de7e00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8de806b660 0x7f8de80cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:45908/0 (socket says 172.21.15.55:45908) 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de7e00640 1 -- 172.21.15.55:0/2137368462 learned_addr learned my addr 172.21.15.55:0/2137368462 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de7400640 1 -- 172.21.15.55:0/2137368462 >> v2:172.21.15.162:3300/0 conn(0x7f8de815dec0 msgr2=0x7f8de806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:57.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.011+0000 7f8de7400640 1 --2- 172.21.15.55:0/2137368462 >> v2:172.21.15.162:3300/0 conn(0x7f8de815dec0 0x7f8de806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.012+0000 7f8de7400640 1 -- 172.21.15.55:0/2137368462 >> v1:172.21.15.162:6789/0 conn(0x7f8de806b660 legacy=0x7f8de80cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.012+0000 7f8de7400640 1 -- 172.21.15.55:0/2137368462 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de815e5c0 con 0x7f8de81614b0 2024-05-10T03:31:57.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.013+0000 7f8de7400640 1 --2- 172.21.15.55:0/2137368462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de80aedd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097a0 tx=0x7f8dd802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e935da71f87e510 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.013+0000 7f8de6000640 1 -- 172.21.15.55:0/2137368462 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8032440 con 0x7f8de81614b0 2024-05-10T03:31:57.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.013+0000 7f8de6000640 1 -- 172.21.15.55:0/2137368462 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd80325a0 con 0x7f8de81614b0 2024-05-10T03:31:57.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.013+0000 7f8de6000640 1 -- 172.21.15.55:0/2137368462 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8032930 con 0x7f8de81614b0 2024-05-10T03:31:57.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 -- 172.21.15.55:0/2137368462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 msgr2=0x7f8de80aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 --2- 172.21.15.55:0/2137368462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de80aedd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097a0 tx=0x7f8dd802a180 comp rx=0 tx=0).stop 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 -- 172.21.15.55:0/2137368462 shutdown_connections 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 --2- 172.21.15.55:0/2137368462 >> v2:172.21.15.162:3300/0 conn(0x7f8de815dec0 0x7f8de806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 --2- 172.21.15.55:0/2137368462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de80aedd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.014+0000 7f8dee400640 1 -- 172.21.15.55:0/2137368462 >> 172.21.15.55:0/2137368462 conn(0x7f8de8070900 msgr2=0x7f8de8070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.015+0000 7f8dee400640 1 -- 172.21.15.55:0/2137368462 shutdown_connections 2024-05-10T03:31:57.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.015+0000 7f8dee400640 1 -- 172.21.15.55:0/2137368462 wait complete. 2024-05-10T03:31:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.017+0000 7f8dee400640 1 Processor -- start 2024-05-10T03:31:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.017+0000 7f8dee400640 1 -- start start 2024-05-10T03:31:57.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8de815dec0 0x7f8de815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de81f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de815f6b0 con 0x7f8de806b660 2024-05-10T03:31:57.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de8156a40 con 0x7f8de815dec0 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8dee400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de81f7de0 con 0x7f8de81614b0 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42630/0 (socket says 172.21.15.55:42630) 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 -- 172.21.15.55:0/3792676275 learned_addr learned my addr 172.21.15.55:0/3792676275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 -- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 msgr2=0x7f8de81f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de81f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 -- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8de815dec0 msgr2=0x7f8de815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8de815dec0 0x7f8de815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.019+0000 7f8de7400640 1 -- 172.21.15.55:0/3792676275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8009400 con 0x7f8de806b660 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8de7400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80098c0 tx=0x7f8dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8030b80 con 0x7f8de806b660 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd8032e90 con 0x7f8de806b660 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de81f8030 con 0x7f8de806b660 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8006e10 con 0x7f8de806b660 2024-05-10T03:31:57.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.020+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de815eff0 con 0x7f8de806b660 2024-05-10T03:31:57.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.022+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de815f7f0 con 0x7f8de806b660 2024-05-10T03:31:57.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.023+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8dd8039040 con 0x7f8de806b660 2024-05-10T03:31:57.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.024+0000 7f8ddf400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 0x7f8dc806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.024+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8dd80a9fc0 con 0x7f8de806b660 2024-05-10T03:31:57.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.025+0000 7f8de6a00640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 0x7f8dc806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.027+0000 7f8de6a00640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 0x7f8dc806d140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006a90 tx=0x7f8dd4006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.029+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8de815f7f0 con 0x7f8de806b660 2024-05-10T03:31:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.228+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8de815f7f0 con 0x7f8de806b660 2024-05-10T03:31:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.231+0000 7f8ddf400640 1 -- 172.21.15.55:0/3792676275 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f8de815f7f0 con 0x7f8de806b660 2024-05-10T03:31:57.236 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:31:57.236 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":40,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:31:28.386857+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:31:24.993809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-10T03:31:26.385874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 msgr2=0x7f8dc806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 0x7f8dc806d140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006a90 tx=0x7f8dd4006a20 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 msgr2=0x7f8de8157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80098c0 tx=0x7f8dd80046c0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 shutdown_connections 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8de81614b0 0x7f8de81f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8dc806ac30 0x7f8dc806d140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8de815dec0 0x7f8de815fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 --2- 172.21.15.55:0/3792676275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8de806b660 0x7f8de8157310 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 >> 172.21.15.55:0/3792676275 conn(0x7f8de8070900 msgr2=0x7f8de815bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 shutdown_connections 2024-05-10T03:31:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.234+0000 7f8dee400640 1 -- 172.21.15.55:0/3792676275 wait complete. 2024-05-10T03:31:57.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:31:57.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.543+0000 7f81c2800640 1 Processor -- start 2024-05-10T03:31:57.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.543+0000 7f81c2800640 1 -- start start 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81bc15b9d0 0x7f81bc06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c2800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81bc06b660 0x7f81bc13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c2800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc15bde0 con 0x7f81bc13e390 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c2800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc13e7a0 con 0x7f81bc15b9d0 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c2800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc13eae0 con 0x7f81bc06b660 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81bc15b9d0 0x7f81bc06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c1400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f81bc13e390 0x7f81bc1414c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54174/0 (socket says 172.21.15.55:54174) 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c1400640 1 -- 172.21.15.55:0/577813337 learned_addr learned my addr 172.21.15.55:0/577813337 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:57.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.546+0000 7f81c0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81bc15b9d0 0x7f81bc06b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42632/0 (socket says 172.21.15.55:42632) 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.547+0000 7f81c1e00640 1 --2- 172.21.15.55:0/577813337 >> v2:172.21.15.162:3300/0 conn(0x7f81bc06b660 0x7f81bc13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.547+0000 7f81c0a00640 1 -- 172.21.15.55:0/577813337 >> v2:172.21.15.162:3300/0 conn(0x7f81bc06b660 msgr2=0x7f81bc13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.547+0000 7f81c0a00640 1 --2- 172.21.15.55:0/577813337 >> v2:172.21.15.162:3300/0 conn(0x7f81bc06b660 0x7f81bc13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.547+0000 7f81c0a00640 1 -- 172.21.15.55:0/577813337 >> v1:172.21.15.55:6789/0 conn(0x7f81bc13e390 legacy=0x7f81bc1414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.547+0000 7f81c0a00640 1 -- 172.21.15.55:0/577813337 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc06ba70 con 0x7f81bc15b9d0 2024-05-10T03:31:57.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.548+0000 7f81c0a00640 1 --2- 172.21.15.55:0/577813337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc15b9d0 0x7f81bc06b120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0cda30 tx=0x7f81a802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221b5887da3593f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.548+0000 7f81bbe00640 1 -- 172.21.15.55:0/577813337 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a802ae50 con 0x7f81bc15b9d0 2024-05-10T03:31:57.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.548+0000 7f81bbe00640 1 -- 172.21.15.55:0/577813337 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8032440 con 0x7f81bc15b9d0 2024-05-10T03:31:57.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.548+0000 7f81bbe00640 1 -- 172.21.15.55:0/577813337 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80327d0 con 0x7f81bc15b9d0 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 -- 172.21.15.55:0/577813337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc15b9d0 msgr2=0x7f81bc06b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 --2- 172.21.15.55:0/577813337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc15b9d0 0x7f81bc06b120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0cda30 tx=0x7f81a802a100 comp rx=0 tx=0).stop 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 -- 172.21.15.55:0/577813337 shutdown_connections 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 --2- 172.21.15.55:0/577813337 >> v2:172.21.15.162:3300/0 conn(0x7f81bc06b660 0x7f81bc13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 --2- 172.21.15.55:0/577813337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc15b9d0 0x7f81bc06b120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 -- 172.21.15.55:0/577813337 >> 172.21.15.55:0/577813337 conn(0x7f81bc06c730 msgr2=0x7f81bc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 -- 172.21.15.55:0/577813337 shutdown_connections 2024-05-10T03:31:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.549+0000 7f81c2800640 1 -- 172.21.15.55:0/577813337 wait complete. 2024-05-10T03:31:57.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.552+0000 7f81c2800640 1 Processor -- start 2024-05-10T03:31:57.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.552+0000 7f81c2800640 1 -- start start 2024-05-10T03:31:57.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.553+0000 7f81c2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.553+0000 7f81c2800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81bc13e390 0x7f81bc153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c2800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81bc15b9d0 0x7f81bc154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c2800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc1398f0 con 0x7f81bc06b660 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c2800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc152ba0 con 0x7f81bc13e390 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c2800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc13b600 con 0x7f81bc15b9d0 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42648/0 (socket says 172.21.15.55:42648) 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 -- 172.21.15.55:0/1070502584 learned_addr learned my addr 172.21.15.55:0/1070502584 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 -- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81bc15b9d0 msgr2=0x7f81bc154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81bc15b9d0 0x7f81bc154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c0a00640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81bc13e390 0x7f81bc153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 -- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81bc13e390 msgr2=0x7f81bc153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81bc13e390 0x7f81bc153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.554+0000 7f81c1400640 1 -- 172.21.15.55:0/1070502584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a80093a0 con 0x7f81bc06b660 2024-05-10T03:31:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81c1400640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c440 tx=0x7f81b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0005ce0 con 0x7f81bc06b660 2024-05-10T03:31:57.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b0005e40 con 0x7f81bc06b660 2024-05-10T03:31:57.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc152df0 con 0x7f81bc06b660 2024-05-10T03:31:57.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00072f0 con 0x7f81bc06b660 2024-05-10T03:31:57.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.555+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1533b0 con 0x7f81bc06b660 2024-05-10T03:31:57.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.556+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc1382d0 con 0x7f81bc06b660 2024-05-10T03:31:57.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.557+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f81b0007450 con 0x7f81bc06b660 2024-05-10T03:31:57.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.558+0000 7f81b9600640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 0x7f819c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:57.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.559+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f81b0088880 con 0x7f81bc06b660 2024-05-10T03:31:57.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.559+0000 7f81c0a00640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 0x7f819c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:57.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.560+0000 7f81c0a00640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 0x7f819c06d140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:57.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.564+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f81bc1382d0 con 0x7f81bc06b660 2024-05-10T03:31:57.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.802+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f81bc15bde0 con 0x7f81bc06b660 2024-05-10T03:31:57.809 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:31:57.805+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:31:57.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.827+0000 7f81b9600640 1 -- 172.21.15.55:0/1070502584 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f81bc15bde0 con 0x7f81bc06b660 2024-05-10T03:31:57.831 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 msgr2=0x7f819c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 0x7f819c06d140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f81a8034000 tx=0x7f81a802e000 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 msgr2=0x7f81bc151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f81b000c440 tx=0x7f81b000c900 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 shutdown_connections 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81bc15b9d0 0x7f81bc154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f819c06ac30 0x7f819c06d140 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81bc13e390 0x7f81bc153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 --2- 172.21.15.55:0/1070502584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81bc06b660 0x7f81bc151530 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 >> 172.21.15.55:0/1070502584 conn(0x7f81bc06c730 msgr2=0x7f81bc071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 shutdown_connections 2024-05-10T03:31:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:57.831+0000 7f81c2800640 1 -- 172.21.15.55:0/1070502584 wait complete. 2024-05-10T03:31:57.840 INFO:tasks.ceph.mds.a.smithi055.stderr: -2687> 2024-05-10T03:31:28.159+0000 7f4e4ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:31:57.866 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:31:57.979 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:31:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.142+0000 7fdee6e00640 1 Processor -- start 2024-05-10T03:31:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.142+0000 7fdee6e00640 1 -- start start 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee01611c0 0x7fdee00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdee006b1b0 0x7fdee006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee006a110 con 0x7fdee00b6570 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00b60a0 con 0x7fdee006b1b0 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee006bb00 con 0x7fdee01611c0 2024-05-10T03:31:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdee00b6570 0x7fdee00b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54190/0 (socket says 172.21.15.55:54190) 2024-05-10T03:31:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.144+0000 7fdee6400640 1 -- 172.21.15.55:0/4207912220 learned_addr learned my addr 172.21.15.55:0/4207912220 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdee5a00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee01611c0 0x7fdee00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdee5000640 1 --2- 172.21.15.55:0/4207912220 >> v2:172.21.15.162:3300/0 conn(0x7fdee006b1b0 0x7fdee006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775630975 0 0) 0x7fdee006a110 con 0x7fdee00b6570 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8003520 con 0x7fdee00b6570 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570473133 0 0) 0x7fdec8003520 con 0x7fdee00b6570 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee01611c0 msgr2=0x7fdee00b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee01611c0 0x7fdee00b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 >> v2:172.21.15.162:3300/0 conn(0x7fdee006b1b0 msgr2=0x7fdee006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 --2- 172.21.15.55:0/4207912220 >> v2:172.21.15.162:3300/0 conn(0x7fdee006b1b0 0x7fdee006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee006a3f0 con 0x7fdee00b6570 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fded8001670 con 0x7fdee00b6570 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 >> v1:172.21.15.55:6789/0 conn(0x7fdee00b6570 legacy=0x7fdee00b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.145+0000 7fdedfe00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdec80039e0 0x7fdec8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdedfe00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdec80047c0 0x7fdec8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdedfe00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdec800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b60a0 con 0x7fdec8005110 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec800bec0 con 0x7fdec80047c0 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec800c030 con 0x7fdec80039e0 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdec800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee5a00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdec80039e0 0x7fdec8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee5000640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdec80047c0 0x7fdec8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 -- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdec80039e0 msgr2=0x7fdec8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdec80039e0 0x7fdec8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 -- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdec80047c0 msgr2=0x7fdec8004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:58.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdec80047c0 0x7fdec8004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.146+0000 7fdee6400640 1 -- 172.21.15.55:0/4207912220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8003520 con 0x7fdec8005110 2024-05-10T03:31:58.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.147+0000 7fdee5000640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdec80047c0 0x7fdec8004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.148+0000 7fdee6400640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdec800b980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cd6a0 tx=0x7fdec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beb9394be9e56c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.148+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8005ad0 con 0x7fdec8005110 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.148+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded802d070 con 0x7fdec8005110 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.148+0000 7fdedfe00640 1 -- 172.21.15.55:0/4207912220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8005e70 con 0x7fdec8005110 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 -- 172.21.15.55:0/4207912220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 msgr2=0x7fdec800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 --2- 172.21.15.55:0/4207912220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdec800b980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cd6a0 tx=0x7fdec80057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6400640 1 -- 172.21.15.55:0/4207912220 reap_dead start 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 -- 172.21.15.55:0/4207912220 shutdown_connections 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 -- 172.21.15.55:0/4207912220 >> 172.21.15.55:0/4207912220 conn(0x7fdee00d06f0 msgr2=0x7fdee00d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:58.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 -- 172.21.15.55:0/4207912220 shutdown_connections 2024-05-10T03:31:58.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.149+0000 7fdee6e00640 1 -- 172.21.15.55:0/4207912220 wait complete. 2024-05-10T03:31:58.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.151+0000 7fdee6e00640 1 Processor -- start 2024-05-10T03:31:58.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.152+0000 7fdee6e00640 1 -- start start 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdee00cb860 0x7fdee014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee00cbe80 0x7fdee00cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee014fdd0 con 0x7fdec8005110 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ca570 con 0x7fdee00cb860 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.153+0000 7fdee6e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cc7d0 con 0x7fdee00cbe80 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42662/0 (socket says 172.21.15.55:42662) 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 -- 172.21.15.55:0/3113548490 learned_addr learned my addr 172.21.15.55:0/3113548490 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 -- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee00cbe80 msgr2=0x7fdee00cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee00cbe80 0x7fdee00cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 -- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdee00cb860 msgr2=0x7fdee014e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5000640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdee00cb860 0x7fdee014e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdee00cb860 0x7fdee014e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 -- 172.21.15.55:0/3113548490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00ca7c0 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.154+0000 7fdee5a00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fded00114f0 tx=0x7fded00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.155+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0015930 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.155+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0015a90 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.155+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00caaa0 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.155+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0015e60 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.155+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00cb060 con 0x7fdec8005110 2024-05-10T03:31:58.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.156+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee004e290 con 0x7fdec8005110 2024-05-10T03:31:58.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.157+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fded001d400 con 0x7fdec8005110 2024-05-10T03:31:58.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.158+0000 7fdedd600640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 0x7fdec406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:58.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.159+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fded0089d70 con 0x7fdec8005110 2024-05-10T03:31:58.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.159+0000 7fdee5000640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 0x7fdec406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:58.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.161+0000 7fdee5000640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 0x7fdec406d140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdee015e380 tx=0x7fded400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:58.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.163+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fded0054d20 con 0x7fdec8005110 2024-05-10T03:31:58.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.403+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdee006a730 con 0x7fdec8005110 2024-05-10T03:31:58.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.830+0000 7fdedd600640 1 -- 172.21.15.55:0/3113548490 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fded0058340 con 0x7fdec8005110 2024-05-10T03:31:58.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 msgr2=0x7fdec406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 0x7fdec406d140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdee015e380 tx=0x7fded400b040 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 msgr2=0x7fdee00c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fded00114f0 tx=0x7fded00119b0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 shutdown_connections 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdee00cbe80 0x7fdee00cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdec406ac30 0x7fdec406d140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdee00cb860 0x7fdee014e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 --2- 172.21.15.55:0/3113548490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec8005110 0x7fdee00c8070 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 >> 172.21.15.55:0/3113548490 conn(0x7fdee00d06f0 msgr2=0x7fdee0160400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.834+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 shutdown_connections 2024-05-10T03:31:58.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:58.835+0000 7fdee6e00640 1 -- 172.21.15.55:0/3113548490 wait complete. 2024-05-10T03:31:58.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:31:59.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.141+0000 7f8af3e00640 1 Processor -- start 2024-05-10T03:31:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.141+0000 7f8af3e00640 1 -- start start 2024-05-10T03:31:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8af415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af4072220 con 0x7f8af406b830 2024-05-10T03:31:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40ac980 con 0x7f8af40ce840 2024-05-10T03:31:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40acc60 con 0x7f8af4071cb0 2024-05-10T03:31:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8af406b830 0x7f8af406ff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54206/0 (socket says 172.21.15.55:54206) 2024-05-10T03:31:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.144+0000 7f8af3400640 1 -- 172.21.15.55:0/2977962353 learned_addr learned my addr 172.21.15.55:0/2977962353 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:31:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.145+0000 7f8af2a00640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8af415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.145+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509920214 0 0) 0x7f8af4072220 con 0x7f8af406b830 2024-05-10T03:31:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.145+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad8003520 con 0x7f8af406b830 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.145+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228200457 0 0) 0x7f8af40ac980 con 0x7f8af40ce840 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.145+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af4072220 con 0x7f8af40ce840 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235575822 0 0) 0x7f8ad8003520 con 0x7f8af406b830 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 msgr2=0x7f8af415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8af415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 >> v1:172.21.15.162:6789/0 conn(0x7f8af40ce840 legacy=0x7f8af406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40acf40 con 0x7f8af406b830 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8aec0019f0 con 0x7f8af406b830 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2a00640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8af415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 >> v1:172.21.15.55:6789/0 conn(0x7f8af406b830 legacy=0x7f8af406ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad8003b30 0x7f8ad8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8ad80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40acc60 con 0x7f8ad8004d00 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad800bab0 con 0x7f8ad8003b30 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad800bc70 con 0x7f8af4071cb0 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af3400640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.146+0000 7f8af2a00640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad8003b30 0x7f8ad8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8aebe00640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8ad80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8af3400640 1 -- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 msgr2=0x7f8ad80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:59.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8af3400640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af4071cb0 0x7f8ad80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8af3400640 1 -- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad8003b30 msgr2=0x7f8ad8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:31:59.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8af3400640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad8003b30 0x7f8ad8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.147+0000 7f8af3400640 1 -- 172.21.15.55:0/2977962353 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009400 con 0x7f8ad8004d00 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.148+0000 7f8af3400640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00a710 tx=0x7f8ad80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd54f2add2d38d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02b930 con 0x7f8ad8004d00 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec02ba90 con 0x7f8ad8004d00 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af2000640 1 -- 172.21.15.55:0/2977962353 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02be80 con 0x7f8ad8004d00 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af3e00640 1 -- 172.21.15.55:0/2977962353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad8004d00 msgr2=0x7f8ad8005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af3e00640 1 --2- 172.21.15.55:0/2977962353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad8004d00 0x7f8ad8005110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00a710 tx=0x7f8ad80057f0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af3400640 1 -- 172.21.15.55:0/2977962353 reap_dead start 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af3e00640 1 -- 172.21.15.55:0/2977962353 shutdown_connections 2024-05-10T03:31:59.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.149+0000 7f8af3e00640 1 -- 172.21.15.55:0/2977962353 >> 172.21.15.55:0/2977962353 conn(0x7f8af406c070 msgr2=0x7f8af406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:31:59.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.150+0000 7f8af3e00640 1 -- 172.21.15.55:0/2977962353 shutdown_connections 2024-05-10T03:31:59.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.150+0000 7f8af3e00640 1 -- 172.21.15.55:0/2977962353 wait complete. 2024-05-10T03:31:59.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.152+0000 7f8af3e00640 1 Processor -- start 2024-05-10T03:31:59.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.152+0000 7f8af3e00640 1 -- start start 2024-05-10T03:31:59.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af415d6f0 0x7f8af40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af40cce70 0x7f8af41f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4160bb0 con 0x7f8af415e080 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40cd280 con 0x7f8af415d6f0 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41605f0 con 0x7f8af40cce70 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42674/0 (socket says 172.21.15.55:42674) 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 -- 172.21.15.55:0/1957615451 learned_addr learned my addr 172.21.15.55:0/1957615451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 -- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af40cce70 msgr2=0x7f8af41f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af40cce70 0x7f8af41f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 -- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af415d6f0 msgr2=0x7f8af40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af415d6f0 0x7f8af40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:31:59.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.154+0000 7f8af2a00640 1 -- 172.21.15.55:0/1957615451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af41608d0 con 0x7f8af415e080 2024-05-10T03:31:59.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8af2a00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4011ad0 tx=0x7f8ae4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:59.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400fc50 con 0x7f8af415e080 2024-05-10T03:31:59.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae400fdb0 con 0x7f8af415e080 2024-05-10T03:31:59.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4014660 con 0x7f8af415e080 2024-05-10T03:31:59.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af4155590 con 0x7f8af415e080 2024-05-10T03:31:59.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.155+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af4151560 con 0x7f8af415e080 2024-05-10T03:31:59.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.156+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4155720 con 0x7f8af415e080 2024-05-10T03:31:59.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.157+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8ae40147c0 con 0x7f8af415e080 2024-05-10T03:31:59.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.158+0000 7f8aeaa00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 0x7f8ad006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:31:59.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.158+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f8ae4089f10 con 0x7f8af415e080 2024-05-10T03:31:59.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.159+0000 7f8aebe00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 0x7f8ad006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:31:59.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.160+0000 7f8aebe00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 0x7f8ad006d110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001850 tx=0x7f8adc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:31:59.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.162+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8ae4054eb0 con 0x7f8af415e080 2024-05-10T03:31:59.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:31:59.359+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8af4067220 con 0x7f8af415e080 2024-05-10T03:32:00.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.195+0000 7f8aeaa00640 1 -- 172.21.15.55:0/1957615451 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ae40584d0 con 0x7f8af415e080 2024-05-10T03:32:00.200 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:32:00.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 msgr2=0x7f8ad006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 0x7f8ad006d110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001850 tx=0x7f8adc009040 comp rx=0 tx=0).stop 2024-05-10T03:32:00.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 msgr2=0x7f8af4151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4011ad0 tx=0x7f8ae4011f90 comp rx=0 tx=0).stop 2024-05-10T03:32:00.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 shutdown_connections 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af40cce70 0x7f8af41f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad006ac00 0x7f8ad006d110 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af415d6f0 0x7f8af40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 --2- 172.21.15.55:0/1957615451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af415e080 0x7f8af4151020 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.200+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 >> 172.21.15.55:0/1957615451 conn(0x7f8af406c070 msgr2=0x7f8af406fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.201+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 shutdown_connections 2024-05-10T03:32:00.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.201+0000 7f8af3e00640 1 -- 172.21.15.55:0/1957615451 wait complete. 2024-05-10T03:32:00.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:32:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.506+0000 7f151c600640 1 Processor -- start 2024-05-10T03:32:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.506+0000 7f151c600640 1 -- start start 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f151c600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15140714d0 0x7f15140cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f151c600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140ad730 con 0x7f151406b7c0 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f151c600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151406bbd0 con 0x7f151415b820 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f151c600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140cd380 con 0x7f15140714d0 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f1519c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f151406b7c0 0x7f151406a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54220/0 (socket says 172.21.15.55:54220) 2024-05-10T03:32:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f1519c00640 1 -- 172.21.15.55:0/2375877377 learned_addr learned my addr 172.21.15.55:0/2375877377 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.509+0000 7f151a600640 1 --2- 172.21.15.55:0/2375877377 >> v2:172.21.15.55:3300/0 conn(0x7f15140714d0 0x7f15140cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.510+0000 7f1513e00640 1 -- 172.21.15.55:0/2375877377 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309114404 0 0) 0x7f1508001110 con 0x7f151406b7c0 2024-05-10T03:32:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.510+0000 7f1513e00640 1 -- 172.21.15.55:0/2375877377 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14fc003520 con 0x7f151406b7c0 2024-05-10T03:32:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.510+0000 7f151a600640 1 -- 172.21.15.55:0/2375877377 >> v1:172.21.15.162:6789/0 conn(0x7f151415b820 legacy=0x7f1514070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.510+0000 7f151a600640 1 -- 172.21.15.55:0/2375877377 >> v1:172.21.15.55:6789/0 conn(0x7f151406b7c0 legacy=0x7f151406a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.510+0000 7f151a600640 1 -- 172.21.15.55:0/2375877377 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151415bc30 con 0x7f15140714d0 2024-05-10T03:32:00.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.511+0000 7f151a600640 1 --2- 172.21.15.55:0/2375877377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140cce40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f150c009d90 tx=0x7f150c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d09e77b767a002 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:00.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.512+0000 7f1513e00640 1 -- 172.21.15.55:0/2375877377 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c007740 con 0x7f15140714d0 2024-05-10T03:32:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.512+0000 7f1513e00640 1 -- 172.21.15.55:0/2375877377 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c02d050 con 0x7f15140714d0 2024-05-10T03:32:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.512+0000 7f1513e00640 1 -- 172.21.15.55:0/2375877377 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c007ac0 con 0x7f15140714d0 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.512+0000 7f151c600640 1 -- 172.21.15.55:0/2375877377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 msgr2=0x7f15140cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.512+0000 7f151c600640 1 --2- 172.21.15.55:0/2375877377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140cce40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f150c009d90 tx=0x7f150c02a6f0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.513+0000 7f151c600640 1 -- 172.21.15.55:0/2375877377 shutdown_connections 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.513+0000 7f151c600640 1 --2- 172.21.15.55:0/2375877377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140cce40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.513+0000 7f151c600640 1 -- 172.21.15.55:0/2375877377 >> 172.21.15.55:0/2375877377 conn(0x7f15140b0cf0 msgr2=0x7f15140b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.513+0000 7f151c600640 1 -- 172.21.15.55:0/2375877377 shutdown_connections 2024-05-10T03:32:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.513+0000 7f151c600640 1 -- 172.21.15.55:0/2375877377 wait complete. 2024-05-10T03:32:00.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.515+0000 7f151c600640 1 Processor -- start 2024-05-10T03:32:00.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.515+0000 7f151c600640 1 -- start start 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 0x7f151409aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f151415b820 0x7f15141533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140c9c00 con 0x7f15140714d0 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514099e10 con 0x7f151406b7c0 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f151c600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151409a0f0 con 0x7f151415b820 2024-05-10T03:32:00.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f1519200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f1519200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42690/0 (socket says 172.21.15.55:42690) 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.517+0000 7f1519200640 1 -- 172.21.15.55:0/1357890170 learned_addr learned my addr 172.21.15.55:0/1357890170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519c00640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 0x7f151409aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f151a600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f151415b820 0x7f15141533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 -- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f151415b820 msgr2=0x7f15141533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f151415b820 0x7f15141533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 -- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 msgr2=0x7f151409aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 0x7f151409aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 -- 172.21.15.55:0/1357890170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c009a70 con 0x7f15140714d0 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519c00640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 0x7f151409aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:00.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.518+0000 7f1519200640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f150000c400 tx=0x7f150000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.519+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000052d0 con 0x7f15140714d0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.519+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1500005430 con 0x7f15140714d0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.519+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151409a3d0 con 0x7f15140714d0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.519+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500003830 con 0x7f15140714d0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.519+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151409a990 con 0x7f15140714d0 2024-05-10T03:32:00.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.520+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151409ab50 con 0x7f15140714d0 2024-05-10T03:32:00.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.521+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1500003990 con 0x7f15140714d0 2024-05-10T03:32:00.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.522+0000 7f1511600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 0x7f14f406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:00.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.522+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5677+0+0 (secure 0 0 0) 0x7f1500088810 con 0x7f15140714d0 2024-05-10T03:32:00.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.523+0000 7f1519c00640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 0x7f14f406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:00.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.524+0000 7f1519c00640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 0x7f14f406d110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1508001270 tx=0x7f1508009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:00.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.527+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1500053920 con 0x7f15140714d0 2024-05-10T03:32:00.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:00.726+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15140b1a40 con 0x7f15140714d0 2024-05-10T03:32:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.214+0000 7f1511600640 1 -- 172.21.15.55:0/1357890170 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f15000170d0 con 0x7f15140714d0 2024-05-10T03:32:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:32:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 msgr2=0x7f14f406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 0x7f14f406d110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1508001270 tx=0x7f1508009040 comp rx=0 tx=0).stop 2024-05-10T03:32:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 msgr2=0x7f15140c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f150000c400 tx=0x7f150000c8c0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 shutdown_connections 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f151415b820 0x7f15141533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15140714d0 0x7f15140c90a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f14f406ac00 0x7f14f406d110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 --2- 172.21.15.55:0/1357890170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f151406b7c0 0x7f151409aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.218+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 >> 172.21.15.55:0/1357890170 conn(0x7f15140b0cf0 msgr2=0x7f15140b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.219+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 shutdown_connections 2024-05-10T03:32:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.219+0000 7f151c600640 1 -- 172.21.15.55:0/1357890170 wait complete. 2024-05-10T03:32:01.254 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:32:01.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.531+0000 7f0d6e600640 1 Processor -- start 2024-05-10T03:32:01.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.531+0000 7f0d6e600640 1 -- start start 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.532+0000 7f0d6e600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d680b6c80 0x7f0d6815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.532+0000 7f0d6e600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d680b8be0 con 0x7f0d681611c0 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.532+0000 7f0d6e600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6806a110 con 0x7f0d680b87d0 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.532+0000 7f0d6e600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680b60a0 con 0x7f0d680b6c80 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d6d200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0d681611c0 0x7f0d6806b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54230/0 (socket says 172.21.15.55:54230) 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d6d200640 1 -- 172.21.15.55:0/3155903603 learned_addr learned my addr 172.21.15.55:0/3155903603 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:01.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d6dc00640 1 --2- 172.21.15.55:0/3155903603 >> v2:172.21.15.55:3300/0 conn(0x7f0d680b6c80 0x7f0d6815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d67400640 1 -- 172.21.15.55:0/3155903603 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842257648 0 0) 0x7f0d680b8be0 con 0x7f0d681611c0 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d67400640 1 -- 172.21.15.55:0/3155903603 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d4c003520 con 0x7f0d681611c0 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d6dc00640 1 -- 172.21.15.55:0/3155903603 >> v1:172.21.15.162:6789/0 conn(0x7f0d680b87d0 legacy=0x7f0d680b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.533+0000 7f0d6dc00640 1 -- 172.21.15.55:0/3155903603 >> v1:172.21.15.55:6789/0 conn(0x7f0d681611c0 legacy=0x7f0d6806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.534+0000 7f0d6dc00640 1 -- 172.21.15.55:0/3155903603 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6806a3f0 con 0x7f0d680b6c80 2024-05-10T03:32:01.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.535+0000 7f0d6dc00640 1 --2- 172.21.15.55:0/3155903603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815ccf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009d90 tx=0x7f0d6002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a21030a525cff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.535+0000 7f0d67400640 1 -- 172.21.15.55:0/3155903603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60007740 con 0x7f0d680b6c80 2024-05-10T03:32:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.535+0000 7f0d67400640 1 -- 172.21.15.55:0/3155903603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6002d050 con 0x7f0d680b6c80 2024-05-10T03:32:01.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.535+0000 7f0d67400640 1 -- 172.21.15.55:0/3155903603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60007ac0 con 0x7f0d680b6c80 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 -- 172.21.15.55:0/3155903603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 msgr2=0x7f0d6815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3155903603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815ccf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009d90 tx=0x7f0d6002a6f0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 -- 172.21.15.55:0/3155903603 shutdown_connections 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3155903603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815ccf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 -- 172.21.15.55:0/3155903603 >> 172.21.15.55:0/3155903603 conn(0x7f0d680d06f0 msgr2=0x7f0d680d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 -- 172.21.15.55:0/3155903603 shutdown_connections 2024-05-10T03:32:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.536+0000 7f0d6e600640 1 -- 172.21.15.55:0/3155903603 wait complete. 2024-05-10T03:32:01.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.538+0000 7f0d6e600640 1 Processor -- start 2024-05-10T03:32:01.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.538+0000 7f0d6e600640 1 -- start start 2024-05-10T03:32:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d680b87d0 0x7f0d680c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d6815ee10 0x7f0d680ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680ab2c0 con 0x7f0d680b6c80 2024-05-10T03:32:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680b1720 con 0x7f0d680b87d0 2024-05-10T03:32:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680a9af0 con 0x7f0d6815ee10 2024-05-10T03:32:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42702/0 (socket says 172.21.15.55:42702) 2024-05-10T03:32:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 -- 172.21.15.55:0/3209339725 learned_addr learned my addr 172.21.15.55:0/3209339725 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6dc00640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d6815ee10 0x7f0d680ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d67e00640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d680b87d0 0x7f0d680c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 -- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d6815ee10 msgr2=0x7f0d680ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d6815ee10 0x7f0d680ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 -- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d680b87d0 msgr2=0x7f0d680c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d680b87d0 0x7f0d680c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 -- 172.21.15.55:0/3209339725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009a70 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.541+0000 7f0d6d200640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800c410 tx=0x7f0d5800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.542+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58005ce0 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.542+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d58005e40 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.542+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580072f0 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.542+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680cc580 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.542+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680cc820 con 0x7f0d680b6c80 2024-05-10T03:32:01.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.543+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68067740 con 0x7f0d680b6c80 2024-05-10T03:32:01.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.544+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d58007450 con 0x7f0d680b6c80 2024-05-10T03:32:01.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.545+0000 7f0d64c00640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 0x7f0d4406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:01.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.545+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0d580888e0 con 0x7f0d680b6c80 2024-05-10T03:32:01.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.546+0000 7f0d67e00640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 0x7f0d4406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.547+0000 7f0d67e00640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 0x7f0d4406d110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007fd0 tx=0x7f0d50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:01.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.550+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0d58053c10 con 0x7f0d680b6c80 2024-05-10T03:32:01.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.752+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0d680a9c60 con 0x7f0d680b6c80 2024-05-10T03:32:01.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.763+0000 7f0d64c00640 1 -- 172.21.15.55:0/3209339725 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d580170a0 con 0x7f0d680b6c80 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 msgr2=0x7f0d4406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 0x7f0d4406d110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007fd0 tx=0x7f0d50009040 comp rx=0 tx=0).stop 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 msgr2=0x7f0d6815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800c410 tx=0x7f0d5800c8d0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 shutdown_connections 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d6815ee10 0x7f0d680ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0d4406ac00 0x7f0d4406d110 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d680b87d0 0x7f0d680c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 --2- 172.21.15.55:0/3209339725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d680b6c80 0x7f0d6815e640 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 >> 172.21.15.55:0/3209339725 conn(0x7f0d680d06f0 msgr2=0x7f0d680cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 shutdown_connections 2024-05-10T03:32:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:01.766+0000 7f0d6e600640 1 -- 172.21.15.55:0/3209339725 wait complete. 2024-05-10T03:32:01.795 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-05-10T03:32:02.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.075+0000 7f3a96c00640 1 Processor -- start 2024-05-10T03:32:02.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.076+0000 7f3a96c00640 1 -- start start 2024-05-10T03:32:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a900713f0 0x7f3a90071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9015a8c0 con 0x7f3a900b1f60 2024-05-10T03:32:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a90070fd0 con 0x7f3a900713f0 2024-05-10T03:32:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90070350 con 0x7f3a900cd940 2024-05-10T03:32:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a94e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a900713f0 0x7f3a90071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3a900b1f60 0x7f3a900ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54242/0 (socket says 172.21.15.55:54242) 2024-05-10T03:32:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a96200640 1 -- 172.21.15.55:0/2912668672 learned_addr learned my addr 172.21.15.55:0/2912668672 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a95800640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a94e00640 1 -- 172.21.15.55:0/2912668672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 msgr2=0x7f3a9015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a94e00640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a94e00640 1 -- 172.21.15.55:0/2912668672 >> v1:172.21.15.55:6789/0 conn(0x7f3a900b1f60 legacy=0x7f3a900ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a94e00640 1 -- 172.21.15.55:0/2912668672 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90070630 con 0x7f3a900713f0 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a95800640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:02.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.078+0000 7f3a8fe00640 1 -- 172.21.15.55:0/2912668672 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062514539 0 0) 0x7f3a9015a8c0 con 0x7f3a900b1f60 2024-05-10T03:32:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.079+0000 7f3a94e00640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900713f0 0x7f3a90071800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009d90 tx=0x7f3a8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9643968f05971803 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.080+0000 7f3a8fe00640 1 -- 172.21.15.55:0/2912668672 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007740 con 0x7f3a900713f0 2024-05-10T03:32:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.080+0000 7f3a8fe00640 1 -- 172.21.15.55:0/2912668672 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8402d050 con 0x7f3a900713f0 2024-05-10T03:32:02.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.080+0000 7f3a8fe00640 1 -- 172.21.15.55:0/2912668672 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007ac0 con 0x7f3a900713f0 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.080+0000 7f3a96c00640 1 -- 172.21.15.55:0/2912668672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900713f0 msgr2=0x7f3a90071800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.080+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900713f0 0x7f3a90071800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009d90 tx=0x7f3a8402a660 comp rx=0 tx=0).stop 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 -- 172.21.15.55:0/2912668672 shutdown_connections 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900713f0 0x7f3a90071800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2912668672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 -- 172.21.15.55:0/2912668672 >> 172.21.15.55:0/2912668672 conn(0x7f3a900b1290 msgr2=0x7f3a9006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 -- 172.21.15.55:0/2912668672 shutdown_connections 2024-05-10T03:32:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.081+0000 7f3a96c00640 1 -- 172.21.15.55:0/2912668672 wait complete. 2024-05-10T03:32:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.084+0000 7f3a96c00640 1 Processor -- start 2024-05-10T03:32:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.084+0000 7f3a96c00640 1 -- start start 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a900713f0 0x7f3a901f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90154830 con 0x7f3a900b1f60 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901e5760 con 0x7f3a900713f0 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a96c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901f6760 con 0x7f3a900cd940 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a94e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a94e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42718/0 (socket says 172.21.15.55:42718) 2024-05-10T03:32:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.085+0000 7f3a94e00640 1 -- 172.21.15.55:0/2286430788 learned_addr learned my addr 172.21.15.55:0/2286430788 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 -- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 msgr2=0x7f3a9015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a95800640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a900713f0 0x7f3a901f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 -- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a900713f0 msgr2=0x7f3a901f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a900713f0 0x7f3a901f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 -- 172.21.15.55:0/2286430788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84009a70 con 0x7f3a900b1f60 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a94e00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009eb0 tx=0x7f3a84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031950 con 0x7f3a900b1f60 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84031ab0 con 0x7f3a900b1f60 2024-05-10T03:32:02.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031e20 con 0x7f3a900b1f60 2024-05-10T03:32:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.086+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901f3720 con 0x7f3a900b1f60 2024-05-10T03:32:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.087+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a901f3ce0 con 0x7f3a900b1f60 2024-05-10T03:32:02.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.088+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9004e290 con 0x7f3a900b1f60 2024-05-10T03:32:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.089+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a8403b410 con 0x7f3a900b1f60 2024-05-10T03:32:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.091+0000 7f3a8d600640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 0x7f3a7406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.091+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3a840aa5c0 con 0x7f3a900b1f60 2024-05-10T03:32:02.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.091+0000 7f3a95800640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 0x7f3a7406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.092+0000 7f3a95800640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 0x7f3a7406d110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a80006fd0 tx=0x7f3a80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.094+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a84075830 con 0x7f3a900b1f60 2024-05-10T03:32:02.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.298+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) v1 -- 0x7f3a9015d0d0 con 0x7f3a900b1f60 2024-05-10T03:32:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.308+0000 7f3a8d600640 1 -- 172.21.15.55:0/2286430788 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v13) v1 ==== 106+0+0 (secure 0 0 0) 0x7f3a84078e50 con 0x7f3a900b1f60 2024-05-10T03:32:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 msgr2=0x7f3a7406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 0x7f3a7406d110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a80006fd0 tx=0x7f3a80008040 comp rx=0 tx=0).stop 2024-05-10T03:32:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 msgr2=0x7f3a901542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009eb0 tx=0x7f3a84004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 shutdown_connections 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a900cd940 0x7f3a9015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a900b1f60 0x7f3a901542f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a7406ac00 0x7f3a7406d110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 --2- 172.21.15.55:0/2286430788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a900713f0 0x7f3a901f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 >> 172.21.15.55:0/2286430788 conn(0x7f3a900b1290 msgr2=0x7f3a900b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 shutdown_connections 2024-05-10T03:32:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.312+0000 7f3a96c00640 1 -- 172.21.15.55:0/2286430788 wait complete. 2024-05-10T03:32:02.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-05-10T03:32:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.614+0000 7f2246a00640 1 Processor -- start 2024-05-10T03:32:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.614+0000 7f2246a00640 1 -- start start 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.616+0000 7f2246a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f224015fe50 0x7f2240141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.616+0000 7f2246a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22400b2550 0x7f224006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2246a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224006ad20 con 0x7f22400b2140 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2246a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2240141f50 con 0x7f224015fe50 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2246a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224006a110 con 0x7f22400b2550 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2237e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f22400b2140 0x7f2240142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54250/0 (socket says 172.21.15.55:54250) 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2237e00640 1 -- 172.21.15.55:0/662152834 learned_addr learned my addr 172.21.15.55:0/662152834 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f223fe00640 1 --2- 172.21.15.55:0/662152834 >> v2:172.21.15.55:3300/0 conn(0x7f224015fe50 0x7f2240141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f2244a00640 1 --2- 172.21.15.55:0/662152834 >> v2:172.21.15.162:3300/0 conn(0x7f22400b2550 0x7f224006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.617+0000 7f223f400640 1 -- 172.21.15.55:0/662152834 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288348262 0 0) 0x7f224006ad20 con 0x7f22400b2140 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.618+0000 7f223fe00640 1 -- 172.21.15.55:0/662152834 >> v2:172.21.15.162:3300/0 conn(0x7f22400b2550 msgr2=0x7f224006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.618+0000 7f223fe00640 1 --2- 172.21.15.55:0/662152834 >> v2:172.21.15.162:3300/0 conn(0x7f22400b2550 0x7f224006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.618+0000 7f223fe00640 1 -- 172.21.15.55:0/662152834 >> v1:172.21.15.55:6789/0 conn(0x7f22400b2140 legacy=0x7f2240142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.618+0000 7f223fe00640 1 -- 172.21.15.55:0/662152834 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224006a3f0 con 0x7f224015fe50 2024-05-10T03:32:02.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.619+0000 7f223fe00640 1 --2- 172.21.15.55:0/662152834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f224015fe50 0x7f2240141a10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f224006d130 tx=0x7f223002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885a6a65ae96860 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.619+0000 7f223f400640 1 -- 172.21.15.55:0/662152834 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230032440 con 0x7f224015fe50 2024-05-10T03:32:02.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.619+0000 7f223f400640 1 -- 172.21.15.55:0/662152834 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22300325a0 con 0x7f224015fe50 2024-05-10T03:32:02.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.619+0000 7f223f400640 1 -- 172.21.15.55:0/662152834 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230032930 con 0x7f224015fe50 2024-05-10T03:32:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 -- 172.21.15.55:0/662152834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f224015fe50 msgr2=0x7f2240141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 --2- 172.21.15.55:0/662152834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f224015fe50 0x7f2240141a10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f224006d130 tx=0x7f223002a250 comp rx=0 tx=0).stop 2024-05-10T03:32:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 -- 172.21.15.55:0/662152834 shutdown_connections 2024-05-10T03:32:02.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 --2- 172.21.15.55:0/662152834 >> v2:172.21.15.162:3300/0 conn(0x7f22400b2550 0x7f224006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 --2- 172.21.15.55:0/662152834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f224015fe50 0x7f2240141a10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 -- 172.21.15.55:0/662152834 >> 172.21.15.55:0/662152834 conn(0x7f22400cbb90 msgr2=0x7f22400cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:02.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 -- 172.21.15.55:0/662152834 shutdown_connections 2024-05-10T03:32:02.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.620+0000 7f2246a00640 1 -- 172.21.15.55:0/662152834 wait complete. 2024-05-10T03:32:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.623+0000 7f2246a00640 1 Processor -- start 2024-05-10T03:32:02.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.623+0000 7f2246a00640 1 -- start start 2024-05-10T03:32:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 0x7f22400ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f224015fe50 0x7f22400ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400ad4d0 con 0x7f22400b2140 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401e55d0 con 0x7f22400b2550 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f2246a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224015d210 con 0x7f224015fe50 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f223fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.624+0000 7f223fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42728/0 (socket says 172.21.15.55:42728) 2024-05-10T03:32:02.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 -- 172.21.15.55:0/451204332 learned_addr learned my addr 172.21.15.55:0/451204332 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 -- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f224015fe50 msgr2=0x7f22400ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f224015fe50 0x7f22400ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f2237e00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 0x7f22400ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 -- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 msgr2=0x7f22400ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:02.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 0x7f22400ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 -- 172.21.15.55:0/451204332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2230009400 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f2237e00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 0x7f22400ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.625+0000 7f223fe00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2230009890 tx=0x7f22300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.626+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230004230 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.626+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22300043b0 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.626+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230004520 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.626+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224013cfe0 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.626+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22400acb30 con 0x7f22400b2140 2024-05-10T03:32:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.627+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22400ad610 con 0x7f22400b2140 2024-05-10T03:32:02.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.628+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2230039050 con 0x7f22400b2140 2024-05-10T03:32:02.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.629+0000 7f223cc00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 0x7f221c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:02.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.629+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f22300a98f0 con 0x7f22400b2140 2024-05-10T03:32:02.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.630+0000 7f2237e00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 0x7f221c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:02.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.631+0000 7f2237e00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 0x7f221c06d110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22400b09f0 tx=0x7f2228009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:02.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.635+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f22400ad610 con 0x7f22400b2140 2024-05-10T03:32:02.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:02.909+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) v1 -- 0x7f22400ad610 con 0x7f22400b2140 2024-05-10T03:32:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.216+0000 7f223cc00640 1 -- 172.21.15.55:0/451204332 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v158) v1 ==== 144+0+0 (secure 0 0 0) 0x7f22400ad610 con 0x7f22400b2140 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 msgr2=0x7f221c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 0x7f221c06d110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f22400b09f0 tx=0x7f2228009040 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 msgr2=0x7f224015af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2230009890 tx=0x7f22300046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 shutdown_connections 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f224015fe50 0x7f22400ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f221c06ac00 0x7f221c06d110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22400b2550 0x7f22400ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 --2- 172.21.15.55:0/451204332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22400b2140 0x7f224015af00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.219+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 >> 172.21.15.55:0/451204332 conn(0x7f22400cbb90 msgr2=0x7f2240140e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.220+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 shutdown_connections 2024-05-10T03:32:03.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.220+0000 7f2246a00640 1 -- 172.21.15.55:0/451204332 wait complete. 2024-05-10T03:32:03.253 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2024-05-10T03:32:03.254 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2024-05-10T03:32:03.254 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:32:03.254 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:03.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.540+0000 7fa38f200640 1 Processor -- start 2024-05-10T03:32:03.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.540+0000 7fa38f200640 1 -- start start 2024-05-10T03:32:03.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.542+0000 7fa38f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa39015e190 0x7fa3900ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.542+0000 7fa38f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3900b2370 0x7fa39015a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.542+0000 7fa38f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39015d740 con 0x7fa3900b1f60 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3900b2780 con 0x7fa3900b2370 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015ada0 con 0x7fa39015e190 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38e800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa3900b2370 0x7fa39015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa385400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa3900b1f60 0x7fa39015a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38948/0 (socket says 172.21.15.55:38948) 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa385400640 1 -- 172.21.15.55:0/1389328698 learned_addr learned my addr 172.21.15.55:0/1389328698 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38de00640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa39015e190 0x7fa3900ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38e800640 1 -- 172.21.15.55:0/1389328698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa39015e190 msgr2=0x7fa3900ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38e800640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa39015e190 0x7fa3900ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38e800640 1 -- 172.21.15.55:0/1389328698 >> v1:172.21.15.55:6789/0 conn(0x7fa3900b1f60 legacy=0x7fa39015a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38e800640 1 -- 172.21.15.55:0/1389328698 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900b2a20 con 0x7fa3900b2370 2024-05-10T03:32:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.543+0000 7fa38d400640 1 -- 172.21.15.55:0/1389328698 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648801166 0 0) 0x7fa39015d740 con 0x7fa3900b1f60 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.545+0000 7fa38e800640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3900b2370 0x7fa39015a860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa388009d90 tx=0x7fa38802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cace7de84a3a9f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.545+0000 7fa38d400640 1 -- 172.21.15.55:0/1389328698 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388031600 con 0x7fa3900b2370 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.545+0000 7fa38d400640 1 -- 172.21.15.55:0/1389328698 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38802d050 con 0x7fa3900b2370 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.545+0000 7fa38d400640 1 -- 172.21.15.55:0/1389328698 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388031980 con 0x7fa3900b2370 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 -- 172.21.15.55:0/1389328698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3900b2370 msgr2=0x7fa39015a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3900b2370 0x7fa39015a860 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa388009d90 tx=0x7fa38802a5d0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 -- 172.21.15.55:0/1389328698 shutdown_connections 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3900b2370 0x7fa39015a860 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 --2- 172.21.15.55:0/1389328698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa39015e190 0x7fa3900ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 -- 172.21.15.55:0/1389328698 >> 172.21.15.55:0/1389328698 conn(0x7fa39006a110 msgr2=0x7fa39006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:03.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 -- 172.21.15.55:0/1389328698 shutdown_connections 2024-05-10T03:32:03.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.546+0000 7fa38f200640 1 -- 172.21.15.55:0/1389328698 wait complete. 2024-05-10T03:32:03.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.548+0000 7fa38f200640 1 Processor -- start 2024-05-10T03:32:03.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.548+0000 7fa38f200640 1 -- start start 2024-05-10T03:32:03.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3900b1f60 0x7fa3901549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3900b2370 0x7fa3901e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901f4900 con 0x7fa39015e190 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901e7fe0 con 0x7fa3900b1f60 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901f6820 con 0x7fa3900b2370 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.550+0000 7fa38e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41000/0 (socket says 172.21.15.55:41000) 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 -- 172.21.15.55:0/2524232754 learned_addr learned my addr 172.21.15.55:0/2524232754 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa385400640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3900b2370 0x7fa3901e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38de00640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3900b1f60 0x7fa3901549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 -- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3900b2370 msgr2=0x7fa3901e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3900b2370 0x7fa3901e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 -- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3900b1f60 msgr2=0x7fa3901549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3900b1f60 0x7fa3901549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 -- 172.21.15.55:0/2524232754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388009a70 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.551+0000 7fa38e800640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa388009eb0 tx=0x7fa388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.552+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38803b8e0 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.552+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38803ba40 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.552+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38803bdd0 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.552+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3901e8230 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.552+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3901e87f0 con 0x7fa39015e190 2024-05-10T03:32:03.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.553+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390066ce0 con 0x7fa39015e190 2024-05-10T03:32:03.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.554+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa38804d3f0 con 0x7fa39015e190 2024-05-10T03:32:03.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.555+0000 7fa386a00640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 0x7fa36c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:03.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.555+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa3880aa810 con 0x7fa39015e190 2024-05-10T03:32:03.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.556+0000 7fa38de00640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 0x7fa36c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:03.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.557+0000 7fa38de00640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 0x7fa36c06d160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:03.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.560+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa388075c90 con 0x7fa39015e190 2024-05-10T03:32:03.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.760+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa39004e290 con 0x7fa39015e190 2024-05-10T03:32:03.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.762+0000 7fa386a00640 1 -- 172.21.15.55:0/2524232754 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12721 (secure 0 0 0) 0x7fa3880792b0 con 0x7fa39015e190 2024-05-10T03:32:03.766 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:03.766 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":43,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:01.186113+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6835/455504925":"2024-05-11T03:31:57.805136+0000","172.21.15.55:6834/455504925":"2024-05-11T03:31:57.805136+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 msgr2=0x7fa36c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 0x7fa36c06d160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380008040 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 msgr2=0x7fa3901f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa388009eb0 tx=0x7fa388004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 shutdown_connections 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa39015e190 0x7fa3901f62e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3900b2370 0x7fa3901e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa36c06ac50 0x7fa36c06d160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 --2- 172.21.15.55:0/2524232754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3900b1f60 0x7fa3901549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.765+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 >> 172.21.15.55:0/2524232754 conn(0x7fa39006a110 msgr2=0x7fa3900b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.766+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 shutdown_connections 2024-05-10T03:32:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:03.766+0000 7fa38f200640 1 -- 172.21.15.55:0/2524232754 wait complete. 2024-05-10T03:32:03.798 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:03.798 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.085+0000 7f671d800640 1 Processor -- start 2024-05-10T03:32:04.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.085+0000 7f671d800640 1 -- start start 2024-05-10T03:32:04.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 0x7f671806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671d800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f671806b840 0x7f67180700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671d800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671815fdf0 con 0x7f6718071e00 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671d800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671815f1e0 con 0x7f671806b840 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671815f4c0 con 0x7f67180719f0 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f670fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f671806b840 0x7f67180700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671ce00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6718071e00 0x7f6718070620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38962/0 (socket says 172.21.15.55:38962) 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.088+0000 7f671ce00640 1 -- 172.21.15.55:0/1601365263 learned_addr learned my addr 172.21.15.55:0/1601365263 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f6717e00640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 0x7f671806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f670fe00640 1 -- 172.21.15.55:0/1601365263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 msgr2=0x7f671806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f670fe00640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 0x7f671806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f670fe00640 1 -- 172.21.15.55:0/1601365263 >> v1:172.21.15.55:6789/0 conn(0x7f6718071e00 legacy=0x7f6718070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f670fe00640 1 -- 172.21.15.55:0/1601365263 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671815f7a0 con 0x7f671806b840 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f6717400640 1 -- 172.21.15.55:0/1601365263 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499588468 0 0) 0x7f671815fdf0 con 0x7f6718071e00 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.089+0000 7f6717e00640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 0x7f671806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.090+0000 7f670fe00640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f671806b840 0x7f67180700e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f67000097a0 tx=0x7f670002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c779369a2ffdf39d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.091+0000 7f6717400640 1 -- 172.21.15.55:0/1601365263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670002ae50 con 0x7f671806b840 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.091+0000 7f6717400640 1 -- 172.21.15.55:0/1601365263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700032440 con 0x7f671806b840 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.091+0000 7f6717400640 1 -- 172.21.15.55:0/1601365263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000327d0 con 0x7f671806b840 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.091+0000 7f671d800640 1 -- 172.21.15.55:0/1601365263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f671806b840 msgr2=0x7f67180700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.091+0000 7f671d800640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f671806b840 0x7f67180700e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f67000097a0 tx=0x7f670002a100 comp rx=0 tx=0).stop 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 -- 172.21.15.55:0/1601365263 shutdown_connections 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f671806b840 0x7f67180700e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 --2- 172.21.15.55:0/1601365263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67180719f0 0x7f671806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 -- 172.21.15.55:0/1601365263 >> 172.21.15.55:0/1601365263 conn(0x7f671806c750 msgr2=0x7f67180b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 -- 172.21.15.55:0/1601365263 shutdown_connections 2024-05-10T03:32:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.092+0000 7f671d800640 1 -- 172.21.15.55:0/1601365263 wait complete. 2024-05-10T03:32:04.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.094+0000 7f671d800640 1 Processor -- start 2024-05-10T03:32:04.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.094+0000 7f671d800640 1 -- start start 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671806b840 0x7f6718137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67180719f0 0x7f67180b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67181f4460 con 0x7f6718071e00 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180b3eb0 con 0x7f67180719f0 2024-05-10T03:32:04.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718137a60 con 0x7f671806b840 2024-05-10T03:32:04.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41030/0 (socket says 172.21.15.55:41030) 2024-05-10T03:32:04.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f671ce00640 1 -- 172.21.15.55:0/409712670 learned_addr learned my addr 172.21.15.55:0/409712670 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.096+0000 7f670fe00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67180719f0 0x7f67180b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f6717e00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671806b840 0x7f6718137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 -- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671806b840 msgr2=0x7f6718137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671806b840 0x7f6718137520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 -- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67180719f0 msgr2=0x7f67180b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67180719f0 0x7f67180b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 -- 172.21.15.55:0/409712670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009400 con 0x7f6718071e00 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671ce00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f671000c780 tx=0x7f671000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000ce30 con 0x7f6718071e00 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000a640 con 0x7f6718071e00 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000aa30 con 0x7f6718071e00 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67180c3df0 con 0x7f6718071e00 2024-05-10T03:32:04.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.097+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67180c43b0 con 0x7f6718071e00 2024-05-10T03:32:04.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.099+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671804e290 con 0x7f6718071e00 2024-05-10T03:32:04.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.100+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f671000ab90 con 0x7f6718071e00 2024-05-10T03:32:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.101+0000 7f6714c00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 0x7f66f406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.101+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f67100886d0 con 0x7f6718071e00 2024-05-10T03:32:04.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.101+0000 7f6717e00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 0x7f66f406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.103+0000 7f6717e00640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 0x7f66f406d110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67080096d0 tx=0x7f6708008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.105+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f67100539d0 con 0x7f6718071e00 2024-05-10T03:32:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.304+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67180cd370 con 0x7f6718071e00 2024-05-10T03:32:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.305+0000 7f6714c00640 1 -- 172.21.15.55:0/409712670 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6710018020 con 0x7f6718071e00 2024-05-10T03:32:04.308 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 msgr2=0x7f66f406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 0x7f66f406d110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67080096d0 tx=0x7f6708008040 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 msgr2=0x7f67180ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f671000c780 tx=0x7f671000cc40 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 shutdown_connections 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6718071e00 0x7f67180ad930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f66f406ac00 0x7f66f406d110 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67180719f0 0x7f67180b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 --2- 172.21.15.55:0/409712670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671806b840 0x7f6718137520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.308+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 >> 172.21.15.55:0/409712670 conn(0x7f671806c750 msgr2=0x7f6718143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.309+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 shutdown_connections 2024-05-10T03:32:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.309+0000 7f671d800640 1 -- 172.21.15.55:0/409712670 wait complete. 2024-05-10T03:32:04.343 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-05-10T03:32:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.624+0000 7f4a13e00640 1 Processor -- start 2024-05-10T03:32:04.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.624+0000 7f4a13e00640 1 -- start start 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.626+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140ce5f0 0x7f4a14157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.626+0000 7f4a13e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4a141613c0 0x7f4a140b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a13e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1415edc0 con 0x7f4a1415ddd0 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a13e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a141623d0 con 0x7f4a141613c0 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a13e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415e1e0 con 0x7f4a140ce5f0 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a12a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4a1415ddd0 0x7f4a140cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38978/0 (socket says 172.21.15.55:38978) 2024-05-10T03:32:04.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a12a00640 1 -- 172.21.15.55:0/1171485838 learned_addr learned my addr 172.21.15.55:0/1171485838 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a13400640 1 --2- 172.21.15.55:0/1171485838 >> v2:172.21.15.162:3300/0 conn(0x7f4a141613c0 0x7f4a140b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345498700 0 0) 0x7f4a1415edc0 con 0x7f4a1415ddd0 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f8003520 con 0x7f4a1415ddd0 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.627+0000 7f4a0a000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140ce5f0 0x7f4a14157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456912757 0 0) 0x7f49f8003520 con 0x7f4a1415ddd0 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140ce5f0 msgr2=0x7f4a14157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140ce5f0 0x7f4a14157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 >> v2:172.21.15.162:3300/0 conn(0x7f4a141613c0 msgr2=0x7f4a140b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 --2- 172.21.15.55:0/1171485838 >> v2:172.21.15.162:3300/0 conn(0x7f4a141613c0 0x7f4a140b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140cecd0 con 0x7f4a1415ddd0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f49fc001670 con 0x7f4a1415ddd0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 >> v1:172.21.15.55:6789/0 conn(0x7f4a1415ddd0 legacy=0x7f4a140cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f80039e0 0x7f49f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49f80047c0 0x7f49f8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f8005110 0x7f49f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141623d0 con 0x7f49f80039e0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415e1e0 con 0x7f49f8005110 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f800bef0 con 0x7f49f80047c0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.628+0000 7f4a12a00640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f80039e0 0x7f49f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a0a000640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49f80047c0 0x7f49f8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a12a00640 1 -- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49f80047c0 msgr2=0x7f49f8004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a12a00640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49f80047c0 0x7f49f8004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a12a00640 1 -- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f8005110 msgr2=0x7f49f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:04.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a12a00640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f8005110 0x7f49f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.629+0000 7f4a12a00640 1 -- 172.21.15.55:0/1171485838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8003520 con 0x7f49f80039e0 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.630+0000 7f4a12a00640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f80039e0 0x7f49f8003e50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00c210 tx=0x7f49f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b617de90d9d608f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.630+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02e070 con 0x7f49f80039e0 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.630+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc0058d0 con 0x7f49f80039e0 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.630+0000 7f4a12000640 1 -- 172.21.15.55:0/1171485838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc005ca0 con 0x7f49f80039e0 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 -- 172.21.15.55:0/1171485838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f80039e0 msgr2=0x7f49f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 --2- 172.21.15.55:0/1171485838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f80039e0 0x7f49f8003e50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00c210 tx=0x7f49f80057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13400640 1 -- 172.21.15.55:0/1171485838 reap_dead start 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 -- 172.21.15.55:0/1171485838 shutdown_connections 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 -- 172.21.15.55:0/1171485838 >> 172.21.15.55:0/1171485838 conn(0x7f4a140d0370 msgr2=0x7f4a140d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 -- 172.21.15.55:0/1171485838 shutdown_connections 2024-05-10T03:32:04.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.631+0000 7f4a13e00640 1 -- 172.21.15.55:0/1171485838 wait complete. 2024-05-10T03:32:04.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.634+0000 7f4a13e00640 1 Processor -- start 2024-05-10T03:32:04.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.634+0000 7f4a13e00640 1 -- start start 2024-05-10T03:32:04.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f80039e0 0x7f4a1414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140aa440 0x7f4a140b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141f4bd0 con 0x7f4a140b3500 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415a340 con 0x7f49f80039e0 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.635+0000 7f4a13e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415b1a0 con 0x7f4a140aa440 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41048/0 (socket says 172.21.15.55:41048) 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 -- 172.21.15.55:0/3408553585 learned_addr learned my addr 172.21.15.55:0/3408553585 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a12a00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f80039e0 0x7f4a1414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a0a000640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140aa440 0x7f4a140b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 -- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140aa440 msgr2=0x7f4a140b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140aa440 0x7f4a140b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 -- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f80039e0 msgr2=0x7f4a1414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:04.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f80039e0 0x7f4a1414b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:04.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.636+0000 7f4a13400640 1 -- 172.21.15.55:0/3408553585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1414ba70 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a13400640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4a1415b9a0 tx=0x7f4a0c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00c680 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c00c7e0 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a140b3910 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00cb90 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.637+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1415a870 con 0x7f4a140b3500 2024-05-10T03:32:04.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.638+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1415b330 con 0x7f4a140b3500 2024-05-10T03:32:04.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.639+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a0c00ccf0 con 0x7f4a140b3500 2024-05-10T03:32:04.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.640+0000 7f4a0b400640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 0x7f49f006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:04.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.641+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4a0c088cd0 con 0x7f4a140b3500 2024-05-10T03:32:04.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.641+0000 7f4a12a00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 0x7f49f006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:04.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.643+0000 7f4a12a00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 0x7f49f006d110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49fc033000 tx=0x7f49fc009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:04.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.645+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a0c053fd0 con 0x7f4a140b3500 2024-05-10T03:32:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:04.919+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) v1 -- 0x7f4a1404e290 con 0x7f4a140b3500 2024-05-10T03:32:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.230+0000 7f4a0b400640 1 -- 172.21.15.55:0/3408553585 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v160) v1 ==== 156+0+0 (secure 0 0 0) 0x7f4a0c0575f0 con 0x7f4a140b3500 2024-05-10T03:32:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 msgr2=0x7f49f006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 0x7f49f006d110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49fc033000 tx=0x7f49fc009030 comp rx=0 tx=0).stop 2024-05-10T03:32:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 msgr2=0x7f4a141f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4a1415b9a0 tx=0x7f4a0c00ec00 comp rx=0 tx=0).stop 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 shutdown_connections 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a140b3500 0x7f4a141f3af0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a140aa440 0x7f4a140b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49f006ac00 0x7f49f006d110 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 --2- 172.21.15.55:0/3408553585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f80039e0 0x7f4a1414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 >> 172.21.15.55:0/3408553585 conn(0x7f4a140d0370 msgr2=0x7f4a14157470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 shutdown_connections 2024-05-10T03:32:05.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.234+0000 7f4a13e00640 1 -- 172.21.15.55:0/3408553585 wait complete. 2024-05-10T03:32:05.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:32:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.558+0000 7ff3ce200640 1 Processor -- start 2024-05-10T03:32:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.558+0000 7ff3ce200640 1 -- start start 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.561+0000 7ff3ce200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3c80ce260 0x7ff3c815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.561+0000 7ff3ce200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b1bc0 0x7ff3c806a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.561+0000 7ff3ce200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c80ceb00 con 0x7ff3c80b2110 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.561+0000 7ff3ce200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c815dcd0 con 0x7ff3c80ce260 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.561+0000 7ff3ce200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80d2150 con 0x7ff3c80b1bc0 2024-05-10T03:32:05.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3c80ce260 0x7ff3c815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3cd800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b1bc0 0x7ff3c806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3c80ce260 0x7ff3c815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46732/0 (socket says 172.21.15.55:46732) 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 -- 172.21.15.55:0/3736607883 learned_addr learned my addr 172.21.15.55:0/3736607883 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 -- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b1bc0 msgr2=0x7ff3c806a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 --2- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b1bc0 0x7ff3c806a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 -- 172.21.15.55:0/3736607883 >> v1:172.21.15.162:6789/0 conn(0x7ff3c80b2110 legacy=0x7ff3c80cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.562+0000 7ff3c7e00640 1 -- 172.21.15.55:0/3736607883 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80b2910 con 0x7ff3c80ce260 2024-05-10T03:32:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.564+0000 7ff3c7e00640 1 --2- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80ce260 0x7ff3c815d790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e50 tx=0x7ff3b002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9157341622deb0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.564+0000 7ff3c7400640 1 -- 172.21.15.55:0/3736607883 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b002aea0 con 0x7ff3c80ce260 2024-05-10T03:32:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.564+0000 7ff3c7400640 1 -- 172.21.15.55:0/3736607883 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b002d050 con 0x7ff3c80ce260 2024-05-10T03:32:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.564+0000 7ff3c7400640 1 -- 172.21.15.55:0/3736607883 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0031820 con 0x7ff3c80ce260 2024-05-10T03:32:05.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 -- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80ce260 msgr2=0x7ff3c815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 --2- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80ce260 0x7ff3c815d790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e50 tx=0x7ff3b002a680 comp rx=0 tx=0).stop 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 -- 172.21.15.55:0/3736607883 shutdown_connections 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 --2- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b1bc0 0x7ff3c806a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 --2- 172.21.15.55:0/3736607883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80ce260 0x7ff3c815d790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 -- 172.21.15.55:0/3736607883 >> 172.21.15.55:0/3736607883 conn(0x7ff3c80d30d0 msgr2=0x7ff3c80d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 -- 172.21.15.55:0/3736607883 shutdown_connections 2024-05-10T03:32:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.565+0000 7ff3ce200640 1 -- 172.21.15.55:0/3736607883 wait complete. 2024-05-10T03:32:05.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.568+0000 7ff3ce200640 1 Processor -- start 2024-05-10T03:32:05.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.568+0000 7ff3ce200640 1 -- start start 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 0x7ff3c815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b2110 0x7ff3c81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80cc040 con 0x7ff3c80ce260 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8160000 con 0x7ff3c80b1bc0 2024-05-10T03:32:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3ce200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c815fdb0 con 0x7ff3c80b2110 2024-05-10T03:32:05.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3cd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3cd800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41052/0 (socket says 172.21.15.55:41052) 2024-05-10T03:32:05.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3cd800640 1 -- 172.21.15.55:0/2183209589 learned_addr learned my addr 172.21.15.55:0/2183209589 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:05.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3cce00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 0x7ff3c815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.570+0000 7ff3c7e00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b2110 0x7ff3c81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 -- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b2110 msgr2=0x7ff3c81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b2110 0x7ff3c81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 -- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 msgr2=0x7ff3c815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 0x7ff3c815f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 -- 172.21.15.55:0/2183209589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009b30 con 0x7ff3c80ce260 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cce00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 0x7ff3c815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3cd800640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e7f0 tx=0x7ff3c000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0009510 con 0x7ff3c80ce260 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c000ee40 con 0x7ff3c80ce260 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c8151c10 con 0x7ff3c80ce260 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000cb80 con 0x7ff3c80ce260 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.571+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8160530 con 0x7ff3c80ce260 2024-05-10T03:32:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.573+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c804e290 con 0x7ff3c80ce260 2024-05-10T03:32:05.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.574+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff3c000cce0 con 0x7ff3c80ce260 2024-05-10T03:32:05.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.575+0000 7ff3c4c00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 0x7ff3a406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.575+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff3c0089650 con 0x7ff3c80ce260 2024-05-10T03:32:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.575+0000 7ff3cce00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 0x7ff3a406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:05.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.577+0000 7ff3cce00640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 0x7ff3a406d110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8001660 tx=0x7ff3b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:05.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.579+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff3c0054950 con 0x7ff3c80ce260 2024-05-10T03:32:05.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.777+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff3c80cd490 con 0x7ff3c80ce260 2024-05-10T03:32:05.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.787+0000 7ff3c4c00640 1 -- 172.21.15.55:0/2183209589 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3c0057f70 con 0x7ff3c80ce260 2024-05-10T03:32:05.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 msgr2=0x7ff3a406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 0x7ff3a406d110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8001660 tx=0x7ff3b8009040 comp rx=0 tx=0).stop 2024-05-10T03:32:05.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 msgr2=0x7ff3c815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e7f0 tx=0x7ff3c000ebf0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 shutdown_connections 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c80ce260 0x7ff3c815a370 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3c80b2110 0x7ff3c81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff3a406ac00 0x7ff3a406d110 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 --2- 172.21.15.55:0/2183209589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3c80b1bc0 0x7ff3c815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 >> 172.21.15.55:0/2183209589 conn(0x7ff3c80d30d0 msgr2=0x7ff3c80b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 shutdown_connections 2024-05-10T03:32:05.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:05.790+0000 7ff3ce200640 1 -- 172.21.15.55:0/2183209589 wait complete. 2024-05-10T03:32:05.818 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:32:05.819 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:05.925 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:06.039 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:32:06.039 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:06.039 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:06.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:32:06.099 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:06.099 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:06.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:32:06.156 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:06.156 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:06.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:32:06.217 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:06.217 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:06.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:32:06.274 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:32:06.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:06.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.557+0000 7f3a41600640 1 Processor -- start 2024-05-10T03:32:06.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.557+0000 7f3a41600640 1 -- start start 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a41600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a3c071db0 0x7f3a3c158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a41600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a3c06a4b0 0x7f3a3c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a41600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0708b0 con 0x7f3a3c0b3860 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a41600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b2490 con 0x7f3a3c071db0 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a41600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c070050 con 0x7f3a3c06a4b0 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a3b400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3a3c0b3860 0x7f3a3c0b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38992/0 (socket says 172.21.15.55:38992) 2024-05-10T03:32:06.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a3b400640 1 -- 172.21.15.55:0/3141371951 learned_addr learned my addr 172.21.15.55:0/3141371951 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:06.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.559+0000 7f3a3aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a3c071db0 0x7f3a3c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a3a000640 1 --2- 172.21.15.55:0/3141371951 >> v2:172.21.15.162:3300/0 conn(0x7f3a3c06a4b0 0x7f3a3c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a3aa00640 1 -- 172.21.15.55:0/3141371951 >> v2:172.21.15.162:3300/0 conn(0x7f3a3c06a4b0 msgr2=0x7f3a3c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:06.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a3aa00640 1 --2- 172.21.15.55:0/3141371951 >> v2:172.21.15.162:3300/0 conn(0x7f3a3c06a4b0 0x7f3a3c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a3aa00640 1 -- 172.21.15.55:0/3141371951 >> v1:172.21.15.55:6789/0 conn(0x7f3a3c0b3860 legacy=0x7f3a3c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a39600640 1 -- 172.21.15.55:0/3141371951 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996989075 0 0) 0x7f3a3c0708b0 con 0x7f3a3c0b3860 2024-05-10T03:32:06.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.560+0000 7f3a3aa00640 1 -- 172.21.15.55:0/3141371951 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c15b940 con 0x7f3a3c071db0 2024-05-10T03:32:06.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.561+0000 7f3a3aa00640 1 --2- 172.21.15.55:0/3141371951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c158150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0cce20 tx=0x7f3a2c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909ef096a310eee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:06.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.561+0000 7f3a39600640 1 -- 172.21.15.55:0/3141371951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c02add0 con 0x7f3a3c071db0 2024-05-10T03:32:06.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.561+0000 7f3a39600640 1 -- 172.21.15.55:0/3141371951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c032440 con 0x7f3a3c071db0 2024-05-10T03:32:06.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.561+0000 7f3a39600640 1 -- 172.21.15.55:0/3141371951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0327d0 con 0x7f3a3c071db0 2024-05-10T03:32:06.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 -- 172.21.15.55:0/3141371951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 msgr2=0x7f3a3c158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 --2- 172.21.15.55:0/3141371951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c158150 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0cce20 tx=0x7f3a2c02a080 comp rx=0 tx=0).stop 2024-05-10T03:32:06.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 -- 172.21.15.55:0/3141371951 shutdown_connections 2024-05-10T03:32:06.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 --2- 172.21.15.55:0/3141371951 >> v2:172.21.15.162:3300/0 conn(0x7f3a3c06a4b0 0x7f3a3c0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 --2- 172.21.15.55:0/3141371951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c158150 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 -- 172.21.15.55:0/3141371951 >> 172.21.15.55:0/3141371951 conn(0x7f3a3c0cbb90 msgr2=0x7f3a3c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:06.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 -- 172.21.15.55:0/3141371951 shutdown_connections 2024-05-10T03:32:06.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.562+0000 7f3a41600640 1 -- 172.21.15.55:0/3141371951 wait complete. 2024-05-10T03:32:06.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.564+0000 7f3a41600640 1 Processor -- start 2024-05-10T03:32:06.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.564+0000 7f3a41600640 1 -- start start 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a3c06a4b0 0x7f3a3c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3c0b3860 0x7f3a3c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c15bdd0 con 0x7f3a3c071db0 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c15d530 con 0x7f3a3c06a4b0 2024-05-10T03:32:06.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a41600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c0ae050 con 0x7f3a3c0b3860 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a3a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41070/0 (socket says 172.21.15.55:41070) 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.566+0000 7f3a3a000640 1 -- 172.21.15.55:0/1939132996 learned_addr learned my addr 172.21.15.55:0/1939132996 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3aa00640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a3c06a4b0 0x7f3a3c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3b400640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3c0b3860 0x7f3a3c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 -- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3c0b3860 msgr2=0x7f3a3c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3c0b3860 0x7f3a3c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 -- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a3c06a4b0 msgr2=0x7f3a3c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a3c06a4b0 0x7f3a3c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 -- 172.21.15.55:0/1939132996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c009400 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a3a000640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d5c0 tx=0x7f3a2400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.567+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a240125f0 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.568+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24012750 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.568+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012b00 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.568+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0ae330 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.568+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0ae8f0 con 0x7f3a3c071db0 2024-05-10T03:32:06.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.569+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c156c90 con 0x7f3a3c071db0 2024-05-10T03:32:06.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.570+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a24012c60 con 0x7f3a3c071db0 2024-05-10T03:32:06.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.571+0000 7f3a32a00640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 0x7f3a1806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:06.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.571+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3a24089650 con 0x7f3a3c071db0 2024-05-10T03:32:06.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.572+0000 7f3a3aa00640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 0x7f3a1806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:06.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.573+0000 7f3a3aa00640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 0x7f3a1806d110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:06.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.576+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a240549f0 con 0x7f3a3c071db0 2024-05-10T03:32:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.817+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a3c066f60 con 0x7f3a3c071db0 2024-05-10T03:32:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.818+0000 7f3a32a00640 1 -- 172.21.15.55:0/1939132996 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f3a24058010 con 0x7f3a3c071db0 2024-05-10T03:32:06.821 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:06.821 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/3083484054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3083484054},{"type":"v1","addr":"172.21.15.162:6836","nonce":3083484054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2188329377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":2188329377},{"type":"v1","addr":"172.21.15.162:6837","nonce":2188329377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1245720066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1245720066},{"type":"v1","addr":"172.21.15.55:6833","nonce":1245720066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1998180276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1998180276},{"type":"v1","addr":"172.21.15.55:6835","nonce":1998180276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2024-05-10T03:32:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 18 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 msgr2=0x7f3a1806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 0x7f3a1806d110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 msgr2=0x7f3a3c0cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d5c0 tx=0x7f3a2400da80 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 shutdown_connections 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3c0b3860 0x7f3a3c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a3c071db0 0x7f3a3c0cd7a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a1806ac00 0x7f3a1806d110 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 --2- 172.21.15.55:0/1939132996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a3c06a4b0 0x7f3a3c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:06.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.820+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 >> 172.21.15.55:0/1939132996 conn(0x7f3a3c0cbb90 msgr2=0x7f3a3c1577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.821+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 shutdown_connections 2024-05-10T03:32:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:06.821+0000 7f3a41600640 1 -- 172.21.15.55:0/1939132996 wait complete. 2024-05-10T03:32:06.849 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:32:06.850 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:32:06.850 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:32:06.850 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:06.850 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:32:06.850 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:32:06.850 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:32:06.850 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:06.851 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:32:06.851 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:32:06.851 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:32:06.851 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:06.851 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:32:06.851 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:32:06.851 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:32:06.852 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:06.879 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:32:06.879 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:32:06.882 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:32:06.883 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:32:06.884 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:32:06.884 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:32:06.889 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:32:06.889 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:32:06.929 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:32:06.929 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:06.935 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:32:06.935 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:06.936 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:32:06.936 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:06.941 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:32:06.941 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:06.941 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:32:07.152 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:32:07.154 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:32:07.161 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:32:07.161 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:32:07.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.199+0000 7fb111600640 1 Processor -- start 2024-05-10T03:32:07.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.199+0000 7fb111600640 1 -- start start 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb111600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb10c071db0 0x7fb10c158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb111600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb10c06a4b0 0x7fb10c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb111600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0708b0 con 0x7fb10c0b3860 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb111600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0b2490 con 0x7fb10c071db0 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb111600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c070050 con 0x7fb10c06a4b0 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb10aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb10c071db0 0x7fb10c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb10aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb10c071db0 0x7fb10c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41130/0 (socket says 172.21.15.55:41130) 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.201+0000 7fb10aa00640 1 -- 172.21.15.55:0/3311795783 learned_addr learned my addr 172.21.15.55:0/3311795783 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:07.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.202+0000 7fb10a000640 1 --2- 172.21.15.55:0/3311795783 >> v2:172.21.15.162:3300/0 conn(0x7fb10c06a4b0 0x7fb10c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.202+0000 7fb10aa00640 1 -- 172.21.15.55:0/3311795783 >> v2:172.21.15.162:3300/0 conn(0x7fb10c06a4b0 msgr2=0x7fb10c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:07.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.202+0000 7fb10aa00640 1 --2- 172.21.15.55:0/3311795783 >> v2:172.21.15.162:3300/0 conn(0x7fb10c06a4b0 0x7fb10c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:07.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.202+0000 7fb10aa00640 1 -- 172.21.15.55:0/3311795783 >> v1:172.21.15.55:6789/0 conn(0x7fb10c0b3860 legacy=0x7fb10c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:07.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.202+0000 7fb10aa00640 1 -- 172.21.15.55:0/3311795783 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c15b940 con 0x7fb10c071db0 2024-05-10T03:32:07.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.203+0000 7fb10aa00640 1 --2- 172.21.15.55:0/3311795783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c071db0 0x7fb10c158150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0cce20 tx=0x7fb0fc02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95472856903b650 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:07.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.203+0000 7fb109600640 1 -- 172.21.15.55:0/3311795783 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc032440 con 0x7fb10c071db0 2024-05-10T03:32:07.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.203+0000 7fb109600640 1 -- 172.21.15.55:0/3311795783 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0325a0 con 0x7fb10c071db0 2024-05-10T03:32:07.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.203+0000 7fb109600640 1 -- 172.21.15.55:0/3311795783 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc032930 con 0x7fb10c071db0 2024-05-10T03:32:07.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 -- 172.21.15.55:0/3311795783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c071db0 msgr2=0x7fb10c158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 --2- 172.21.15.55:0/3311795783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c071db0 0x7fb10c158150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0cce20 tx=0x7fb0fc02a160 comp rx=0 tx=0).stop 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 -- 172.21.15.55:0/3311795783 shutdown_connections 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 --2- 172.21.15.55:0/3311795783 >> v2:172.21.15.162:3300/0 conn(0x7fb10c06a4b0 0x7fb10c0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 --2- 172.21.15.55:0/3311795783 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c071db0 0x7fb10c158150 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 -- 172.21.15.55:0/3311795783 >> 172.21.15.55:0/3311795783 conn(0x7fb10c0cbb90 msgr2=0x7fb10c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 -- 172.21.15.55:0/3311795783 shutdown_connections 2024-05-10T03:32:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.204+0000 7fb111600640 1 -- 172.21.15.55:0/3311795783 wait complete. 2024-05-10T03:32:07.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.207+0000 7fb111600640 1 Processor -- start 2024-05-10T03:32:07.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.207+0000 7fb111600640 1 -- start start 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c071db0 0x7fb10c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b3860 0x7fb10c0a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c132640 con 0x7fb10c06a4b0 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0aabb0 con 0x7fb10c071db0 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.208+0000 7fb111600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c131770 con 0x7fb10c0b3860 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41138/0 (socket says 172.21.15.55:41138) 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 -- 172.21.15.55:0/2890961397 learned_addr learned my addr 172.21.15.55:0/2890961397 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10b400640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b3860 0x7fb10c0a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10a000640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c071db0 0x7fb10c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 -- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b3860 msgr2=0x7fb10c0a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b3860 0x7fb10c0a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 -- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c071db0 msgr2=0x7fb10c156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c071db0 0x7fb10c156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.209+0000 7fb10aa00640 1 -- 172.21.15.55:0/2890961397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0094d0 con 0x7fb10c06a4b0 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.210+0000 7fb10aa00640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc02a1a0 tx=0x7fb0fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.210+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc032440 con 0x7fb10c06a4b0 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.210+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc030ba0 con 0x7fb10c06a4b0 2024-05-10T03:32:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.210+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc03d440 con 0x7fb10c06a4b0 2024-05-10T03:32:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.211+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c1319c0 con 0x7fb10c06a4b0 2024-05-10T03:32:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.211+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c131f80 con 0x7fb10c06a4b0 2024-05-10T03:32:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.212+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c132780 con 0x7fb10c06a4b0 2024-05-10T03:32:07.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.213+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb0fc039040 con 0x7fb10c06a4b0 2024-05-10T03:32:07.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.214+0000 7fb102a00640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 0x7fb0ec071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:07.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.214+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb0fc0a9da0 con 0x7fb10c06a4b0 2024-05-10T03:32:07.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.214+0000 7fb10a000640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 0x7fb0ec071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.215+0000 7fb10a000640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 0x7fb0ec071520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006fd0 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.218+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb10c132780 con 0x7fb10c06a4b0 2024-05-10T03:32:07.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:07.431+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb10c132780 con 0x7fb10c06a4b0 2024-05-10T03:32:08.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.255+0000 7fb102a00640 1 -- 172.21.15.55:0/2890961397 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb10c132780 con 0x7fb10c06a4b0 2024-05-10T03:32:08.259 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:32:08.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.258+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 msgr2=0x7fb0ec071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.258+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 0x7fb0ec071520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006fd0 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2024-05-10T03:32:08.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.258+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 msgr2=0x7fb10c0aa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.258+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc02a1a0 tx=0x7fb0fc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 shutdown_connections 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb10c0b3860 0x7fb10c0a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb0ec06f010 0x7fb0ec071520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb10c071db0 0x7fb10c156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 --2- 172.21.15.55:0/2890961397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb10c06a4b0 0x7fb10c0aa2b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 >> 172.21.15.55:0/2890961397 conn(0x7fb10c0cbb90 msgr2=0x7fb10c06b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 shutdown_connections 2024-05-10T03:32:08.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.259+0000 7fb111600640 1 -- 172.21.15.55:0/2890961397 wait complete. 2024-05-10T03:32:08.290 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:32:08.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.566+0000 7f8e40e00640 1 Processor -- start 2024-05-10T03:32:08.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.566+0000 7f8e40e00640 1 -- start start 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.568+0000 7f8e40e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e3c15fe50 0x7f8e3c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.568+0000 7f8e40e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.568+0000 7f8e40e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c06ad20 con 0x7f8e3c0b2140 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.568+0000 7f8e40e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c141f50 con 0x7f8e3c0b2550 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.568+0000 7f8e40e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c06a110 con 0x7f8e3c15fe50 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41152/0 (socket says 172.21.15.55:41152) 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 -- 172.21.15.55:0/2707284426 learned_addr learned my addr 172.21.15.55:0/2707284426 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:08.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e39e00640 1 --2- 172.21.15.55:0/2707284426 >> v2:172.21.15.162:3300/0 conn(0x7f8e3c15fe50 0x7f8e3c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:08.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 -- 172.21.15.55:0/2707284426 >> v2:172.21.15.162:3300/0 conn(0x7f8e3c15fe50 msgr2=0x7f8e3c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:08.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 --2- 172.21.15.55:0/2707284426 >> v2:172.21.15.162:3300/0 conn(0x7f8e3c15fe50 0x7f8e3c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 -- 172.21.15.55:0/2707284426 >> v1:172.21.15.162:6789/0 conn(0x7f8e3c0b2140 legacy=0x7f8e3c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:08.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.569+0000 7f8e3a800640 1 -- 172.21.15.55:0/2707284426 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c06a3f0 con 0x7f8e3c0b2550 2024-05-10T03:32:08.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.570+0000 7f8e3a800640 1 --2- 172.21.15.55:0/2707284426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c06e520 tx=0x7f8e3402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c14d311e6421ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:08.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.571+0000 7f8e38a00640 1 -- 172.21.15.55:0/2707284426 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34032440 con 0x7f8e3c0b2550 2024-05-10T03:32:08.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.571+0000 7f8e38a00640 1 -- 172.21.15.55:0/2707284426 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e340325a0 con 0x7f8e3c0b2550 2024-05-10T03:32:08.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.571+0000 7f8e38a00640 1 -- 172.21.15.55:0/2707284426 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34032930 con 0x7f8e3c0b2550 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 -- 172.21.15.55:0/2707284426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2550 msgr2=0x7f8e3c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 --2- 172.21.15.55:0/2707284426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c06e520 tx=0x7f8e3402a2c0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 -- 172.21.15.55:0/2707284426 shutdown_connections 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 --2- 172.21.15.55:0/2707284426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c06b1b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 --2- 172.21.15.55:0/2707284426 >> v2:172.21.15.162:3300/0 conn(0x7f8e3c15fe50 0x7f8e3c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 -- 172.21.15.55:0/2707284426 >> 172.21.15.55:0/2707284426 conn(0x7f8e3c0cbb90 msgr2=0x7f8e3c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 -- 172.21.15.55:0/2707284426 shutdown_connections 2024-05-10T03:32:08.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.572+0000 7f8e40e00640 1 -- 172.21.15.55:0/2707284426 wait complete. 2024-05-10T03:32:08.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.574+0000 7f8e40e00640 1 Processor -- start 2024-05-10T03:32:08.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.574+0000 7f8e40e00640 1 -- start start 2024-05-10T03:32:08.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c13e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e3c15fe50 0x7f8e3c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c15b3a0 con 0x7f8e3c0b2140 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c15e1f0 con 0x7f8e3c0b2550 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e40e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c13ea60 con 0x7f8e3c15fe50 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41164/0 (socket says 172.21.15.55:41164) 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.576+0000 7f8e39e00640 1 -- 172.21.15.55:0/3681881382 learned_addr learned my addr 172.21.15.55:0/3681881382 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39400640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c13e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 -- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e3c15fe50 msgr2=0x7f8e3c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e3c15fe50 0x7f8e3c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 -- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 msgr2=0x7f8e3c13e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c13e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 -- 172.21.15.55:0/3681881382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340094d0 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39400640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c13e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e39e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007e40 tx=0x7f8e2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.577+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c014620 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.578+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00ee90 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.578+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0149c0 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.578+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c15c460 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.578+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c15ca20 con 0x7f8e3c0b2140 2024-05-10T03:32:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.579+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c15c100 con 0x7f8e3c0b2140 2024-05-10T03:32:08.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.580+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e2c014b20 con 0x7f8e3c0b2140 2024-05-10T03:32:08.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.581+0000 7f8e32000640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 0x7f8e1c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:08.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.581+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8e2c089c00 con 0x7f8e3c0b2140 2024-05-10T03:32:08.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.581+0000 7f8e39400640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 0x7f8e1c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:08.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.583+0000 7f8e39400640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 0x7f8e1c06d140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0b0970 tx=0x7f8e28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:08.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.586+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8e2c054ec0 con 0x7f8e3c0b2140 2024-05-10T03:32:08.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.865+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8e3c066f60 con 0x7f8e3c0b2140 2024-05-10T03:32:08.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.866+0000 7f8e32000640 1 -- 172.21.15.55:0/3681881382 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8e2c0584e0 con 0x7f8e3c0b2140 2024-05-10T03:32:08.869 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.870 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:32:08.871 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:08.872 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.873 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.874 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:32:08.875 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:08.877 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.878 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:08.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.880 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:32:08.881 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:32:08.882 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 msgr2=0x7f8e1c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 0x7f8e1c06d140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0b0970 tx=0x7f8e28008040 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 msgr2=0x7f8e3c13b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007e40 tx=0x7f8e2c00e4b0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 shutdown_connections 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e3c15fe50 0x7f8e3c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e1c06ac30 0x7f8e1c06d140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e3c0b2550 0x7f8e3c13e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 --2- 172.21.15.55:0/3681881382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e3c0b2140 0x7f8e3c13b230 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 >> 172.21.15.55:0/3681881382 conn(0x7f8e3c0cbb90 msgr2=0x7f8e3c15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 shutdown_connections 2024-05-10T03:32:08.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:08.869+0000 7f8e40e00640 1 -- 172.21.15.55:0/3681881382 wait complete. 2024-05-10T03:32:08.898 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.182+0000 7f3c59000640 1 Processor -- start 2024-05-10T03:32:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.182+0000 7f3c59000640 1 -- start start 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.184+0000 7f3c59000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c54144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.184+0000 7f3c59000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c540b3190 0x7f3c540cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.184+0000 7f3c59000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5406d350 con 0x7f3c5406d7e0 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.184+0000 7f3c59000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5406c740 con 0x7f3c540b3190 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.184+0000 7f3c59000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5406ca20 con 0x7f3c54144710 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c4be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3c5406d7e0 0x7f3c540b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39016/0 (socket says 172.21.15.55:39016) 2024-05-10T03:32:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c4be00640 1 -- 172.21.15.55:0/2909803498 learned_addr learned my addr 172.21.15.55:0/2909803498 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:09.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c53e00640 1 --2- 172.21.15.55:0/2909803498 >> v2:172.21.15.162:3300/0 conn(0x7f3c540b3190 0x7f3c540cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467037074 0 0) 0x7f3c5406d350 con 0x7f3c5406d7e0 2024-05-10T03:32:09.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c53400640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c54144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.185+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c38003520 con 0x7f3c5406d7e0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3835976920 0 0) 0x7f3c38003520 con 0x7f3c5406d7e0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 msgr2=0x7f3c54144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c54144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 >> v2:172.21.15.162:3300/0 conn(0x7f3c540b3190 msgr2=0x7f3c540cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 --2- 172.21.15.55:0/2909803498 >> v2:172.21.15.162:3300/0 conn(0x7f3c540b3190 0x7f3c540cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5406cd00 con 0x7f3c5406d7e0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f3c3c001670 con 0x7f3c5406d7e0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 >> v1:172.21.15.55:6789/0 conn(0x7f3c5406d7e0 legacy=0x7f3c540b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c380039e0 0x7f3c38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c380047c0 0x7f3c38004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c380051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5406c740 con 0x7f3c380047c0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5406ca20 con 0x7f3c380039e0 2024-05-10T03:32:09.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.186+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3800bb10 con 0x7f3c54144710 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c380047c0 0x7f3c38004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c53400640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c380039e0 0x7f3c38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c53e00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c380051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 -- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 msgr2=0x7f3c380051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c380051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 -- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c380039e0 msgr2=0x7f3c38003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c380039e0 0x7f3c38003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c4be00640 1 -- 172.21.15.55:0/2909803498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38003520 con 0x7f3c380047c0 2024-05-10T03:32:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.187+0000 7f3c53e00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c54144710 0x7f3c380051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:09.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.188+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c380047c0 0x7f3c38004bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ae870 tx=0x7f3c380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e7177b2bd416 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.188+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c005ad0 con 0x7f3c380047c0 2024-05-10T03:32:09.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.188+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c02d070 con 0x7f3c380047c0 2024-05-10T03:32:09.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.188+0000 7f3c52a00640 1 -- 172.21.15.55:0/2909803498 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c005e70 con 0x7f3c380047c0 2024-05-10T03:32:09.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 -- 172.21.15.55:0/2909803498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c380047c0 msgr2=0x7f3c38004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 --2- 172.21.15.55:0/2909803498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c380047c0 0x7f3c38004bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c540ae870 tx=0x7f3c380057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c53e00640 1 -- 172.21.15.55:0/2909803498 reap_dead start 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 -- 172.21.15.55:0/2909803498 shutdown_connections 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 -- 172.21.15.55:0/2909803498 >> 172.21.15.55:0/2909803498 conn(0x7f3c5406a110 msgr2=0x7f3c5406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 -- 172.21.15.55:0/2909803498 shutdown_connections 2024-05-10T03:32:09.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.189+0000 7f3c59000640 1 -- 172.21.15.55:0/2909803498 wait complete. 2024-05-10T03:32:09.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.192+0000 7f3c59000640 1 Processor -- start 2024-05-10T03:32:09.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.192+0000 7f3c59000640 1 -- start start 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.193+0000 7f3c59000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c380047c0 0x7f3c541f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.193+0000 7f3c59000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c59000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541f7600 0x7f3c541f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c59000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5415de80 con 0x7f3c540cbac0 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c59000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54143540 con 0x7f3c541f7600 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c59000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54142f10 con 0x7f3c380047c0 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41192/0 (socket says 172.21.15.55:41192) 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 -- 172.21.15.55:0/2784094207 learned_addr learned my addr 172.21.15.55:0/2784094207 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c53400640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c380047c0 0x7f3c541f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c53e00640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541f7600 0x7f3c541f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 -- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c380047c0 msgr2=0x7f3c541f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c380047c0 0x7f3c541f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 -- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541f7600 msgr2=0x7f3c541f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541f7600 0x7f3c541f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.194+0000 7f3c4be00640 1 -- 172.21.15.55:0/2784094207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54140690 con 0x7f3c540cbac0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c4be00640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c001930 tx=0x7f3c3c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0049e0 con 0x7f3c540cbac0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c004b40 con 0x7f3c540cbac0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c007070 con 0x7f3c540cbac0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c54140970 con 0x7f3c540cbac0 2024-05-10T03:32:09.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.195+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5415dbc0 con 0x7f3c540cbac0 2024-05-10T03:32:09.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.196+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c541f43b0 con 0x7f3c540cbac0 2024-05-10T03:32:09.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.197+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c3c02f030 con 0x7f3c540cbac0 2024-05-10T03:32:09.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.198+0000 7f3c4b400640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 0x7f3c3006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.198+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3c3c0a8a10 con 0x7f3c540cbac0 2024-05-10T03:32:09.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.199+0000 7f3c53400640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 0x7f3c3006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.200+0000 7f3c53400640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 0x7f3c3006d110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009160 tx=0x7f3c44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.203+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3c3c073d50 con 0x7f3c540cbac0 2024-05-10T03:32:09.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.405+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c541f43b0 con 0x7f3c540cbac0 2024-05-10T03:32:09.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.407+0000 7f3c4b400640 1 -- 172.21.15.55:0/2784094207 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+12594 (secure 0 0 0) 0x7f3c3c007d30 con 0x7f3c540cbac0 2024-05-10T03:32:09.411 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:09.411 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":48,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:08.236161+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.410+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 msgr2=0x7f3c3006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.410+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 0x7f3c3006d110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009160 tx=0x7f3c44008040 comp rx=0 tx=0).stop 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.410+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 msgr2=0x7f3c541f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.410+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c001930 tx=0x7f3c3c001a00 comp rx=0 tx=0).stop 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 shutdown_connections 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3006ac00 0x7f3c3006d110 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541f7600 0x7f3c541f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540cbac0 0x7f3c541f69b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 --2- 172.21.15.55:0/2784094207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c380047c0 0x7f3c541f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 >> 172.21.15.55:0/2784094207 conn(0x7f3c5406a110 msgr2=0x7f3c540cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:09.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 shutdown_connections 2024-05-10T03:32:09.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.411+0000 7f3c59000640 1 -- 172.21.15.55:0/2784094207 wait complete. 2024-05-10T03:32:09.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:09.441 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:09.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.728+0000 7f6441000640 1 Processor -- start 2024-05-10T03:32:09.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.729+0000 7f6441000640 1 -- start start 2024-05-10T03:32:09.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f643c15e920 0x7f643c0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 0x7f643c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f643c0710f0 0x7f643c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c15a900 con 0x7f643c15e920 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0b2a30 con 0x7f643c0710f0 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f6441000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c15eec0 con 0x7f643c06b120 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.731+0000 7f643b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f643c15e920 0x7f643c0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f643c15e920 0x7f643c0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41202/0 (socket says 172.21.15.55:41202) 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 -- 172.21.15.55:0/2240265069 learned_addr learned my addr 172.21.15.55:0/2240265069 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643be00640 1 --2- 172.21.15.55:0/2240265069 >> v2:172.21.15.162:3300/0 conn(0x7f643c0710f0 0x7f643c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f6433e00640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 0x7f643c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 -- 172.21.15.55:0/2240265069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 msgr2=0x7f643c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 0x7f643c06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 -- 172.21.15.55:0/2240265069 >> v2:172.21.15.162:3300/0 conn(0x7f643c0710f0 msgr2=0x7f643c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 --2- 172.21.15.55:0/2240265069 >> v2:172.21.15.162:3300/0 conn(0x7f643c0710f0 0x7f643c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f643b400640 1 -- 172.21.15.55:0/2240265069 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c06ba70 con 0x7f643c15e920 2024-05-10T03:32:09.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.732+0000 7f6433e00640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 0x7f643c06b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.733+0000 7f643b400640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c15e920 0x7f643c0cc240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f643c0cc0b0 tx=0x7f642c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3076c886cb5013 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.734+0000 7f643aa00640 1 -- 172.21.15.55:0/2240265069 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c032440 con 0x7f643c15e920 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.734+0000 7f643aa00640 1 -- 172.21.15.55:0/2240265069 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0325a0 con 0x7f643c15e920 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.734+0000 7f643aa00640 1 -- 172.21.15.55:0/2240265069 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c032930 con 0x7f643c15e920 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.734+0000 7f6441000640 1 -- 172.21.15.55:0/2240265069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c15e920 msgr2=0x7f643c0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.734+0000 7f6441000640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c15e920 0x7f643c0cc240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f643c0cc0b0 tx=0x7f642c02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 -- 172.21.15.55:0/2240265069 shutdown_connections 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 --2- 172.21.15.55:0/2240265069 >> v2:172.21.15.162:3300/0 conn(0x7f643c0710f0 0x7f643c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c06b120 0x7f643c06b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 --2- 172.21.15.55:0/2240265069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c15e920 0x7f643c0cc240 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 -- 172.21.15.55:0/2240265069 >> 172.21.15.55:0/2240265069 conn(0x7f643c06c750 msgr2=0x7f643c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:09.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 -- 172.21.15.55:0/2240265069 shutdown_connections 2024-05-10T03:32:09.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.735+0000 7f6441000640 1 -- 172.21.15.55:0/2240265069 wait complete. 2024-05-10T03:32:09.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.737+0000 7f6441000640 1 Processor -- start 2024-05-10T03:32:09.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.737+0000 7f6441000640 1 -- start start 2024-05-10T03:32:09.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c0710f0 0x7f643c156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 0x7f643c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c152220 con 0x7f643c06b120 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1582f0 con 0x7f643c15e920 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6441000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c13fb70 con 0x7f643c0710f0 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f643b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f643b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41216/0 (socket says 172.21.15.55:41216) 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f643b400640 1 -- 172.21.15.55:0/1616652976 learned_addr learned my addr 172.21.15.55:0/1616652976 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.739+0000 7f6433e00640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c0710f0 0x7f643c156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643be00640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 0x7f643c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 -- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c0710f0 msgr2=0x7f643c156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c0710f0 0x7f643c156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 -- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 msgr2=0x7f643c156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 0x7f643c156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 -- 172.21.15.55:0/1616652976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0094d0 con 0x7f643c06b120 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643be00640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 0x7f643c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f643b400640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c032440 con 0x7f643c06b120 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c030ba0 con 0x7f643c06b120 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c03d440 con 0x7f643c06b120 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.740+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c154be0 con 0x7f643c06b120 2024-05-10T03:32:09.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.741+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c154e30 con 0x7f643c06b120 2024-05-10T03:32:09.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.742+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c04e290 con 0x7f643c06b120 2024-05-10T03:32:09.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.743+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f642c039040 con 0x7f643c06b120 2024-05-10T03:32:09.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.744+0000 7f6433400640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 0x7f641806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:09.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.744+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f642c0a9d50 con 0x7f643c06b120 2024-05-10T03:32:09.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.745+0000 7f6433e00640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 0x7f641806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:09.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.746+0000 7f6433e00640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 0x7f641806d140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f643c0cca50 tx=0x7f6424008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:09.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.749+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f642c0750a0 con 0x7f643c06b120 2024-05-10T03:32:09.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.945+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f643c0b2810 con 0x7f643c06b120 2024-05-10T03:32:09.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.946+0000 7f6433400640 1 -- 172.21.15.55:0/1616652976 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f642c0370a0 con 0x7f643c06b120 2024-05-10T03:32:09.949 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:09.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 msgr2=0x7f641806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 0x7f641806d140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f643c0cca50 tx=0x7f6424008040 comp rx=0 tx=0).stop 2024-05-10T03:32:09.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 msgr2=0x7f643c155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f642c000c00 tx=0x7f642c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 shutdown_connections 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f641806ac30 0x7f641806d140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f643c15e920 0x7f643c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f643c0710f0 0x7f643c156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 --2- 172.21.15.55:0/1616652976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c06b120 0x7f643c155bf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 >> 172.21.15.55:0/1616652976 conn(0x7f643c06c750 msgr2=0x7f643c141ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 shutdown_connections 2024-05-10T03:32:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:09.949+0000 7f6441000640 1 -- 172.21.15.55:0/1616652976 wait complete. 2024-05-10T03:32:09.977 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:32:10.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.255+0000 7f0f0d800640 1 Processor -- start 2024-05-10T03:32:10.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.255+0000 7f0f0d800640 1 -- start start 2024-05-10T03:32:10.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.257+0000 7f0f0d800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f080ce880 0x7f0f0806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.257+0000 7f0f0d800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f0806b660 0x7f0f080b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.257+0000 7f0f0d800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0806ba70 con 0x7f0f0815adc0 2024-05-10T03:32:10.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.257+0000 7f0f0d800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0815a0d0 con 0x7f0f0806b660 2024-05-10T03:32:10.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.257+0000 7f0f0d800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f080cec90 con 0x7f0f080ce880 2024-05-10T03:32:10.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f0806b660 0x7f0f080b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f0806b660 0x7f0f080b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41224/0 (socket says 172.21.15.55:41224) 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 -- 172.21.15.55:0/1844102288 learned_addr learned my addr 172.21.15.55:0/1844102288 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0effe00640 1 --2- 172.21.15.55:0/1844102288 >> v2:172.21.15.162:3300/0 conn(0x7f0f080ce880 0x7f0f0806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 -- 172.21.15.55:0/1844102288 >> v2:172.21.15.162:3300/0 conn(0x7f0f080ce880 msgr2=0x7f0f0806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 --2- 172.21.15.55:0/1844102288 >> v2:172.21.15.162:3300/0 conn(0x7f0f080ce880 0x7f0f0806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 -- 172.21.15.55:0/1844102288 >> v1:172.21.15.162:6789/0 conn(0x7f0f0815adc0 legacy=0x7f0f0815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.258+0000 7f0f0ce00640 1 -- 172.21.15.55:0/1844102288 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f080b4330 con 0x7f0f0806b660 2024-05-10T03:32:10.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.259+0000 7f0f0ce00640 1 --2- 172.21.15.55:0/1844102288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f080b2610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f08071f90 tx=0x7f0f0002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a277914f25ee21ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.260+0000 7f0f07400640 1 -- 172.21.15.55:0/1844102288 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00031600 con 0x7f0f0806b660 2024-05-10T03:32:10.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.260+0000 7f0f07400640 1 -- 172.21.15.55:0/1844102288 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0002d050 con 0x7f0f0806b660 2024-05-10T03:32:10.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.260+0000 7f0f07400640 1 -- 172.21.15.55:0/1844102288 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00031980 con 0x7f0f0806b660 2024-05-10T03:32:10.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 -- 172.21.15.55:0/1844102288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 msgr2=0x7f0f080b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1844102288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f080b2610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0f08071f90 tx=0x7f0f0002a6f0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 -- 172.21.15.55:0/1844102288 shutdown_connections 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1844102288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f080b2610 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1844102288 >> v2:172.21.15.162:3300/0 conn(0x7f0f080ce880 0x7f0f0806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 -- 172.21.15.55:0/1844102288 >> 172.21.15.55:0/1844102288 conn(0x7f0f0806c730 msgr2=0x7f0f080b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.261+0000 7f0f0d800640 1 -- 172.21.15.55:0/1844102288 shutdown_connections 2024-05-10T03:32:10.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.262+0000 7f0f0d800640 1 -- 172.21.15.55:0/1844102288 wait complete. 2024-05-10T03:32:10.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.264+0000 7f0f0d800640 1 Processor -- start 2024-05-10T03:32:10.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.264+0000 7f0f0d800640 1 -- start start 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 0x7f0f080b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f0815adc0 0x7f0f080b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f080aedc0 con 0x7f0f0806b660 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f080affe0 con 0x7f0f080ce880 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f0d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08141960 con 0x7f0f0815adc0 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41230/0 (socket says 172.21.15.55:41230) 2024-05-10T03:32:10.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0f07e00640 1 -- 172.21.15.55:0/1994550043 learned_addr learned my addr 172.21.15.55:0/1994550043 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.266+0000 7f0effe00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 0x7f0f080b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f0ce00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f0815adc0 0x7f0f080b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 -- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f0815adc0 msgr2=0x7f0f080b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f0815adc0 0x7f0f080b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 -- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 msgr2=0x7f0f080b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 0x7f0f080b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 -- 172.21.15.55:0/1994550043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00009b30 con 0x7f0f0806b660 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0effe00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 0x7f0f080b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f07e00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b3f0 tx=0x7f0ef800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8011830 con 0x7f0f0806b660 2024-05-10T03:32:10.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef8011990 con 0x7f0f0806b660 2024-05-10T03:32:10.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8011d20 con 0x7f0f0806b660 2024-05-10T03:32:10.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f08140d50 con 0x7f0f0806b660 2024-05-10T03:32:10.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.267+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f08141310 con 0x7f0f0806b660 2024-05-10T03:32:10.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.269+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0804e290 con 0x7f0f0806b660 2024-05-10T03:32:10.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.270+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ef801a410 con 0x7f0f0806b660 2024-05-10T03:32:10.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.271+0000 7f0f04c00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 0x7f0ee406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.271+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0ef8088d60 con 0x7f0f0806b660 2024-05-10T03:32:10.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.272+0000 7f0effe00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 0x7f0ee406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.273+0000 7f0effe00640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 0x7f0ee406d140 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f080cbae0 tx=0x7f0ef0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.276+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ef8054130 con 0x7f0f0806b660 2024-05-10T03:32:10.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.475+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0f080ce480 con 0x7f0f0806b660 2024-05-10T03:32:10.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.476+0000 7f0f04c00640 1 -- 172.21.15.55:0/1994550043 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0ef8057750 con 0x7f0f0806b660 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:10.480 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:32:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 msgr2=0x7f0ee406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 0x7f0ee406d140 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f080cbae0 tx=0x7f0ef0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 msgr2=0x7f0f081539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b3f0 tx=0x7f0ef800b8b0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 shutdown_connections 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f0815adc0 0x7f0f080b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee406ac30 0x7f0ee406d140 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f080ce880 0x7f0f080b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 --2- 172.21.15.55:0/1994550043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f0806b660 0x7f0f081539c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 >> 172.21.15.55:0/1994550043 conn(0x7f0f0806c730 msgr2=0x7f0f0806ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 shutdown_connections 2024-05-10T03:32:10.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.479+0000 7f0f0d800640 1 -- 172.21.15.55:0/1994550043 wait complete. 2024-05-10T03:32:10.509 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:32:10.509 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:32:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.792+0000 7fa2fe400640 1 Processor -- start 2024-05-10T03:32:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.792+0000 7fa2fe400640 1 -- start start 2024-05-10T03:32:10.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.795+0000 7fa2fe400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa2f806b7c0 0x7fa2f815b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.795+0000 7fa2fe400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f80cce40 0x7fa2f806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.795+0000 7fa2fe400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f80ad730 con 0x7fa2f806ad10 2024-05-10T03:32:10.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.795+0000 7fa2fe400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f806b120 con 0x7fa2f806b7c0 2024-05-10T03:32:10.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.795+0000 7fa2fe400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f806bbd0 con 0x7fa2f80cce40 2024-05-10T03:32:10.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.796+0000 7fa2f7400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa2f806b7c0 0x7fa2f815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.796+0000 7fa2f7400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa2f806b7c0 0x7fa2f815b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46892/0 (socket says 172.21.15.55:46892) 2024-05-10T03:32:10.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.796+0000 7fa2f7400640 1 -- 172.21.15.55:0/15743621 learned_addr learned my addr 172.21.15.55:0/15743621 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:10.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.796+0000 7fa2eea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f80cce40 0x7fa2f806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.797+0000 7fa2f7400640 1 -- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f80cce40 msgr2=0x7fa2f806a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:10.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.797+0000 7fa2f7400640 1 --2- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f80cce40 0x7fa2f806a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.797+0000 7fa2f7400640 1 -- 172.21.15.55:0/15743621 >> v1:172.21.15.162:6789/0 conn(0x7fa2f806ad10 legacy=0x7fa2f8070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.797+0000 7fa2f7400640 1 -- 172.21.15.55:0/15743621 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80ad170 con 0x7fa2f806b7c0 2024-05-10T03:32:10.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.799+0000 7fa2f7400640 1 --2- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806b7c0 0x7fa2f815b820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e50 tx=0x7fa2e802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcb249b60744e37a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.800+0000 7fa2f6a00640 1 -- 172.21.15.55:0/15743621 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8031600 con 0x7fa2f806b7c0 2024-05-10T03:32:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.800+0000 7fa2f6a00640 1 -- 172.21.15.55:0/15743621 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e802d050 con 0x7fa2f806b7c0 2024-05-10T03:32:10.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.800+0000 7fa2f6a00640 1 -- 172.21.15.55:0/15743621 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8031980 con 0x7fa2f806b7c0 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 -- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806b7c0 msgr2=0x7fa2f815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 --2- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806b7c0 0x7fa2f815b820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e50 tx=0x7fa2e802a710 comp rx=0 tx=0).stop 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 -- 172.21.15.55:0/15743621 shutdown_connections 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 --2- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f80cce40 0x7fa2f806a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 --2- 172.21.15.55:0/15743621 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806b7c0 0x7fa2f815b820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 -- 172.21.15.55:0/15743621 >> 172.21.15.55:0/15743621 conn(0x7fa2f80b0cf0 msgr2=0x7fa2f80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:10.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 -- 172.21.15.55:0/15743621 shutdown_connections 2024-05-10T03:32:10.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.801+0000 7fa2fe400640 1 -- 172.21.15.55:0/15743621 wait complete. 2024-05-10T03:32:10.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.804+0000 7fa2fe400640 1 Processor -- start 2024-05-10T03:32:10.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.805+0000 7fa2fe400640 1 -- start start 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 0x7fa2f809a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f806b7c0 0x7fa2f815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81534c0 con 0x7fa2f80cce40 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80ca4d0 con 0x7fa2f806ad10 2024-05-10T03:32:10.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2fe400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80b2b00 con 0x7fa2f806b7c0 2024-05-10T03:32:10.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41232/0 (socket says 172.21.15.55:41232) 2024-05-10T03:32:10.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.807+0000 7fa2f7e00640 1 -- 172.21.15.55:0/2137502803 learned_addr learned my addr 172.21.15.55:0/2137502803 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:10.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 0x7fa2f809a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2eea00640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f806b7c0 0x7fa2f815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 -- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f806b7c0 msgr2=0x7fa2f815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f806b7c0 0x7fa2f815e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 -- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 msgr2=0x7fa2f809a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 0x7fa2f809a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 -- 172.21.15.55:0/2137502803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009b30 con 0x7fa2f80cce40 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 0x7fa2f809a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:10.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.808+0000 7fa2f7e00640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c3f0 tx=0x7fa2f000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.809+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00052d0 con 0x7fa2f80cce40 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.809+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0005430 con 0x7fa2f80cce40 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.809+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0003830 con 0x7fa2f80cce40 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.809+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f80b2d40 con 0x7fa2f80cce40 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.809+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f80b3270 con 0x7fa2f80cce40 2024-05-10T03:32:10.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.810+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f80b1a40 con 0x7fa2f80cce40 2024-05-10T03:32:10.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.811+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa2f0003990 con 0x7fa2f80cce40 2024-05-10T03:32:10.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.813+0000 7fa2efe00640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 0x7fa2d406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:10.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.813+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa2f0088540 con 0x7fa2f80cce40 2024-05-10T03:32:10.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.813+0000 7fa2f7400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 0x7fa2d406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:10.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.815+0000 7fa2f7400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 0x7fa2d406d110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8033000 tx=0x7fa2e8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:10.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:10.818+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa2f0053890 con 0x7fa2f80cce40 2024-05-10T03:32:11.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.020+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa2f80ad730 con 0x7fa2f80cce40 2024-05-10T03:32:11.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.286+0000 7fa2efe00640 1 -- 172.21.15.55:0/2137502803 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa2f80ad730 con 0x7fa2f80cce40 2024-05-10T03:32:11.290 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:32:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 msgr2=0x7fa2d406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 0x7fa2d406d110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8033000 tx=0x7fa2e8009000 comp rx=0 tx=0).stop 2024-05-10T03:32:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 msgr2=0x7fa2f809bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000c3f0 tx=0x7fa2f000c8b0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 shutdown_connections 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f80cce40 0x7fa2f809bec0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2f806b7c0 0x7fa2f815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa2d406ac00 0x7fa2d406d110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 --2- 172.21.15.55:0/2137502803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2f806ad10 0x7fa2f809a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.293+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 >> 172.21.15.55:0/2137502803 conn(0x7fa2f80b0cf0 msgr2=0x7fa2f80cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.294+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 shutdown_connections 2024-05-10T03:32:11.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.296+0000 7fa2fe400640 1 -- 172.21.15.55:0/2137502803 wait complete. 2024-05-10T03:32:11.327 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:32:11.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.598+0000 7f8a07e00640 1 Processor -- start 2024-05-10T03:32:11.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.598+0000 7f8a07e00640 1 -- start start 2024-05-10T03:32:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.601+0000 7f8a07e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a080cc230 0x7f8a080b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.601+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a08141360 0x7f8a08141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.601+0000 7f8a07e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a080b3f80 con 0x7f8a0815fe80 2024-05-10T03:32:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.601+0000 7f8a07e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a080b4540 con 0x7f8a080cc230 2024-05-10T03:32:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.601+0000 7f8a07e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0815c1d0 con 0x7f8a08141360 2024-05-10T03:32:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.602+0000 7f89ffe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a080cc230 0x7f8a080b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.602+0000 7f8a07400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a08141360 0x7f8a08141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.602+0000 7f89ffe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a080cc230 0x7f8a080b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46910/0 (socket says 172.21.15.55:46910) 2024-05-10T03:32:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.602+0000 7f89ffe00640 1 -- 172.21.15.55:0/2297140260 learned_addr learned my addr 172.21.15.55:0/2297140260 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.603+0000 7f89ffe00640 1 -- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a08141360 msgr2=0x7f8a08141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.603+0000 7f89ffe00640 1 --2- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a08141360 0x7f8a08141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.603+0000 7f89ffe00640 1 -- 172.21.15.55:0/2297140260 >> v1:172.21.15.162:6789/0 conn(0x7f8a0815fe80 legacy=0x7f8a0815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.603+0000 7f89ffe00640 1 -- 172.21.15.55:0/2297140260 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080b4260 con 0x7f8a080cc230 2024-05-10T03:32:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.604+0000 7f89ffe00640 1 --2- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080b2e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78666841b4f5acd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.605+0000 7f8a06000640 1 -- 172.21.15.55:0/2297140260 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0032440 con 0x7f8a080cc230 2024-05-10T03:32:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.605+0000 7f8a06000640 1 -- 172.21.15.55:0/2297140260 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f00325a0 con 0x7f8a080cc230 2024-05-10T03:32:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.605+0000 7f8a06000640 1 -- 172.21.15.55:0/2297140260 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0032930 con 0x7f8a080cc230 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 -- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 msgr2=0x7f8a080b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080b2e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009870 tx=0x7f89f002a250 comp rx=0 tx=0).stop 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 -- 172.21.15.55:0/2297140260 shutdown_connections 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a08141360 0x7f8a08141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2297140260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080b2e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 -- 172.21.15.55:0/2297140260 >> 172.21.15.55:0/2297140260 conn(0x7f8a080b1810 msgr2=0x7f8a0806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 -- 172.21.15.55:0/2297140260 shutdown_connections 2024-05-10T03:32:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.606+0000 7f8a07e00640 1 -- 172.21.15.55:0/2297140260 wait complete. 2024-05-10T03:32:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.609+0000 7f8a07e00640 1 Processor -- start 2024-05-10T03:32:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.609+0000 7f8a07e00640 1 -- start start 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0815fe80 0x7f8a080ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080ac100 con 0x7f8a08141360 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080a8a60 con 0x7f8a080cc230 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a07e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080a8cb0 con 0x7f8a0815fe80 2024-05-10T03:32:11.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41242/0 (socket says 172.21.15.55:41242) 2024-05-10T03:32:11.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f89ffe00640 1 -- 172.21.15.55:0/2040973970 learned_addr learned my addr 172.21.15.55:0/2040973970 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.612+0000 7f8a06a00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f8a07400640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0815fe80 0x7f8a080ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 -- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0815fe80 msgr2=0x7f8a080ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0815fe80 0x7f8a080ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 -- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 msgr2=0x7f8a080aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:11.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 -- 172.21.15.55:0/2040973970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00094d0 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89ffe00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0032440 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0030980 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.613+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a080a8f90 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.614+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0030d10 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.614+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a080a9550 con 0x7f8a08141360 2024-05-10T03:32:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.615+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a080b0360 con 0x7f8a08141360 2024-05-10T03:32:11.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.616+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f89f0039040 con 0x7f8a08141360 2024-05-10T03:32:11.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.617+0000 7f89fea00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 0x7f89e406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:11.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.617+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f89f00aa640 con 0x7f8a08141360 2024-05-10T03:32:11.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.618+0000 7f8a06a00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 0x7f89e406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:11.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.619+0000 7f8a06a00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 0x7f89e406d140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000f60 tx=0x7f89f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:11.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.622+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f89f0075790 con 0x7f8a08141360 2024-05-10T03:32:11.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:11.820+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8a080cd320 con 0x7f8a08141360 2024-05-10T03:32:12.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.297+0000 7f89fea00640 1 -- 172.21.15.55:0/2040973970 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f89f0078db0 con 0x7f8a08141360 2024-05-10T03:32:12.303 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:32:12.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.307+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 msgr2=0x7f89e406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.307+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 0x7f89e406d140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000f60 tx=0x7f89f8009040 comp rx=0 tx=0).stop 2024-05-10T03:32:12.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.307+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 msgr2=0x7f8a080cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.307+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f00046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 shutdown_connections 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0815fe80 0x7f8a080ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a08141360 0x7f8a080cb970 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f89e406ac30 0x7f89e406d140 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 --2- 172.21.15.55:0/2040973970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a080cc230 0x7f8a080aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 >> 172.21.15.55:0/2040973970 conn(0x7f8a080b1810 msgr2=0x7f8a080afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 shutdown_connections 2024-05-10T03:32:12.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.311+0000 7f8a07e00640 1 -- 172.21.15.55:0/2040973970 wait complete. 2024-05-10T03:32:12.359 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:32:12.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.605+0000 7ff4bbe00640 1 Processor -- start 2024-05-10T03:32:12.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.606+0000 7ff4bbe00640 1 -- start start 2024-05-10T03:32:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4bc0cc240 0x7ff4bc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bbe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc0b2a30 con 0x7ff4bc06b830 2024-05-10T03:32:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bbe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc15eec0 con 0x7ff4bc15e920 2024-05-10T03:32:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bbe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4bc15a900 con 0x7ff4bc0cc240 2024-05-10T03:32:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4bc0cc240 0x7ff4bc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:12.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4bc0cc240 0x7ff4bc0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41258/0 (socket says 172.21.15.55:41258) 2024-05-10T03:32:12.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4bb400640 1 -- 172.21.15.55:0/1784430278 learned_addr learned my addr 172.21.15.55:0/1784430278 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:12.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.608+0000 7ff4b3e00640 1 --1- 172.21.15.55:0/1784430278 >> v1:172.21.15.55:6789/0 conn(0x7ff4bc06b830 0x7ff4bc0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39028/0 (socket says 172.21.15.55:39028) 2024-05-10T03:32:12.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.609+0000 7ff4bb400640 1 -- 172.21.15.55:0/1784430278 >> v1:172.21.15.162:6789/0 conn(0x7ff4bc15e920 legacy=0x7ff4bc06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.609+0000 7ff4bb400640 1 -- 172.21.15.55:0/1784430278 >> v1:172.21.15.55:6789/0 conn(0x7ff4bc06b830 legacy=0x7ff4bc0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.609+0000 7ff4bb400640 1 -- 172.21.15.55:0/1784430278 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc15b8b0 con 0x7ff4bc0cc240 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.610+0000 7ff4bb400640 1 --2- 172.21.15.55:0/1784430278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc0ce280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009e50 tx=0x7ff4b402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca5810098db3cbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.610+0000 7ff4ba000640 1 -- 172.21.15.55:0/1784430278 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4031600 con 0x7ff4bc0cc240 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.610+0000 7ff4ba000640 1 -- 172.21.15.55:0/1784430278 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b402d050 con 0x7ff4bc0cc240 2024-05-10T03:32:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.610+0000 7ff4ba000640 1 -- 172.21.15.55:0/1784430278 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4031980 con 0x7ff4bc0cc240 2024-05-10T03:32:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1784430278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 msgr2=0x7ff4bc0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1784430278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc0ce280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009e50 tx=0x7ff4b402a760 comp rx=0 tx=0).stop 2024-05-10T03:32:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1784430278 shutdown_connections 2024-05-10T03:32:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1784430278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc0ce280 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1784430278 >> 172.21.15.55:0/1784430278 conn(0x7ff4bc06c750 msgr2=0x7ff4bc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1784430278 shutdown_connections 2024-05-10T03:32:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.611+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1784430278 wait complete. 2024-05-10T03:32:12.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.613+0000 7ff4bbe00640 1 Processor -- start 2024-05-10T03:32:12.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.614+0000 7ff4bbe00640 1 -- start start 2024-05-10T03:32:12.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:12.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4bc15e920 0x7ff4bc13bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:12.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc13c2e0 0x7ff4bc1f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:12.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0ca690 con 0x7ff4bc0cc240 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc1f7b80 con 0x7ff4bc13c2e0 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4bbe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc14d070 con 0x7ff4bc15e920 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4baa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4baa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41268/0 (socket says 172.21.15.55:41268) 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.615+0000 7ff4baa00640 1 -- 172.21.15.55:0/1211799690 learned_addr learned my addr 172.21.15.55:0/1211799690 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4b3e00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4bc15e920 0x7ff4bc13bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4bb400640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc13c2e0 0x7ff4bc1f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 -- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4bc15e920 msgr2=0x7ff4bc13bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4bc15e920 0x7ff4bc13bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 -- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc13c2e0 msgr2=0x7ff4bc1f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc13c2e0 0x7ff4bc1f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 -- 172.21.15.55:0/1211799690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009b30 con 0x7ff4bc0cc240 2024-05-10T03:32:12.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.616+0000 7ff4baa00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b3f0 tx=0x7ff4ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.617+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011830 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.617+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac011990 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.617+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc1e3e20 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.617+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011d40 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.617+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc0c8870 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.618+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4bc04e290 con 0x7ff4bc0cc240 2024-05-10T03:32:12.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.620+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4ac004020 con 0x7ff4bc0cc240 2024-05-10T03:32:12.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.620+0000 7ff4b2a00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 0x7ff49806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:12.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.621+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5929+0+0 (secure 0 0 0) 0x7ff4ac089790 con 0x7ff4bc0cc240 2024-05-10T03:32:12.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.621+0000 7ff4b3e00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 0x7ff49806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.622+0000 7ff4b3e00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 0x7ff49806d110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4000f60 tx=0x7ff4a4007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:12.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.625+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff4ac054870 con 0x7ff4bc0cc240 2024-05-10T03:32:12.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.867+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff4bc0b2300 con 0x7ff4bc0cc240 2024-05-10T03:32:12.885 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 7 and data pool 8 2024-05-10T03:32:12.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.882+0000 7ff4b2a00640 1 -- 172.21.15.55:0/1211799690 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff4ac057e90 con 0x7ff4bc0cc240 2024-05-10T03:32:12.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.886+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 msgr2=0x7ff49806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.886+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 0x7ff49806d110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4000f60 tx=0x7ff4a4007ca0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.886+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 msgr2=0x7ff4bc159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:12.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.886+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b3f0 tx=0x7ff4ac00b8b0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 shutdown_connections 2024-05-10T03:32:12.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff49806ac00 0x7ff49806d110 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4000f60 tx=0x7ff4a4007ca0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc13c2e0 0x7ff4bc1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4bc15e920 0x7ff4bc13bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 --2- 172.21.15.55:0/1211799690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4bc0cc240 0x7ff4bc159df0 secure :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b3f0 tx=0x7ff4ac00b8b0 comp rx=0 tx=0).stop 2024-05-10T03:32:12.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.887+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 >> 172.21.15.55:0/1211799690 conn(0x7ff4bc06c750 msgr2=0x7ff4bc06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:12.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.891+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 shutdown_connections 2024-05-10T03:32:12.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:12.891+0000 7ff4bbe00640 1 -- 172.21.15.55:0/1211799690 wait complete. 2024-05-10T03:32:12.944 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.187+0000 7f296aa00640 1 Processor -- start 2024-05-10T03:32:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.188+0000 7f296aa00640 1 -- start start 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f296aa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29641549e0 0x7f29640b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f296aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29640b3390 0x7f29641612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f296aa00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640b37a0 con 0x7f2964161830 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f296aa00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2964161c40 con 0x7f29640b3390 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f296aa00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b39f0 con 0x7f29641549e0 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f2968c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29640b3390 0x7f29641612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f2968c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29640b3390 0x7f29641612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40160/0 (socket says 172.21.15.55:40160) 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.190+0000 7f2968c00640 1 -- 172.21.15.55:0/1527310879 learned_addr learned my addr 172.21.15.55:0/1527310879 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2969600640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29641549e0 0x7f29640b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2968c00640 1 -- 172.21.15.55:0/1527310879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29641549e0 msgr2=0x7f29640b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2968c00640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29641549e0 0x7f29640b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2968c00640 1 -- 172.21.15.55:0/1527310879 >> v1:172.21.15.55:6789/0 conn(0x7f2964161830 legacy=0x7f2964071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2968c00640 1 -- 172.21.15.55:0/1527310879 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640b3cd0 con 0x7f29640b3390 2024-05-10T03:32:13.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.191+0000 7f2963e00640 1 -- 172.21.15.55:0/1527310879 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130568536 0 0) 0x7f29640b37a0 con 0x7f2964161830 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.192+0000 7f2968c00640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f29641612f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2950009870 tx=0x7f295002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6bc68f1dc49bda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.193+0000 7f2963e00640 1 -- 172.21.15.55:0/1527310879 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950032440 con 0x7f29640b3390 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.193+0000 7f2963e00640 1 -- 172.21.15.55:0/1527310879 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29500325a0 con 0x7f29640b3390 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.193+0000 7f2963e00640 1 -- 172.21.15.55:0/1527310879 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950032930 con 0x7f29640b3390 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.193+0000 7f296aa00640 1 -- 172.21.15.55:0/1527310879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 msgr2=0x7f29641612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.193+0000 7f296aa00640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f29641612f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2950009870 tx=0x7f295002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 -- 172.21.15.55:0/1527310879 shutdown_connections 2024-05-10T03:32:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f29641612f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 --2- 172.21.15.55:0/1527310879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29641549e0 0x7f29640b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 -- 172.21.15.55:0/1527310879 >> 172.21.15.55:0/1527310879 conn(0x7f296406a110 msgr2=0x7f296406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:13.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 -- 172.21.15.55:0/1527310879 shutdown_connections 2024-05-10T03:32:13.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.194+0000 7f296aa00640 1 -- 172.21.15.55:0/1527310879 wait complete. 2024-05-10T03:32:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.196+0000 7f296aa00640 1 Processor -- start 2024-05-10T03:32:13.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.196+0000 7f296aa00640 1 -- start start 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29641549e0 0x7f2964149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2964161830 0x7f29641e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641445d0 con 0x7f29640b3390 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964148550 con 0x7f29641549e0 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f296aa00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296414e280 con 0x7f2964161830 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f2969600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f2969600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40176/0 (socket says 172.21.15.55:40176) 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.198+0000 7f2969600640 1 -- 172.21.15.55:0/564189465 learned_addr learned my addr 172.21.15.55:0/564189465 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 -- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2964161830 msgr2=0x7f29641e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2964161830 0x7f29641e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 -- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29641549e0 msgr2=0x7f2964149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29641549e0 0x7f2964149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 -- 172.21.15.55:0/564189465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500094d0 con 0x7f29640b3390 2024-05-10T03:32:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2969600640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f295800d5d0 tx=0x7f295800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004240 con 0x7f29640b3390 2024-05-10T03:32:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29580043a0 con 0x7f29640b3390 2024-05-10T03:32:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958003830 con 0x7f29640b3390 2024-05-10T03:32:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.199+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296414a040 con 0x7f29640b3390 2024-05-10T03:32:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.200+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296414e7b0 con 0x7f29640b3390 2024-05-10T03:32:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.201+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296404e290 con 0x7f29640b3390 2024-05-10T03:32:13.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.202+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2958003990 con 0x7f29640b3390 2024-05-10T03:32:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.203+0000 7f2961600640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 0x7f294406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.203+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2958089770 con 0x7f29640b3390 2024-05-10T03:32:13.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.204+0000 7f2968c00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 0x7f294406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.205+0000 7f2968c00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 0x7f294406d110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2950034000 tx=0x7f295002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.208+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2958054780 con 0x7f29640b3390 2024-05-10T03:32:13.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.410+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f296406fdc0 con 0x7f29640b3390 2024-05-10T03:32:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.413+0000 7f2961600640 1 -- 172.21.15.55:0/564189465 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f2958057da0 con 0x7f29640b3390 2024-05-10T03:32:13.417 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:13.417 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":51,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:12.868903+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 msgr2=0x7f294406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 0x7f294406d110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2950034000 tx=0x7f295002c000 comp rx=0 tx=0).stop 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 msgr2=0x7f2964147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f295800d5d0 tx=0x7f295800da90 comp rx=0 tx=0).stop 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 shutdown_connections 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2964161830 0x7f29641e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f294406ac00 0x7f294406d110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29641549e0 0x7f2964149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 --2- 172.21.15.55:0/564189465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29640b3390 0x7f2964147cc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 >> 172.21.15.55:0/564189465 conn(0x7f296406a110 msgr2=0x7f296406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 shutdown_connections 2024-05-10T03:32:13.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.416+0000 7f296aa00640 1 -- 172.21.15.55:0/564189465 wait complete. 2024-05-10T03:32:13.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.720+0000 7f9867e00640 1 Processor -- start 2024-05-10T03:32:13.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.720+0000 7f9867e00640 1 -- start start 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.722+0000 7f9867e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 0x7f9868168510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.722+0000 7f9867e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986806bac0 con 0x7f98680b6680 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.722+0000 7f9867e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9868161fc0 con 0x7f98680b2170 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.722+0000 7f9867e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680d2b20 con 0x7f986815d5e0 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.723+0000 7f9866a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f98680b6680 0x7f98681611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60922/0 (socket says 172.21.15.55:60922) 2024-05-10T03:32:13.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.723+0000 7f9866a00640 1 -- 172.21.15.55:0/4105250976 learned_addr learned my addr 172.21.15.55:0/4105250976 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.723+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537081977 0 0) 0x7f986806bac0 con 0x7f98680b6680 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.723+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c003520 con 0x7f98680b6680 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.723+0000 7f9867400640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 0x7f9868168510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715373605 0 0) 0x7f984c003520 con 0x7f98680b6680 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 msgr2=0x7f9868168510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 0x7f9868168510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 >> v1:172.21.15.162:6789/0 conn(0x7f98680b2170 legacy=0x7f98681618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680d25a0 con 0x7f98680b6680 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f98580019f0 con 0x7f98680b6680 2024-05-10T03:32:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 >> v1:172.21.15.55:6789/0 conn(0x7f98680b6680 legacy=0x7f98681611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f984c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 0x7f984c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c004e20 0x7f984c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680d2b20 con 0x7f984c003a10 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bbb0 con 0x7f984c004e20 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c00bd70 con 0x7f986815d5e0 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.724+0000 7f9866a00640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f984c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.725+0000 7f9866a00640 1 -- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 msgr2=0x7f984c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.725+0000 7f9866a00640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f986815d5e0 0x7f984c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.725+0000 7f9866a00640 1 -- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c004e20 msgr2=0x7f984c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.725+0000 7f9866a00640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c004e20 0x7f984c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.725+0000 7f9866a00640 1 -- 172.21.15.55:0/4105250976 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c003520 con 0x7f984c003a10 2024-05-10T03:32:13.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.726+0000 7f9866a00640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f984c003e80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f985800a820 tx=0x7f984c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b374408a5ccb05c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.726+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985802ba30 con 0x7f984c003a10 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.726+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985802bb90 con 0x7f984c003a10 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.726+0000 7f9865600640 1 -- 172.21.15.55:0/4105250976 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858034470 con 0x7f984c003a10 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 -- 172.21.15.55:0/4105250976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 msgr2=0x7f984c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 --2- 172.21.15.55:0/4105250976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f984c003e80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f985800a820 tx=0x7f984c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867400640 1 -- 172.21.15.55:0/4105250976 reap_dead start 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 -- 172.21.15.55:0/4105250976 shutdown_connections 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 -- 172.21.15.55:0/4105250976 >> 172.21.15.55:0/4105250976 conn(0x7f98680b5730 msgr2=0x7f98680d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 -- 172.21.15.55:0/4105250976 shutdown_connections 2024-05-10T03:32:13.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.727+0000 7f9867e00640 1 -- 172.21.15.55:0/4105250976 wait complete. 2024-05-10T03:32:13.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.729+0000 7f9867e00640 1 Processor -- start 2024-05-10T03:32:13.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.729+0000 7f9867e00640 1 -- start start 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98680cb870 0x7f98680a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98680a9fe0 0x7f986814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986814bf20 con 0x7f984c003a10 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680cc070 con 0x7f98680a9fe0 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9867e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986814dca0 con 0x7f98680cb870 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9866a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9866a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40186/0 (socket says 172.21.15.55:40186) 2024-05-10T03:32:13.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9866a00640 1 -- 172.21.15.55:0/3235596006 learned_addr learned my addr 172.21.15.55:0/3235596006 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9866a00640 1 -- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98680cb870 msgr2=0x7f98680a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.731+0000 7f9866a00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98680cb870 0x7f98680a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9866a00640 1 -- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98680a9fe0 msgr2=0x7f986814acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9866a00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98680a9fe0 0x7f986814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9866a00640 1 -- 172.21.15.55:0/3235596006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680cced0 con 0x7f984c003a10 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9866a00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f985802b410 tx=0x7f98680d2730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858038070 con 0x7f984c003a10 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858034670 con 0x7f984c003a10 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98680aa3f0 con 0x7f984c003a10 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985803b040 con 0x7f984c003a10 2024-05-10T03:32:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.732+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98680cc5a0 con 0x7f984c003a10 2024-05-10T03:32:13.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.733+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986804e290 con 0x7f984c003a10 2024-05-10T03:32:13.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.735+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9858031070 con 0x7f984c003a10 2024-05-10T03:32:13.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.735+0000 7f985ea00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 0x7f984406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:13.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.736+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98580a99e0 con 0x7f984c003a10 2024-05-10T03:32:13.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.736+0000 7f9866000640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 0x7f984406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:13.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.737+0000 7f9866000640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 0x7f984406d110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98680cd500 tx=0x7f9850007990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:13.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.740+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9858074950 con 0x7f984c003a10 2024-05-10T03:32:13.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.933+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98680d35b0 con 0x7f984c003a10 2024-05-10T03:32:13.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.937+0000 7f985ea00640 1 -- 172.21.15.55:0/3235596006 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f9858045070 con 0x7f984c003a10 2024-05-10T03:32:13.940 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:13.940 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":52,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:13.875880+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.939+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 msgr2=0x7f984406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.939+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 0x7f984406d110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98680cd500 tx=0x7f9850007990 comp rx=0 tx=0).stop 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.939+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 msgr2=0x7f986814d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.939+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f985802b410 tx=0x7f98680d2730 comp rx=0 tx=0).stop 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 shutdown_connections 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f984406ac00 0x7f984406d110 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98680a9fe0 0x7f986814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98680cb870 0x7f98680a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 --2- 172.21.15.55:0/3235596006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f984c003a10 0x7f986814d760 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:13.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 >> 172.21.15.55:0/3235596006 conn(0x7f98680b5730 msgr2=0x7f98680b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:13.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 shutdown_connections 2024-05-10T03:32:13.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:13.940+0000 7f9867e00640 1 -- 172.21.15.55:0/3235596006 wait complete. 2024-05-10T03:32:13.971 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:32:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.251+0000 7fcdad800640 1 Processor -- start 2024-05-10T03:32:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.251+0000 7fcdad800640 1 -- start start 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.253+0000 7fcdad800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcda80710f0 0x7fcda81405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcdad800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda815fdf0 con 0x7fcda8071e00 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcdad800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda815f1e0 con 0x7fcda80719f0 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcdad800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda815f4c0 con 0x7fcda80710f0 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcd9fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcda80710f0 0x7fcda81405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcdace00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcda8071e00 0x7fcda8140b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60924/0 (socket says 172.21.15.55:60924) 2024-05-10T03:32:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcdace00640 1 -- 172.21.15.55:0/3315289094 learned_addr learned my addr 172.21.15.55:0/3315289094 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcd9fe00640 1 -- 172.21.15.55:0/3315289094 >> v1:172.21.15.162:6789/0 conn(0x7fcda80719f0 legacy=0x7fcda806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcd9fe00640 1 -- 172.21.15.55:0/3315289094 >> v1:172.21.15.55:6789/0 conn(0x7fcda8071e00 legacy=0x7fcda8140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.254+0000 7fcda7400640 1 -- 172.21.15.55:0/3315289094 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450856597 0 0) 0x7fcda815fdf0 con 0x7fcda8071e00 2024-05-10T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.255+0000 7fcd9fe00640 1 -- 172.21.15.55:0/3315289094 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda815f7a0 con 0x7fcda80710f0 2024-05-10T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.256+0000 7fcd9fe00640 1 --2- 172.21.15.55:0/3315289094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80710f0 0x7fcda81405f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd900097a0 tx=0x7fcd9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfffc346a0e8792 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.256+0000 7fcda7400640 1 -- 172.21.15.55:0/3315289094 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90032440 con 0x7fcda80710f0 2024-05-10T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.256+0000 7fcda7400640 1 -- 172.21.15.55:0/3315289094 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900325a0 con 0x7fcda80710f0 2024-05-10T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.256+0000 7fcda7400640 1 -- 172.21.15.55:0/3315289094 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90032930 con 0x7fcda80710f0 2024-05-10T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 -- 172.21.15.55:0/3315289094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80710f0 msgr2=0x7fcda81405f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 --2- 172.21.15.55:0/3315289094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80710f0 0x7fcda81405f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd900097a0 tx=0x7fcd9002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 -- 172.21.15.55:0/3315289094 shutdown_connections 2024-05-10T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 --2- 172.21.15.55:0/3315289094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80710f0 0x7fcda81405f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 -- 172.21.15.55:0/3315289094 >> 172.21.15.55:0/3315289094 conn(0x7fcda806c750 msgr2=0x7fcda80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.257+0000 7fcdad800640 1 -- 172.21.15.55:0/3315289094 shutdown_connections 2024-05-10T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.258+0000 7fcdad800640 1 -- 172.21.15.55:0/3315289094 wait complete. 2024-05-10T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.260+0000 7fcdad800640 1 Processor -- start 2024-05-10T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.260+0000 7fcdad800640 1 -- start start 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda80710f0 0x7fcda81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8071e00 0x7fcda8135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80a91d0 con 0x7fcda80719f0 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda81f1e40 con 0x7fcda80710f0 2024-05-10T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.261+0000 7fcdad800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda8156e60 con 0x7fcda8071e00 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40210/0 (socket says 172.21.15.55:40210) 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 -- 172.21.15.55:0/2140412465 learned_addr learned my addr 172.21.15.55:0/2140412465 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcdace00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8071e00 0x7fcda8135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcda7e00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda80710f0 0x7fcda81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 -- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8071e00 msgr2=0x7fcda8135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8071e00 0x7fcda8135270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 -- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda80710f0 msgr2=0x7fcda81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda80710f0 0x7fcda81567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.262+0000 7fcd9fe00640 1 -- 172.21.15.55:0/2140412465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90009400 con 0x7fcda80719f0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcd9fe00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c00 tx=0x7fcd900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90004230 con 0x7fcda80719f0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90004390 con 0x7fcda80719f0 2024-05-10T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90030ac0 con 0x7fcda80719f0 2024-05-10T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda81570b0 con 0x7fcda80719f0 2024-05-10T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.263+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda8157670 con 0x7fcda80719f0 2024-05-10T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.264+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda81442a0 con 0x7fcda80719f0 2024-05-10T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.266+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcd90039050 con 0x7fcda80719f0 2024-05-10T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.267+0000 7fcda4c00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 0x7fcd8406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.267+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd900a9e10 con 0x7fcda80719f0 2024-05-10T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.268+0000 7fcda7e00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 0x7fcd8406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.269+0000 7fcda7e00640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 0x7fcd8406d110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd98000f60 tx=0x7fcd98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.272+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcd900ad030 con 0x7fcda80719f0 2024-05-10T03:32:14.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:14.501+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcda804e290 con 0x7fcda80719f0 2024-05-10T03:32:15.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.342+0000 7fcda4c00640 1 -- 172.21.15.55:0/2140412465 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd9004b070 con 0x7fcda80719f0 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.346+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 msgr2=0x7fcd8406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.346+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 0x7fcd8406d110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd98000f60 tx=0x7fcd98009040 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.346+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 msgr2=0x7fcda8134d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.346+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c00 tx=0x7fcd900046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 shutdown_connections 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8071e00 0x7fcda8135270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcda80719f0 0x7fcda8134d30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcd8406ac00 0x7fcd8406d110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 --2- 172.21.15.55:0/2140412465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda80710f0 0x7fcda81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 >> 172.21.15.55:0/2140412465 conn(0x7fcda806c750 msgr2=0x7fcda8143860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:15.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 shutdown_connections 2024-05-10T03:32:15.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.347+0000 7fcdad800640 1 -- 172.21.15.55:0/2140412465 wait complete. 2024-05-10T03:32:15.383 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:32:15.383 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:32:15.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.668+0000 7f4696800640 1 Processor -- start 2024-05-10T03:32:15.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.668+0000 7f4696800640 1 -- start start 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.670+0000 7f4696800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f46900719f0 0x7f469006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.670+0000 7f4696800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4690071e00 0x7f4690141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.670+0000 7f4696800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469015fdf0 con 0x7f469006b840 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.670+0000 7f4696800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469015f1e0 con 0x7f4690071e00 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.670+0000 7f4696800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469015f4c0 con 0x7f46900719f0 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4690071e00 0x7f4690141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4690071e00 0x7f4690141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40214/0 (socket says 172.21.15.55:40214) 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 -- 172.21.15.55:0/3603739908 learned_addr learned my addr 172.21.15.55:0/3603739908 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:15.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695400640 1 --2- 172.21.15.55:0/3603739908 >> v2:172.21.15.162:3300/0 conn(0x7f46900719f0 0x7f469006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 -- 172.21.15.55:0/3603739908 >> v2:172.21.15.162:3300/0 conn(0x7f46900719f0 msgr2=0x7f469006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:15.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 --2- 172.21.15.55:0/3603739908 >> v2:172.21.15.162:3300/0 conn(0x7f46900719f0 0x7f469006b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 -- 172.21.15.55:0/3603739908 >> v1:172.21.15.162:6789/0 conn(0x7f469006b840 legacy=0x7f46900700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:15.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.671+0000 7f4695e00640 1 -- 172.21.15.55:0/3603739908 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469015f7a0 con 0x7f4690071e00 2024-05-10T03:32:15.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.672+0000 7f4695e00640 1 --2- 172.21.15.55:0/3603739908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f4690141600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f46900b41f0 tx=0x7f468802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd172c8e711 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:15.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f468fe00640 1 -- 172.21.15.55:0/3603739908 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688031600 con 0x7f4690071e00 2024-05-10T03:32:15.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f468fe00640 1 -- 172.21.15.55:0/3603739908 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468802d050 con 0x7f4690071e00 2024-05-10T03:32:15.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f468fe00640 1 -- 172.21.15.55:0/3603739908 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688031980 con 0x7f4690071e00 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 -- 172.21.15.55:0/3603739908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 msgr2=0x7f4690141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 --2- 172.21.15.55:0/3603739908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f4690141600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f46900b41f0 tx=0x7f468802a660 comp rx=0 tx=0).stop 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 -- 172.21.15.55:0/3603739908 shutdown_connections 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 --2- 172.21.15.55:0/3603739908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f4690141600 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 --2- 172.21.15.55:0/3603739908 >> v2:172.21.15.162:3300/0 conn(0x7f46900719f0 0x7f469006b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.673+0000 7f4696800640 1 -- 172.21.15.55:0/3603739908 >> 172.21.15.55:0/3603739908 conn(0x7f469006c750 msgr2=0x7f46900b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.674+0000 7f4696800640 1 -- 172.21.15.55:0/3603739908 shutdown_connections 2024-05-10T03:32:15.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.674+0000 7f4696800640 1 -- 172.21.15.55:0/3603739908 wait complete. 2024-05-10T03:32:15.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.676+0000 7f4696800640 1 Processor -- start 2024-05-10T03:32:15.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.676+0000 7f4696800640 1 -- start start 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 0x7f4690161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46900719f0 0x7f46901f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469015ad00 con 0x7f4690071e00 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469015aaf0 con 0x7f469006b840 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4696800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4690165290 con 0x7f46900719f0 2024-05-10T03:32:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4695e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4695e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40228/0 (socket says 172.21.15.55:40228) 2024-05-10T03:32:15.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.678+0000 7f4695e00640 1 -- 172.21.15.55:0/529562654 learned_addr learned my addr 172.21.15.55:0/529562654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695400640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 0x7f4690161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4694a00640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46900719f0 0x7f46901f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 -- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46900719f0 msgr2=0x7f46901f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46900719f0 0x7f46901f7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 -- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 msgr2=0x7f4690161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 0x7f4690161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 -- 172.21.15.55:0/529562654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688009b30 con 0x7f4690071e00 2024-05-10T03:32:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695400640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 0x7f4690161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4695e00640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4688034040 tx=0x7f4688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468803d860 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468803d9e0 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469015d7a0 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.679+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468803dd90 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.680+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4690164fb0 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.681+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46901597c0 con 0x7f4690071e00 2024-05-10T03:32:15.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.682+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f468803b400 con 0x7f4690071e00 2024-05-10T03:32:15.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.683+0000 7f468d600640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 0x7f467006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:15.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.683+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f468802ae40 con 0x7f4690071e00 2024-05-10T03:32:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.683+0000 7f4695400640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 0x7f467006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:15.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.685+0000 7f4695400640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 0x7f467006d110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:15.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.687+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f46880754f0 con 0x7f4690071e00 2024-05-10T03:32:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:15.929+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f469015dc80 con 0x7f4690071e00 2024-05-10T03:32:16.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.346+0000 7f468d600640 1 -- 172.21.15.55:0/529562654 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f469015dc80 con 0x7f4690071e00 2024-05-10T03:32:16.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 msgr2=0x7f467006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 0x7f467006d110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684008040 comp rx=0 tx=0).stop 2024-05-10T03:32:16.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 msgr2=0x7f46901f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4688034040 tx=0x7f4688004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 shutdown_connections 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4690071e00 0x7f46901f7a40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46900719f0 0x7f46901f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467006ac00 0x7f467006d110 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 --2- 172.21.15.55:0/529562654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469006b840 0x7f4690161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 >> 172.21.15.55:0/529562654 conn(0x7f469006c750 msgr2=0x7f46900b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.350+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 shutdown_connections 2024-05-10T03:32:16.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.351+0000 7f4696800640 1 -- 172.21.15.55:0/529562654 wait complete. 2024-05-10T03:32:16.386 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:32:16.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:16.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.664+0000 7fc90b000640 1 Processor -- start 2024-05-10T03:32:16.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.664+0000 7fc90b000640 1 -- start start 2024-05-10T03:32:16.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.666+0000 7fc90b000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc9040d1880 0x7fc9040b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.666+0000 7fc90b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.666+0000 7fc90b000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90406b420 con 0x7fc90406af30 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.666+0000 7fc90b000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90406b700 con 0x7fc9040d1880 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.666+0000 7fc90b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90406b9e0 con 0x7fc90406ab20 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.667+0000 7fc909200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.667+0000 7fc909200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47942/0 (socket says 172.21.15.55:47942) 2024-05-10T03:32:16.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.667+0000 7fc909200640 1 -- 172.21.15.55:0/3784376491 learned_addr learned my addr 172.21.15.55:0/3784376491 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:16.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.667+0000 7fc909c00640 1 --2- 172.21.15.55:0/3784376491 >> v2:172.21.15.162:3300/0 conn(0x7fc9040d1880 0x7fc9040b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.668+0000 7fc909200640 1 -- 172.21.15.55:0/3784376491 >> v2:172.21.15.162:3300/0 conn(0x7fc9040d1880 msgr2=0x7fc9040b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:16.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.668+0000 7fc909200640 1 --2- 172.21.15.55:0/3784376491 >> v2:172.21.15.162:3300/0 conn(0x7fc9040d1880 0x7fc9040b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.668+0000 7fc909200640 1 -- 172.21.15.55:0/3784376491 >> v1:172.21.15.162:6789/0 conn(0x7fc90406af30 legacy=0x7fc9040d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.668+0000 7fc909200640 1 -- 172.21.15.55:0/3784376491 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040b6a50 con 0x7fc90406ab20 2024-05-10T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.670+0000 7fc909200640 1 --2- 172.21.15.55:0/3784376491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009870 tx=0x7fc8f002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4406588f692812b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:16.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.670+0000 7fc903e00640 1 -- 172.21.15.55:0/3784376491 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f002ae80 con 0x7fc90406ab20 2024-05-10T03:32:16.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.670+0000 7fc903e00640 1 -- 172.21.15.55:0/3784376491 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0032440 con 0x7fc90406ab20 2024-05-10T03:32:16.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.670+0000 7fc903e00640 1 -- 172.21.15.55:0/3784376491 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00327b0 con 0x7fc90406ab20 2024-05-10T03:32:16.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 -- 172.21.15.55:0/3784376491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 msgr2=0x7fc9040ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 --2- 172.21.15.55:0/3784376491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009870 tx=0x7fc8f002a130 comp rx=0 tx=0).stop 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 -- 172.21.15.55:0/3784376491 shutdown_connections 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 --2- 172.21.15.55:0/3784376491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040ce250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 --2- 172.21.15.55:0/3784376491 >> v2:172.21.15.162:3300/0 conn(0x7fc9040d1880 0x7fc9040b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 -- 172.21.15.55:0/3784376491 >> 172.21.15.55:0/3784376491 conn(0x7fc9040d31a0 msgr2=0x7fc9040d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 -- 172.21.15.55:0/3784376491 shutdown_connections 2024-05-10T03:32:16.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.671+0000 7fc90b000640 1 -- 172.21.15.55:0/3784376491 wait complete. 2024-05-10T03:32:16.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.674+0000 7fc90b000640 1 Processor -- start 2024-05-10T03:32:16.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.674+0000 7fc90b000640 1 -- start start 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9040d1880 0x7fc9040cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90415e0b0 con 0x7fc90406af30 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041f4550 con 0x7fc9040d1880 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.675+0000 7fc90b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040ca6c0 con 0x7fc90406ab20 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40244/0 (socket says 172.21.15.55:40244) 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 -- 172.21.15.55:0/3221498223 learned_addr learned my addr 172.21.15.55:0/3221498223 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909c00640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc90a600640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9040d1880 0x7fc9040cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 -- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 msgr2=0x7fc9040c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 -- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9040d1880 msgr2=0x7fc9040cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9040d1880 0x7fc9040cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.676+0000 7fc909200640 1 -- 172.21.15.55:0/3221498223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00094d0 con 0x7fc90406af30 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc909200640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f003d830 con 0x7fc90406af30 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f003d990 con 0x7fc90406af30 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040ca910 con 0x7fc90406af30 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f003dd40 con 0x7fc90406af30 2024-05-10T03:32:16.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.677+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040caed0 con 0x7fc90406af30 2024-05-10T03:32:16.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.679+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90404e290 con 0x7fc90406af30 2024-05-10T03:32:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.680+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc8f0039040 con 0x7fc90406af30 2024-05-10T03:32:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.681+0000 7fc901600640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 0x7fc8e406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.681+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc8f00a9fe0 con 0x7fc90406af30 2024-05-10T03:32:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.681+0000 7fc909c00640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 0x7fc8e406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.683+0000 7fc909c00640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 0x7fc8e406d140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009a30 tx=0x7fc8f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.685+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc8f0074f70 con 0x7fc90406af30 2024-05-10T03:32:16.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.929+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc904066be0 con 0x7fc90406af30 2024-05-10T03:32:16.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.931+0000 7fc901600640 1 -- 172.21.15.55:0/3221498223 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fc8f0078590 con 0x7fc90406af30 2024-05-10T03:32:16.935 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:16.935 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:16.935 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:16.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.933+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 msgr2=0x7fc8e406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.933+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 0x7fc8e406d140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009a30 tx=0x7fc8f4008040 comp rx=0 tx=0).stop 2024-05-10T03:32:16.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.933+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 msgr2=0x7fc904154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:16.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.933+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f00046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 shutdown_connections 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8e406ac30 0x7fc8e406d140 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9040d1880 0x7fc9040cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc90406af30 0x7fc904154550 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 --2- 172.21.15.55:0/3221498223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc90406ab20 0x7fc9040c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 >> 172.21.15.55:0/3221498223 conn(0x7fc9040d31a0 msgr2=0x7fc904160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 shutdown_connections 2024-05-10T03:32:16.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:16.934+0000 7fc90b000640 1 -- 172.21.15.55:0/3221498223 wait complete. 2024-05-10T03:32:16.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:17.049 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:32:17.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:32:17.247 INFO:tasks.ceph.mds.c.smithi055.stderr:2024-05-10T03:32:17.244+0000 7f09f7600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:32:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.251+0000 7f7841400640 1 Processor -- start 2024-05-10T03:32:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.252+0000 7f7841400640 1 -- start start 2024-05-10T03:32:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.254+0000 7f7841400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f783c15fe50 0x7f783c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.254+0000 7f7841400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c06ad20 con 0x7f783c0b2140 2024-05-10T03:32:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.254+0000 7f7841400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783c141f50 con 0x7f783c0b2550 2024-05-10T03:32:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.254+0000 7f7841400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783c06a110 con 0x7f783c15fe50 2024-05-10T03:32:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7833e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f783c0b2140 0x7f783c142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60930/0 (socket says 172.21.15.55:60930) 2024-05-10T03:32:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7833e00640 1 -- 172.21.15.55:0/4068195563 learned_addr learned my addr 172.21.15.55:0/4068195563 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f783a800640 1 --2- 172.21.15.55:0/4068195563 >> v2:172.21.15.162:3300/0 conn(0x7f783c15fe50 0x7f783c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901878344 0 0) 0x7f783c06ad20 con 0x7f783c0b2140 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7820003520 con 0x7f783c0b2140 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901090561 0 0) 0x7f7820003520 con 0x7f783c0b2140 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 >> v2:172.21.15.162:3300/0 conn(0x7f783c15fe50 msgr2=0x7f783c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.255+0000 7f7839e00640 1 --2- 172.21.15.55:0/4068195563 >> v2:172.21.15.162:3300/0 conn(0x7f783c15fe50 0x7f783c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 >> v1:172.21.15.162:6789/0 conn(0x7f783c0b2550 legacy=0x7f783c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c06a3f0 con 0x7f783c0b2140 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7824001670 con 0x7f783c0b2140 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 >> v1:172.21.15.55:6789/0 conn(0x7f783c0b2140 legacy=0x7f783c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7820003970 0x7f7820003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 0x7f7820004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7820005180 0x7f782000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c06a110 con 0x7f78200047c0 2024-05-10T03:32:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782000c110 con 0x7f7820003970 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782000c2d0 con 0x7f7820005180 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7833e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 0x7f7820004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f783a800640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7820003970 0x7f7820003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7833e00640 1 -- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7820005180 msgr2=0x7f782000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.256+0000 7f7833e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7820005180 0x7f782000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.257+0000 7f7833e00640 1 -- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7820003970 msgr2=0x7f7820003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.257+0000 7f7833e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7820003970 0x7f7820003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.257+0000 7f7833e00640 1 -- 172.21.15.55:0/4068195563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820003520 con 0x7f78200047c0 2024-05-10T03:32:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.257+0000 7f783b200640 1 -- 172.21.15.55:0/4068195563 reap_dead start 2024-05-10T03:32:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7833e00640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 0x7f7820004bd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f782400c190 tx=0x7f78200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8684c9cad9320f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824005ad0 con 0x7f78200047c0 2024-05-10T03:32:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782402d070 con 0x7f78200047c0 2024-05-10T03:32:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7839e00640 1 -- 172.21.15.55:0/4068195563 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824007070 con 0x7f78200047c0 2024-05-10T03:32:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7841400640 1 -- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 msgr2=0x7f7820004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.258+0000 7f7841400640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 0x7f7820004bd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f782400c190 tx=0x7f78200057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.259+0000 7f7841400640 1 -- 172.21.15.55:0/4068195563 shutdown_connections 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.259+0000 7f7841400640 1 --2- 172.21.15.55:0/4068195563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78200047c0 0x7f7820004bd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.259+0000 7f7841400640 1 -- 172.21.15.55:0/4068195563 >> 172.21.15.55:0/4068195563 conn(0x7f783c0cbb90 msgr2=0x7f783c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.259+0000 7f7841400640 1 -- 172.21.15.55:0/4068195563 shutdown_connections 2024-05-10T03:32:17.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.259+0000 7f7841400640 1 -- 172.21.15.55:0/4068195563 wait complete. 2024-05-10T03:32:17.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.261+0000 7f7841400640 1 Processor -- start 2024-05-10T03:32:17.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.261+0000 7f7841400640 1 -- start start 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 0x7f783c13c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f783c15c8c0 0x7f783c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c0ae1d0 con 0x7f783c070920 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c071420 con 0x7f78200047c0 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7841400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c071270 con 0x7f783c15c8c0 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7833e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7833e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40264/0 (socket says 172.21.15.55:40264) 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7833e00640 1 -- 172.21.15.55:0/1860158131 learned_addr learned my addr 172.21.15.55:0/1860158131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f783a800640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 0x7f783c13c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f7833e00640 1 -- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f783c15c8c0 msgr2=0x7f783c15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.263+0000 7f783b200640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f783c15c8c0 0x7f783c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7833e00640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f783c15c8c0 0x7f783c15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7833e00640 1 -- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 msgr2=0x7f783c13c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7833e00640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 0x7f783c13c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7833e00640 1 -- 172.21.15.55:0/1860158131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0aca00 con 0x7f783c070920 2024-05-10T03:32:17.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f783a800640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 0x7f783c13c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:17.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7833e00640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7824001930 tx=0x7f783c06ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824004aa0 con 0x7f783c070920 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824004c20 con 0x7f783c070920 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c15d210 con 0x7f783c070920 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824007070 con 0x7f783c070920 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.264+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c071950 con 0x7f783c070920 2024-05-10T03:32:17.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.266+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c066f60 con 0x7f783c070920 2024-05-10T03:32:17.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.267+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f782402f030 con 0x7f783c070920 2024-05-10T03:32:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.268+0000 7f7832a00640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 0x7f781806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.268+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f78240a8d20 con 0x7f783c070920 2024-05-10T03:32:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.269+0000 7f783a800640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 0x7f781806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.270+0000 7f783a800640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 0x7f781806d110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f782c009a60 tx=0x7f782c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.273+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7824077840 con 0x7f783c070920 2024-05-10T03:32:17.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.475+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f783c066f60 con 0x7f783c070920 2024-05-10T03:32:17.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.477+0000 7f7832a00640 1 -- 172.21.15.55:0/1860158131 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f78240773d0 con 0x7f783c070920 2024-05-10T03:32:17.481 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:17.481 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":52,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:13.875880+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:17.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 msgr2=0x7f781806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 0x7f781806d110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f782c009a60 tx=0x7f782c008040 comp rx=0 tx=0).stop 2024-05-10T03:32:17.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 msgr2=0x7f783c070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7824001930 tx=0x7f783c06ac90 comp rx=0 tx=0).stop 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 shutdown_connections 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f783c15c8c0 0x7f783c15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f783c070920 0x7f783c070d30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f781806ac00 0x7f781806d110 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 --2- 172.21.15.55:0/1860158131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78200047c0 0x7f783c13c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 >> 172.21.15.55:0/1860158131 conn(0x7f783c0cbb90 msgr2=0x7f783c140d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 shutdown_connections 2024-05-10T03:32:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.480+0000 7f7841400640 1 -- 172.21.15.55:0/1860158131 wait complete. 2024-05-10T03:32:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:32:17.509 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:32:17.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.792+0000 7fb4f8c00640 1 Processor -- start 2024-05-10T03:32:17.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.792+0000 7fb4f8c00640 1 -- start start 2024-05-10T03:32:17.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.795+0000 7fb4f8c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 0x7fb4f4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.795+0000 7fb4f8c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 0x7fb4f4142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.795+0000 7fb4f8c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f406ad20 con 0x7fb4f40b2550 2024-05-10T03:32:17.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.795+0000 7fb4f8c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f4141f50 con 0x7fb4f415fe50 2024-05-10T03:32:17.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.795+0000 7fb4f8c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f406a110 con 0x7fb4f40b2140 2024-05-10T03:32:17.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.796+0000 7fb4f1e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 0x7fb4f4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.796+0000 7fb4f1e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 0x7fb4f4141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:57542/0 (socket says 172.21.15.55:57542) 2024-05-10T03:32:17.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.796+0000 7fb4f1e00640 1 -- 172.21.15.55:0/4011387965 learned_addr learned my addr 172.21.15.55:0/4011387965 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:17.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.796+0000 7fb4e9400640 1 --2- 172.21.15.55:0/4011387965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 0x7fb4f4142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.797+0000 7fb4e9400640 1 -- 172.21.15.55:0/4011387965 >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 msgr2=0x7fb4f4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:17.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.797+0000 7fb4e9400640 1 --2- 172.21.15.55:0/4011387965 >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 0x7fb4f4141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.797+0000 7fb4e9400640 1 -- 172.21.15.55:0/4011387965 >> v1:172.21.15.162:6789/0 conn(0x7fb4f40b2550 legacy=0x7fb4f406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.797+0000 7fb4e9400640 1 -- 172.21.15.55:0/4011387965 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f406a3f0 con 0x7fb4f40b2140 2024-05-10T03:32:17.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.798+0000 7fb4e9400640 1 --2- 172.21.15.55:0/4011387965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 0x7fb4f4142440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097a0 tx=0x7fb4dc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf7d9d7ea20ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.799+0000 7fb4f1400640 1 -- 172.21.15.55:0/4011387965 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc032440 con 0x7fb4f40b2140 2024-05-10T03:32:17.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.799+0000 7fb4f1400640 1 -- 172.21.15.55:0/4011387965 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc0325a0 con 0x7fb4f40b2140 2024-05-10T03:32:17.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.799+0000 7fb4f1400640 1 -- 172.21.15.55:0/4011387965 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc032930 con 0x7fb4f40b2140 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.800+0000 7fb4f8c00640 1 -- 172.21.15.55:0/4011387965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 msgr2=0x7fb4f4142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.800+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/4011387965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 0x7fb4f4142440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0097a0 tx=0x7fb4dc02a210 comp rx=0 tx=0).stop 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 -- 172.21.15.55:0/4011387965 shutdown_connections 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/4011387965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f40b2140 0x7fb4f4142440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/4011387965 >> v2:172.21.15.162:3300/0 conn(0x7fb4f415fe50 0x7fb4f4141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 -- 172.21.15.55:0/4011387965 >> 172.21.15.55:0/4011387965 conn(0x7fb4f40cbb90 msgr2=0x7fb4f40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 -- 172.21.15.55:0/4011387965 shutdown_connections 2024-05-10T03:32:17.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.801+0000 7fb4f8c00640 1 -- 172.21.15.55:0/4011387965 wait complete. 2024-05-10T03:32:17.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.804+0000 7fb4f8c00640 1 Processor -- start 2024-05-10T03:32:17.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.804+0000 7fb4f8c00640 1 -- start start 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.806+0000 7fb4f8c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 0x7fb4f41429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f8c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f8c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 0x7fb4f41447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f8c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41fb420 con 0x7fb4f40b2550 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f8c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41f7de0 con 0x7fb4f40b2140 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f8c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f413b400 con 0x7fb4f415fe50 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40276/0 (socket says 172.21.15.55:40276) 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4e9400640 1 -- 172.21.15.55:0/3816502707 learned_addr learned my addr 172.21.15.55:0/3816502707 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f2800640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 0x7fb4f41447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.807+0000 7fb4f1e00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 0x7fb4f41429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 -- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 msgr2=0x7fb4f41447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 0x7fb4f41447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 -- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 msgr2=0x7fb4f41429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 0x7fb4f41429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 -- 172.21.15.55:0/3816502707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009400 con 0x7fb4f40b2550 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4f2800640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 0x7fb4f41447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4f1e00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 0x7fb4f41429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:17.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4e9400640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc035040 tx=0x7fb4dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc006e10 con 0x7fb4f40b2550 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc032e90 con 0x7fb4f40b2550 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc030be0 con 0x7fb4f40b2550 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f41fb590 con 0x7fb4f40b2550 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.808+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f41fb7e0 con 0x7fb4f40b2550 2024-05-10T03:32:17.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.810+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f4066f60 con 0x7fb4f40b2550 2024-05-10T03:32:17.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.811+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb4dc039040 con 0x7fb4f40b2550 2024-05-10T03:32:17.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.812+0000 7fb4eaa00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 0x7fb4d006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:17.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.812+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb4dc0a9f40 con 0x7fb4f40b2550 2024-05-10T03:32:17.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.812+0000 7fb4f1e00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 0x7fb4d006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:17.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.814+0000 7fb4f1e00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 0x7fb4d006d110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009a20 tx=0x7fb4e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:17.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:17.817+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb4dc075060 con 0x7fb4f40b2550 2024-05-10T03:32:18.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.013+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f4067220 con 0x7fb4d006ac00 2024-05-10T03:32:18.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.038+0000 7fb4eaa00640 1 -- 172.21.15.55:0/3816502707 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275960 (secure 0 0 0) 0x7fb4f4067220 con 0x7fb4d006ac00 2024-05-10T03:32:18.045 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:18.047 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 msgr2=0x7fb4d006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 0x7fb4d006d110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009a20 tx=0x7fb4e4008040 comp rx=0 tx=0).stop 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 msgr2=0x7fb4f4140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc035040 tx=0x7fb4dc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 shutdown_connections 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4f415fe50 0x7fb4f41447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4f40b2550 0x7fb4f4140090 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb4d006ac00 0x7fb4d006d110 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 --2- 172.21.15.55:0/3816502707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4f40b2140 0x7fb4f41429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 >> 172.21.15.55:0/3816502707 conn(0x7fb4f40cbb90 msgr2=0x7fb4f40af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:18.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 shutdown_connections 2024-05-10T03:32:18.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.045+0000 7fb4f8c00640 1 -- 172.21.15.55:0/3816502707 wait complete. 2024-05-10T03:32:18.078 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-05-10T03:32:17.008381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238240,"kb_used_data":5680,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736480,"statfs":{"total":773094113280,"available":772850155520,"internally_reserved":0,"allocated":5816320,"data_stored":3177830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.727360"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299792+0000","last_change":"2024-05-10T03:32:11.299792+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299792+0000","last_undegraded":"2024-05-10T03:32:11.299792+0000","last_fullsized":"2024-05-10T03:32:11.299792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:09:45.664808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291695+0000","last_change":"2024-05-10T03:32:11.291695+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291695+0000","last_undegraded":"2024-05-10T03:32:11.291695+0000","last_fullsized":"2024-05-10T03:32:11.291695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.292711+0000","last_change":"2024-05-10T03:32:11.292711+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.292711+0000","last_undegraded":"2024-05-10T03:32:11.292711+0000","last_fullsized":"2024-05-10T03:32:11.292711+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286375+0000","last_change":"2024-05-10T03:32:11.286375+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286375+0000","last_undegraded":"2024-05-10T03:32:11.286375+0000","last_fullsized":"2024-05-10T03:32:11.286375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:43.411173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.292787+0000","last_change":"2024-05-10T03:32:11.292787+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.292787+0000","last_undegraded":"2024-05-10T03:32:11.292787+0000","last_fullsized":"2024-05-10T03:32:11.292787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.288579+0000","last_change":"2024-05-10T03:32:11.288579+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.288579+0000","last_undegraded":"2024-05-10T03:32:11.288579+0000","last_fullsized":"2024-05-10T03:32:11.288579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297360+0000","last_change":"2024-05-10T03:32:11.297360+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297360+0000","last_undegraded":"2024-05-10T03:32:11.297360+0000","last_fullsized":"2024-05-10T03:32:11.297360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:34.518049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291798+0000","last_change":"2024-05-10T03:32:11.291798+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291798+0000","last_undegraded":"2024-05-10T03:32:11.291798+0000","last_fullsized":"2024-05-10T03:32:11.291798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.303307+0000","last_change":"2024-05-10T03:32:11.303307+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.303307+0000","last_undegraded":"2024-05-10T03:32:11.303307+0000","last_fullsized":"2024-05-10T03:32:11.303307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.288117+0000","last_change":"2024-05-10T03:32:11.288117+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.288117+0000","last_undegraded":"2024-05-10T03:32:11.288117+0000","last_fullsized":"2024-05-10T03:32:11.288117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.303047+0000","last_change":"2024-05-10T03:32:11.303047+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.303047+0000","last_undegraded":"2024-05-10T03:32:11.303047+0000","last_fullsized":"2024-05-10T03:32:11.303047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296004+0000","last_change":"2024-05-10T03:32:11.296004+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296004+0000","last_undegraded":"2024-05-10T03:32:11.296004+0000","last_fullsized":"2024-05-10T03:32:11.296004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:46.848796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291326+0000","last_change":"2024-05-10T03:32:11.291326+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291326+0000","last_undegraded":"2024-05-10T03:32:11.291326+0000","last_fullsized":"2024-05-10T03:32:11.291326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:48:20.732699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296893+0000","last_change":"2024-05-10T03:32:11.296893+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296893+0000","last_undegraded":"2024-05-10T03:32:11.296893+0000","last_fullsized":"2024-05-10T03:32:11.296893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.300620+0000","last_change":"2024-05-10T03:32:11.300620+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.300620+0000","last_undegraded":"2024-05-10T03:32:11.300620+0000","last_fullsized":"2024-05-10T03:32:11.300620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295635+0000","last_change":"2024-05-10T03:32:11.295635+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295635+0000","last_undegraded":"2024-05-10T03:32:11.295635+0000","last_fullsized":"2024-05-10T03:32:11.295635+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:56.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290350+0000","last_change":"2024-05-10T03:32:11.290350+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290350+0000","last_undegraded":"2024-05-10T03:32:11.290350+0000","last_fullsized":"2024-05-10T03:32:11.290350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299957+0000","last_change":"2024-05-10T03:32:11.299957+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299957+0000","last_undegraded":"2024-05-10T03:32:11.299957+0000","last_fullsized":"2024-05-10T03:32:11.299957+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:42:58.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297574+0000","last_change":"2024-05-10T03:32:11.297574+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297574+0000","last_undegraded":"2024-05-10T03:32:11.297574+0000","last_fullsized":"2024-05-10T03:32:11.297574+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297642+0000","last_change":"2024-05-10T03:32:11.297642+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297642+0000","last_undegraded":"2024-05-10T03:32:11.297642+0000","last_fullsized":"2024-05-10T03:32:11.297642+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291456+0000","last_change":"2024-05-10T03:32:11.291456+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291456+0000","last_undegraded":"2024-05-10T03:32:11.291456+0000","last_fullsized":"2024-05-10T03:32:11.291456+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296224+0000","last_change":"2024-05-10T03:32:11.296224+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296224+0000","last_undegraded":"2024-05-10T03:32:11.296224+0000","last_fullsized":"2024-05-10T03:32:11.296224+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:04.809817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.289069+0000","last_change":"2024-05-10T03:32:11.289069+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.289069+0000","last_undegraded":"2024-05-10T03:32:11.289069+0000","last_fullsized":"2024-05-10T03:32:11.289069+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:39.030457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299567+0000","last_change":"2024-05-10T03:32:11.299567+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299567+0000","last_undegraded":"2024-05-10T03:32:11.299567+0000","last_fullsized":"2024-05-10T03:32:11.299567+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:35:55.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.298216+0000","last_change":"2024-05-10T03:32:11.298216+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.298216+0000","last_undegraded":"2024-05-10T03:32:11.298216+0000","last_fullsized":"2024-05-10T03:32:11.298216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.300679+0000","last_change":"2024-05-10T03:32:11.300679+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.300679+0000","last_undegraded":"2024-05-10T03:32:11.300679+0000","last_fullsized":"2024-05-10T03:32:11.300679+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290328+0000","last_change":"2024-05-10T03:32:11.290328+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290328+0000","last_undegraded":"2024-05-10T03:32:11.290328+0000","last_fullsized":"2024-05-10T03:32:11.290328+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:28.939240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295926+0000","last_change":"2024-05-10T03:32:11.295926+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295926+0000","last_undegraded":"2024-05-10T03:32:11.295926+0000","last_fullsized":"2024-05-10T03:32:11.295926+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:37.925978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295569+0000","last_change":"2024-05-10T03:32:11.295569+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295569+0000","last_undegraded":"2024-05-10T03:32:11.295569+0000","last_fullsized":"2024-05-10T03:32:11.295569+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:12.822129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286817+0000","last_change":"2024-05-10T03:32:11.286817+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286817+0000","last_undegraded":"2024-05-10T03:32:11.286817+0000","last_fullsized":"2024-05-10T03:32:11.286817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286410+0000","last_change":"2024-05-10T03:32:11.286410+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286410+0000","last_undegraded":"2024-05-10T03:32:11.286410+0000","last_fullsized":"2024-05-10T03:32:11.286410+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:42.125952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290420+0000","last_change":"2024-05-10T03:32:11.290420+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290420+0000","last_undegraded":"2024-05-10T03:32:11.290420+0000","last_fullsized":"2024-05-10T03:32:11.290420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:42.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-10T03:32:08.348410+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:08.348410+0000","last_peered":"2024-05-10T03:32:08.348410+0000","last_clean":"2024-05-10T03:32:08.348410+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:08.348410+0000","last_undegraded":"2024-05-10T03:32:08.348410+0000","last_fullsized":"2024-05-10T03:32:08.348410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1175552,"data_stored":841539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2230000000000001,"15min":1.2270000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.367,"15min":1.365},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8100000000000001,"5min":1.877,"15min":1.877},"last":1.262}]},{"osd":1,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.266,"15min":1.274},"min":{"1min":0.84999999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.667,"5min":1.925,"15min":1.925},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1349999999999998,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4410000000000001,"15min":1.4490000000000001},"min":{"1min":0.81200000000000006,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.819,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3899999999999999,"15min":1.3919999999999999},"min":{"1min":1.0489999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.3520000000000001,"15min":1.3340000000000001},"min":{"1min":1.0109999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1120000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.49,"15min":1.484},"min":{"1min":1.071,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.1339999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5169999999999999,"15min":1.506},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.6539999999999999}]},{"osd":5,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.546,"15min":1.516},"min":{"1min":1.2030000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1779999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.327},{"interface":"front","average":{"1min":1.76,"5min":1.712,"15min":1.704},"min":{"1min":1.383,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.5009999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.762,"5min":1.7370000000000001,"15min":1.7330000000000001},"min":{"1min":1.3939999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.653,"15min":1.641},"min":{"1min":1.0600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.7669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.073,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.03},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.329,"15min":1.351},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8640000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4590000000000001,"5min":3.113,"15min":3.113},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4750000000000001,"15min":1.4970000000000001},"min":{"1min":1.046,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.944,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.327}]},{"osd":2,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.246,"15min":1.232},"min":{"1min":0.93000000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.444},{"interface":"front","average":{"1min":1.248,"5min":1.482,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5589999999999999,"5min":3.569,"15min":3.569},"last":1.698}]},{"osd":3,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.591,"15min":1.6279999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.863},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.306,"15min":1.304},"min":{"1min":1.0409999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.744,"5min":1.931,"15min":1.931},"last":1.5049999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4359999999999999,"15min":1.4299999999999999},"min":{"1min":1.069,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.8759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5640000000000001,"15min":1.585},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.819,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.9610000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.1259999999999999,"5min":1,"15min":1},"max":{"1min":1.857,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.004,"5min":2.472,"15min":2.472},"last":1.772}]},{"osd":7,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.665,"15min":1.671},"min":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":1.9199999999999999,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":2.1499999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.651,"15min":1.655},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0070000000000001,"5min":2.359,"15min":2.359},"last":1.75}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.452},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.25}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.1559999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.004}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7030000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.841}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.806},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.6080000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":1.885}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.173,"15min":1.177},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.1359999999999999,"15min":1.139},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5629999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.165,"15min":1.1559999999999999},"min":{"1min":0.79100000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.734,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.278,"15min":1.2829999999999999},"min":{"1min":0.85699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.994,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.302}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8280000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.042},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9179999999999999,"5min":2.29,"15min":2.29},"last":1.718}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.458,"15min":1.476},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.76,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5900000000000001,"15min":1.589},"min":{"1min":1.2729999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":1.9530000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5009999999999999,"15min":1.5049999999999999},"min":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.8129999999999999,"5min":2.512,"15min":2.512},"last":1.536},{"interface":"front","average":{"1min":1.456,"5min":1.552,"15min":1.5680000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.982,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5900000000000001,"15min":1.5940000000000001},"min":{"1min":1.153,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1139999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7829999999999999,"15min":1.7969999999999999},"min":{"1min":1.1040000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0409999999999999,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.8500000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6930000000000001,"15min":1.7030000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.069,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.601},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7609999999999999,"15min":1.7509999999999999},"min":{"1min":1.4730000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.282,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.655}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.95399999999999996,"15min":0.97599999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4850000000000001,"5min":1.708,"15min":1.708},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"min":{"1min":0.55800000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4370000000000001,"5min":1.603,"15min":1.603},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.24,"15min":1.254},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.925,"5min":1.964,"15min":1.964},"last":0.94599999999999995}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1779999999999999,"15min":1.1839999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.897,"5min":1.99,"15min":1.99},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":1.369},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.919,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.95099999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.8080000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.3879999999999999,"15min":1.3779999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.1349999999999998,"5min":2.464,"15min":2.464},"last":1.284}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.6479999999999999,"15min":1.6859999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.109,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.462,"15min":1.484},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0499999999999998,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.024}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.2559999999999998,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.589},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.774,"15min":1.804},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4729999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.417}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.5629999999999999,"15min":1.5900000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.1379999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.571,"5min":1.6479999999999999,"15min":1.661},"min":{"1min":1.052,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.2690000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.48}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.494,"5min":1.772,"15min":1.772},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.431,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5600000000000001,"5min":1.621,"15min":1.621},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1930000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6930000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1240000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5009999999999999,"5min":2.169,"15min":2.169},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.25,"15min":1.2629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9650000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.526,"15min":1.548},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.133,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.516,"15min":1.5569999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.821,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.175}]},{"osd":5,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.036,"5min":2.508,"15min":2.508},"last":1.008},{"interface":"front","average":{"1min":1.546,"5min":1.6140000000000001,"15min":1.625},"min":{"1min":1.069,"5min":1.004,"15min":1.004},"max":{"1min":2.1190000000000002,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1140000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1139999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5820000000000001,"15min":1.5980000000000001},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.982,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.0549999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.641,"15min":1.6599999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.1360000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.369},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.659,"15min":1.679},"min":{"1min":1.052,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.169,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.129}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.54600000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.256,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.488,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.1699999999999999,"15min":1.1910000000000001},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4330000000000001,"5min":2.444,"15min":2.444},"last":1.413}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.3,"15min":1.3280000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.556,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3100000000000001},"min":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7789999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.9399999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.673,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":2.3159999999999998},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5669999999999999,"15min":1.597},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":1.8220000000000001,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3660000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.5600000000000001,"15min":1.5980000000000001},"min":{"1min":1.069,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":2.2450000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.6200000000000001,"15min":1.647},"min":{"1min":1.1080000000000001,"5min":1.077,"15min":1.077},"max":{"1min":1.9770000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":2.0190000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.6990000000000001,"15min":1.7370000000000001},"min":{"1min":0.94999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0550000000000002,"5min":3.48,"15min":3.48},"last":2.4470000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.716,"15min":1.748},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.0800000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.7170000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.7270000000000001,"15min":1.7470000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0910000000000002,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":2.1859999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.762,"15min":1.796},"min":{"1min":1.173,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.7909999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":2.5449999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":841539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.999,"15min":1.036},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5629999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":1.1599999999999999,"15min":1.2110000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.054}]},{"osd":2,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5389999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.498},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4550000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.226}]},{"osd":3,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.089,"15min":1.1140000000000001},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.448,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.929,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.78}]},{"osd":4,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.6040000000000001,"15min":1.6539999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0209999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.431,"15min":1.446},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.661,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.464,"15min":1.486},"min":{"1min":0.85799999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9710000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.401},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6080000000000001,"15min":1.647},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8999999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.6279999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6299999999999999,"15min":1.6539999999999999},"min":{"1min":1.028,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.206,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.7150000000000001,"15min":1.7629999999999999},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.972,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.6659999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.77,"15min":1.8200000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.946,"5min":3.149,"15min":3.149},"last":1.458},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5609999999999999,"15min":1.581},"min":{"1min":0.91500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.99,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.714}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:32:18.084 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:32:18.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.406+0000 7f00d7000640 1 Processor -- start 2024-05-10T03:32:18.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.406+0000 7f00d7000640 1 -- start start 2024-05-10T03:32:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d7000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d7000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f00d01618b0 0x7f00d006aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d7000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d00d25a0 con 0x7f00d00b2170 2024-05-10T03:32:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d7000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d00d2b20 con 0x7f00d01618b0 2024-05-10T03:32:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d7000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d015db20 con 0x7f00d00b6680 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d5200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f00d00b2170 0x7f00d01611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60938/0 (socket says 172.21.15.55:60938) 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.408+0000 7f00d5200640 1 -- 172.21.15.55:0/2512866716 learned_addr learned my addr 172.21.15.55:0/2512866716 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d6600640 1 --2- 172.21.15.55:0/2512866716 >> v2:172.21.15.55:3300/0 conn(0x7f00d01618b0 0x7f00d006aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d5c00640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00cfe00640 1 -- 172.21.15.55:0/2512866716 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207311419 0 0) 0x7f00d00d25a0 con 0x7f00d00b2170 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00cfe00640 1 -- 172.21.15.55:0/2512866716 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b8003520 con 0x7f00d00b2170 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d6600640 1 -- 172.21.15.55:0/2512866716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 msgr2=0x7f00d015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d6600640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d6600640 1 -- 172.21.15.55:0/2512866716 >> v1:172.21.15.55:6789/0 conn(0x7f00d00b2170 legacy=0x7f00d01611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.409+0000 7f00d6600640 1 -- 172.21.15.55:0/2512866716 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00b28d0 con 0x7f00d01618b0 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.410+0000 7f00d6600640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d01618b0 0x7f00d006aab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f00c80097a0 tx=0x7f00c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762a7a4ce4de82b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00cfe00640 1 -- 172.21.15.55:0/2512866716 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8032440 con 0x7f00d01618b0 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00cfe00640 1 -- 172.21.15.55:0/2512866716 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c80325a0 con 0x7f00d01618b0 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00cfe00640 1 -- 172.21.15.55:0/2512866716 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8032910 con 0x7f00d01618b0 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 -- 172.21.15.55:0/2512866716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d01618b0 msgr2=0x7f00d006aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d01618b0 0x7f00d006aab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f00c80097a0 tx=0x7f00c802a180 comp rx=0 tx=0).stop 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 -- 172.21.15.55:0/2512866716 shutdown_connections 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d01618b0 0x7f00d006aab0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 --2- 172.21.15.55:0/2512866716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.411+0000 7f00d7000640 1 -- 172.21.15.55:0/2512866716 >> 172.21.15.55:0/2512866716 conn(0x7f00d00b5730 msgr2=0x7f00d00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:18.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.412+0000 7f00d7000640 1 -- 172.21.15.55:0/2512866716 shutdown_connections 2024-05-10T03:32:18.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.412+0000 7f00d7000640 1 -- 172.21.15.55:0/2512866716 wait complete. 2024-05-10T03:32:18.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.414+0000 7f00d7000640 1 Processor -- start 2024-05-10T03:32:18.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.414+0000 7f00d7000640 1 -- start start 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01618b0 0x7f00d006ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0153f00 con 0x7f00d00b2170 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d0155660 con 0x7f00d01618b0 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d7000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d015eb60 con 0x7f00d00b6680 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40292/0 (socket says 172.21.15.55:40292) 2024-05-10T03:32:18.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 -- 172.21.15.55:0/664730137 learned_addr learned my addr 172.21.15.55:0/664730137 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 -- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 msgr2=0x7f00d006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 -- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01618b0 msgr2=0x7f00d006ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01618b0 0x7f00d006ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.416+0000 7f00d5c00640 1 -- 172.21.15.55:0/664730137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c8009400 con 0x7f00d00b2170 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00d5c00640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f00c4007e40 tx=0x7f00c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4014620 con 0x7f00d00b2170 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c400ee90 con 0x7f00d00b2170 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40149c0 con 0x7f00d00b2170 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d015edb0 con 0x7f00d00b2170 2024-05-10T03:32:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.417+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d015f370 con 0x7f00d00b2170 2024-05-10T03:32:18.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.419+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d0067740 con 0x7f00d00b2170 2024-05-10T03:32:18.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.420+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f00c4014b20 con 0x7f00d00b2170 2024-05-10T03:32:18.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.420+0000 7f00cd600640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 0x7f00b006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:18.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.421+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f00c4089e40 con 0x7f00d00b2170 2024-05-10T03:32:18.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.421+0000 7f00d5200640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 0x7f00b006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:18.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.422+0000 7f00d5200640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 0x7f00b006d110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000f60 tx=0x7f00bc0078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:18.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.425+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f00c4054ee0 con 0x7f00d00b2170 2024-05-10T03:32:18.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.622+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00d0067c00 con 0x7f00b006ac00 2024-05-10T03:32:18.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.644+0000 7f00cd600640 1 -- 172.21.15.55:0/664730137 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275960 (secure 0 0 0) 0x7f00d0067c00 con 0x7f00b006ac00 2024-05-10T03:32:18.653 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:18.655 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:32:18.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 msgr2=0x7f00b006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 0x7f00b006d110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000f60 tx=0x7f00bc0078f0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 msgr2=0x7f00d015e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f00c4007e40 tx=0x7f00c400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 shutdown_connections 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00b006ac00 0x7f00b006d110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00d01618b0 0x7f00d006ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00d00b6680 0x7f00d006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 --2- 172.21.15.55:0/664730137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00d00b2170 0x7f00d015e130 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:18.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.650+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 >> 172.21.15.55:0/664730137 conn(0x7f00d00b5730 msgr2=0x7f00d015c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:18.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.651+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 shutdown_connections 2024-05-10T03:32:18.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:18.651+0000 7f00d7000640 1 -- 172.21.15.55:0/664730137 wait complete. 2024-05-10T03:32:18.684 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-05-10T03:32:17.008381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238240,"kb_used_data":5680,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754736480,"statfs":{"total":773094113280,"available":772850155520,"internally_reserved":0,"allocated":5816320,"data_stored":3177830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.727360"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299792+0000","last_change":"2024-05-10T03:32:11.299792+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299792+0000","last_undegraded":"2024-05-10T03:32:11.299792+0000","last_fullsized":"2024-05-10T03:32:11.299792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:09:45.664808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291695+0000","last_change":"2024-05-10T03:32:11.291695+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291695+0000","last_undegraded":"2024-05-10T03:32:11.291695+0000","last_fullsized":"2024-05-10T03:32:11.291695+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.292711+0000","last_change":"2024-05-10T03:32:11.292711+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.292711+0000","last_undegraded":"2024-05-10T03:32:11.292711+0000","last_fullsized":"2024-05-10T03:32:11.292711+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286375+0000","last_change":"2024-05-10T03:32:11.286375+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286375+0000","last_undegraded":"2024-05-10T03:32:11.286375+0000","last_fullsized":"2024-05-10T03:32:11.286375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:43.411173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.292787+0000","last_change":"2024-05-10T03:32:11.292787+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.292787+0000","last_undegraded":"2024-05-10T03:32:11.292787+0000","last_fullsized":"2024-05-10T03:32:11.292787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.288579+0000","last_change":"2024-05-10T03:32:11.288579+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.288579+0000","last_undegraded":"2024-05-10T03:32:11.288579+0000","last_fullsized":"2024-05-10T03:32:11.288579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297360+0000","last_change":"2024-05-10T03:32:11.297360+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297360+0000","last_undegraded":"2024-05-10T03:32:11.297360+0000","last_fullsized":"2024-05-10T03:32:11.297360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:34.518049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291798+0000","last_change":"2024-05-10T03:32:11.291798+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291798+0000","last_undegraded":"2024-05-10T03:32:11.291798+0000","last_fullsized":"2024-05-10T03:32:11.291798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.303307+0000","last_change":"2024-05-10T03:32:11.303307+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.303307+0000","last_undegraded":"2024-05-10T03:32:11.303307+0000","last_fullsized":"2024-05-10T03:32:11.303307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.288117+0000","last_change":"2024-05-10T03:32:11.288117+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.288117+0000","last_undegraded":"2024-05-10T03:32:11.288117+0000","last_fullsized":"2024-05-10T03:32:11.288117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.303047+0000","last_change":"2024-05-10T03:32:11.303047+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.303047+0000","last_undegraded":"2024-05-10T03:32:11.303047+0000","last_fullsized":"2024-05-10T03:32:11.303047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296004+0000","last_change":"2024-05-10T03:32:11.296004+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296004+0000","last_undegraded":"2024-05-10T03:32:11.296004+0000","last_fullsized":"2024-05-10T03:32:11.296004+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:46.848796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291326+0000","last_change":"2024-05-10T03:32:11.291326+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291326+0000","last_undegraded":"2024-05-10T03:32:11.291326+0000","last_fullsized":"2024-05-10T03:32:11.291326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:48:20.732699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296893+0000","last_change":"2024-05-10T03:32:11.296893+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296893+0000","last_undegraded":"2024-05-10T03:32:11.296893+0000","last_fullsized":"2024-05-10T03:32:11.296893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.300620+0000","last_change":"2024-05-10T03:32:11.300620+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.300620+0000","last_undegraded":"2024-05-10T03:32:11.300620+0000","last_fullsized":"2024-05-10T03:32:11.300620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295635+0000","last_change":"2024-05-10T03:32:11.295635+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295635+0000","last_undegraded":"2024-05-10T03:32:11.295635+0000","last_fullsized":"2024-05-10T03:32:11.295635+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:56.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290350+0000","last_change":"2024-05-10T03:32:11.290350+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290350+0000","last_undegraded":"2024-05-10T03:32:11.290350+0000","last_fullsized":"2024-05-10T03:32:11.290350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299957+0000","last_change":"2024-05-10T03:32:11.299957+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299957+0000","last_undegraded":"2024-05-10T03:32:11.299957+0000","last_fullsized":"2024-05-10T03:32:11.299957+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:42:58.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297574+0000","last_change":"2024-05-10T03:32:11.297574+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297574+0000","last_undegraded":"2024-05-10T03:32:11.297574+0000","last_fullsized":"2024-05-10T03:32:11.297574+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.297642+0000","last_change":"2024-05-10T03:32:11.297642+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.297642+0000","last_undegraded":"2024-05-10T03:32:11.297642+0000","last_fullsized":"2024-05-10T03:32:11.297642+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.291456+0000","last_change":"2024-05-10T03:32:11.291456+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.291456+0000","last_undegraded":"2024-05-10T03:32:11.291456+0000","last_fullsized":"2024-05-10T03:32:11.291456+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.296224+0000","last_change":"2024-05-10T03:32:11.296224+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.296224+0000","last_undegraded":"2024-05-10T03:32:11.296224+0000","last_fullsized":"2024-05-10T03:32:11.296224+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:04.809817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.289069+0000","last_change":"2024-05-10T03:32:11.289069+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.289069+0000","last_undegraded":"2024-05-10T03:32:11.289069+0000","last_fullsized":"2024-05-10T03:32:11.289069+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:39.030457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.299567+0000","last_change":"2024-05-10T03:32:11.299567+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.299567+0000","last_undegraded":"2024-05-10T03:32:11.299567+0000","last_fullsized":"2024-05-10T03:32:11.299567+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:35:55.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.298216+0000","last_change":"2024-05-10T03:32:11.298216+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.298216+0000","last_undegraded":"2024-05-10T03:32:11.298216+0000","last_fullsized":"2024-05-10T03:32:11.298216+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.300679+0000","last_change":"2024-05-10T03:32:11.300679+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.300679+0000","last_undegraded":"2024-05-10T03:32:11.300679+0000","last_fullsized":"2024-05-10T03:32:11.300679+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290328+0000","last_change":"2024-05-10T03:32:11.290328+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290328+0000","last_undegraded":"2024-05-10T03:32:11.290328+0000","last_fullsized":"2024-05-10T03:32:11.290328+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:28.939240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295926+0000","last_change":"2024-05-10T03:32:11.295926+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295926+0000","last_undegraded":"2024-05-10T03:32:11.295926+0000","last_fullsized":"2024-05-10T03:32:11.295926+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:37.925978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.295569+0000","last_change":"2024-05-10T03:32:11.295569+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.295569+0000","last_undegraded":"2024-05-10T03:32:11.295569+0000","last_fullsized":"2024-05-10T03:32:11.295569+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:12.822129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286817+0000","last_change":"2024-05-10T03:32:11.286817+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286817+0000","last_undegraded":"2024-05-10T03:32:11.286817+0000","last_fullsized":"2024-05-10T03:32:11.286817+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.286410+0000","last_change":"2024-05-10T03:32:11.286410+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.286410+0000","last_undegraded":"2024-05-10T03:32:11.286410+0000","last_fullsized":"2024-05-10T03:32:11.286410+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:42.125952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-10T03:32:12.278239+0000","last_change":"2024-05-10T03:32:12.278239+0000","last_active":"2024-05-10T03:32:12.278239+0000","last_peered":"2024-05-10T03:32:12.278239+0000","last_clean":"2024-05-10T03:32:12.278239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:12.278239+0000","last_undegraded":"2024-05-10T03:32:12.278239+0000","last_fullsized":"2024-05-10T03:32:12.278239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-05-10T03:32:11.290420+0000","last_change":"2024-05-10T03:32:11.290420+0000","last_active":"2024-05-10T03:32:11.264273+0000","last_peered":"2024-05-10T03:32:11.264273+0000","last_clean":"2024-05-10T03:32:11.264273+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-10T03:32:11.290420+0000","last_undegraded":"2024-05-10T03:32:11.290420+0000","last_fullsized":"2024-05-10T03:32:11.290420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:42.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-10T03:32:08.348410+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:08.348410+0000","last_peered":"2024-05-10T03:32:08.348410+0000","last_clean":"2024-05-10T03:32:08.348410+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:08.348410+0000","last_undegraded":"2024-05-10T03:32:08.348410+0000","last_fullsized":"2024-05-10T03:32:08.348410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1175552,"data_stored":841539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2230000000000001,"15min":1.2270000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.367,"15min":1.365},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8100000000000001,"5min":1.877,"15min":1.877},"last":1.262}]},{"osd":1,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.266,"15min":1.274},"min":{"1min":0.84999999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.667,"5min":1.925,"15min":1.925},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1349999999999998,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4410000000000001,"15min":1.4490000000000001},"min":{"1min":0.81200000000000006,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.819,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3899999999999999,"15min":1.3919999999999999},"min":{"1min":1.0489999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.3520000000000001,"15min":1.3340000000000001},"min":{"1min":1.0109999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1120000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.49,"15min":1.484},"min":{"1min":1.071,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.1339999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5169999999999999,"15min":1.506},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.6539999999999999}]},{"osd":5,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.546,"15min":1.516},"min":{"1min":1.2030000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1779999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.327},{"interface":"front","average":{"1min":1.76,"5min":1.712,"15min":1.704},"min":{"1min":1.383,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.5009999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.762,"5min":1.7370000000000001,"15min":1.7330000000000001},"min":{"1min":1.3939999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.653,"15min":1.641},"min":{"1min":1.0600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.7669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.073,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.03},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.329,"15min":1.351},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8640000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4590000000000001,"5min":3.113,"15min":3.113},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4750000000000001,"15min":1.4970000000000001},"min":{"1min":1.046,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.944,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.327}]},{"osd":2,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.246,"15min":1.232},"min":{"1min":0.93000000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.444},{"interface":"front","average":{"1min":1.248,"5min":1.482,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5589999999999999,"5min":3.569,"15min":3.569},"last":1.698}]},{"osd":3,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.591,"15min":1.6279999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.863},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.306,"15min":1.304},"min":{"1min":1.0409999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.744,"5min":1.931,"15min":1.931},"last":1.5049999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4359999999999999,"15min":1.4299999999999999},"min":{"1min":1.069,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.8759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5640000000000001,"15min":1.585},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.819,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.9610000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.1259999999999999,"5min":1,"15min":1},"max":{"1min":1.857,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.004,"5min":2.472,"15min":2.472},"last":1.772}]},{"osd":7,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.665,"15min":1.671},"min":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":1.9199999999999999,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":2.1499999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.651,"15min":1.655},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0070000000000001,"5min":2.359,"15min":2.359},"last":1.75}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.452},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.25}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.1559999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.004}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7030000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.841}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.806},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.6080000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.9299999999999999},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":1.885}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.173,"15min":1.177},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.1359999999999999,"15min":1.139},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5629999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.165,"15min":1.1559999999999999},"min":{"1min":0.79100000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.734,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.278,"15min":1.2829999999999999},"min":{"1min":0.85699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.994,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.302}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8280000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.042},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9179999999999999,"5min":2.29,"15min":2.29},"last":1.718}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.458,"15min":1.476},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.76,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5900000000000001,"15min":1.589},"min":{"1min":1.2729999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":1.9530000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5009999999999999,"15min":1.5049999999999999},"min":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.8129999999999999,"5min":2.512,"15min":2.512},"last":1.536},{"interface":"front","average":{"1min":1.456,"5min":1.552,"15min":1.5680000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.982,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5900000000000001,"15min":1.5940000000000001},"min":{"1min":1.153,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1139999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7829999999999999,"15min":1.7969999999999999},"min":{"1min":1.1040000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0409999999999999,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.8500000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6930000000000001,"15min":1.7030000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.069,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.601},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7609999999999999,"15min":1.7509999999999999},"min":{"1min":1.4730000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.282,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.655}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.95399999999999996,"15min":0.97599999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4850000000000001,"5min":1.708,"15min":1.708},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"min":{"1min":0.55800000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4370000000000001,"5min":1.603,"15min":1.603},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.24,"15min":1.254},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.925,"5min":1.964,"15min":1.964},"last":0.94599999999999995}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1779999999999999,"15min":1.1839999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.897,"5min":1.99,"15min":1.99},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":1.369},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.919,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.95099999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.8080000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.3879999999999999,"15min":1.3779999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.1349999999999998,"5min":2.464,"15min":2.464},"last":1.284}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.6479999999999999,"15min":1.6859999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.109,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.462,"15min":1.484},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0499999999999998,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.024}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.2559999999999998,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.589},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.774,"15min":1.804},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4729999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.417}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.5629999999999999,"15min":1.5900000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.1379999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.571,"5min":1.6479999999999999,"15min":1.661},"min":{"1min":1.052,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.2690000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.48}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":569344,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.494,"5min":1.772,"15min":1.772},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.431,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5600000000000001,"5min":1.621,"15min":1.621},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1930000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6930000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1240000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5009999999999999,"5min":2.169,"15min":2.169},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.25,"15min":1.2629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9650000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.526,"15min":1.548},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.133,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.516,"15min":1.5569999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.821,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.175}]},{"osd":5,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.036,"5min":2.508,"15min":2.508},"last":1.008},{"interface":"front","average":{"1min":1.546,"5min":1.6140000000000001,"15min":1.625},"min":{"1min":1.069,"5min":1.004,"15min":1.004},"max":{"1min":2.1190000000000002,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1140000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1139999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5820000000000001,"15min":1.5980000000000001},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.982,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.0549999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.641,"15min":1.6599999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.1360000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.369},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.659,"15min":1.679},"min":{"1min":1.052,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.169,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.129}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":581632,"data_stored":251171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.54600000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.256,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.488,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.1699999999999999,"15min":1.1910000000000001},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4330000000000001,"5min":2.444,"15min":2.444},"last":1.413}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.3,"15min":1.3280000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.556,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3100000000000001},"min":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7789999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.9399999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.673,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":2.3159999999999998},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5669999999999999,"15min":1.597},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":1.8220000000000001,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3660000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.5600000000000001,"15min":1.5980000000000001},"min":{"1min":1.069,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":2.2450000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.6200000000000001,"15min":1.647},"min":{"1min":1.1080000000000001,"5min":1.077,"15min":1.077},"max":{"1min":1.9770000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":2.0190000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.6990000000000001,"15min":1.7370000000000001},"min":{"1min":0.94999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0550000000000002,"5min":3.48,"15min":3.48},"last":2.4470000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.716,"15min":1.748},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.0800000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.7170000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.7270000000000001,"15min":1.7470000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0910000000000002,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":2.1859999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.762,"15min":1.796},"min":{"1min":1.173,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.7909999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":2.5449999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1175552,"data_stored":841539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.999,"15min":1.036},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5629999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":1.1599999999999999,"15min":1.2110000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.054}]},{"osd":2,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5389999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.498},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4550000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.226}]},{"osd":3,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.089,"15min":1.1140000000000001},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.448,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.929,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.78}]},{"osd":4,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.6040000000000001,"15min":1.6539999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0209999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.431,"15min":1.446},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.661,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.464,"15min":1.486},"min":{"1min":0.85799999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9710000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.401},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6080000000000001,"15min":1.647},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8999999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.6279999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6299999999999999,"15min":1.6539999999999999},"min":{"1min":1.028,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.206,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.7150000000000001,"15min":1.7629999999999999},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.972,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.6659999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.77,"15min":1.8200000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.946,"5min":3.149,"15min":3.149},"last":1.458},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5609999999999999,"15min":1.581},"min":{"1min":0.91500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.99,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.714}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:32:18.689 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:32:19.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.011+0000 7feeb4a00640 1 Processor -- start 2024-05-10T03:32:19.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.011+0000 7feeb4a00640 1 -- start start 2024-05-10T03:32:19.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.013+0000 7feeb4a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feeb0144710 0x7feeb0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.013+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.013+0000 7feeb4a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb006d350 con 0x7feeb00b2fc0 2024-05-10T03:32:19.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.013+0000 7feeb4a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb006c740 con 0x7feeb0144710 2024-05-10T03:32:19.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.013+0000 7feeb4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb006ca20 con 0x7feeb006d7e0 2024-05-10T03:32:19.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.014+0000 7feea7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.014+0000 7feea7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:48026/0 (socket says 172.21.15.55:48026) 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.014+0000 7feea7e00640 1 -- 172.21.15.55:0/1131489449 learned_addr learned my addr 172.21.15.55:0/1131489449 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.014+0000 7feeae800640 1 --2- 172.21.15.55:0/1131489449 >> v2:172.21.15.162:3300/0 conn(0x7feeb0144710 0x7feeb0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.015+0000 7feea7e00640 1 -- 172.21.15.55:0/1131489449 >> v2:172.21.15.162:3300/0 conn(0x7feeb0144710 msgr2=0x7feeb0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.015+0000 7feea7e00640 1 --2- 172.21.15.55:0/1131489449 >> v2:172.21.15.162:3300/0 conn(0x7feeb0144710 0x7feeb0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.015+0000 7feea7e00640 1 -- 172.21.15.55:0/1131489449 >> v1:172.21.15.162:6789/0 conn(0x7feeb00b2fc0 legacy=0x7feeb015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.015+0000 7feea7e00640 1 -- 172.21.15.55:0/1131489449 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb006cd00 con 0x7feeb006d7e0 2024-05-10T03:32:19.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.016+0000 7feea7e00640 1 --2- 172.21.15.55:0/1131489449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee98009870 tx=0x7fee9802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b08e85a9f79b636f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.017+0000 7feeade00640 1 -- 172.21.15.55:0/1131489449 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98032440 con 0x7feeb006d7e0 2024-05-10T03:32:19.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.017+0000 7feeade00640 1 -- 172.21.15.55:0/1131489449 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee980325a0 con 0x7feeb006d7e0 2024-05-10T03:32:19.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.017+0000 7feeade00640 1 -- 172.21.15.55:0/1131489449 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98032930 con 0x7feeb006d7e0 2024-05-10T03:32:19.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 -- 172.21.15.55:0/1131489449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 msgr2=0x7feeb00b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 --2- 172.21.15.55:0/1131489449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fee98009870 tx=0x7fee9802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 -- 172.21.15.55:0/1131489449 shutdown_connections 2024-05-10T03:32:19.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 --2- 172.21.15.55:0/1131489449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb00b2a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 --2- 172.21.15.55:0/1131489449 >> v2:172.21.15.162:3300/0 conn(0x7feeb0144710 0x7feeb0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 -- 172.21.15.55:0/1131489449 >> 172.21.15.55:0/1131489449 conn(0x7feeb006a110 msgr2=0x7feeb006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:19.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 -- 172.21.15.55:0/1131489449 shutdown_connections 2024-05-10T03:32:19.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.018+0000 7feeb4a00640 1 -- 172.21.15.55:0/1131489449 wait complete. 2024-05-10T03:32:19.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.021+0000 7feeb4a00640 1 Processor -- start 2024-05-10T03:32:19.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.021+0000 7feeb4a00640 1 -- start start 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 0x7feeb00c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0070440 con 0x7feeb00b2fc0 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01f72e0 con 0x7feeb0144710 2024-05-10T03:32:19.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.022+0000 7feeb4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01f80b0 con 0x7feeb006d7e0 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40302/0 (socket says 172.21.15.55:40302) 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 -- 172.21.15.55:0/2969300656 learned_addr learned my addr 172.21.15.55:0/2969300656 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feeae800640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feeaf200640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 0x7feeb00c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 -- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 msgr2=0x7feeb013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb013ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 -- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 msgr2=0x7feeb00c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 0x7feeb00c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feea7e00640 1 -- 172.21.15.55:0/2969300656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980094d0 con 0x7feeb00b2fc0 2024-05-10T03:32:19.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.023+0000 7feeaf200640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 0x7feeb00c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feea7e00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98032440 con 0x7feeb00b2fc0 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98030980 con 0x7feeb00b2fc0 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98030d10 con 0x7feeb00b2fc0 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb01f7530 con 0x7feeb00b2fc0 2024-05-10T03:32:19.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.024+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb01f7af0 con 0x7feeb00b2fc0 2024-05-10T03:32:19.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.026+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb0066ce0 con 0x7feeb00b2fc0 2024-05-10T03:32:19.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.026+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fee98039040 con 0x7feeb00b2fc0 2024-05-10T03:32:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.027+0000 7feea6a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 0x7fee8c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.027+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee980a9d30 con 0x7feeb00b2fc0 2024-05-10T03:32:19.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.028+0000 7feeae800640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 0x7fee8c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.029+0000 7feeae800640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 0x7fee8c06d110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feeb00b4ad0 tx=0x7feea0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.032+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fee98074e60 con 0x7feeb00b2fc0 2024-05-10T03:32:19.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.238+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feeb00b4b00 con 0x7fee8c06ac00 2024-05-10T03:32:19.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.262+0000 7feea6a00640 1 -- 172.21.15.55:0/2969300656 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286654 (secure 0 0 0) 0x7feeb00b4b00 con 0x7fee8c06ac00 2024-05-10T03:32:19.269 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:19.270 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:32:19.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 msgr2=0x7fee8c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 0x7fee8c06d110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feeb00b4ad0 tx=0x7feea0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:19.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 msgr2=0x7feeb0142a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee980046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 shutdown_connections 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fee8c06ac00 0x7fee8c06d110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feeb0144710 0x7feeb00c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feeb00b2fc0 0x7feeb0142a10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 --2- 172.21.15.55:0/2969300656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feeb006d7e0 0x7feeb013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 >> 172.21.15.55:0/2969300656 conn(0x7feeb006a110 msgr2=0x7feeb015e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 shutdown_connections 2024-05-10T03:32:19.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.268+0000 7feeb4a00640 1 -- 172.21.15.55:0/2969300656 wait complete. 2024-05-10T03:32:19.302 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-05-10T03:32:19.009076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242756,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731964,"statfs":{"total":773094113280,"available":772845531136,"internally_reserved":0,"allocated":6201344,"data_stored":3357308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.720375"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898678+0000","last_change":"2024-05-10T03:32:12.914232+0000","last_active":"2024-05-10T03:32:13.898678+0000","last_peered":"2024-05-10T03:32:13.898678+0000","last_clean":"2024-05-10T03:32:13.898678+0000","last_became_active":"2024-05-10T03:32:12.913664+0000","last_became_peered":"2024-05-10T03:32:12.913664+0000","last_unstale":"2024-05-10T03:32:13.898678+0000","last_undegraded":"2024-05-10T03:32:13.898678+0000","last_fullsized":"2024-05-10T03:32:13.898678+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178978+0000","last_change":"2024-05-10T03:32:12.922843+0000","last_active":"2024-05-10T03:32:14.178978+0000","last_peered":"2024-05-10T03:32:14.178978+0000","last_clean":"2024-05-10T03:32:14.178978+0000","last_became_active":"2024-05-10T03:32:12.922531+0000","last_became_peered":"2024-05-10T03:32:12.922531+0000","last_unstale":"2024-05-10T03:32:14.178978+0000","last_undegraded":"2024-05-10T03:32:14.178978+0000","last_fullsized":"2024-05-10T03:32:14.178978+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175288+0000","last_change":"2024-05-10T03:32:12.918902+0000","last_active":"2024-05-10T03:32:14.175288+0000","last_peered":"2024-05-10T03:32:14.175288+0000","last_clean":"2024-05-10T03:32:14.175288+0000","last_became_active":"2024-05-10T03:32:12.918598+0000","last_became_peered":"2024-05-10T03:32:12.918598+0000","last_unstale":"2024-05-10T03:32:14.175288+0000","last_undegraded":"2024-05-10T03:32:14.175288+0000","last_fullsized":"2024-05-10T03:32:14.175288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178493+0000","last_change":"2024-05-10T03:32:12.924668+0000","last_active":"2024-05-10T03:32:14.178493+0000","last_peered":"2024-05-10T03:32:14.178493+0000","last_clean":"2024-05-10T03:32:14.178493+0000","last_became_active":"2024-05-10T03:32:12.923917+0000","last_became_peered":"2024-05-10T03:32:12.923917+0000","last_unstale":"2024-05-10T03:32:14.178493+0000","last_undegraded":"2024-05-10T03:32:14.178493+0000","last_fullsized":"2024-05-10T03:32:14.178493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902676+0000","last_change":"2024-05-10T03:32:12.915697+0000","last_active":"2024-05-10T03:32:13.902676+0000","last_peered":"2024-05-10T03:32:13.902676+0000","last_clean":"2024-05-10T03:32:13.902676+0000","last_became_active":"2024-05-10T03:32:12.915337+0000","last_became_peered":"2024-05-10T03:32:12.915337+0000","last_unstale":"2024-05-10T03:32:13.902676+0000","last_undegraded":"2024-05-10T03:32:13.902676+0000","last_fullsized":"2024-05-10T03:32:13.902676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898620+0000","last_change":"2024-05-10T03:32:12.910337+0000","last_active":"2024-05-10T03:32:13.898620+0000","last_peered":"2024-05-10T03:32:13.898620+0000","last_clean":"2024-05-10T03:32:13.898620+0000","last_became_active":"2024-05-10T03:32:12.910049+0000","last_became_peered":"2024-05-10T03:32:12.910049+0000","last_unstale":"2024-05-10T03:32:13.898620+0000","last_undegraded":"2024-05-10T03:32:13.898620+0000","last_fullsized":"2024-05-10T03:32:13.898620+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174738+0000","last_change":"2024-05-10T03:32:12.918643+0000","last_active":"2024-05-10T03:32:14.174738+0000","last_peered":"2024-05-10T03:32:14.174738+0000","last_clean":"2024-05-10T03:32:14.174738+0000","last_became_active":"2024-05-10T03:32:12.918296+0000","last_became_peered":"2024-05-10T03:32:12.918296+0000","last_unstale":"2024-05-10T03:32:14.174738+0000","last_undegraded":"2024-05-10T03:32:14.174738+0000","last_fullsized":"2024-05-10T03:32:14.174738+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177500+0000","last_change":"2024-05-10T03:32:12.919824+0000","last_active":"2024-05-10T03:32:14.177500+0000","last_peered":"2024-05-10T03:32:14.177500+0000","last_clean":"2024-05-10T03:32:14.177500+0000","last_became_active":"2024-05-10T03:32:12.919521+0000","last_became_peered":"2024-05-10T03:32:12.919521+0000","last_unstale":"2024-05-10T03:32:14.177500+0000","last_undegraded":"2024-05-10T03:32:14.177500+0000","last_fullsized":"2024-05-10T03:32:14.177500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:45.737960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178502+0000","last_change":"2024-05-10T03:32:12.923819+0000","last_active":"2024-05-10T03:32:14.178502+0000","last_peered":"2024-05-10T03:32:14.178502+0000","last_clean":"2024-05-10T03:32:14.178502+0000","last_became_active":"2024-05-10T03:32:12.922895+0000","last_became_peered":"2024-05-10T03:32:12.922895+0000","last_unstale":"2024-05-10T03:32:14.178502+0000","last_undegraded":"2024-05-10T03:32:14.178502+0000","last_fullsized":"2024-05-10T03:32:14.178502+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:21.171840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898711+0000","last_change":"2024-05-10T03:32:12.908590+0000","last_active":"2024-05-10T03:32:13.898711+0000","last_peered":"2024-05-10T03:32:13.898711+0000","last_clean":"2024-05-10T03:32:13.898711+0000","last_became_active":"2024-05-10T03:32:12.908181+0000","last_became_peered":"2024-05-10T03:32:12.908181+0000","last_unstale":"2024-05-10T03:32:13.898711+0000","last_undegraded":"2024-05-10T03:32:13.898711+0000","last_fullsized":"2024-05-10T03:32:13.898711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116836+0000","last_change":"2024-05-10T03:32:12.921052+0000","last_active":"2024-05-10T03:32:14.116836+0000","last_peered":"2024-05-10T03:32:14.116836+0000","last_clean":"2024-05-10T03:32:14.116836+0000","last_became_active":"2024-05-10T03:32:12.918022+0000","last_became_peered":"2024-05-10T03:32:12.918022+0000","last_unstale":"2024-05-10T03:32:14.116836+0000","last_undegraded":"2024-05-10T03:32:14.116836+0000","last_fullsized":"2024-05-10T03:32:14.116836+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:13.248072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178356+0000","last_change":"2024-05-10T03:32:12.921186+0000","last_active":"2024-05-10T03:32:14.178356+0000","last_peered":"2024-05-10T03:32:14.178356+0000","last_clean":"2024-05-10T03:32:14.178356+0000","last_became_active":"2024-05-10T03:32:12.920915+0000","last_became_peered":"2024-05-10T03:32:12.920915+0000","last_unstale":"2024-05-10T03:32:14.178356+0000","last_undegraded":"2024-05-10T03:32:14.178356+0000","last_fullsized":"2024-05-10T03:32:14.178356+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:51.954514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898529+0000","last_change":"2024-05-10T03:32:12.908427+0000","last_active":"2024-05-10T03:32:13.898529+0000","last_peered":"2024-05-10T03:32:13.898529+0000","last_clean":"2024-05-10T03:32:13.898529+0000","last_became_active":"2024-05-10T03:32:12.908141+0000","last_became_peered":"2024-05-10T03:32:12.908141+0000","last_unstale":"2024-05-10T03:32:13.898529+0000","last_undegraded":"2024-05-10T03:32:13.898529+0000","last_fullsized":"2024-05-10T03:32:13.898529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175216+0000","last_change":"2024-05-10T03:32:12.915903+0000","last_active":"2024-05-10T03:32:14.175216+0000","last_peered":"2024-05-10T03:32:14.175216+0000","last_clean":"2024-05-10T03:32:14.175216+0000","last_became_active":"2024-05-10T03:32:12.915485+0000","last_became_peered":"2024-05-10T03:32:12.915485+0000","last_unstale":"2024-05-10T03:32:14.175216+0000","last_undegraded":"2024-05-10T03:32:14.175216+0000","last_fullsized":"2024-05-10T03:32:14.175216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:04.967182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177018+0000","last_change":"2024-05-10T03:32:12.921319+0000","last_active":"2024-05-10T03:32:14.177018+0000","last_peered":"2024-05-10T03:32:14.177018+0000","last_clean":"2024-05-10T03:32:14.177018+0000","last_became_active":"2024-05-10T03:32:12.920917+0000","last_became_peered":"2024-05-10T03:32:12.920917+0000","last_unstale":"2024-05-10T03:32:14.177018+0000","last_undegraded":"2024-05-10T03:32:14.177018+0000","last_fullsized":"2024-05-10T03:32:14.177018+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898412+0000","last_change":"2024-05-10T03:32:12.909443+0000","last_active":"2024-05-10T03:32:13.898412+0000","last_peered":"2024-05-10T03:32:13.898412+0000","last_clean":"2024-05-10T03:32:13.898412+0000","last_became_active":"2024-05-10T03:32:12.908121+0000","last_became_peered":"2024-05-10T03:32:12.908121+0000","last_unstale":"2024-05-10T03:32:13.898412+0000","last_undegraded":"2024-05-10T03:32:13.898412+0000","last_fullsized":"2024-05-10T03:32:13.898412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175155+0000","last_change":"2024-05-10T03:32:12.924889+0000","last_active":"2024-05-10T03:32:14.175155+0000","last_peered":"2024-05-10T03:32:14.175155+0000","last_clean":"2024-05-10T03:32:14.175155+0000","last_became_active":"2024-05-10T03:32:12.920754+0000","last_became_peered":"2024-05-10T03:32:12.920754+0000","last_unstale":"2024-05-10T03:32:14.175155+0000","last_undegraded":"2024-05-10T03:32:14.175155+0000","last_fullsized":"2024-05-10T03:32:14.175155+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179224+0000","last_change":"2024-05-10T03:32:12.922850+0000","last_active":"2024-05-10T03:32:14.179224+0000","last_peered":"2024-05-10T03:32:14.179224+0000","last_clean":"2024-05-10T03:32:14.179224+0000","last_became_active":"2024-05-10T03:32:12.922543+0000","last_became_peered":"2024-05-10T03:32:12.922543+0000","last_unstale":"2024-05-10T03:32:14.179224+0000","last_undegraded":"2024-05-10T03:32:14.179224+0000","last_fullsized":"2024-05-10T03:32:14.179224+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175667+0000","last_change":"2024-05-10T03:32:12.915732+0000","last_active":"2024-05-10T03:32:14.175667+0000","last_peered":"2024-05-10T03:32:14.175667+0000","last_clean":"2024-05-10T03:32:14.175667+0000","last_became_active":"2024-05-10T03:32:12.914507+0000","last_became_peered":"2024-05-10T03:32:12.914507+0000","last_unstale":"2024-05-10T03:32:14.175667+0000","last_undegraded":"2024-05-10T03:32:14.175667+0000","last_fullsized":"2024-05-10T03:32:14.175667+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:03.419392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178102+0000","last_change":"2024-05-10T03:32:12.919811+0000","last_active":"2024-05-10T03:32:14.178102+0000","last_peered":"2024-05-10T03:32:14.178102+0000","last_clean":"2024-05-10T03:32:14.178102+0000","last_became_active":"2024-05-10T03:32:12.919469+0000","last_became_peered":"2024-05-10T03:32:12.919469+0000","last_unstale":"2024-05-10T03:32:14.178102+0000","last_undegraded":"2024-05-10T03:32:14.178102+0000","last_fullsized":"2024-05-10T03:32:14.178102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177895+0000","last_change":"2024-05-10T03:32:12.926306+0000","last_active":"2024-05-10T03:32:14.177895+0000","last_peered":"2024-05-10T03:32:14.177895+0000","last_clean":"2024-05-10T03:32:14.177895+0000","last_became_active":"2024-05-10T03:32:12.925942+0000","last_became_peered":"2024-05-10T03:32:12.925942+0000","last_unstale":"2024-05-10T03:32:14.177895+0000","last_undegraded":"2024-05-10T03:32:14.177895+0000","last_fullsized":"2024-05-10T03:32:14.177895+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177762+0000","last_change":"2024-05-10T03:32:12.922894+0000","last_active":"2024-05-10T03:32:14.177762+0000","last_peered":"2024-05-10T03:32:14.177762+0000","last_clean":"2024-05-10T03:32:14.177762+0000","last_became_active":"2024-05-10T03:32:12.922244+0000","last_became_peered":"2024-05-10T03:32:12.922244+0000","last_unstale":"2024-05-10T03:32:14.177762+0000","last_undegraded":"2024-05-10T03:32:14.177762+0000","last_fullsized":"2024-05-10T03:32:14.177762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:03.341641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116024+0000","last_change":"2024-05-10T03:32:12.921743+0000","last_active":"2024-05-10T03:32:14.116024+0000","last_peered":"2024-05-10T03:32:14.116024+0000","last_clean":"2024-05-10T03:32:14.116024+0000","last_became_active":"2024-05-10T03:32:12.918101+0000","last_became_peered":"2024-05-10T03:32:12.918101+0000","last_unstale":"2024-05-10T03:32:14.116024+0000","last_undegraded":"2024-05-10T03:32:14.116024+0000","last_fullsized":"2024-05-10T03:32:14.116024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:48.926764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174969+0000","last_change":"2024-05-10T03:32:12.915425+0000","last_active":"2024-05-10T03:32:14.174969+0000","last_peered":"2024-05-10T03:32:14.174969+0000","last_clean":"2024-05-10T03:32:14.174969+0000","last_became_active":"2024-05-10T03:32:12.914547+0000","last_became_peered":"2024-05-10T03:32:12.914547+0000","last_unstale":"2024-05-10T03:32:14.174969+0000","last_undegraded":"2024-05-10T03:32:14.174969+0000","last_fullsized":"2024-05-10T03:32:14.174969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175394+0000","last_change":"2024-05-10T03:32:12.925418+0000","last_active":"2024-05-10T03:32:14.175394+0000","last_peered":"2024-05-10T03:32:14.175394+0000","last_clean":"2024-05-10T03:32:14.175394+0000","last_became_active":"2024-05-10T03:32:12.924225+0000","last_became_peered":"2024-05-10T03:32:12.924225+0000","last_unstale":"2024-05-10T03:32:14.175394+0000","last_undegraded":"2024-05-10T03:32:14.175394+0000","last_fullsized":"2024-05-10T03:32:14.175394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:26.673168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178116+0000","last_change":"2024-05-10T03:32:12.910349+0000","last_active":"2024-05-10T03:32:14.178116+0000","last_peered":"2024-05-10T03:32:14.178116+0000","last_clean":"2024-05-10T03:32:14.178116+0000","last_became_active":"2024-05-10T03:32:12.910056+0000","last_became_peered":"2024-05-10T03:32:12.910056+0000","last_unstale":"2024-05-10T03:32:14.178116+0000","last_undegraded":"2024-05-10T03:32:14.178116+0000","last_fullsized":"2024-05-10T03:32:14.178116+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:08.718135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899066+0000","last_change":"2024-05-10T03:32:12.919859+0000","last_active":"2024-05-10T03:32:13.899066+0000","last_peered":"2024-05-10T03:32:13.899066+0000","last_clean":"2024-05-10T03:32:13.899066+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.899066+0000","last_undegraded":"2024-05-10T03:32:13.899066+0000","last_fullsized":"2024-05-10T03:32:13.899066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178414+0000","last_change":"2024-05-10T03:32:12.927253+0000","last_active":"2024-05-10T03:32:14.178414+0000","last_peered":"2024-05-10T03:32:14.178414+0000","last_clean":"2024-05-10T03:32:14.178414+0000","last_became_active":"2024-05-10T03:32:12.926930+0000","last_became_peered":"2024-05-10T03:32:12.926930+0000","last_unstale":"2024-05-10T03:32:14.178414+0000","last_undegraded":"2024-05-10T03:32:14.178414+0000","last_fullsized":"2024-05-10T03:32:14.178414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176263+0000","last_change":"2024-05-10T03:32:12.921104+0000","last_active":"2024-05-10T03:32:14.176263+0000","last_peered":"2024-05-10T03:32:14.176263+0000","last_clean":"2024-05-10T03:32:14.176263+0000","last_became_active":"2024-05-10T03:32:12.920745+0000","last_became_peered":"2024-05-10T03:32:12.920745+0000","last_unstale":"2024-05-10T03:32:14.176263+0000","last_undegraded":"2024-05-10T03:32:14.176263+0000","last_fullsized":"2024-05-10T03:32:14.176263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:02.965291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177679+0000","last_change":"2024-05-10T03:32:12.927250+0000","last_active":"2024-05-10T03:32:14.177679+0000","last_peered":"2024-05-10T03:32:14.177679+0000","last_clean":"2024-05-10T03:32:14.177679+0000","last_became_active":"2024-05-10T03:32:12.926910+0000","last_became_peered":"2024-05-10T03:32:12.926910+0000","last_unstale":"2024-05-10T03:32:14.177679+0000","last_undegraded":"2024-05-10T03:32:14.177679+0000","last_fullsized":"2024-05-10T03:32:14.177679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174982+0000","last_change":"2024-05-10T03:32:12.917812+0000","last_active":"2024-05-10T03:32:14.174982+0000","last_peered":"2024-05-10T03:32:14.174982+0000","last_clean":"2024-05-10T03:32:14.174982+0000","last_became_active":"2024-05-10T03:32:12.913954+0000","last_became_peered":"2024-05-10T03:32:12.913954+0000","last_unstale":"2024-05-10T03:32:14.174982+0000","last_undegraded":"2024-05-10T03:32:14.174982+0000","last_fullsized":"2024-05-10T03:32:14.174982+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899746+0000","last_change":"2024-05-10T03:32:12.902462+0000","last_active":"2024-05-10T03:32:13.899746+0000","last_peered":"2024-05-10T03:32:13.899746+0000","last_clean":"2024-05-10T03:32:13.899746+0000","last_became_active":"2024-05-10T03:32:12.901977+0000","last_became_peered":"2024-05-10T03:32:12.901977+0000","last_unstale":"2024-05-10T03:32:13.899746+0000","last_undegraded":"2024-05-10T03:32:13.899746+0000","last_fullsized":"2024-05-10T03:32:13.899746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:05.823783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176963+0000","last_change":"2024-05-10T03:32:12.923673+0000","last_active":"2024-05-10T03:32:14.176963+0000","last_peered":"2024-05-10T03:32:14.176963+0000","last_clean":"2024-05-10T03:32:14.176963+0000","last_became_active":"2024-05-10T03:32:12.923367+0000","last_became_peered":"2024-05-10T03:32:12.923367+0000","last_unstale":"2024-05-10T03:32:14.176963+0000","last_undegraded":"2024-05-10T03:32:14.176963+0000","last_fullsized":"2024-05-10T03:32:14.176963+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927108+0000","last_change":"2024-05-10T03:32:12.321598+0000","last_active":"2024-05-10T03:32:12.927108+0000","last_peered":"2024-05-10T03:32:12.927108+0000","last_clean":"2024-05-10T03:32:12.927108+0000","last_became_active":"2024-05-10T03:32:12.320780+0000","last_became_peered":"2024-05-10T03:32:12.320780+0000","last_unstale":"2024-05-10T03:32:12.927108+0000","last_undegraded":"2024-05-10T03:32:12.927108+0000","last_fullsized":"2024-05-10T03:32:12.927108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:09:45.664808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900214+0000","last_change":"2024-05-10T03:32:12.912792+0000","last_active":"2024-05-10T03:32:13.900214+0000","last_peered":"2024-05-10T03:32:13.900214+0000","last_clean":"2024-05-10T03:32:13.900214+0000","last_became_active":"2024-05-10T03:32:12.907370+0000","last_became_peered":"2024-05-10T03:32:12.907370+0000","last_unstale":"2024-05-10T03:32:13.900214+0000","last_undegraded":"2024-05-10T03:32:13.900214+0000","last_fullsized":"2024-05-10T03:32:13.900214+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.929409+0000","last_change":"2024-05-10T03:32:12.323974+0000","last_active":"2024-05-10T03:32:12.929409+0000","last_peered":"2024-05-10T03:32:12.929409+0000","last_clean":"2024-05-10T03:32:12.929409+0000","last_became_active":"2024-05-10T03:32:12.323651+0000","last_became_peered":"2024-05-10T03:32:12.323651+0000","last_unstale":"2024-05-10T03:32:12.929409+0000","last_undegraded":"2024-05-10T03:32:12.929409+0000","last_fullsized":"2024-05-10T03:32:12.929409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177373+0000","last_change":"2024-05-10T03:32:12.926741+0000","last_active":"2024-05-10T03:32:14.177373+0000","last_peered":"2024-05-10T03:32:14.177373+0000","last_clean":"2024-05-10T03:32:14.177373+0000","last_became_active":"2024-05-10T03:32:12.926261+0000","last_became_peered":"2024-05-10T03:32:12.926261+0000","last_unstale":"2024-05-10T03:32:14.177373+0000","last_undegraded":"2024-05-10T03:32:14.177373+0000","last_fullsized":"2024-05-10T03:32:14.177373+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.922236+0000","last_change":"2024-05-10T03:32:12.327381+0000","last_active":"2024-05-10T03:32:12.922236+0000","last_peered":"2024-05-10T03:32:12.922236+0000","last_clean":"2024-05-10T03:32:12.922236+0000","last_became_active":"2024-05-10T03:32:12.327048+0000","last_became_peered":"2024-05-10T03:32:12.327048+0000","last_unstale":"2024-05-10T03:32:12.922236+0000","last_undegraded":"2024-05-10T03:32:12.922236+0000","last_fullsized":"2024-05-10T03:32:12.922236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177770+0000","last_change":"2024-05-10T03:32:12.910467+0000","last_active":"2024-05-10T03:32:14.177770+0000","last_peered":"2024-05-10T03:32:14.177770+0000","last_clean":"2024-05-10T03:32:14.177770+0000","last_became_active":"2024-05-10T03:32:12.910188+0000","last_became_peered":"2024-05-10T03:32:12.910188+0000","last_unstale":"2024-05-10T03:32:14.177770+0000","last_undegraded":"2024-05-10T03:32:14.177770+0000","last_fullsized":"2024-05-10T03:32:14.177770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923121+0000","last_change":"2024-05-10T03:32:12.325376+0000","last_active":"2024-05-10T03:32:12.923121+0000","last_peered":"2024-05-10T03:32:12.923121+0000","last_clean":"2024-05-10T03:32:12.923121+0000","last_became_active":"2024-05-10T03:32:12.324035+0000","last_became_peered":"2024-05-10T03:32:12.324035+0000","last_unstale":"2024-05-10T03:32:12.923121+0000","last_undegraded":"2024-05-10T03:32:12.923121+0000","last_fullsized":"2024-05-10T03:32:12.923121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:43.411173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.903007+0000","last_change":"2024-05-10T03:32:12.924458+0000","last_active":"2024-05-10T03:32:13.903007+0000","last_peered":"2024-05-10T03:32:13.903007+0000","last_clean":"2024-05-10T03:32:13.903007+0000","last_became_active":"2024-05-10T03:32:12.923064+0000","last_became_peered":"2024-05-10T03:32:12.923064+0000","last_unstale":"2024-05-10T03:32:13.903007+0000","last_undegraded":"2024-05-10T03:32:13.903007+0000","last_fullsized":"2024-05-10T03:32:13.903007+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930846+0000","last_change":"2024-05-10T03:32:12.328698+0000","last_active":"2024-05-10T03:32:12.930846+0000","last_peered":"2024-05-10T03:32:12.930846+0000","last_clean":"2024-05-10T03:32:12.930846+0000","last_became_active":"2024-05-10T03:32:12.328069+0000","last_became_peered":"2024-05-10T03:32:12.328069+0000","last_unstale":"2024-05-10T03:32:12.930846+0000","last_undegraded":"2024-05-10T03:32:12.930846+0000","last_fullsized":"2024-05-10T03:32:12.930846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178392+0000","last_change":"2024-05-10T03:32:12.929164+0000","last_active":"2024-05-10T03:32:14.178392+0000","last_peered":"2024-05-10T03:32:14.178392+0000","last_clean":"2024-05-10T03:32:14.178392+0000","last_became_active":"2024-05-10T03:32:12.928657+0000","last_became_peered":"2024-05-10T03:32:12.928657+0000","last_unstale":"2024-05-10T03:32:14.178392+0000","last_undegraded":"2024-05-10T03:32:14.178392+0000","last_fullsized":"2024-05-10T03:32:14.178392+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923421+0000","last_change":"2024-05-10T03:32:12.317756+0000","last_active":"2024-05-10T03:32:12.923421+0000","last_peered":"2024-05-10T03:32:12.923421+0000","last_clean":"2024-05-10T03:32:12.923421+0000","last_became_active":"2024-05-10T03:32:12.317442+0000","last_became_peered":"2024-05-10T03:32:12.317442+0000","last_unstale":"2024-05-10T03:32:12.923421+0000","last_undegraded":"2024-05-10T03:32:12.923421+0000","last_fullsized":"2024-05-10T03:32:12.923421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178416+0000","last_change":"2024-05-10T03:32:12.904312+0000","last_active":"2024-05-10T03:32:14.178416+0000","last_peered":"2024-05-10T03:32:14.178416+0000","last_clean":"2024-05-10T03:32:14.178416+0000","last_became_active":"2024-05-10T03:32:12.904017+0000","last_became_peered":"2024-05-10T03:32:12.904017+0000","last_unstale":"2024-05-10T03:32:14.178416+0000","last_undegraded":"2024-05-10T03:32:14.178416+0000","last_fullsized":"2024-05-10T03:32:14.178416+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:00:06.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906708+0000","last_change":"2024-05-10T03:32:12.324225+0000","last_active":"2024-05-10T03:32:12.906708+0000","last_peered":"2024-05-10T03:32:12.906708+0000","last_clean":"2024-05-10T03:32:12.906708+0000","last_became_active":"2024-05-10T03:32:12.323907+0000","last_became_peered":"2024-05-10T03:32:12.323907+0000","last_unstale":"2024-05-10T03:32:12.906708+0000","last_undegraded":"2024-05-10T03:32:12.906708+0000","last_fullsized":"2024-05-10T03:32:12.906708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:34.518049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178666+0000","last_change":"2024-05-10T03:32:12.931385+0000","last_active":"2024-05-10T03:32:14.178666+0000","last_peered":"2024-05-10T03:32:14.178666+0000","last_clean":"2024-05-10T03:32:14.178666+0000","last_became_active":"2024-05-10T03:32:12.931015+0000","last_became_peered":"2024-05-10T03:32:12.931015+0000","last_unstale":"2024-05-10T03:32:14.178666+0000","last_undegraded":"2024-05-10T03:32:14.178666+0000","last_fullsized":"2024-05-10T03:32:14.178666+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907110+0000","last_change":"2024-05-10T03:32:12.335472+0000","last_active":"2024-05-10T03:32:12.907110+0000","last_peered":"2024-05-10T03:32:12.907110+0000","last_clean":"2024-05-10T03:32:12.907110+0000","last_became_active":"2024-05-10T03:32:12.335158+0000","last_became_peered":"2024-05-10T03:32:12.335158+0000","last_unstale":"2024-05-10T03:32:12.907110+0000","last_undegraded":"2024-05-10T03:32:12.907110+0000","last_fullsized":"2024-05-10T03:32:12.907110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902078+0000","last_change":"2024-05-10T03:32:12.907425+0000","last_active":"2024-05-10T03:32:13.902078+0000","last_peered":"2024-05-10T03:32:13.902078+0000","last_clean":"2024-05-10T03:32:13.902078+0000","last_became_active":"2024-05-10T03:32:12.904255+0000","last_became_peered":"2024-05-10T03:32:12.904255+0000","last_unstale":"2024-05-10T03:32:13.902078+0000","last_undegraded":"2024-05-10T03:32:13.902078+0000","last_fullsized":"2024-05-10T03:32:13.902078+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910427+0000","last_change":"2024-05-10T03:32:12.326744+0000","last_active":"2024-05-10T03:32:12.910427+0000","last_peered":"2024-05-10T03:32:12.910427+0000","last_clean":"2024-05-10T03:32:12.910427+0000","last_became_active":"2024-05-10T03:32:12.326336+0000","last_became_peered":"2024-05-10T03:32:12.326336+0000","last_unstale":"2024-05-10T03:32:12.910427+0000","last_undegraded":"2024-05-10T03:32:12.910427+0000","last_fullsized":"2024-05-10T03:32:12.910427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899885+0000","last_change":"2024-05-10T03:32:12.904290+0000","last_active":"2024-05-10T03:32:13.899885+0000","last_peered":"2024-05-10T03:32:13.899885+0000","last_clean":"2024-05-10T03:32:13.899885+0000","last_became_active":"2024-05-10T03:32:12.903859+0000","last_became_peered":"2024-05-10T03:32:12.903859+0000","last_unstale":"2024-05-10T03:32:13.899885+0000","last_undegraded":"2024-05-10T03:32:13.899885+0000","last_fullsized":"2024-05-10T03:32:13.899885+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:38.939944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895544+0000","last_change":"2024-05-10T03:32:12.326240+0000","last_active":"2024-05-10T03:32:12.895544+0000","last_peered":"2024-05-10T03:32:12.895544+0000","last_clean":"2024-05-10T03:32:12.895544+0000","last_became_active":"2024-05-10T03:32:12.325869+0000","last_became_peered":"2024-05-10T03:32:12.325869+0000","last_unstale":"2024-05-10T03:32:12.895544+0000","last_undegraded":"2024-05-10T03:32:12.895544+0000","last_fullsized":"2024-05-10T03:32:12.895544+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175662+0000","last_change":"2024-05-10T03:32:12.924590+0000","last_active":"2024-05-10T03:32:14.175662+0000","last_peered":"2024-05-10T03:32:14.175662+0000","last_clean":"2024-05-10T03:32:14.175662+0000","last_became_active":"2024-05-10T03:32:12.924100+0000","last_became_peered":"2024-05-10T03:32:12.924100+0000","last_unstale":"2024-05-10T03:32:14.175662+0000","last_undegraded":"2024-05-10T03:32:14.175662+0000","last_fullsized":"2024-05-10T03:32:14.175662+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:07.133902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.928529+0000","last_change":"2024-05-10T03:32:12.331932+0000","last_active":"2024-05-10T03:32:12.928529+0000","last_peered":"2024-05-10T03:32:12.928529+0000","last_clean":"2024-05-10T03:32:12.928529+0000","last_became_active":"2024-05-10T03:32:12.331589+0000","last_became_peered":"2024-05-10T03:32:12.331589+0000","last_unstale":"2024-05-10T03:32:12.928529+0000","last_undegraded":"2024-05-10T03:32:12.928529+0000","last_fullsized":"2024-05-10T03:32:12.928529+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902755+0000","last_change":"2024-05-10T03:32:12.904583+0000","last_active":"2024-05-10T03:32:13.902755+0000","last_peered":"2024-05-10T03:32:13.902755+0000","last_clean":"2024-05-10T03:32:13.902755+0000","last_became_active":"2024-05-10T03:32:12.903287+0000","last_became_peered":"2024-05-10T03:32:12.903287+0000","last_unstale":"2024-05-10T03:32:13.902755+0000","last_undegraded":"2024-05-10T03:32:13.902755+0000","last_fullsized":"2024-05-10T03:32:13.902755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934219+0000","last_change":"2024-05-10T03:32:12.322783+0000","last_active":"2024-05-10T03:32:12.934219+0000","last_peered":"2024-05-10T03:32:12.934219+0000","last_clean":"2024-05-10T03:32:12.934219+0000","last_became_active":"2024-05-10T03:32:12.320656+0000","last_became_peered":"2024-05-10T03:32:12.320656+0000","last_unstale":"2024-05-10T03:32:12.934219+0000","last_undegraded":"2024-05-10T03:32:12.934219+0000","last_fullsized":"2024-05-10T03:32:12.934219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:46.848796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900290+0000","last_change":"2024-05-10T03:32:12.920087+0000","last_active":"2024-05-10T03:32:13.900290+0000","last_peered":"2024-05-10T03:32:13.900290+0000","last_clean":"2024-05-10T03:32:13.900290+0000","last_became_active":"2024-05-10T03:32:12.919738+0000","last_became_peered":"2024-05-10T03:32:12.919738+0000","last_unstale":"2024-05-10T03:32:13.900290+0000","last_undegraded":"2024-05-10T03:32:13.900290+0000","last_fullsized":"2024-05-10T03:32:13.900290+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:36.708699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930884+0000","last_change":"2024-05-10T03:32:12.329191+0000","last_active":"2024-05-10T03:32:12.930884+0000","last_peered":"2024-05-10T03:32:12.930884+0000","last_clean":"2024-05-10T03:32:12.930884+0000","last_became_active":"2024-05-10T03:32:12.326891+0000","last_became_peered":"2024-05-10T03:32:12.326891+0000","last_unstale":"2024-05-10T03:32:12.930884+0000","last_undegraded":"2024-05-10T03:32:12.930884+0000","last_fullsized":"2024-05-10T03:32:12.930884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:48:20.732699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175971+0000","last_change":"2024-05-10T03:32:12.915905+0000","last_active":"2024-05-10T03:32:14.175971+0000","last_peered":"2024-05-10T03:32:14.175971+0000","last_clean":"2024-05-10T03:32:14.175971+0000","last_became_active":"2024-05-10T03:32:12.915314+0000","last_became_peered":"2024-05-10T03:32:12.915314+0000","last_unstale":"2024-05-10T03:32:14.175971+0000","last_undegraded":"2024-05-10T03:32:14.175971+0000","last_fullsized":"2024-05-10T03:32:14.175971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:38:53.287475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909280+0000","last_change":"2024-05-10T03:32:12.340972+0000","last_active":"2024-05-10T03:32:12.909280+0000","last_peered":"2024-05-10T03:32:12.909280+0000","last_clean":"2024-05-10T03:32:12.909280+0000","last_became_active":"2024-05-10T03:32:12.340465+0000","last_became_peered":"2024-05-10T03:32:12.340465+0000","last_unstale":"2024-05-10T03:32:12.909280+0000","last_undegraded":"2024-05-10T03:32:12.909280+0000","last_fullsized":"2024-05-10T03:32:12.909280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178707+0000","last_change":"2024-05-10T03:32:12.917579+0000","last_active":"2024-05-10T03:32:14.178707+0000","last_peered":"2024-05-10T03:32:14.178707+0000","last_clean":"2024-05-10T03:32:14.178707+0000","last_became_active":"2024-05-10T03:32:12.917261+0000","last_became_peered":"2024-05-10T03:32:12.917261+0000","last_unstale":"2024-05-10T03:32:14.178707+0000","last_undegraded":"2024-05-10T03:32:14.178707+0000","last_fullsized":"2024-05-10T03:32:14.178707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:16.513183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907440+0000","last_change":"2024-05-10T03:32:12.328850+0000","last_active":"2024-05-10T03:32:12.907440+0000","last_peered":"2024-05-10T03:32:12.907440+0000","last_clean":"2024-05-10T03:32:12.907440+0000","last_became_active":"2024-05-10T03:32:12.328541+0000","last_became_peered":"2024-05-10T03:32:12.328541+0000","last_unstale":"2024-05-10T03:32:12.907440+0000","last_undegraded":"2024-05-10T03:32:12.907440+0000","last_fullsized":"2024-05-10T03:32:12.907440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175276+0000","last_change":"2024-05-10T03:32:12.925092+0000","last_active":"2024-05-10T03:32:14.175276+0000","last_peered":"2024-05-10T03:32:14.175276+0000","last_clean":"2024-05-10T03:32:14.175276+0000","last_became_active":"2024-05-10T03:32:12.920412+0000","last_became_peered":"2024-05-10T03:32:12.920412+0000","last_unstale":"2024-05-10T03:32:14.175276+0000","last_undegraded":"2024-05-10T03:32:14.175276+0000","last_fullsized":"2024-05-10T03:32:14.175276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:10.962423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.926806+0000","last_change":"2024-05-10T03:32:12.330153+0000","last_active":"2024-05-10T03:32:12.926806+0000","last_peered":"2024-05-10T03:32:12.926806+0000","last_clean":"2024-05-10T03:32:12.926806+0000","last_became_active":"2024-05-10T03:32:12.328275+0000","last_became_peered":"2024-05-10T03:32:12.328275+0000","last_unstale":"2024-05-10T03:32:12.926806+0000","last_undegraded":"2024-05-10T03:32:12.926806+0000","last_fullsized":"2024-05-10T03:32:12.926806+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:56.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178945+0000","last_change":"2024-05-10T03:32:12.912037+0000","last_active":"2024-05-10T03:32:14.178945+0000","last_peered":"2024-05-10T03:32:14.178945+0000","last_clean":"2024-05-10T03:32:14.178945+0000","last_became_active":"2024-05-10T03:32:12.911220+0000","last_became_peered":"2024-05-10T03:32:12.911220+0000","last_unstale":"2024-05-10T03:32:14.178945+0000","last_undegraded":"2024-05-10T03:32:14.178945+0000","last_fullsized":"2024-05-10T03:32:14.178945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934217+0000","last_change":"2024-05-10T03:32:12.332211+0000","last_active":"2024-05-10T03:32:12.934217+0000","last_peered":"2024-05-10T03:32:12.934217+0000","last_clean":"2024-05-10T03:32:12.934217+0000","last_became_active":"2024-05-10T03:32:12.331391+0000","last_became_peered":"2024-05-10T03:32:12.331391+0000","last_unstale":"2024-05-10T03:32:12.934217+0000","last_undegraded":"2024-05-10T03:32:12.934217+0000","last_fullsized":"2024-05-10T03:32:12.934217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898530+0000","last_change":"2024-05-10T03:32:12.919886+0000","last_active":"2024-05-10T03:32:13.898530+0000","last_peered":"2024-05-10T03:32:13.898530+0000","last_clean":"2024-05-10T03:32:13.898530+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.898530+0000","last_undegraded":"2024-05-10T03:32:13.898530+0000","last_fullsized":"2024-05-10T03:32:13.898530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907502+0000","last_change":"2024-05-10T03:32:12.320826+0000","last_active":"2024-05-10T03:32:12.907502+0000","last_peered":"2024-05-10T03:32:12.907502+0000","last_clean":"2024-05-10T03:32:12.907502+0000","last_became_active":"2024-05-10T03:32:12.320490+0000","last_became_peered":"2024-05-10T03:32:12.320490+0000","last_unstale":"2024-05-10T03:32:12.907502+0000","last_undegraded":"2024-05-10T03:32:12.907502+0000","last_fullsized":"2024-05-10T03:32:12.907502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:42:58.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902476+0000","last_change":"2024-05-10T03:32:12.918118+0000","last_active":"2024-05-10T03:32:13.902476+0000","last_peered":"2024-05-10T03:32:13.902476+0000","last_clean":"2024-05-10T03:32:13.902476+0000","last_became_active":"2024-05-10T03:32:12.917768+0000","last_became_peered":"2024-05-10T03:32:12.917768+0000","last_unstale":"2024-05-10T03:32:13.902476+0000","last_undegraded":"2024-05-10T03:32:13.902476+0000","last_fullsized":"2024-05-10T03:32:13.902476+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.924200+0000","last_change":"2024-05-10T03:32:12.321740+0000","last_active":"2024-05-10T03:32:12.924200+0000","last_peered":"2024-05-10T03:32:12.924200+0000","last_clean":"2024-05-10T03:32:12.924200+0000","last_became_active":"2024-05-10T03:32:12.321433+0000","last_became_peered":"2024-05-10T03:32:12.321433+0000","last_unstale":"2024-05-10T03:32:12.924200+0000","last_undegraded":"2024-05-10T03:32:12.924200+0000","last_fullsized":"2024-05-10T03:32:12.924200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898874+0000","last_change":"2024-05-10T03:32:12.917048+0000","last_active":"2024-05-10T03:32:13.898874+0000","last_peered":"2024-05-10T03:32:13.898874+0000","last_clean":"2024-05-10T03:32:13.898874+0000","last_became_active":"2024-05-10T03:32:12.916721+0000","last_became_peered":"2024-05-10T03:32:12.916721+0000","last_unstale":"2024-05-10T03:32:13.898874+0000","last_undegraded":"2024-05-10T03:32:13.898874+0000","last_fullsized":"2024-05-10T03:32:13.898874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910839+0000","last_change":"2024-05-10T03:32:12.326718+0000","last_active":"2024-05-10T03:32:12.910839+0000","last_peered":"2024-05-10T03:32:12.910839+0000","last_clean":"2024-05-10T03:32:12.910839+0000","last_became_active":"2024-05-10T03:32:12.325934+0000","last_became_peered":"2024-05-10T03:32:12.325934+0000","last_unstale":"2024-05-10T03:32:12.910839+0000","last_undegraded":"2024-05-10T03:32:12.910839+0000","last_fullsized":"2024-05-10T03:32:12.910839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900458+0000","last_change":"2024-05-10T03:32:12.908080+0000","last_active":"2024-05-10T03:32:13.900458+0000","last_peered":"2024-05-10T03:32:13.900458+0000","last_clean":"2024-05-10T03:32:13.900458+0000","last_became_active":"2024-05-10T03:32:12.907698+0000","last_became_peered":"2024-05-10T03:32:12.907698+0000","last_unstale":"2024-05-10T03:32:13.900458+0000","last_undegraded":"2024-05-10T03:32:13.900458+0000","last_fullsized":"2024-05-10T03:32:13.900458+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927874+0000","last_change":"2024-05-10T03:32:12.321882+0000","last_active":"2024-05-10T03:32:12.927874+0000","last_peered":"2024-05-10T03:32:12.927874+0000","last_clean":"2024-05-10T03:32:12.927874+0000","last_became_active":"2024-05-10T03:32:12.321451+0000","last_became_peered":"2024-05-10T03:32:12.321451+0000","last_unstale":"2024-05-10T03:32:12.927874+0000","last_undegraded":"2024-05-10T03:32:12.927874+0000","last_fullsized":"2024-05-10T03:32:12.927874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179016+0000","last_change":"2024-05-10T03:32:12.918914+0000","last_active":"2024-05-10T03:32:14.179016+0000","last_peered":"2024-05-10T03:32:14.179016+0000","last_clean":"2024-05-10T03:32:14.179016+0000","last_became_active":"2024-05-10T03:32:12.918226+0000","last_became_peered":"2024-05-10T03:32:12.918226+0000","last_unstale":"2024-05-10T03:32:14.179016+0000","last_undegraded":"2024-05-10T03:32:14.179016+0000","last_fullsized":"2024-05-10T03:32:14.179016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894269+0000","last_change":"2024-05-10T03:32:12.323189+0000","last_active":"2024-05-10T03:32:12.894269+0000","last_peered":"2024-05-10T03:32:12.894269+0000","last_clean":"2024-05-10T03:32:12.894269+0000","last_became_active":"2024-05-10T03:32:12.322864+0000","last_became_peered":"2024-05-10T03:32:12.322864+0000","last_unstale":"2024-05-10T03:32:12.894269+0000","last_undegraded":"2024-05-10T03:32:12.894269+0000","last_fullsized":"2024-05-10T03:32:12.894269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:04.809817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175964+0000","last_change":"2024-05-10T03:32:12.925481+0000","last_active":"2024-05-10T03:32:14.175964+0000","last_peered":"2024-05-10T03:32:14.175964+0000","last_clean":"2024-05-10T03:32:14.175964+0000","last_became_active":"2024-05-10T03:32:12.925134+0000","last_became_peered":"2024-05-10T03:32:12.925134+0000","last_unstale":"2024-05-10T03:32:14.175964+0000","last_undegraded":"2024-05-10T03:32:14.175964+0000","last_fullsized":"2024-05-10T03:32:14.175964+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:13.132758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906033+0000","last_change":"2024-05-10T03:32:12.327618+0000","last_active":"2024-05-10T03:32:12.906033+0000","last_peered":"2024-05-10T03:32:12.906033+0000","last_clean":"2024-05-10T03:32:12.906033+0000","last_became_active":"2024-05-10T03:32:12.327215+0000","last_became_peered":"2024-05-10T03:32:12.327215+0000","last_unstale":"2024-05-10T03:32:12.906033+0000","last_undegraded":"2024-05-10T03:32:12.906033+0000","last_fullsized":"2024-05-10T03:32:12.906033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:39.030457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899016+0000","last_change":"2024-05-10T03:32:12.911714+0000","last_active":"2024-05-10T03:32:13.899016+0000","last_peered":"2024-05-10T03:32:13.899016+0000","last_clean":"2024-05-10T03:32:13.899016+0000","last_became_active":"2024-05-10T03:32:12.908258+0000","last_became_peered":"2024-05-10T03:32:12.908258+0000","last_unstale":"2024-05-10T03:32:13.899016+0000","last_undegraded":"2024-05-10T03:32:13.899016+0000","last_fullsized":"2024-05-10T03:32:13.899016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.882028+0000","last_change":"2024-05-10T03:32:12.325007+0000","last_active":"2024-05-10T03:32:12.882028+0000","last_peered":"2024-05-10T03:32:12.882028+0000","last_clean":"2024-05-10T03:32:12.882028+0000","last_became_active":"2024-05-10T03:32:12.324692+0000","last_became_peered":"2024-05-10T03:32:12.324692+0000","last_unstale":"2024-05-10T03:32:12.882028+0000","last_undegraded":"2024-05-10T03:32:12.882028+0000","last_fullsized":"2024-05-10T03:32:12.882028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:35:55.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899740+0000","last_change":"2024-05-10T03:32:12.899378+0000","last_active":"2024-05-10T03:32:13.899740+0000","last_peered":"2024-05-10T03:32:13.899740+0000","last_clean":"2024-05-10T03:32:13.899740+0000","last_became_active":"2024-05-10T03:32:12.898752+0000","last_became_peered":"2024-05-10T03:32:12.898752+0000","last_unstale":"2024-05-10T03:32:13.899740+0000","last_undegraded":"2024-05-10T03:32:13.899740+0000","last_fullsized":"2024-05-10T03:32:13.899740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.931779+0000","last_change":"2024-05-10T03:32:12.332881+0000","last_active":"2024-05-10T03:32:12.931779+0000","last_peered":"2024-05-10T03:32:12.931779+0000","last_clean":"2024-05-10T03:32:12.931779+0000","last_became_active":"2024-05-10T03:32:12.332577+0000","last_became_peered":"2024-05-10T03:32:12.332577+0000","last_unstale":"2024-05-10T03:32:12.931779+0000","last_undegraded":"2024-05-10T03:32:12.931779+0000","last_fullsized":"2024-05-10T03:32:12.931779+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898806+0000","last_change":"2024-05-10T03:32:12.901260+0000","last_active":"2024-05-10T03:32:13.898806+0000","last_peered":"2024-05-10T03:32:13.898806+0000","last_clean":"2024-05-10T03:32:13.898806+0000","last_became_active":"2024-05-10T03:32:12.900971+0000","last_became_peered":"2024-05-10T03:32:12.900971+0000","last_unstale":"2024-05-10T03:32:13.898806+0000","last_undegraded":"2024-05-10T03:32:13.898806+0000","last_fullsized":"2024-05-10T03:32:13.898806+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909685+0000","last_change":"2024-05-10T03:32:12.340985+0000","last_active":"2024-05-10T03:32:12.909685+0000","last_peered":"2024-05-10T03:32:12.909685+0000","last_clean":"2024-05-10T03:32:12.909685+0000","last_became_active":"2024-05-10T03:32:12.340613+0000","last_became_peered":"2024-05-10T03:32:12.340613+0000","last_unstale":"2024-05-10T03:32:12.909685+0000","last_undegraded":"2024-05-10T03:32:12.909685+0000","last_fullsized":"2024-05-10T03:32:12.909685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178003+0000","last_change":"2024-05-10T03:32:12.905988+0000","last_active":"2024-05-10T03:32:14.178003+0000","last_peered":"2024-05-10T03:32:14.178003+0000","last_clean":"2024-05-10T03:32:14.178003+0000","last_became_active":"2024-05-10T03:32:12.905703+0000","last_became_peered":"2024-05-10T03:32:12.905703+0000","last_unstale":"2024-05-10T03:32:14.178003+0000","last_undegraded":"2024-05-10T03:32:14.178003+0000","last_fullsized":"2024-05-10T03:32:14.178003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:25:53.297139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.911269+0000","last_change":"2024-05-10T03:32:12.329354+0000","last_active":"2024-05-10T03:32:12.911269+0000","last_peered":"2024-05-10T03:32:12.911269+0000","last_clean":"2024-05-10T03:32:12.911269+0000","last_became_active":"2024-05-10T03:32:12.328206+0000","last_became_peered":"2024-05-10T03:32:12.328206+0000","last_unstale":"2024-05-10T03:32:12.911269+0000","last_undegraded":"2024-05-10T03:32:12.911269+0000","last_fullsized":"2024-05-10T03:32:12.911269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:28.939240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176002+0000","last_change":"2024-05-10T03:32:12.917543+0000","last_active":"2024-05-10T03:32:14.176002+0000","last_peered":"2024-05-10T03:32:14.176002+0000","last_clean":"2024-05-10T03:32:14.176002+0000","last_became_active":"2024-05-10T03:32:12.917311+0000","last_became_peered":"2024-05-10T03:32:12.917311+0000","last_unstale":"2024-05-10T03:32:14.176002+0000","last_undegraded":"2024-05-10T03:32:14.176002+0000","last_fullsized":"2024-05-10T03:32:14.176002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:09:39.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894845+0000","last_change":"2024-05-10T03:32:12.323909+0000","last_active":"2024-05-10T03:32:12.894845+0000","last_peered":"2024-05-10T03:32:12.894845+0000","last_clean":"2024-05-10T03:32:12.894845+0000","last_became_active":"2024-05-10T03:32:12.321976+0000","last_became_peered":"2024-05-10T03:32:12.321976+0000","last_unstale":"2024-05-10T03:32:12.894845+0000","last_undegraded":"2024-05-10T03:32:12.894845+0000","last_fullsized":"2024-05-10T03:32:12.894845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:37.925978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900409+0000","last_change":"2024-05-10T03:32:12.911863+0000","last_active":"2024-05-10T03:32:13.900409+0000","last_peered":"2024-05-10T03:32:13.900409+0000","last_clean":"2024-05-10T03:32:13.900409+0000","last_became_active":"2024-05-10T03:32:12.911565+0000","last_became_peered":"2024-05-10T03:32:12.911565+0000","last_unstale":"2024-05-10T03:32:13.900409+0000","last_undegraded":"2024-05-10T03:32:13.900409+0000","last_fullsized":"2024-05-10T03:32:13.900409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894283+0000","last_change":"2024-05-10T03:32:12.323028+0000","last_active":"2024-05-10T03:32:12.894283+0000","last_peered":"2024-05-10T03:32:12.894283+0000","last_clean":"2024-05-10T03:32:12.894283+0000","last_became_active":"2024-05-10T03:32:12.322679+0000","last_became_peered":"2024-05-10T03:32:12.322679+0000","last_unstale":"2024-05-10T03:32:12.894283+0000","last_undegraded":"2024-05-10T03:32:12.894283+0000","last_fullsized":"2024-05-10T03:32:12.894283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:12.822129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175892+0000","last_change":"2024-05-10T03:32:12.912422+0000","last_active":"2024-05-10T03:32:14.175892+0000","last_peered":"2024-05-10T03:32:14.175892+0000","last_clean":"2024-05-10T03:32:14.175892+0000","last_became_active":"2024-05-10T03:32:12.911930+0000","last_became_peered":"2024-05-10T03:32:12.911930+0000","last_unstale":"2024-05-10T03:32:14.175892+0000","last_undegraded":"2024-05-10T03:32:14.175892+0000","last_fullsized":"2024-05-10T03:32:14.175892+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895035+0000","last_change":"2024-05-10T03:32:12.326371+0000","last_active":"2024-05-10T03:32:12.895035+0000","last_peered":"2024-05-10T03:32:12.895035+0000","last_clean":"2024-05-10T03:32:12.895035+0000","last_became_active":"2024-05-10T03:32:12.325858+0000","last_became_peered":"2024-05-10T03:32:12.325858+0000","last_unstale":"2024-05-10T03:32:12.895035+0000","last_undegraded":"2024-05-10T03:32:12.895035+0000","last_fullsized":"2024-05-10T03:32:12.895035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178757+0000","last_change":"2024-05-10T03:32:12.912144+0000","last_active":"2024-05-10T03:32:14.178757+0000","last_peered":"2024-05-10T03:32:14.178757+0000","last_clean":"2024-05-10T03:32:14.178757+0000","last_became_active":"2024-05-10T03:32:12.911805+0000","last_became_peered":"2024-05-10T03:32:12.911805+0000","last_unstale":"2024-05-10T03:32:14.178757+0000","last_undegraded":"2024-05-10T03:32:14.178757+0000","last_fullsized":"2024-05-10T03:32:14.178757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.886727+0000","last_change":"2024-05-10T03:32:12.319651+0000","last_active":"2024-05-10T03:32:12.886727+0000","last_peered":"2024-05-10T03:32:12.886727+0000","last_clean":"2024-05-10T03:32:12.886727+0000","last_became_active":"2024-05-10T03:32:12.319302+0000","last_became_peered":"2024-05-10T03:32:12.319302+0000","last_unstale":"2024-05-10T03:32:12.886727+0000","last_undegraded":"2024-05-10T03:32:12.886727+0000","last_fullsized":"2024-05-10T03:32:12.886727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:42.125952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176756+0000","last_change":"2024-05-10T03:32:12.923331+0000","last_active":"2024-05-10T03:32:14.176756+0000","last_peered":"2024-05-10T03:32:14.176756+0000","last_clean":"2024-05-10T03:32:14.176756+0000","last_became_active":"2024-05-10T03:32:12.922968+0000","last_became_peered":"2024-05-10T03:32:12.922968+0000","last_unstale":"2024-05-10T03:32:14.176756+0000","last_undegraded":"2024-05-10T03:32:14.176756+0000","last_fullsized":"2024-05-10T03:32:14.176756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.912330+0000","last_change":"2024-05-10T03:32:12.327328+0000","last_active":"2024-05-10T03:32:12.912330+0000","last_peered":"2024-05-10T03:32:12.912330+0000","last_clean":"2024-05-10T03:32:12.912330+0000","last_became_active":"2024-05-10T03:32:12.326415+0000","last_became_peered":"2024-05-10T03:32:12.326415+0000","last_unstale":"2024-05-10T03:32:12.912330+0000","last_undegraded":"2024-05-10T03:32:12.912330+0000","last_fullsized":"2024-05-10T03:32:12.912330+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:42.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.905988+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:12.905988+0000","last_peered":"2024-05-10T03:32:12.905988+0000","last_clean":"2024-05-10T03:32:12.905988+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:12.905988+0000","last_undegraded":"2024-05-10T03:32:12.905988+0000","last_fullsized":"2024-05-10T03:32:12.905988+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2230000000000001,"15min":1.2270000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.367,"15min":1.365},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8100000000000001,"5min":1.877,"15min":1.877},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.266,"15min":1.274},"min":{"1min":0.84999999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.667,"5min":1.925,"15min":1.925},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1349999999999998,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4410000000000001,"15min":1.4490000000000001},"min":{"1min":0.81200000000000006,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.819,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3899999999999999,"15min":1.3919999999999999},"min":{"1min":1.0489999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.505},{"interface":"front","average":{"1min":1.46,"5min":1.3520000000000001,"15min":1.3340000000000001},"min":{"1min":1.0109999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1120000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.49,"15min":1.484},"min":{"1min":1.071,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.1339999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.583,"5min":1.5169999999999999,"15min":1.506},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.437}]},{"osd":5,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.546,"15min":1.516},"min":{"1min":1.2030000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1779999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.76,"5min":1.712,"15min":1.704},"min":{"1min":1.383,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.762,"5min":1.7370000000000001,"15min":1.7330000000000001},"min":{"1min":1.3939999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.653,"15min":1.641},"min":{"1min":1.0600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":272133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.073,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.329,"15min":1.351},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8640000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.996}]},{"osd":1,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4590000000000001,"5min":3.113,"15min":3.113},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4750000000000001,"15min":1.4970000000000001},"min":{"1min":1.046,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.944,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.131}]},{"osd":2,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.246,"15min":1.232},"min":{"1min":0.93000000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.482,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5589999999999999,"5min":3.569,"15min":3.569},"last":1.28}]},{"osd":3,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.591,"15min":1.6279999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.48},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.306,"15min":1.304},"min":{"1min":1.0409999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.744,"5min":1.931,"15min":1.931},"last":1.4099999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4359999999999999,"15min":1.4299999999999999},"min":{"1min":1.069,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.8759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5640000000000001,"15min":1.585},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.819,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.6579999999999999}]},{"osd":5,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.1259999999999999,"5min":1,"15min":1},"max":{"1min":1.857,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.004,"5min":2.472,"15min":2.472},"last":1.8859999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.665,"15min":1.671},"min":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":1.9199999999999999,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.651,"15min":1.655},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0070000000000001,"5min":2.359,"15min":2.359},"last":1.821}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":271481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.321}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.446},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.7549999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.512},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.9830000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.921}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.849},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":1.6020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":622592,"data_stored":272063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.173,"15min":1.177},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.1359999999999999,"15min":1.139},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5629999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.165,"15min":1.1559999999999999},"min":{"1min":0.79100000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.734,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.278,"15min":1.2829999999999999},"min":{"1min":0.85699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.994,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.302}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8280000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.042},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9179999999999999,"5min":2.29,"15min":2.29},"last":1.718}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.458,"15min":1.476},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.76,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5900000000000001,"15min":1.589},"min":{"1min":1.2729999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":1.9530000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5009999999999999,"15min":1.5049999999999999},"min":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.8129999999999999,"5min":2.512,"15min":2.512},"last":1.536},{"interface":"front","average":{"1min":1.456,"5min":1.552,"15min":1.5680000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.982,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5900000000000001,"15min":1.5940000000000001},"min":{"1min":1.153,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1139999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7829999999999999,"15min":1.7969999999999999},"min":{"1min":1.1040000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0409999999999999,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.8500000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6930000000000001,"15min":1.7030000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.069,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.601},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7609999999999999,"15min":1.7509999999999999},"min":{"1min":1.4730000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.282,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.655}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":634880,"data_stored":273067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.95399999999999996,"15min":0.97599999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4850000000000001,"5min":1.708,"15min":1.708},"last":1.575}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"min":{"1min":0.55800000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4370000000000001,"5min":1.603,"15min":1.603},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.24,"15min":1.254},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.925,"5min":1.964,"15min":1.964},"last":1.05}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1779999999999999,"15min":1.1839999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.897,"5min":1.99,"15min":1.99},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":1.369},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.919,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.369}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.95099999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.8080000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.417},{"interface":"front","average":{"1min":1.452,"5min":1.3879999999999999,"15min":1.3779999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.1349999999999998,"5min":2.464,"15min":2.464},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.6479999999999999,"15min":1.6859999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.109,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.536},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.462,"15min":1.484},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0499999999999998,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.2559999999999998,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.774,"15min":1.804},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4729999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.5629999999999999,"15min":1.5900000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.1379999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.806},{"interface":"front","average":{"1min":1.571,"5min":1.6479999999999999,"15min":1.661},"min":{"1min":1.052,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.2690000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.6319999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":630784,"data_stored":273045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.494,"5min":1.772,"15min":1.772},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.431,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.88900000000000001}]},{"osd":1,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5600000000000001,"5min":1.621,"15min":1.621},"last":1.425},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1930000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6930000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3420000000000001}]},{"osd":3,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1240000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5009999999999999,"5min":2.169,"15min":2.169},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.25,"15min":1.2629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9650000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.526,"15min":1.548},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.133,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.516,"15min":1.5569999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.821,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.2410000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.036,"5min":2.508,"15min":2.508},"last":1.887},{"interface":"front","average":{"1min":1.546,"5min":1.6140000000000001,"15min":1.625},"min":{"1min":1.069,"5min":1.004,"15min":1.004},"max":{"1min":2.1190000000000002,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0089999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1139999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.782},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5820000000000001,"15min":1.5980000000000001},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.982,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.9710000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.641,"15min":1.6599999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.1360000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.665},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.659,"15min":1.679},"min":{"1min":1.052,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.169,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.7370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":626688,"data_stored":271641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.54600000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.256,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.755}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.488,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.1699999999999999,"15min":1.1910000000000001},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4330000000000001,"5min":2.444,"15min":2.444},"last":1.004}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.3,"15min":1.3280000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.556,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3100000000000001},"min":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7789999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.673,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.155},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5669999999999999,"15min":1.597},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":1.8220000000000001,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.5600000000000001,"15min":1.5980000000000001},"min":{"1min":1.069,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.173},{"interface":"front","average":{"1min":1.458,"5min":1.6200000000000001,"15min":1.647},"min":{"1min":1.1080000000000001,"5min":1.077,"15min":1.077},"max":{"1min":1.9770000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.47}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.6990000000000001,"15min":1.7370000000000001},"min":{"1min":0.94999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0550000000000002,"5min":3.48,"15min":3.48},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.716,"15min":1.748},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.0800000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.5569999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.7270000000000001,"15min":1.7470000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0910000000000002,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.762,"15min":1.796},"min":{"1min":1.173,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.7909999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.3300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.999,"15min":1.036},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5629999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":1.1599999999999999,"15min":1.2110000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5389999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.593},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4550000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.089,"15min":1.1140000000000001},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.448,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.929,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.159}]},{"osd":4,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.6040000000000001,"15min":1.6539999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0209999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.655},{"interface":"front","average":{"1min":1.343,"5min":1.431,"15min":1.446},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.661,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.464,"15min":1.486},"min":{"1min":0.85799999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9710000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6080000000000001,"15min":1.647},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8999999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.7490000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6299999999999999,"15min":1.6539999999999999},"min":{"1min":1.028,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.206,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.7150000000000001,"15min":1.7629999999999999},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.972,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.5369999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.77,"15min":1.8200000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.946,"5min":3.149,"15min":3.149},"last":1.381},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5609999999999999,"15min":1.581},"min":{"1min":0.91500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.99,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.8069999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:32:19.307 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-10T03:32:19.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.621+0000 7f5411400640 1 Processor -- start 2024-05-10T03:32:19.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.621+0000 7f5411400640 1 -- start start 2024-05-10T03:32:19.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.623+0000 7f5411400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f540c144710 0x7f540c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.623+0000 7f5411400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f540c0b3190 0x7f540c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.623+0000 7f5411400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c06d350 con 0x7f540c06d7e0 2024-05-10T03:32:19.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.623+0000 7f5411400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c06c740 con 0x7f540c0b3190 2024-05-10T03:32:19.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.624+0000 7f5411400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c06ca20 con 0x7f540c144710 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.624+0000 7f5410a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f540c0b3190 0x7f540c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.624+0000 7f5410a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f540c0b3190 0x7f540c0cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40314/0 (socket says 172.21.15.55:40314) 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.624+0000 7f5410a00640 1 -- 172.21.15.55:0/231403331 learned_addr learned my addr 172.21.15.55:0/231403331 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.624+0000 7f540be00640 1 --2- 172.21.15.55:0/231403331 >> v2:172.21.15.162:3300/0 conn(0x7f540c144710 0x7f540c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.625+0000 7f5410a00640 1 -- 172.21.15.55:0/231403331 >> v2:172.21.15.162:3300/0 conn(0x7f540c144710 msgr2=0x7f540c144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.625+0000 7f5410a00640 1 --2- 172.21.15.55:0/231403331 >> v2:172.21.15.162:3300/0 conn(0x7f540c144710 0x7f540c144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.625+0000 7f5410a00640 1 -- 172.21.15.55:0/231403331 >> v1:172.21.15.55:6789/0 conn(0x7f540c06d7e0 legacy=0x7f540c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.625+0000 7f5410a00640 1 -- 172.21.15.55:0/231403331 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c06cd00 con 0x7f540c0b3190 2024-05-10T03:32:19.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.626+0000 7f5410a00640 1 --2- 172.21.15.55:0/231403331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c0cd9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f540c072240 tx=0x7f540402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7a035f4900a31c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.626+0000 7f540aa00640 1 -- 172.21.15.55:0/231403331 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404031600 con 0x7f540c0b3190 2024-05-10T03:32:19.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.626+0000 7f540aa00640 1 -- 172.21.15.55:0/231403331 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540402d050 con 0x7f540c0b3190 2024-05-10T03:32:19.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.626+0000 7f540aa00640 1 -- 172.21.15.55:0/231403331 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404031980 con 0x7f540c0b3190 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 -- 172.21.15.55:0/231403331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 msgr2=0x7f540c0cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 --2- 172.21.15.55:0/231403331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c0cd9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f540c072240 tx=0x7f540402a610 comp rx=0 tx=0).stop 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 -- 172.21.15.55:0/231403331 shutdown_connections 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 --2- 172.21.15.55:0/231403331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c0cd9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 --2- 172.21.15.55:0/231403331 >> v2:172.21.15.162:3300/0 conn(0x7f540c144710 0x7f540c144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 -- 172.21.15.55:0/231403331 >> 172.21.15.55:0/231403331 conn(0x7f540c06a110 msgr2=0x7f540c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 -- 172.21.15.55:0/231403331 shutdown_connections 2024-05-10T03:32:19.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.627+0000 7f5411400640 1 -- 172.21.15.55:0/231403331 wait complete. 2024-05-10T03:32:19.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.629+0000 7f5411400640 1 Processor -- start 2024-05-10T03:32:19.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.630+0000 7f5411400640 1 -- start start 2024-05-10T03:32:19.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 0x7f540c15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f540c144710 0x7f540c136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c136db0 con 0x7f540c0b3190 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c138d40 con 0x7f540c06d7e0 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f5411400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c135450 con 0x7f540c144710 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f540b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f540b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40332/0 (socket says 172.21.15.55:40332) 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.631+0000 7f540b400640 1 -- 172.21.15.55:0/2074572882 learned_addr learned my addr 172.21.15.55:0/2074572882 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:19.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540be00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 0x7f540c15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f5410a00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f540c144710 0x7f540c136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 -- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f540c144710 msgr2=0x7f540c136020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f540c144710 0x7f540c136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 -- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 msgr2=0x7f540c15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 0x7f540c15dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 -- 172.21.15.55:0/2074572882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404009b30 con 0x7f540c0b3190 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540be00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 0x7f540c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f540b400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c400 tx=0x7f53f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.632+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f40052d0 con 0x7f540c0b3190 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.633+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f4005430 con 0x7f540c0b3190 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.633+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003830 con 0x7f540c0b3190 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.633+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c151590 con 0x7f540c0b3190 2024-05-10T03:32:19.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.633+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c15d500 con 0x7f540c0b3190 2024-05-10T03:32:19.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.634+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c066ce0 con 0x7f540c0b3190 2024-05-10T03:32:19.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.635+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f53f4003990 con 0x7f540c0b3190 2024-05-10T03:32:19.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.636+0000 7f5403e00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 0x7f53e806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:19.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.636+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f53f4088a90 con 0x7f540c0b3190 2024-05-10T03:32:19.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.637+0000 7f540be00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 0x7f53e806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:19.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.638+0000 7f540be00640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 0x7f53e806d140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006fd0 tx=0x7f53fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:19.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.641+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f53f4053a90 con 0x7f540c0b3190 2024-05-10T03:32:19.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.924+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f540c04e290 con 0x7f540c0b3190 2024-05-10T03:32:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.925+0000 7f5403e00640 1 -- 172.21.15.55:0/2074572882 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1013 (secure 0 0 0) 0x7f53f40570b0 con 0x7f540c0b3190 2024-05-10T03:32:19.928 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:19.928 INFO:teuthology.orchestra.run.smithi055.stdout:{"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1715311771,"num_in_osds":8,"osd_in_since":1715311761,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":248582144,"bytes_avail":772845531136,"bytes_total":773094113280,"write_bytes_sec":2133,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5739}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-10T03:30:50.973110+0000","services":{}},"progress_events":{}} 2024-05-10T03:32:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 msgr2=0x7f53e806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 0x7f53e806d140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006fd0 tx=0x7f53fc008040 comp rx=0 tx=0).stop 2024-05-10T03:32:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 msgr2=0x7f540c1f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c400 tx=0x7f53f400c8c0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 shutdown_connections 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f540c144710 0x7f540c136020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f540c0b3190 0x7f540c1f5280 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f53e806ac30 0x7f53e806d140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 --2- 172.21.15.55:0/2074572882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f540c06d7e0 0x7f540c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 >> 172.21.15.55:0/2074572882 conn(0x7f540c06a110 msgr2=0x7f540c06b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 shutdown_connections 2024-05-10T03:32:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:19.928+0000 7f5411400640 1 -- 172.21.15.55:0/2074572882 wait complete. 2024-05-10T03:32:19.958 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-10T03:32:19.958 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:32:20.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.237+0000 7f3237a00640 1 Processor -- start 2024-05-10T03:32:20.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.237+0000 7f3237a00640 1 -- start start 2024-05-10T03:32:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300ad7b0 0x7f3230161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3230161c70 0x7f32300719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32300adbc0 0x7f3230071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32301623a0 con 0x7f32300adbc0 2024-05-10T03:32:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32300cdd60 con 0x7f3230161c70 2024-05-10T03:32:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.240+0000 7f3237a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ce040 con 0x7f32300ad7b0 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3235a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32300adbc0 0x7f3230071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3235a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f32300adbc0 0x7f3230071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40336/0 (socket says 172.21.15.55:40336) 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3235a00640 1 -- 172.21.15.55:0/4078494103 learned_addr learned my addr 172.21.15.55:0/4078494103 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3227e00640 1 --2- 172.21.15.55:0/4078494103 >> v2:172.21.15.162:3300/0 conn(0x7f3230161c70 0x7f32300719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3235000640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300ad7b0 0x7f3230161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.241+0000 7f3235a00640 1 -- 172.21.15.55:0/4078494103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300ad7b0 msgr2=0x7f3230161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.242+0000 7f3235a00640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300ad7b0 0x7f3230161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.242+0000 7f3235a00640 1 -- 172.21.15.55:0/4078494103 >> v2:172.21.15.162:3300/0 conn(0x7f3230161c70 msgr2=0x7f32300719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.242+0000 7f3235a00640 1 --2- 172.21.15.55:0/4078494103 >> v2:172.21.15.162:3300/0 conn(0x7f3230161c70 0x7f32300719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.242+0000 7f3235a00640 1 -- 172.21.15.55:0/4078494103 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300ce320 con 0x7f32300adbc0 2024-05-10T03:32:20.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.243+0000 7f3235a00640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32300adbc0 0x7f3230071f20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f323006d200 tx=0x7f322802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e6df2280ed8078 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.244+0000 7f322fe00640 1 -- 172.21.15.55:0/4078494103 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322802ae90 con 0x7f32300adbc0 2024-05-10T03:32:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.244+0000 7f322fe00640 1 -- 172.21.15.55:0/4078494103 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228032440 con 0x7f32300adbc0 2024-05-10T03:32:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.244+0000 7f322fe00640 1 -- 172.21.15.55:0/4078494103 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280327d0 con 0x7f32300adbc0 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.244+0000 7f3237a00640 1 -- 172.21.15.55:0/4078494103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32300adbc0 msgr2=0x7f3230071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.244+0000 7f3237a00640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32300adbc0 0x7f3230071f20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f323006d200 tx=0x7f322802a140 comp rx=0 tx=0).stop 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 -- 172.21.15.55:0/4078494103 shutdown_connections 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32300adbc0 0x7f3230071f20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 --2- 172.21.15.55:0/4078494103 >> v2:172.21.15.162:3300/0 conn(0x7f3230161c70 0x7f32300719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 --2- 172.21.15.55:0/4078494103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300ad7b0 0x7f3230161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 -- 172.21.15.55:0/4078494103 >> 172.21.15.55:0/4078494103 conn(0x7f32300cbb90 msgr2=0x7f32300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 -- 172.21.15.55:0/4078494103 shutdown_connections 2024-05-10T03:32:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.245+0000 7f3237a00640 1 -- 172.21.15.55:0/4078494103 wait complete. 2024-05-10T03:32:20.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.247+0000 7f3237a00640 1 Processor -- start 2024-05-10T03:32:20.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.248+0000 7f3237a00640 1 -- start start 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32300ad7b0 0x7f32301508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300adbc0 0x7f3230145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230151620 con 0x7f3230161c70 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323014cb00 con 0x7f32300ad7b0 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.249+0000 7f3237a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32301510f0 con 0x7f32300adbc0 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40340/0 (socket says 172.21.15.55:40340) 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 -- 172.21.15.55:0/1552663874 learned_addr learned my addr 172.21.15.55:0/1552663874 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235000640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32300ad7b0 0x7f32301508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3227e00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300adbc0 0x7f3230145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 -- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300adbc0 msgr2=0x7f3230145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300adbc0 0x7f3230145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 -- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32300ad7b0 msgr2=0x7f32301508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32300ad7b0 0x7f32301508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.250+0000 7f3235a00640 1 -- 172.21.15.55:0/1552663874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280094d0 con 0x7f3230161c70 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f3235a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3228009960 tx=0x7f32280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322803d830 con 0x7f3230161c70 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322803d990 con 0x7f3230161c70 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32301e7510 con 0x7f3230161c70 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322803dd40 con 0x7f3230161c70 2024-05-10T03:32:20.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.251+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230151340 con 0x7f3230161c70 2024-05-10T03:32:20.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.253+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32301e7370 con 0x7f3230161c70 2024-05-10T03:32:20.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.254+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3228039040 con 0x7f3230161c70 2024-05-10T03:32:20.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.256+0000 7f322d600640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 0x7f320c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:20.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.256+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32280a9790 con 0x7f3230161c70 2024-05-10T03:32:20.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.256+0000 7f3235000640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 0x7f320c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:20.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.258+0000 7f3235000640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 0x7f320c06d110 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f32200073f0 tx=0x7f322000d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:20.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.261+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f32301e7370 con 0x7f3230161c70 2024-05-10T03:32:20.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.465+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32301e7370 con 0x7f320c06ac00 2024-05-10T03:32:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.490+0000 7f322d600640 1 -- 172.21.15.55:0/1552663874 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286654 (secure 0 0 0) 0x7f32301e7370 con 0x7f320c06ac00 2024-05-10T03:32:20.497 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 msgr2=0x7f320c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 0x7f320c06d110 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f32200073f0 tx=0x7f322000d040 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 msgr2=0x7f323006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3228009960 tx=0x7f32280046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 shutdown_connections 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3230161c70 0x7f323006ac70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32300adbc0 0x7f3230145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320c06ac00 0x7f320c06d110 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 --2- 172.21.15.55:0/1552663874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32300ad7b0 0x7f32301508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 >> 172.21.15.55:0/1552663874 conn(0x7f32300cbb90 msgr2=0x7f32300af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.496+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 shutdown_connections 2024-05-10T03:32:20.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:20.497+0000 7f3237a00640 1 -- 172.21.15.55:0/1552663874 wait complete. 2024-05-10T03:32:20.525 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-05-10T03:32:19.009076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242756,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731964,"statfs":{"total":773094113280,"available":772845531136,"internally_reserved":0,"allocated":6201344,"data_stored":3357308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.720375"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898678+0000","last_change":"2024-05-10T03:32:12.914232+0000","last_active":"2024-05-10T03:32:13.898678+0000","last_peered":"2024-05-10T03:32:13.898678+0000","last_clean":"2024-05-10T03:32:13.898678+0000","last_became_active":"2024-05-10T03:32:12.913664+0000","last_became_peered":"2024-05-10T03:32:12.913664+0000","last_unstale":"2024-05-10T03:32:13.898678+0000","last_undegraded":"2024-05-10T03:32:13.898678+0000","last_fullsized":"2024-05-10T03:32:13.898678+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178978+0000","last_change":"2024-05-10T03:32:12.922843+0000","last_active":"2024-05-10T03:32:14.178978+0000","last_peered":"2024-05-10T03:32:14.178978+0000","last_clean":"2024-05-10T03:32:14.178978+0000","last_became_active":"2024-05-10T03:32:12.922531+0000","last_became_peered":"2024-05-10T03:32:12.922531+0000","last_unstale":"2024-05-10T03:32:14.178978+0000","last_undegraded":"2024-05-10T03:32:14.178978+0000","last_fullsized":"2024-05-10T03:32:14.178978+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175288+0000","last_change":"2024-05-10T03:32:12.918902+0000","last_active":"2024-05-10T03:32:14.175288+0000","last_peered":"2024-05-10T03:32:14.175288+0000","last_clean":"2024-05-10T03:32:14.175288+0000","last_became_active":"2024-05-10T03:32:12.918598+0000","last_became_peered":"2024-05-10T03:32:12.918598+0000","last_unstale":"2024-05-10T03:32:14.175288+0000","last_undegraded":"2024-05-10T03:32:14.175288+0000","last_fullsized":"2024-05-10T03:32:14.175288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178493+0000","last_change":"2024-05-10T03:32:12.924668+0000","last_active":"2024-05-10T03:32:14.178493+0000","last_peered":"2024-05-10T03:32:14.178493+0000","last_clean":"2024-05-10T03:32:14.178493+0000","last_became_active":"2024-05-10T03:32:12.923917+0000","last_became_peered":"2024-05-10T03:32:12.923917+0000","last_unstale":"2024-05-10T03:32:14.178493+0000","last_undegraded":"2024-05-10T03:32:14.178493+0000","last_fullsized":"2024-05-10T03:32:14.178493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902676+0000","last_change":"2024-05-10T03:32:12.915697+0000","last_active":"2024-05-10T03:32:13.902676+0000","last_peered":"2024-05-10T03:32:13.902676+0000","last_clean":"2024-05-10T03:32:13.902676+0000","last_became_active":"2024-05-10T03:32:12.915337+0000","last_became_peered":"2024-05-10T03:32:12.915337+0000","last_unstale":"2024-05-10T03:32:13.902676+0000","last_undegraded":"2024-05-10T03:32:13.902676+0000","last_fullsized":"2024-05-10T03:32:13.902676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898620+0000","last_change":"2024-05-10T03:32:12.910337+0000","last_active":"2024-05-10T03:32:13.898620+0000","last_peered":"2024-05-10T03:32:13.898620+0000","last_clean":"2024-05-10T03:32:13.898620+0000","last_became_active":"2024-05-10T03:32:12.910049+0000","last_became_peered":"2024-05-10T03:32:12.910049+0000","last_unstale":"2024-05-10T03:32:13.898620+0000","last_undegraded":"2024-05-10T03:32:13.898620+0000","last_fullsized":"2024-05-10T03:32:13.898620+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174738+0000","last_change":"2024-05-10T03:32:12.918643+0000","last_active":"2024-05-10T03:32:14.174738+0000","last_peered":"2024-05-10T03:32:14.174738+0000","last_clean":"2024-05-10T03:32:14.174738+0000","last_became_active":"2024-05-10T03:32:12.918296+0000","last_became_peered":"2024-05-10T03:32:12.918296+0000","last_unstale":"2024-05-10T03:32:14.174738+0000","last_undegraded":"2024-05-10T03:32:14.174738+0000","last_fullsized":"2024-05-10T03:32:14.174738+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177500+0000","last_change":"2024-05-10T03:32:12.919824+0000","last_active":"2024-05-10T03:32:14.177500+0000","last_peered":"2024-05-10T03:32:14.177500+0000","last_clean":"2024-05-10T03:32:14.177500+0000","last_became_active":"2024-05-10T03:32:12.919521+0000","last_became_peered":"2024-05-10T03:32:12.919521+0000","last_unstale":"2024-05-10T03:32:14.177500+0000","last_undegraded":"2024-05-10T03:32:14.177500+0000","last_fullsized":"2024-05-10T03:32:14.177500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:45.737960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178502+0000","last_change":"2024-05-10T03:32:12.923819+0000","last_active":"2024-05-10T03:32:14.178502+0000","last_peered":"2024-05-10T03:32:14.178502+0000","last_clean":"2024-05-10T03:32:14.178502+0000","last_became_active":"2024-05-10T03:32:12.922895+0000","last_became_peered":"2024-05-10T03:32:12.922895+0000","last_unstale":"2024-05-10T03:32:14.178502+0000","last_undegraded":"2024-05-10T03:32:14.178502+0000","last_fullsized":"2024-05-10T03:32:14.178502+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:21.171840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898711+0000","last_change":"2024-05-10T03:32:12.908590+0000","last_active":"2024-05-10T03:32:13.898711+0000","last_peered":"2024-05-10T03:32:13.898711+0000","last_clean":"2024-05-10T03:32:13.898711+0000","last_became_active":"2024-05-10T03:32:12.908181+0000","last_became_peered":"2024-05-10T03:32:12.908181+0000","last_unstale":"2024-05-10T03:32:13.898711+0000","last_undegraded":"2024-05-10T03:32:13.898711+0000","last_fullsized":"2024-05-10T03:32:13.898711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116836+0000","last_change":"2024-05-10T03:32:12.921052+0000","last_active":"2024-05-10T03:32:14.116836+0000","last_peered":"2024-05-10T03:32:14.116836+0000","last_clean":"2024-05-10T03:32:14.116836+0000","last_became_active":"2024-05-10T03:32:12.918022+0000","last_became_peered":"2024-05-10T03:32:12.918022+0000","last_unstale":"2024-05-10T03:32:14.116836+0000","last_undegraded":"2024-05-10T03:32:14.116836+0000","last_fullsized":"2024-05-10T03:32:14.116836+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:13.248072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178356+0000","last_change":"2024-05-10T03:32:12.921186+0000","last_active":"2024-05-10T03:32:14.178356+0000","last_peered":"2024-05-10T03:32:14.178356+0000","last_clean":"2024-05-10T03:32:14.178356+0000","last_became_active":"2024-05-10T03:32:12.920915+0000","last_became_peered":"2024-05-10T03:32:12.920915+0000","last_unstale":"2024-05-10T03:32:14.178356+0000","last_undegraded":"2024-05-10T03:32:14.178356+0000","last_fullsized":"2024-05-10T03:32:14.178356+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:51.954514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898529+0000","last_change":"2024-05-10T03:32:12.908427+0000","last_active":"2024-05-10T03:32:13.898529+0000","last_peered":"2024-05-10T03:32:13.898529+0000","last_clean":"2024-05-10T03:32:13.898529+0000","last_became_active":"2024-05-10T03:32:12.908141+0000","last_became_peered":"2024-05-10T03:32:12.908141+0000","last_unstale":"2024-05-10T03:32:13.898529+0000","last_undegraded":"2024-05-10T03:32:13.898529+0000","last_fullsized":"2024-05-10T03:32:13.898529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175216+0000","last_change":"2024-05-10T03:32:12.915903+0000","last_active":"2024-05-10T03:32:14.175216+0000","last_peered":"2024-05-10T03:32:14.175216+0000","last_clean":"2024-05-10T03:32:14.175216+0000","last_became_active":"2024-05-10T03:32:12.915485+0000","last_became_peered":"2024-05-10T03:32:12.915485+0000","last_unstale":"2024-05-10T03:32:14.175216+0000","last_undegraded":"2024-05-10T03:32:14.175216+0000","last_fullsized":"2024-05-10T03:32:14.175216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:04.967182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177018+0000","last_change":"2024-05-10T03:32:12.921319+0000","last_active":"2024-05-10T03:32:14.177018+0000","last_peered":"2024-05-10T03:32:14.177018+0000","last_clean":"2024-05-10T03:32:14.177018+0000","last_became_active":"2024-05-10T03:32:12.920917+0000","last_became_peered":"2024-05-10T03:32:12.920917+0000","last_unstale":"2024-05-10T03:32:14.177018+0000","last_undegraded":"2024-05-10T03:32:14.177018+0000","last_fullsized":"2024-05-10T03:32:14.177018+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898412+0000","last_change":"2024-05-10T03:32:12.909443+0000","last_active":"2024-05-10T03:32:13.898412+0000","last_peered":"2024-05-10T03:32:13.898412+0000","last_clean":"2024-05-10T03:32:13.898412+0000","last_became_active":"2024-05-10T03:32:12.908121+0000","last_became_peered":"2024-05-10T03:32:12.908121+0000","last_unstale":"2024-05-10T03:32:13.898412+0000","last_undegraded":"2024-05-10T03:32:13.898412+0000","last_fullsized":"2024-05-10T03:32:13.898412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175155+0000","last_change":"2024-05-10T03:32:12.924889+0000","last_active":"2024-05-10T03:32:14.175155+0000","last_peered":"2024-05-10T03:32:14.175155+0000","last_clean":"2024-05-10T03:32:14.175155+0000","last_became_active":"2024-05-10T03:32:12.920754+0000","last_became_peered":"2024-05-10T03:32:12.920754+0000","last_unstale":"2024-05-10T03:32:14.175155+0000","last_undegraded":"2024-05-10T03:32:14.175155+0000","last_fullsized":"2024-05-10T03:32:14.175155+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179224+0000","last_change":"2024-05-10T03:32:12.922850+0000","last_active":"2024-05-10T03:32:14.179224+0000","last_peered":"2024-05-10T03:32:14.179224+0000","last_clean":"2024-05-10T03:32:14.179224+0000","last_became_active":"2024-05-10T03:32:12.922543+0000","last_became_peered":"2024-05-10T03:32:12.922543+0000","last_unstale":"2024-05-10T03:32:14.179224+0000","last_undegraded":"2024-05-10T03:32:14.179224+0000","last_fullsized":"2024-05-10T03:32:14.179224+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175667+0000","last_change":"2024-05-10T03:32:12.915732+0000","last_active":"2024-05-10T03:32:14.175667+0000","last_peered":"2024-05-10T03:32:14.175667+0000","last_clean":"2024-05-10T03:32:14.175667+0000","last_became_active":"2024-05-10T03:32:12.914507+0000","last_became_peered":"2024-05-10T03:32:12.914507+0000","last_unstale":"2024-05-10T03:32:14.175667+0000","last_undegraded":"2024-05-10T03:32:14.175667+0000","last_fullsized":"2024-05-10T03:32:14.175667+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:03.419392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178102+0000","last_change":"2024-05-10T03:32:12.919811+0000","last_active":"2024-05-10T03:32:14.178102+0000","last_peered":"2024-05-10T03:32:14.178102+0000","last_clean":"2024-05-10T03:32:14.178102+0000","last_became_active":"2024-05-10T03:32:12.919469+0000","last_became_peered":"2024-05-10T03:32:12.919469+0000","last_unstale":"2024-05-10T03:32:14.178102+0000","last_undegraded":"2024-05-10T03:32:14.178102+0000","last_fullsized":"2024-05-10T03:32:14.178102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177895+0000","last_change":"2024-05-10T03:32:12.926306+0000","last_active":"2024-05-10T03:32:14.177895+0000","last_peered":"2024-05-10T03:32:14.177895+0000","last_clean":"2024-05-10T03:32:14.177895+0000","last_became_active":"2024-05-10T03:32:12.925942+0000","last_became_peered":"2024-05-10T03:32:12.925942+0000","last_unstale":"2024-05-10T03:32:14.177895+0000","last_undegraded":"2024-05-10T03:32:14.177895+0000","last_fullsized":"2024-05-10T03:32:14.177895+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177762+0000","last_change":"2024-05-10T03:32:12.922894+0000","last_active":"2024-05-10T03:32:14.177762+0000","last_peered":"2024-05-10T03:32:14.177762+0000","last_clean":"2024-05-10T03:32:14.177762+0000","last_became_active":"2024-05-10T03:32:12.922244+0000","last_became_peered":"2024-05-10T03:32:12.922244+0000","last_unstale":"2024-05-10T03:32:14.177762+0000","last_undegraded":"2024-05-10T03:32:14.177762+0000","last_fullsized":"2024-05-10T03:32:14.177762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:03.341641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116024+0000","last_change":"2024-05-10T03:32:12.921743+0000","last_active":"2024-05-10T03:32:14.116024+0000","last_peered":"2024-05-10T03:32:14.116024+0000","last_clean":"2024-05-10T03:32:14.116024+0000","last_became_active":"2024-05-10T03:32:12.918101+0000","last_became_peered":"2024-05-10T03:32:12.918101+0000","last_unstale":"2024-05-10T03:32:14.116024+0000","last_undegraded":"2024-05-10T03:32:14.116024+0000","last_fullsized":"2024-05-10T03:32:14.116024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:48.926764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174969+0000","last_change":"2024-05-10T03:32:12.915425+0000","last_active":"2024-05-10T03:32:14.174969+0000","last_peered":"2024-05-10T03:32:14.174969+0000","last_clean":"2024-05-10T03:32:14.174969+0000","last_became_active":"2024-05-10T03:32:12.914547+0000","last_became_peered":"2024-05-10T03:32:12.914547+0000","last_unstale":"2024-05-10T03:32:14.174969+0000","last_undegraded":"2024-05-10T03:32:14.174969+0000","last_fullsized":"2024-05-10T03:32:14.174969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175394+0000","last_change":"2024-05-10T03:32:12.925418+0000","last_active":"2024-05-10T03:32:14.175394+0000","last_peered":"2024-05-10T03:32:14.175394+0000","last_clean":"2024-05-10T03:32:14.175394+0000","last_became_active":"2024-05-10T03:32:12.924225+0000","last_became_peered":"2024-05-10T03:32:12.924225+0000","last_unstale":"2024-05-10T03:32:14.175394+0000","last_undegraded":"2024-05-10T03:32:14.175394+0000","last_fullsized":"2024-05-10T03:32:14.175394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:26.673168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178116+0000","last_change":"2024-05-10T03:32:12.910349+0000","last_active":"2024-05-10T03:32:14.178116+0000","last_peered":"2024-05-10T03:32:14.178116+0000","last_clean":"2024-05-10T03:32:14.178116+0000","last_became_active":"2024-05-10T03:32:12.910056+0000","last_became_peered":"2024-05-10T03:32:12.910056+0000","last_unstale":"2024-05-10T03:32:14.178116+0000","last_undegraded":"2024-05-10T03:32:14.178116+0000","last_fullsized":"2024-05-10T03:32:14.178116+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:08.718135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899066+0000","last_change":"2024-05-10T03:32:12.919859+0000","last_active":"2024-05-10T03:32:13.899066+0000","last_peered":"2024-05-10T03:32:13.899066+0000","last_clean":"2024-05-10T03:32:13.899066+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.899066+0000","last_undegraded":"2024-05-10T03:32:13.899066+0000","last_fullsized":"2024-05-10T03:32:13.899066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178414+0000","last_change":"2024-05-10T03:32:12.927253+0000","last_active":"2024-05-10T03:32:14.178414+0000","last_peered":"2024-05-10T03:32:14.178414+0000","last_clean":"2024-05-10T03:32:14.178414+0000","last_became_active":"2024-05-10T03:32:12.926930+0000","last_became_peered":"2024-05-10T03:32:12.926930+0000","last_unstale":"2024-05-10T03:32:14.178414+0000","last_undegraded":"2024-05-10T03:32:14.178414+0000","last_fullsized":"2024-05-10T03:32:14.178414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176263+0000","last_change":"2024-05-10T03:32:12.921104+0000","last_active":"2024-05-10T03:32:14.176263+0000","last_peered":"2024-05-10T03:32:14.176263+0000","last_clean":"2024-05-10T03:32:14.176263+0000","last_became_active":"2024-05-10T03:32:12.920745+0000","last_became_peered":"2024-05-10T03:32:12.920745+0000","last_unstale":"2024-05-10T03:32:14.176263+0000","last_undegraded":"2024-05-10T03:32:14.176263+0000","last_fullsized":"2024-05-10T03:32:14.176263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:02.965291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177679+0000","last_change":"2024-05-10T03:32:12.927250+0000","last_active":"2024-05-10T03:32:14.177679+0000","last_peered":"2024-05-10T03:32:14.177679+0000","last_clean":"2024-05-10T03:32:14.177679+0000","last_became_active":"2024-05-10T03:32:12.926910+0000","last_became_peered":"2024-05-10T03:32:12.926910+0000","last_unstale":"2024-05-10T03:32:14.177679+0000","last_undegraded":"2024-05-10T03:32:14.177679+0000","last_fullsized":"2024-05-10T03:32:14.177679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174982+0000","last_change":"2024-05-10T03:32:12.917812+0000","last_active":"2024-05-10T03:32:14.174982+0000","last_peered":"2024-05-10T03:32:14.174982+0000","last_clean":"2024-05-10T03:32:14.174982+0000","last_became_active":"2024-05-10T03:32:12.913954+0000","last_became_peered":"2024-05-10T03:32:12.913954+0000","last_unstale":"2024-05-10T03:32:14.174982+0000","last_undegraded":"2024-05-10T03:32:14.174982+0000","last_fullsized":"2024-05-10T03:32:14.174982+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899746+0000","last_change":"2024-05-10T03:32:12.902462+0000","last_active":"2024-05-10T03:32:13.899746+0000","last_peered":"2024-05-10T03:32:13.899746+0000","last_clean":"2024-05-10T03:32:13.899746+0000","last_became_active":"2024-05-10T03:32:12.901977+0000","last_became_peered":"2024-05-10T03:32:12.901977+0000","last_unstale":"2024-05-10T03:32:13.899746+0000","last_undegraded":"2024-05-10T03:32:13.899746+0000","last_fullsized":"2024-05-10T03:32:13.899746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:05.823783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176963+0000","last_change":"2024-05-10T03:32:12.923673+0000","last_active":"2024-05-10T03:32:14.176963+0000","last_peered":"2024-05-10T03:32:14.176963+0000","last_clean":"2024-05-10T03:32:14.176963+0000","last_became_active":"2024-05-10T03:32:12.923367+0000","last_became_peered":"2024-05-10T03:32:12.923367+0000","last_unstale":"2024-05-10T03:32:14.176963+0000","last_undegraded":"2024-05-10T03:32:14.176963+0000","last_fullsized":"2024-05-10T03:32:14.176963+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927108+0000","last_change":"2024-05-10T03:32:12.321598+0000","last_active":"2024-05-10T03:32:12.927108+0000","last_peered":"2024-05-10T03:32:12.927108+0000","last_clean":"2024-05-10T03:32:12.927108+0000","last_became_active":"2024-05-10T03:32:12.320780+0000","last_became_peered":"2024-05-10T03:32:12.320780+0000","last_unstale":"2024-05-10T03:32:12.927108+0000","last_undegraded":"2024-05-10T03:32:12.927108+0000","last_fullsized":"2024-05-10T03:32:12.927108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:09:45.664808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900214+0000","last_change":"2024-05-10T03:32:12.912792+0000","last_active":"2024-05-10T03:32:13.900214+0000","last_peered":"2024-05-10T03:32:13.900214+0000","last_clean":"2024-05-10T03:32:13.900214+0000","last_became_active":"2024-05-10T03:32:12.907370+0000","last_became_peered":"2024-05-10T03:32:12.907370+0000","last_unstale":"2024-05-10T03:32:13.900214+0000","last_undegraded":"2024-05-10T03:32:13.900214+0000","last_fullsized":"2024-05-10T03:32:13.900214+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.929409+0000","last_change":"2024-05-10T03:32:12.323974+0000","last_active":"2024-05-10T03:32:12.929409+0000","last_peered":"2024-05-10T03:32:12.929409+0000","last_clean":"2024-05-10T03:32:12.929409+0000","last_became_active":"2024-05-10T03:32:12.323651+0000","last_became_peered":"2024-05-10T03:32:12.323651+0000","last_unstale":"2024-05-10T03:32:12.929409+0000","last_undegraded":"2024-05-10T03:32:12.929409+0000","last_fullsized":"2024-05-10T03:32:12.929409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177373+0000","last_change":"2024-05-10T03:32:12.926741+0000","last_active":"2024-05-10T03:32:14.177373+0000","last_peered":"2024-05-10T03:32:14.177373+0000","last_clean":"2024-05-10T03:32:14.177373+0000","last_became_active":"2024-05-10T03:32:12.926261+0000","last_became_peered":"2024-05-10T03:32:12.926261+0000","last_unstale":"2024-05-10T03:32:14.177373+0000","last_undegraded":"2024-05-10T03:32:14.177373+0000","last_fullsized":"2024-05-10T03:32:14.177373+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.922236+0000","last_change":"2024-05-10T03:32:12.327381+0000","last_active":"2024-05-10T03:32:12.922236+0000","last_peered":"2024-05-10T03:32:12.922236+0000","last_clean":"2024-05-10T03:32:12.922236+0000","last_became_active":"2024-05-10T03:32:12.327048+0000","last_became_peered":"2024-05-10T03:32:12.327048+0000","last_unstale":"2024-05-10T03:32:12.922236+0000","last_undegraded":"2024-05-10T03:32:12.922236+0000","last_fullsized":"2024-05-10T03:32:12.922236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177770+0000","last_change":"2024-05-10T03:32:12.910467+0000","last_active":"2024-05-10T03:32:14.177770+0000","last_peered":"2024-05-10T03:32:14.177770+0000","last_clean":"2024-05-10T03:32:14.177770+0000","last_became_active":"2024-05-10T03:32:12.910188+0000","last_became_peered":"2024-05-10T03:32:12.910188+0000","last_unstale":"2024-05-10T03:32:14.177770+0000","last_undegraded":"2024-05-10T03:32:14.177770+0000","last_fullsized":"2024-05-10T03:32:14.177770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923121+0000","last_change":"2024-05-10T03:32:12.325376+0000","last_active":"2024-05-10T03:32:12.923121+0000","last_peered":"2024-05-10T03:32:12.923121+0000","last_clean":"2024-05-10T03:32:12.923121+0000","last_became_active":"2024-05-10T03:32:12.324035+0000","last_became_peered":"2024-05-10T03:32:12.324035+0000","last_unstale":"2024-05-10T03:32:12.923121+0000","last_undegraded":"2024-05-10T03:32:12.923121+0000","last_fullsized":"2024-05-10T03:32:12.923121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:43.411173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.903007+0000","last_change":"2024-05-10T03:32:12.924458+0000","last_active":"2024-05-10T03:32:13.903007+0000","last_peered":"2024-05-10T03:32:13.903007+0000","last_clean":"2024-05-10T03:32:13.903007+0000","last_became_active":"2024-05-10T03:32:12.923064+0000","last_became_peered":"2024-05-10T03:32:12.923064+0000","last_unstale":"2024-05-10T03:32:13.903007+0000","last_undegraded":"2024-05-10T03:32:13.903007+0000","last_fullsized":"2024-05-10T03:32:13.903007+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930846+0000","last_change":"2024-05-10T03:32:12.328698+0000","last_active":"2024-05-10T03:32:12.930846+0000","last_peered":"2024-05-10T03:32:12.930846+0000","last_clean":"2024-05-10T03:32:12.930846+0000","last_became_active":"2024-05-10T03:32:12.328069+0000","last_became_peered":"2024-05-10T03:32:12.328069+0000","last_unstale":"2024-05-10T03:32:12.930846+0000","last_undegraded":"2024-05-10T03:32:12.930846+0000","last_fullsized":"2024-05-10T03:32:12.930846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178392+0000","last_change":"2024-05-10T03:32:12.929164+0000","last_active":"2024-05-10T03:32:14.178392+0000","last_peered":"2024-05-10T03:32:14.178392+0000","last_clean":"2024-05-10T03:32:14.178392+0000","last_became_active":"2024-05-10T03:32:12.928657+0000","last_became_peered":"2024-05-10T03:32:12.928657+0000","last_unstale":"2024-05-10T03:32:14.178392+0000","last_undegraded":"2024-05-10T03:32:14.178392+0000","last_fullsized":"2024-05-10T03:32:14.178392+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923421+0000","last_change":"2024-05-10T03:32:12.317756+0000","last_active":"2024-05-10T03:32:12.923421+0000","last_peered":"2024-05-10T03:32:12.923421+0000","last_clean":"2024-05-10T03:32:12.923421+0000","last_became_active":"2024-05-10T03:32:12.317442+0000","last_became_peered":"2024-05-10T03:32:12.317442+0000","last_unstale":"2024-05-10T03:32:12.923421+0000","last_undegraded":"2024-05-10T03:32:12.923421+0000","last_fullsized":"2024-05-10T03:32:12.923421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178416+0000","last_change":"2024-05-10T03:32:12.904312+0000","last_active":"2024-05-10T03:32:14.178416+0000","last_peered":"2024-05-10T03:32:14.178416+0000","last_clean":"2024-05-10T03:32:14.178416+0000","last_became_active":"2024-05-10T03:32:12.904017+0000","last_became_peered":"2024-05-10T03:32:12.904017+0000","last_unstale":"2024-05-10T03:32:14.178416+0000","last_undegraded":"2024-05-10T03:32:14.178416+0000","last_fullsized":"2024-05-10T03:32:14.178416+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:00:06.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906708+0000","last_change":"2024-05-10T03:32:12.324225+0000","last_active":"2024-05-10T03:32:12.906708+0000","last_peered":"2024-05-10T03:32:12.906708+0000","last_clean":"2024-05-10T03:32:12.906708+0000","last_became_active":"2024-05-10T03:32:12.323907+0000","last_became_peered":"2024-05-10T03:32:12.323907+0000","last_unstale":"2024-05-10T03:32:12.906708+0000","last_undegraded":"2024-05-10T03:32:12.906708+0000","last_fullsized":"2024-05-10T03:32:12.906708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:34.518049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178666+0000","last_change":"2024-05-10T03:32:12.931385+0000","last_active":"2024-05-10T03:32:14.178666+0000","last_peered":"2024-05-10T03:32:14.178666+0000","last_clean":"2024-05-10T03:32:14.178666+0000","last_became_active":"2024-05-10T03:32:12.931015+0000","last_became_peered":"2024-05-10T03:32:12.931015+0000","last_unstale":"2024-05-10T03:32:14.178666+0000","last_undegraded":"2024-05-10T03:32:14.178666+0000","last_fullsized":"2024-05-10T03:32:14.178666+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907110+0000","last_change":"2024-05-10T03:32:12.335472+0000","last_active":"2024-05-10T03:32:12.907110+0000","last_peered":"2024-05-10T03:32:12.907110+0000","last_clean":"2024-05-10T03:32:12.907110+0000","last_became_active":"2024-05-10T03:32:12.335158+0000","last_became_peered":"2024-05-10T03:32:12.335158+0000","last_unstale":"2024-05-10T03:32:12.907110+0000","last_undegraded":"2024-05-10T03:32:12.907110+0000","last_fullsized":"2024-05-10T03:32:12.907110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902078+0000","last_change":"2024-05-10T03:32:12.907425+0000","last_active":"2024-05-10T03:32:13.902078+0000","last_peered":"2024-05-10T03:32:13.902078+0000","last_clean":"2024-05-10T03:32:13.902078+0000","last_became_active":"2024-05-10T03:32:12.904255+0000","last_became_peered":"2024-05-10T03:32:12.904255+0000","last_unstale":"2024-05-10T03:32:13.902078+0000","last_undegraded":"2024-05-10T03:32:13.902078+0000","last_fullsized":"2024-05-10T03:32:13.902078+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910427+0000","last_change":"2024-05-10T03:32:12.326744+0000","last_active":"2024-05-10T03:32:12.910427+0000","last_peered":"2024-05-10T03:32:12.910427+0000","last_clean":"2024-05-10T03:32:12.910427+0000","last_became_active":"2024-05-10T03:32:12.326336+0000","last_became_peered":"2024-05-10T03:32:12.326336+0000","last_unstale":"2024-05-10T03:32:12.910427+0000","last_undegraded":"2024-05-10T03:32:12.910427+0000","last_fullsized":"2024-05-10T03:32:12.910427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899885+0000","last_change":"2024-05-10T03:32:12.904290+0000","last_active":"2024-05-10T03:32:13.899885+0000","last_peered":"2024-05-10T03:32:13.899885+0000","last_clean":"2024-05-10T03:32:13.899885+0000","last_became_active":"2024-05-10T03:32:12.903859+0000","last_became_peered":"2024-05-10T03:32:12.903859+0000","last_unstale":"2024-05-10T03:32:13.899885+0000","last_undegraded":"2024-05-10T03:32:13.899885+0000","last_fullsized":"2024-05-10T03:32:13.899885+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:38.939944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895544+0000","last_change":"2024-05-10T03:32:12.326240+0000","last_active":"2024-05-10T03:32:12.895544+0000","last_peered":"2024-05-10T03:32:12.895544+0000","last_clean":"2024-05-10T03:32:12.895544+0000","last_became_active":"2024-05-10T03:32:12.325869+0000","last_became_peered":"2024-05-10T03:32:12.325869+0000","last_unstale":"2024-05-10T03:32:12.895544+0000","last_undegraded":"2024-05-10T03:32:12.895544+0000","last_fullsized":"2024-05-10T03:32:12.895544+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175662+0000","last_change":"2024-05-10T03:32:12.924590+0000","last_active":"2024-05-10T03:32:14.175662+0000","last_peered":"2024-05-10T03:32:14.175662+0000","last_clean":"2024-05-10T03:32:14.175662+0000","last_became_active":"2024-05-10T03:32:12.924100+0000","last_became_peered":"2024-05-10T03:32:12.924100+0000","last_unstale":"2024-05-10T03:32:14.175662+0000","last_undegraded":"2024-05-10T03:32:14.175662+0000","last_fullsized":"2024-05-10T03:32:14.175662+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:07.133902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.928529+0000","last_change":"2024-05-10T03:32:12.331932+0000","last_active":"2024-05-10T03:32:12.928529+0000","last_peered":"2024-05-10T03:32:12.928529+0000","last_clean":"2024-05-10T03:32:12.928529+0000","last_became_active":"2024-05-10T03:32:12.331589+0000","last_became_peered":"2024-05-10T03:32:12.331589+0000","last_unstale":"2024-05-10T03:32:12.928529+0000","last_undegraded":"2024-05-10T03:32:12.928529+0000","last_fullsized":"2024-05-10T03:32:12.928529+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902755+0000","last_change":"2024-05-10T03:32:12.904583+0000","last_active":"2024-05-10T03:32:13.902755+0000","last_peered":"2024-05-10T03:32:13.902755+0000","last_clean":"2024-05-10T03:32:13.902755+0000","last_became_active":"2024-05-10T03:32:12.903287+0000","last_became_peered":"2024-05-10T03:32:12.903287+0000","last_unstale":"2024-05-10T03:32:13.902755+0000","last_undegraded":"2024-05-10T03:32:13.902755+0000","last_fullsized":"2024-05-10T03:32:13.902755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934219+0000","last_change":"2024-05-10T03:32:12.322783+0000","last_active":"2024-05-10T03:32:12.934219+0000","last_peered":"2024-05-10T03:32:12.934219+0000","last_clean":"2024-05-10T03:32:12.934219+0000","last_became_active":"2024-05-10T03:32:12.320656+0000","last_became_peered":"2024-05-10T03:32:12.320656+0000","last_unstale":"2024-05-10T03:32:12.934219+0000","last_undegraded":"2024-05-10T03:32:12.934219+0000","last_fullsized":"2024-05-10T03:32:12.934219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:46.848796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900290+0000","last_change":"2024-05-10T03:32:12.920087+0000","last_active":"2024-05-10T03:32:13.900290+0000","last_peered":"2024-05-10T03:32:13.900290+0000","last_clean":"2024-05-10T03:32:13.900290+0000","last_became_active":"2024-05-10T03:32:12.919738+0000","last_became_peered":"2024-05-10T03:32:12.919738+0000","last_unstale":"2024-05-10T03:32:13.900290+0000","last_undegraded":"2024-05-10T03:32:13.900290+0000","last_fullsized":"2024-05-10T03:32:13.900290+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:36.708699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930884+0000","last_change":"2024-05-10T03:32:12.329191+0000","last_active":"2024-05-10T03:32:12.930884+0000","last_peered":"2024-05-10T03:32:12.930884+0000","last_clean":"2024-05-10T03:32:12.930884+0000","last_became_active":"2024-05-10T03:32:12.326891+0000","last_became_peered":"2024-05-10T03:32:12.326891+0000","last_unstale":"2024-05-10T03:32:12.930884+0000","last_undegraded":"2024-05-10T03:32:12.930884+0000","last_fullsized":"2024-05-10T03:32:12.930884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:48:20.732699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175971+0000","last_change":"2024-05-10T03:32:12.915905+0000","last_active":"2024-05-10T03:32:14.175971+0000","last_peered":"2024-05-10T03:32:14.175971+0000","last_clean":"2024-05-10T03:32:14.175971+0000","last_became_active":"2024-05-10T03:32:12.915314+0000","last_became_peered":"2024-05-10T03:32:12.915314+0000","last_unstale":"2024-05-10T03:32:14.175971+0000","last_undegraded":"2024-05-10T03:32:14.175971+0000","last_fullsized":"2024-05-10T03:32:14.175971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:38:53.287475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909280+0000","last_change":"2024-05-10T03:32:12.340972+0000","last_active":"2024-05-10T03:32:12.909280+0000","last_peered":"2024-05-10T03:32:12.909280+0000","last_clean":"2024-05-10T03:32:12.909280+0000","last_became_active":"2024-05-10T03:32:12.340465+0000","last_became_peered":"2024-05-10T03:32:12.340465+0000","last_unstale":"2024-05-10T03:32:12.909280+0000","last_undegraded":"2024-05-10T03:32:12.909280+0000","last_fullsized":"2024-05-10T03:32:12.909280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178707+0000","last_change":"2024-05-10T03:32:12.917579+0000","last_active":"2024-05-10T03:32:14.178707+0000","last_peered":"2024-05-10T03:32:14.178707+0000","last_clean":"2024-05-10T03:32:14.178707+0000","last_became_active":"2024-05-10T03:32:12.917261+0000","last_became_peered":"2024-05-10T03:32:12.917261+0000","last_unstale":"2024-05-10T03:32:14.178707+0000","last_undegraded":"2024-05-10T03:32:14.178707+0000","last_fullsized":"2024-05-10T03:32:14.178707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:16.513183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907440+0000","last_change":"2024-05-10T03:32:12.328850+0000","last_active":"2024-05-10T03:32:12.907440+0000","last_peered":"2024-05-10T03:32:12.907440+0000","last_clean":"2024-05-10T03:32:12.907440+0000","last_became_active":"2024-05-10T03:32:12.328541+0000","last_became_peered":"2024-05-10T03:32:12.328541+0000","last_unstale":"2024-05-10T03:32:12.907440+0000","last_undegraded":"2024-05-10T03:32:12.907440+0000","last_fullsized":"2024-05-10T03:32:12.907440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175276+0000","last_change":"2024-05-10T03:32:12.925092+0000","last_active":"2024-05-10T03:32:14.175276+0000","last_peered":"2024-05-10T03:32:14.175276+0000","last_clean":"2024-05-10T03:32:14.175276+0000","last_became_active":"2024-05-10T03:32:12.920412+0000","last_became_peered":"2024-05-10T03:32:12.920412+0000","last_unstale":"2024-05-10T03:32:14.175276+0000","last_undegraded":"2024-05-10T03:32:14.175276+0000","last_fullsized":"2024-05-10T03:32:14.175276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:10.962423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.926806+0000","last_change":"2024-05-10T03:32:12.330153+0000","last_active":"2024-05-10T03:32:12.926806+0000","last_peered":"2024-05-10T03:32:12.926806+0000","last_clean":"2024-05-10T03:32:12.926806+0000","last_became_active":"2024-05-10T03:32:12.328275+0000","last_became_peered":"2024-05-10T03:32:12.328275+0000","last_unstale":"2024-05-10T03:32:12.926806+0000","last_undegraded":"2024-05-10T03:32:12.926806+0000","last_fullsized":"2024-05-10T03:32:12.926806+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:56.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178945+0000","last_change":"2024-05-10T03:32:12.912037+0000","last_active":"2024-05-10T03:32:14.178945+0000","last_peered":"2024-05-10T03:32:14.178945+0000","last_clean":"2024-05-10T03:32:14.178945+0000","last_became_active":"2024-05-10T03:32:12.911220+0000","last_became_peered":"2024-05-10T03:32:12.911220+0000","last_unstale":"2024-05-10T03:32:14.178945+0000","last_undegraded":"2024-05-10T03:32:14.178945+0000","last_fullsized":"2024-05-10T03:32:14.178945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934217+0000","last_change":"2024-05-10T03:32:12.332211+0000","last_active":"2024-05-10T03:32:12.934217+0000","last_peered":"2024-05-10T03:32:12.934217+0000","last_clean":"2024-05-10T03:32:12.934217+0000","last_became_active":"2024-05-10T03:32:12.331391+0000","last_became_peered":"2024-05-10T03:32:12.331391+0000","last_unstale":"2024-05-10T03:32:12.934217+0000","last_undegraded":"2024-05-10T03:32:12.934217+0000","last_fullsized":"2024-05-10T03:32:12.934217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898530+0000","last_change":"2024-05-10T03:32:12.919886+0000","last_active":"2024-05-10T03:32:13.898530+0000","last_peered":"2024-05-10T03:32:13.898530+0000","last_clean":"2024-05-10T03:32:13.898530+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.898530+0000","last_undegraded":"2024-05-10T03:32:13.898530+0000","last_fullsized":"2024-05-10T03:32:13.898530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907502+0000","last_change":"2024-05-10T03:32:12.320826+0000","last_active":"2024-05-10T03:32:12.907502+0000","last_peered":"2024-05-10T03:32:12.907502+0000","last_clean":"2024-05-10T03:32:12.907502+0000","last_became_active":"2024-05-10T03:32:12.320490+0000","last_became_peered":"2024-05-10T03:32:12.320490+0000","last_unstale":"2024-05-10T03:32:12.907502+0000","last_undegraded":"2024-05-10T03:32:12.907502+0000","last_fullsized":"2024-05-10T03:32:12.907502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:42:58.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902476+0000","last_change":"2024-05-10T03:32:12.918118+0000","last_active":"2024-05-10T03:32:13.902476+0000","last_peered":"2024-05-10T03:32:13.902476+0000","last_clean":"2024-05-10T03:32:13.902476+0000","last_became_active":"2024-05-10T03:32:12.917768+0000","last_became_peered":"2024-05-10T03:32:12.917768+0000","last_unstale":"2024-05-10T03:32:13.902476+0000","last_undegraded":"2024-05-10T03:32:13.902476+0000","last_fullsized":"2024-05-10T03:32:13.902476+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.924200+0000","last_change":"2024-05-10T03:32:12.321740+0000","last_active":"2024-05-10T03:32:12.924200+0000","last_peered":"2024-05-10T03:32:12.924200+0000","last_clean":"2024-05-10T03:32:12.924200+0000","last_became_active":"2024-05-10T03:32:12.321433+0000","last_became_peered":"2024-05-10T03:32:12.321433+0000","last_unstale":"2024-05-10T03:32:12.924200+0000","last_undegraded":"2024-05-10T03:32:12.924200+0000","last_fullsized":"2024-05-10T03:32:12.924200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898874+0000","last_change":"2024-05-10T03:32:12.917048+0000","last_active":"2024-05-10T03:32:13.898874+0000","last_peered":"2024-05-10T03:32:13.898874+0000","last_clean":"2024-05-10T03:32:13.898874+0000","last_became_active":"2024-05-10T03:32:12.916721+0000","last_became_peered":"2024-05-10T03:32:12.916721+0000","last_unstale":"2024-05-10T03:32:13.898874+0000","last_undegraded":"2024-05-10T03:32:13.898874+0000","last_fullsized":"2024-05-10T03:32:13.898874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910839+0000","last_change":"2024-05-10T03:32:12.326718+0000","last_active":"2024-05-10T03:32:12.910839+0000","last_peered":"2024-05-10T03:32:12.910839+0000","last_clean":"2024-05-10T03:32:12.910839+0000","last_became_active":"2024-05-10T03:32:12.325934+0000","last_became_peered":"2024-05-10T03:32:12.325934+0000","last_unstale":"2024-05-10T03:32:12.910839+0000","last_undegraded":"2024-05-10T03:32:12.910839+0000","last_fullsized":"2024-05-10T03:32:12.910839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900458+0000","last_change":"2024-05-10T03:32:12.908080+0000","last_active":"2024-05-10T03:32:13.900458+0000","last_peered":"2024-05-10T03:32:13.900458+0000","last_clean":"2024-05-10T03:32:13.900458+0000","last_became_active":"2024-05-10T03:32:12.907698+0000","last_became_peered":"2024-05-10T03:32:12.907698+0000","last_unstale":"2024-05-10T03:32:13.900458+0000","last_undegraded":"2024-05-10T03:32:13.900458+0000","last_fullsized":"2024-05-10T03:32:13.900458+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927874+0000","last_change":"2024-05-10T03:32:12.321882+0000","last_active":"2024-05-10T03:32:12.927874+0000","last_peered":"2024-05-10T03:32:12.927874+0000","last_clean":"2024-05-10T03:32:12.927874+0000","last_became_active":"2024-05-10T03:32:12.321451+0000","last_became_peered":"2024-05-10T03:32:12.321451+0000","last_unstale":"2024-05-10T03:32:12.927874+0000","last_undegraded":"2024-05-10T03:32:12.927874+0000","last_fullsized":"2024-05-10T03:32:12.927874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179016+0000","last_change":"2024-05-10T03:32:12.918914+0000","last_active":"2024-05-10T03:32:14.179016+0000","last_peered":"2024-05-10T03:32:14.179016+0000","last_clean":"2024-05-10T03:32:14.179016+0000","last_became_active":"2024-05-10T03:32:12.918226+0000","last_became_peered":"2024-05-10T03:32:12.918226+0000","last_unstale":"2024-05-10T03:32:14.179016+0000","last_undegraded":"2024-05-10T03:32:14.179016+0000","last_fullsized":"2024-05-10T03:32:14.179016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894269+0000","last_change":"2024-05-10T03:32:12.323189+0000","last_active":"2024-05-10T03:32:12.894269+0000","last_peered":"2024-05-10T03:32:12.894269+0000","last_clean":"2024-05-10T03:32:12.894269+0000","last_became_active":"2024-05-10T03:32:12.322864+0000","last_became_peered":"2024-05-10T03:32:12.322864+0000","last_unstale":"2024-05-10T03:32:12.894269+0000","last_undegraded":"2024-05-10T03:32:12.894269+0000","last_fullsized":"2024-05-10T03:32:12.894269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:04.809817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175964+0000","last_change":"2024-05-10T03:32:12.925481+0000","last_active":"2024-05-10T03:32:14.175964+0000","last_peered":"2024-05-10T03:32:14.175964+0000","last_clean":"2024-05-10T03:32:14.175964+0000","last_became_active":"2024-05-10T03:32:12.925134+0000","last_became_peered":"2024-05-10T03:32:12.925134+0000","last_unstale":"2024-05-10T03:32:14.175964+0000","last_undegraded":"2024-05-10T03:32:14.175964+0000","last_fullsized":"2024-05-10T03:32:14.175964+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:13.132758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906033+0000","last_change":"2024-05-10T03:32:12.327618+0000","last_active":"2024-05-10T03:32:12.906033+0000","last_peered":"2024-05-10T03:32:12.906033+0000","last_clean":"2024-05-10T03:32:12.906033+0000","last_became_active":"2024-05-10T03:32:12.327215+0000","last_became_peered":"2024-05-10T03:32:12.327215+0000","last_unstale":"2024-05-10T03:32:12.906033+0000","last_undegraded":"2024-05-10T03:32:12.906033+0000","last_fullsized":"2024-05-10T03:32:12.906033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:39.030457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899016+0000","last_change":"2024-05-10T03:32:12.911714+0000","last_active":"2024-05-10T03:32:13.899016+0000","last_peered":"2024-05-10T03:32:13.899016+0000","last_clean":"2024-05-10T03:32:13.899016+0000","last_became_active":"2024-05-10T03:32:12.908258+0000","last_became_peered":"2024-05-10T03:32:12.908258+0000","last_unstale":"2024-05-10T03:32:13.899016+0000","last_undegraded":"2024-05-10T03:32:13.899016+0000","last_fullsized":"2024-05-10T03:32:13.899016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.882028+0000","last_change":"2024-05-10T03:32:12.325007+0000","last_active":"2024-05-10T03:32:12.882028+0000","last_peered":"2024-05-10T03:32:12.882028+0000","last_clean":"2024-05-10T03:32:12.882028+0000","last_became_active":"2024-05-10T03:32:12.324692+0000","last_became_peered":"2024-05-10T03:32:12.324692+0000","last_unstale":"2024-05-10T03:32:12.882028+0000","last_undegraded":"2024-05-10T03:32:12.882028+0000","last_fullsized":"2024-05-10T03:32:12.882028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:35:55.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899740+0000","last_change":"2024-05-10T03:32:12.899378+0000","last_active":"2024-05-10T03:32:13.899740+0000","last_peered":"2024-05-10T03:32:13.899740+0000","last_clean":"2024-05-10T03:32:13.899740+0000","last_became_active":"2024-05-10T03:32:12.898752+0000","last_became_peered":"2024-05-10T03:32:12.898752+0000","last_unstale":"2024-05-10T03:32:13.899740+0000","last_undegraded":"2024-05-10T03:32:13.899740+0000","last_fullsized":"2024-05-10T03:32:13.899740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.931779+0000","last_change":"2024-05-10T03:32:12.332881+0000","last_active":"2024-05-10T03:32:12.931779+0000","last_peered":"2024-05-10T03:32:12.931779+0000","last_clean":"2024-05-10T03:32:12.931779+0000","last_became_active":"2024-05-10T03:32:12.332577+0000","last_became_peered":"2024-05-10T03:32:12.332577+0000","last_unstale":"2024-05-10T03:32:12.931779+0000","last_undegraded":"2024-05-10T03:32:12.931779+0000","last_fullsized":"2024-05-10T03:32:12.931779+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898806+0000","last_change":"2024-05-10T03:32:12.901260+0000","last_active":"2024-05-10T03:32:13.898806+0000","last_peered":"2024-05-10T03:32:13.898806+0000","last_clean":"2024-05-10T03:32:13.898806+0000","last_became_active":"2024-05-10T03:32:12.900971+0000","last_became_peered":"2024-05-10T03:32:12.900971+0000","last_unstale":"2024-05-10T03:32:13.898806+0000","last_undegraded":"2024-05-10T03:32:13.898806+0000","last_fullsized":"2024-05-10T03:32:13.898806+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909685+0000","last_change":"2024-05-10T03:32:12.340985+0000","last_active":"2024-05-10T03:32:12.909685+0000","last_peered":"2024-05-10T03:32:12.909685+0000","last_clean":"2024-05-10T03:32:12.909685+0000","last_became_active":"2024-05-10T03:32:12.340613+0000","last_became_peered":"2024-05-10T03:32:12.340613+0000","last_unstale":"2024-05-10T03:32:12.909685+0000","last_undegraded":"2024-05-10T03:32:12.909685+0000","last_fullsized":"2024-05-10T03:32:12.909685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178003+0000","last_change":"2024-05-10T03:32:12.905988+0000","last_active":"2024-05-10T03:32:14.178003+0000","last_peered":"2024-05-10T03:32:14.178003+0000","last_clean":"2024-05-10T03:32:14.178003+0000","last_became_active":"2024-05-10T03:32:12.905703+0000","last_became_peered":"2024-05-10T03:32:12.905703+0000","last_unstale":"2024-05-10T03:32:14.178003+0000","last_undegraded":"2024-05-10T03:32:14.178003+0000","last_fullsized":"2024-05-10T03:32:14.178003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:25:53.297139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.911269+0000","last_change":"2024-05-10T03:32:12.329354+0000","last_active":"2024-05-10T03:32:12.911269+0000","last_peered":"2024-05-10T03:32:12.911269+0000","last_clean":"2024-05-10T03:32:12.911269+0000","last_became_active":"2024-05-10T03:32:12.328206+0000","last_became_peered":"2024-05-10T03:32:12.328206+0000","last_unstale":"2024-05-10T03:32:12.911269+0000","last_undegraded":"2024-05-10T03:32:12.911269+0000","last_fullsized":"2024-05-10T03:32:12.911269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:28.939240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176002+0000","last_change":"2024-05-10T03:32:12.917543+0000","last_active":"2024-05-10T03:32:14.176002+0000","last_peered":"2024-05-10T03:32:14.176002+0000","last_clean":"2024-05-10T03:32:14.176002+0000","last_became_active":"2024-05-10T03:32:12.917311+0000","last_became_peered":"2024-05-10T03:32:12.917311+0000","last_unstale":"2024-05-10T03:32:14.176002+0000","last_undegraded":"2024-05-10T03:32:14.176002+0000","last_fullsized":"2024-05-10T03:32:14.176002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:09:39.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894845+0000","last_change":"2024-05-10T03:32:12.323909+0000","last_active":"2024-05-10T03:32:12.894845+0000","last_peered":"2024-05-10T03:32:12.894845+0000","last_clean":"2024-05-10T03:32:12.894845+0000","last_became_active":"2024-05-10T03:32:12.321976+0000","last_became_peered":"2024-05-10T03:32:12.321976+0000","last_unstale":"2024-05-10T03:32:12.894845+0000","last_undegraded":"2024-05-10T03:32:12.894845+0000","last_fullsized":"2024-05-10T03:32:12.894845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:37.925978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900409+0000","last_change":"2024-05-10T03:32:12.911863+0000","last_active":"2024-05-10T03:32:13.900409+0000","last_peered":"2024-05-10T03:32:13.900409+0000","last_clean":"2024-05-10T03:32:13.900409+0000","last_became_active":"2024-05-10T03:32:12.911565+0000","last_became_peered":"2024-05-10T03:32:12.911565+0000","last_unstale":"2024-05-10T03:32:13.900409+0000","last_undegraded":"2024-05-10T03:32:13.900409+0000","last_fullsized":"2024-05-10T03:32:13.900409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894283+0000","last_change":"2024-05-10T03:32:12.323028+0000","last_active":"2024-05-10T03:32:12.894283+0000","last_peered":"2024-05-10T03:32:12.894283+0000","last_clean":"2024-05-10T03:32:12.894283+0000","last_became_active":"2024-05-10T03:32:12.322679+0000","last_became_peered":"2024-05-10T03:32:12.322679+0000","last_unstale":"2024-05-10T03:32:12.894283+0000","last_undegraded":"2024-05-10T03:32:12.894283+0000","last_fullsized":"2024-05-10T03:32:12.894283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:12.822129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175892+0000","last_change":"2024-05-10T03:32:12.912422+0000","last_active":"2024-05-10T03:32:14.175892+0000","last_peered":"2024-05-10T03:32:14.175892+0000","last_clean":"2024-05-10T03:32:14.175892+0000","last_became_active":"2024-05-10T03:32:12.911930+0000","last_became_peered":"2024-05-10T03:32:12.911930+0000","last_unstale":"2024-05-10T03:32:14.175892+0000","last_undegraded":"2024-05-10T03:32:14.175892+0000","last_fullsized":"2024-05-10T03:32:14.175892+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895035+0000","last_change":"2024-05-10T03:32:12.326371+0000","last_active":"2024-05-10T03:32:12.895035+0000","last_peered":"2024-05-10T03:32:12.895035+0000","last_clean":"2024-05-10T03:32:12.895035+0000","last_became_active":"2024-05-10T03:32:12.325858+0000","last_became_peered":"2024-05-10T03:32:12.325858+0000","last_unstale":"2024-05-10T03:32:12.895035+0000","last_undegraded":"2024-05-10T03:32:12.895035+0000","last_fullsized":"2024-05-10T03:32:12.895035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178757+0000","last_change":"2024-05-10T03:32:12.912144+0000","last_active":"2024-05-10T03:32:14.178757+0000","last_peered":"2024-05-10T03:32:14.178757+0000","last_clean":"2024-05-10T03:32:14.178757+0000","last_became_active":"2024-05-10T03:32:12.911805+0000","last_became_peered":"2024-05-10T03:32:12.911805+0000","last_unstale":"2024-05-10T03:32:14.178757+0000","last_undegraded":"2024-05-10T03:32:14.178757+0000","last_fullsized":"2024-05-10T03:32:14.178757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.886727+0000","last_change":"2024-05-10T03:32:12.319651+0000","last_active":"2024-05-10T03:32:12.886727+0000","last_peered":"2024-05-10T03:32:12.886727+0000","last_clean":"2024-05-10T03:32:12.886727+0000","last_became_active":"2024-05-10T03:32:12.319302+0000","last_became_peered":"2024-05-10T03:32:12.319302+0000","last_unstale":"2024-05-10T03:32:12.886727+0000","last_undegraded":"2024-05-10T03:32:12.886727+0000","last_fullsized":"2024-05-10T03:32:12.886727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:42.125952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176756+0000","last_change":"2024-05-10T03:32:12.923331+0000","last_active":"2024-05-10T03:32:14.176756+0000","last_peered":"2024-05-10T03:32:14.176756+0000","last_clean":"2024-05-10T03:32:14.176756+0000","last_became_active":"2024-05-10T03:32:12.922968+0000","last_became_peered":"2024-05-10T03:32:12.922968+0000","last_unstale":"2024-05-10T03:32:14.176756+0000","last_undegraded":"2024-05-10T03:32:14.176756+0000","last_fullsized":"2024-05-10T03:32:14.176756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.912330+0000","last_change":"2024-05-10T03:32:12.327328+0000","last_active":"2024-05-10T03:32:12.912330+0000","last_peered":"2024-05-10T03:32:12.912330+0000","last_clean":"2024-05-10T03:32:12.912330+0000","last_became_active":"2024-05-10T03:32:12.326415+0000","last_became_peered":"2024-05-10T03:32:12.326415+0000","last_unstale":"2024-05-10T03:32:12.912330+0000","last_undegraded":"2024-05-10T03:32:12.912330+0000","last_fullsized":"2024-05-10T03:32:12.912330+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:42.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.905988+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:12.905988+0000","last_peered":"2024-05-10T03:32:12.905988+0000","last_clean":"2024-05-10T03:32:12.905988+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:12.905988+0000","last_undegraded":"2024-05-10T03:32:12.905988+0000","last_fullsized":"2024-05-10T03:32:12.905988+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2230000000000001,"15min":1.2270000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.367,"15min":1.365},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8100000000000001,"5min":1.877,"15min":1.877},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.266,"15min":1.274},"min":{"1min":0.84999999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.667,"5min":1.925,"15min":1.925},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1349999999999998,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4410000000000001,"15min":1.4490000000000001},"min":{"1min":0.81200000000000006,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.819,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3899999999999999,"15min":1.3919999999999999},"min":{"1min":1.0489999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.505},{"interface":"front","average":{"1min":1.46,"5min":1.3520000000000001,"15min":1.3340000000000001},"min":{"1min":1.0109999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1120000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.49,"15min":1.484},"min":{"1min":1.071,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.1339999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.583,"5min":1.5169999999999999,"15min":1.506},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.437}]},{"osd":5,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.546,"15min":1.516},"min":{"1min":1.2030000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1779999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.76,"5min":1.712,"15min":1.704},"min":{"1min":1.383,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.762,"5min":1.7370000000000001,"15min":1.7330000000000001},"min":{"1min":1.3939999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.653,"15min":1.641},"min":{"1min":1.0600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.67300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":272133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.073,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.329,"15min":1.351},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8640000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.996}]},{"osd":1,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4590000000000001,"5min":3.113,"15min":3.113},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4750000000000001,"15min":1.4970000000000001},"min":{"1min":1.046,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.944,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.131}]},{"osd":2,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.246,"15min":1.232},"min":{"1min":0.93000000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.482,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5589999999999999,"5min":3.569,"15min":3.569},"last":1.28}]},{"osd":3,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.591,"15min":1.6279999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.48},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.306,"15min":1.304},"min":{"1min":1.0409999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.744,"5min":1.931,"15min":1.931},"last":1.4099999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4359999999999999,"15min":1.4299999999999999},"min":{"1min":1.069,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.8759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5640000000000001,"15min":1.585},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.819,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.6579999999999999}]},{"osd":5,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.1259999999999999,"5min":1,"15min":1},"max":{"1min":1.857,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.004,"5min":2.472,"15min":2.472},"last":1.8859999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.665,"15min":1.671},"min":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":1.9199999999999999,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.651,"15min":1.655},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0070000000000001,"5min":2.359,"15min":2.359},"last":1.821}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":271481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.321}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.446},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.7549999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.512},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.9830000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.921}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.849},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":1.6020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":622592,"data_stored":272063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.173,"15min":1.177},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.117,"5min":1.1359999999999999,"15min":1.139},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5629999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4059999999999999}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.165,"15min":1.1559999999999999},"min":{"1min":0.79100000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.734,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.278,"15min":1.2829999999999999},"min":{"1min":0.85699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.994,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.302}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8280000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.042},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9179999999999999,"5min":2.29,"15min":2.29},"last":1.718}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.458,"15min":1.476},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.76,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5900000000000001,"15min":1.589},"min":{"1min":1.2729999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":1.9530000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5009999999999999,"15min":1.5049999999999999},"min":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.8129999999999999,"5min":2.512,"15min":2.512},"last":1.536},{"interface":"front","average":{"1min":1.456,"5min":1.552,"15min":1.5680000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.982,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.95799999999999996}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5900000000000001,"15min":1.5940000000000001},"min":{"1min":1.153,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1139999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7829999999999999,"15min":1.7969999999999999},"min":{"1min":1.1040000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0409999999999999,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.8500000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6930000000000001,"15min":1.7030000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.069,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.601},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7609999999999999,"15min":1.7509999999999999},"min":{"1min":1.4730000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.282,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.655}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":634880,"data_stored":273067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.95399999999999996,"15min":0.97599999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4850000000000001,"5min":1.708,"15min":1.708},"last":1.575}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"min":{"1min":0.55800000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4370000000000001,"5min":1.603,"15min":1.603},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.24,"15min":1.254},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.925,"5min":1.964,"15min":1.964},"last":1.05}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1779999999999999,"15min":1.1839999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.897,"5min":1.99,"15min":1.99},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":1.369},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.919,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.369}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.95099999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.8080000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.417},{"interface":"front","average":{"1min":1.452,"5min":1.3879999999999999,"15min":1.3779999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.1349999999999998,"5min":2.464,"15min":2.464},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.6479999999999999,"15min":1.6859999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.109,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.536},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.462,"15min":1.484},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0499999999999998,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.2559999999999998,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.774,"15min":1.804},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4729999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.5629999999999999,"15min":1.5900000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.1379999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.806},{"interface":"front","average":{"1min":1.571,"5min":1.6479999999999999,"15min":1.661},"min":{"1min":1.052,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.2690000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.6319999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":630784,"data_stored":273045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.494,"5min":1.772,"15min":1.772},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.431,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.88900000000000001}]},{"osd":1,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5600000000000001,"5min":1.621,"15min":1.621},"last":1.425},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1930000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6930000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.3420000000000001}]},{"osd":3,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1240000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5009999999999999,"5min":2.169,"15min":2.169},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.25,"15min":1.2629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9650000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.526,"15min":1.548},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.133,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.516,"15min":1.5569999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.821,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.2410000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.036,"5min":2.508,"15min":2.508},"last":1.887},{"interface":"front","average":{"1min":1.546,"5min":1.6140000000000001,"15min":1.625},"min":{"1min":1.069,"5min":1.004,"15min":1.004},"max":{"1min":2.1190000000000002,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0089999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1139999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.782},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5820000000000001,"15min":1.5980000000000001},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.982,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.9710000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.641,"15min":1.6599999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.1360000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.665},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.659,"15min":1.679},"min":{"1min":1.052,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.169,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.7370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":626688,"data_stored":271641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.54600000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.256,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.755}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.488,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.1699999999999999,"15min":1.1910000000000001},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4330000000000001,"5min":2.444,"15min":2.444},"last":1.004}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.3,"15min":1.3280000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.556,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3100000000000001},"min":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7789999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.673,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.155},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5669999999999999,"15min":1.597},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":1.8220000000000001,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.5600000000000001,"15min":1.5980000000000001},"min":{"1min":1.069,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.173},{"interface":"front","average":{"1min":1.458,"5min":1.6200000000000001,"15min":1.647},"min":{"1min":1.1080000000000001,"5min":1.077,"15min":1.077},"max":{"1min":1.9770000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.47}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.6990000000000001,"15min":1.7370000000000001},"min":{"1min":0.94999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0550000000000002,"5min":3.48,"15min":3.48},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.716,"15min":1.748},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.0800000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.5569999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.7270000000000001,"15min":1.7470000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0910000000000002,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.762,"15min":1.796},"min":{"1min":1.173,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.7909999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.3300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.999,"15min":1.036},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5629999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":1.1599999999999999,"15min":1.2110000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5389999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.593},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4550000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.089,"15min":1.1140000000000001},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.448,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.929,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.159}]},{"osd":4,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.6040000000000001,"15min":1.6539999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0209999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.655},{"interface":"front","average":{"1min":1.343,"5min":1.431,"15min":1.446},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.661,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.464,"15min":1.486},"min":{"1min":0.85799999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9710000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6080000000000001,"15min":1.647},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8999999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.7490000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6299999999999999,"15min":1.6539999999999999},"min":{"1min":1.028,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.206,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.7150000000000001,"15min":1.7629999999999999},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.972,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.5369999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.77,"15min":1.8200000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.946,"5min":3.149,"15min":3.149},"last":1.381},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5609999999999999,"15min":1.581},"min":{"1min":0.91500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.99,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.8069999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:32:23.532 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:32:23.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.816+0000 7f87b3200640 1 Processor -- start 2024-05-10T03:32:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.816+0000 7f87b3200640 1 -- start start 2024-05-10T03:32:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87b4144710 0x7f87b4144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b406d7e0 0x7f87b40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87b40b2fc0 0x7f87b415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b406d350 con 0x7f87b40b2fc0 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b406c740 con 0x7f87b4144710 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.818+0000 7f87b3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b406ca20 con 0x7f87b406d7e0 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87b40b2fc0 0x7f87b415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87b40b2fc0 0x7f87b415eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52064/0 (socket says 172.21.15.55:52064) 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 -- 172.21.15.55:0/1623479025 learned_addr learned my addr 172.21.15.55:0/1623479025 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87abe00640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b406d7e0 0x7f87b40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b1e00640 1 --2- 172.21.15.55:0/1623479025 >> v2:172.21.15.162:3300/0 conn(0x7f87b4144710 0x7f87b4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 -- 172.21.15.55:0/1623479025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b406d7e0 msgr2=0x7f87b40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b406d7e0 0x7f87b40b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 -- 172.21.15.55:0/1623479025 >> v2:172.21.15.162:3300/0 conn(0x7f87b4144710 msgr2=0x7f87b4144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 --2- 172.21.15.55:0/1623479025 >> v2:172.21.15.162:3300/0 conn(0x7f87b4144710 0x7f87b4144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.819+0000 7f87b2800640 1 -- 172.21.15.55:0/1623479025 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b406cd00 con 0x7f87b40b2fc0 2024-05-10T03:32:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.820+0000 7f87b2800640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b40b2fc0 0x7f87b415eff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f87b4072240 tx=0x7f87ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f78e69d1833f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.821+0000 7f87b1400640 1 -- 172.21.15.55:0/1623479025 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac031600 con 0x7f87b40b2fc0 2024-05-10T03:32:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.821+0000 7f87b1400640 1 -- 172.21.15.55:0/1623479025 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac02d050 con 0x7f87b40b2fc0 2024-05-10T03:32:23.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.821+0000 7f87b1400640 1 -- 172.21.15.55:0/1623479025 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac031980 con 0x7f87b40b2fc0 2024-05-10T03:32:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 -- 172.21.15.55:0/1623479025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b40b2fc0 msgr2=0x7f87b415eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b40b2fc0 0x7f87b415eff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f87b4072240 tx=0x7f87ac02a610 comp rx=0 tx=0).stop 2024-05-10T03:32:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 -- 172.21.15.55:0/1623479025 shutdown_connections 2024-05-10T03:32:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b40b2fc0 0x7f87b415eff0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 --2- 172.21.15.55:0/1623479025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b406d7e0 0x7f87b40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 --2- 172.21.15.55:0/1623479025 >> v2:172.21.15.162:3300/0 conn(0x7f87b4144710 0x7f87b4144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 -- 172.21.15.55:0/1623479025 >> 172.21.15.55:0/1623479025 conn(0x7f87b406a110 msgr2=0x7f87b406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:23.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 -- 172.21.15.55:0/1623479025 shutdown_connections 2024-05-10T03:32:23.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.822+0000 7f87b3200640 1 -- 172.21.15.55:0/1623479025 wait complete. 2024-05-10T03:32:23.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.824+0000 7f87b3200640 1 Processor -- start 2024-05-10T03:32:23.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.825+0000 7f87b3200640 1 -- start start 2024-05-10T03:32:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b40b2fc0 0x7f87b40a9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87b4144710 0x7f87b40a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b415b270 con 0x7f87b406d7e0 2024-05-10T03:32:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40a8820 con 0x7f87b4144710 2024-05-10T03:32:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40a9e90 con 0x7f87b40b2fc0 2024-05-10T03:32:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52066/0 (socket says 172.21.15.55:52066) 2024-05-10T03:32:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.826+0000 7f87b1e00640 1 -- 172.21.15.55:0/490424003 learned_addr learned my addr 172.21.15.55:0/490424003 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b2800640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87b4144710 0x7f87b40a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87abe00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b40b2fc0 0x7f87b40a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 -- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b40b2fc0 msgr2=0x7f87b40a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b40b2fc0 0x7f87b40a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 -- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87b4144710 msgr2=0x7f87b40a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:23.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87b4144710 0x7f87b40a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 -- 172.21.15.55:0/490424003 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009b30 con 0x7f87b406d7e0 2024-05-10T03:32:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.827+0000 7f87b1e00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87a400d5d0 tx=0x7f87a400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.828+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4004240 con 0x7f87b406d7e0 2024-05-10T03:32:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.828+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40043a0 con 0x7f87b406d7e0 2024-05-10T03:32:23.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.828+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4003830 con 0x7f87b406d7e0 2024-05-10T03:32:23.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.828+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40acaa0 con 0x7f87b406d7e0 2024-05-10T03:32:23.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.828+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40acfd0 con 0x7f87b406d7e0 2024-05-10T03:32:23.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.829+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4066ce0 con 0x7f87b406d7e0 2024-05-10T03:32:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.830+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f87a4003990 con 0x7f87b406d7e0 2024-05-10T03:32:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.831+0000 7f87aa000640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 0x7f879006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.831+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f87a40896b0 con 0x7f87b406d7e0 2024-05-10T03:32:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.832+0000 7f87abe00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 0x7f879006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.833+0000 7f87abe00640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 0x7f879006d110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f879c006fd0 tx=0x7f879c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:23.836+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f87a4054670 con 0x7f87b406d7e0 2024-05-10T03:32:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.031+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87b404e310 con 0x7f879006ac00 2024-05-10T03:32:24.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.056+0000 7f87aa000640 1 -- 172.21.15.55:0/490424003 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286531 (secure 0 0 0) 0x7f87b404e310 con 0x7f879006ac00 2024-05-10T03:32:24.064 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:24.066 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:32:24.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.062+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 msgr2=0x7f879006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.062+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 0x7f879006d110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f879c006fd0 tx=0x7f879c008040 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.062+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 msgr2=0x7f87b40a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.062+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87a400d5d0 tx=0x7f87a400da90 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 shutdown_connections 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f879006ac00 0x7f879006d110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87b4144710 0x7f87b40a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87b40b2fc0 0x7f87b40a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 --2- 172.21.15.55:0/490424003 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87b406d7e0 0x7f87b40a8b50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 >> 172.21.15.55:0/490424003 conn(0x7f87b406a110 msgr2=0x7f87b40cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 shutdown_connections 2024-05-10T03:32:24.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.063+0000 7f87b3200640 1 -- 172.21.15.55:0/490424003 wait complete. 2024-05-10T03:32:24.096 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-05-10T03:32:23.419400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242756,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731964,"statfs":{"total":773094113280,"available":772845531136,"internally_reserved":0,"allocated":6201344,"data_stored":3357308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.523918"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898678+0000","last_change":"2024-05-10T03:32:12.914232+0000","last_active":"2024-05-10T03:32:13.898678+0000","last_peered":"2024-05-10T03:32:13.898678+0000","last_clean":"2024-05-10T03:32:13.898678+0000","last_became_active":"2024-05-10T03:32:12.913664+0000","last_became_peered":"2024-05-10T03:32:12.913664+0000","last_unstale":"2024-05-10T03:32:13.898678+0000","last_undegraded":"2024-05-10T03:32:13.898678+0000","last_fullsized":"2024-05-10T03:32:13.898678+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178978+0000","last_change":"2024-05-10T03:32:12.922843+0000","last_active":"2024-05-10T03:32:14.178978+0000","last_peered":"2024-05-10T03:32:14.178978+0000","last_clean":"2024-05-10T03:32:14.178978+0000","last_became_active":"2024-05-10T03:32:12.922531+0000","last_became_peered":"2024-05-10T03:32:12.922531+0000","last_unstale":"2024-05-10T03:32:14.178978+0000","last_undegraded":"2024-05-10T03:32:14.178978+0000","last_fullsized":"2024-05-10T03:32:14.178978+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175288+0000","last_change":"2024-05-10T03:32:12.918902+0000","last_active":"2024-05-10T03:32:14.175288+0000","last_peered":"2024-05-10T03:32:14.175288+0000","last_clean":"2024-05-10T03:32:14.175288+0000","last_became_active":"2024-05-10T03:32:12.918598+0000","last_became_peered":"2024-05-10T03:32:12.918598+0000","last_unstale":"2024-05-10T03:32:14.175288+0000","last_undegraded":"2024-05-10T03:32:14.175288+0000","last_fullsized":"2024-05-10T03:32:14.175288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178493+0000","last_change":"2024-05-10T03:32:12.924668+0000","last_active":"2024-05-10T03:32:14.178493+0000","last_peered":"2024-05-10T03:32:14.178493+0000","last_clean":"2024-05-10T03:32:14.178493+0000","last_became_active":"2024-05-10T03:32:12.923917+0000","last_became_peered":"2024-05-10T03:32:12.923917+0000","last_unstale":"2024-05-10T03:32:14.178493+0000","last_undegraded":"2024-05-10T03:32:14.178493+0000","last_fullsized":"2024-05-10T03:32:14.178493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902676+0000","last_change":"2024-05-10T03:32:12.915697+0000","last_active":"2024-05-10T03:32:13.902676+0000","last_peered":"2024-05-10T03:32:13.902676+0000","last_clean":"2024-05-10T03:32:13.902676+0000","last_became_active":"2024-05-10T03:32:12.915337+0000","last_became_peered":"2024-05-10T03:32:12.915337+0000","last_unstale":"2024-05-10T03:32:13.902676+0000","last_undegraded":"2024-05-10T03:32:13.902676+0000","last_fullsized":"2024-05-10T03:32:13.902676+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898620+0000","last_change":"2024-05-10T03:32:12.910337+0000","last_active":"2024-05-10T03:32:13.898620+0000","last_peered":"2024-05-10T03:32:13.898620+0000","last_clean":"2024-05-10T03:32:13.898620+0000","last_became_active":"2024-05-10T03:32:12.910049+0000","last_became_peered":"2024-05-10T03:32:12.910049+0000","last_unstale":"2024-05-10T03:32:13.898620+0000","last_undegraded":"2024-05-10T03:32:13.898620+0000","last_fullsized":"2024-05-10T03:32:13.898620+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174738+0000","last_change":"2024-05-10T03:32:12.918643+0000","last_active":"2024-05-10T03:32:14.174738+0000","last_peered":"2024-05-10T03:32:14.174738+0000","last_clean":"2024-05-10T03:32:14.174738+0000","last_became_active":"2024-05-10T03:32:12.918296+0000","last_became_peered":"2024-05-10T03:32:12.918296+0000","last_unstale":"2024-05-10T03:32:14.174738+0000","last_undegraded":"2024-05-10T03:32:14.174738+0000","last_fullsized":"2024-05-10T03:32:14.174738+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177500+0000","last_change":"2024-05-10T03:32:12.919824+0000","last_active":"2024-05-10T03:32:14.177500+0000","last_peered":"2024-05-10T03:32:14.177500+0000","last_clean":"2024-05-10T03:32:14.177500+0000","last_became_active":"2024-05-10T03:32:12.919521+0000","last_became_peered":"2024-05-10T03:32:12.919521+0000","last_unstale":"2024-05-10T03:32:14.177500+0000","last_undegraded":"2024-05-10T03:32:14.177500+0000","last_fullsized":"2024-05-10T03:32:14.177500+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:45.737960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178502+0000","last_change":"2024-05-10T03:32:12.923819+0000","last_active":"2024-05-10T03:32:14.178502+0000","last_peered":"2024-05-10T03:32:14.178502+0000","last_clean":"2024-05-10T03:32:14.178502+0000","last_became_active":"2024-05-10T03:32:12.922895+0000","last_became_peered":"2024-05-10T03:32:12.922895+0000","last_unstale":"2024-05-10T03:32:14.178502+0000","last_undegraded":"2024-05-10T03:32:14.178502+0000","last_fullsized":"2024-05-10T03:32:14.178502+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:21.171840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898711+0000","last_change":"2024-05-10T03:32:12.908590+0000","last_active":"2024-05-10T03:32:13.898711+0000","last_peered":"2024-05-10T03:32:13.898711+0000","last_clean":"2024-05-10T03:32:13.898711+0000","last_became_active":"2024-05-10T03:32:12.908181+0000","last_became_peered":"2024-05-10T03:32:12.908181+0000","last_unstale":"2024-05-10T03:32:13.898711+0000","last_undegraded":"2024-05-10T03:32:13.898711+0000","last_fullsized":"2024-05-10T03:32:13.898711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116836+0000","last_change":"2024-05-10T03:32:12.921052+0000","last_active":"2024-05-10T03:32:14.116836+0000","last_peered":"2024-05-10T03:32:14.116836+0000","last_clean":"2024-05-10T03:32:14.116836+0000","last_became_active":"2024-05-10T03:32:12.918022+0000","last_became_peered":"2024-05-10T03:32:12.918022+0000","last_unstale":"2024-05-10T03:32:14.116836+0000","last_undegraded":"2024-05-10T03:32:14.116836+0000","last_fullsized":"2024-05-10T03:32:14.116836+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:13.248072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178356+0000","last_change":"2024-05-10T03:32:12.921186+0000","last_active":"2024-05-10T03:32:14.178356+0000","last_peered":"2024-05-10T03:32:14.178356+0000","last_clean":"2024-05-10T03:32:14.178356+0000","last_became_active":"2024-05-10T03:32:12.920915+0000","last_became_peered":"2024-05-10T03:32:12.920915+0000","last_unstale":"2024-05-10T03:32:14.178356+0000","last_undegraded":"2024-05-10T03:32:14.178356+0000","last_fullsized":"2024-05-10T03:32:14.178356+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:51.954514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898529+0000","last_change":"2024-05-10T03:32:12.908427+0000","last_active":"2024-05-10T03:32:13.898529+0000","last_peered":"2024-05-10T03:32:13.898529+0000","last_clean":"2024-05-10T03:32:13.898529+0000","last_became_active":"2024-05-10T03:32:12.908141+0000","last_became_peered":"2024-05-10T03:32:12.908141+0000","last_unstale":"2024-05-10T03:32:13.898529+0000","last_undegraded":"2024-05-10T03:32:13.898529+0000","last_fullsized":"2024-05-10T03:32:13.898529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175216+0000","last_change":"2024-05-10T03:32:12.915903+0000","last_active":"2024-05-10T03:32:14.175216+0000","last_peered":"2024-05-10T03:32:14.175216+0000","last_clean":"2024-05-10T03:32:14.175216+0000","last_became_active":"2024-05-10T03:32:12.915485+0000","last_became_peered":"2024-05-10T03:32:12.915485+0000","last_unstale":"2024-05-10T03:32:14.175216+0000","last_undegraded":"2024-05-10T03:32:14.175216+0000","last_fullsized":"2024-05-10T03:32:14.175216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:04.967182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177018+0000","last_change":"2024-05-10T03:32:12.921319+0000","last_active":"2024-05-10T03:32:14.177018+0000","last_peered":"2024-05-10T03:32:14.177018+0000","last_clean":"2024-05-10T03:32:14.177018+0000","last_became_active":"2024-05-10T03:32:12.920917+0000","last_became_peered":"2024-05-10T03:32:12.920917+0000","last_unstale":"2024-05-10T03:32:14.177018+0000","last_undegraded":"2024-05-10T03:32:14.177018+0000","last_fullsized":"2024-05-10T03:32:14.177018+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898412+0000","last_change":"2024-05-10T03:32:12.909443+0000","last_active":"2024-05-10T03:32:13.898412+0000","last_peered":"2024-05-10T03:32:13.898412+0000","last_clean":"2024-05-10T03:32:13.898412+0000","last_became_active":"2024-05-10T03:32:12.908121+0000","last_became_peered":"2024-05-10T03:32:12.908121+0000","last_unstale":"2024-05-10T03:32:13.898412+0000","last_undegraded":"2024-05-10T03:32:13.898412+0000","last_fullsized":"2024-05-10T03:32:13.898412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175155+0000","last_change":"2024-05-10T03:32:12.924889+0000","last_active":"2024-05-10T03:32:14.175155+0000","last_peered":"2024-05-10T03:32:14.175155+0000","last_clean":"2024-05-10T03:32:14.175155+0000","last_became_active":"2024-05-10T03:32:12.920754+0000","last_became_peered":"2024-05-10T03:32:12.920754+0000","last_unstale":"2024-05-10T03:32:14.175155+0000","last_undegraded":"2024-05-10T03:32:14.175155+0000","last_fullsized":"2024-05-10T03:32:14.175155+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179224+0000","last_change":"2024-05-10T03:32:12.922850+0000","last_active":"2024-05-10T03:32:14.179224+0000","last_peered":"2024-05-10T03:32:14.179224+0000","last_clean":"2024-05-10T03:32:14.179224+0000","last_became_active":"2024-05-10T03:32:12.922543+0000","last_became_peered":"2024-05-10T03:32:12.922543+0000","last_unstale":"2024-05-10T03:32:14.179224+0000","last_undegraded":"2024-05-10T03:32:14.179224+0000","last_fullsized":"2024-05-10T03:32:14.179224+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175667+0000","last_change":"2024-05-10T03:32:12.915732+0000","last_active":"2024-05-10T03:32:14.175667+0000","last_peered":"2024-05-10T03:32:14.175667+0000","last_clean":"2024-05-10T03:32:14.175667+0000","last_became_active":"2024-05-10T03:32:12.914507+0000","last_became_peered":"2024-05-10T03:32:12.914507+0000","last_unstale":"2024-05-10T03:32:14.175667+0000","last_undegraded":"2024-05-10T03:32:14.175667+0000","last_fullsized":"2024-05-10T03:32:14.175667+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:03.419392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178102+0000","last_change":"2024-05-10T03:32:12.919811+0000","last_active":"2024-05-10T03:32:14.178102+0000","last_peered":"2024-05-10T03:32:14.178102+0000","last_clean":"2024-05-10T03:32:14.178102+0000","last_became_active":"2024-05-10T03:32:12.919469+0000","last_became_peered":"2024-05-10T03:32:12.919469+0000","last_unstale":"2024-05-10T03:32:14.178102+0000","last_undegraded":"2024-05-10T03:32:14.178102+0000","last_fullsized":"2024-05-10T03:32:14.178102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177895+0000","last_change":"2024-05-10T03:32:12.926306+0000","last_active":"2024-05-10T03:32:14.177895+0000","last_peered":"2024-05-10T03:32:14.177895+0000","last_clean":"2024-05-10T03:32:14.177895+0000","last_became_active":"2024-05-10T03:32:12.925942+0000","last_became_peered":"2024-05-10T03:32:12.925942+0000","last_unstale":"2024-05-10T03:32:14.177895+0000","last_undegraded":"2024-05-10T03:32:14.177895+0000","last_fullsized":"2024-05-10T03:32:14.177895+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177762+0000","last_change":"2024-05-10T03:32:12.922894+0000","last_active":"2024-05-10T03:32:14.177762+0000","last_peered":"2024-05-10T03:32:14.177762+0000","last_clean":"2024-05-10T03:32:14.177762+0000","last_became_active":"2024-05-10T03:32:12.922244+0000","last_became_peered":"2024-05-10T03:32:12.922244+0000","last_unstale":"2024-05-10T03:32:14.177762+0000","last_undegraded":"2024-05-10T03:32:14.177762+0000","last_fullsized":"2024-05-10T03:32:14.177762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:03.341641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.116024+0000","last_change":"2024-05-10T03:32:12.921743+0000","last_active":"2024-05-10T03:32:14.116024+0000","last_peered":"2024-05-10T03:32:14.116024+0000","last_clean":"2024-05-10T03:32:14.116024+0000","last_became_active":"2024-05-10T03:32:12.918101+0000","last_became_peered":"2024-05-10T03:32:12.918101+0000","last_unstale":"2024-05-10T03:32:14.116024+0000","last_undegraded":"2024-05-10T03:32:14.116024+0000","last_fullsized":"2024-05-10T03:32:14.116024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:48.926764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174969+0000","last_change":"2024-05-10T03:32:12.915425+0000","last_active":"2024-05-10T03:32:14.174969+0000","last_peered":"2024-05-10T03:32:14.174969+0000","last_clean":"2024-05-10T03:32:14.174969+0000","last_became_active":"2024-05-10T03:32:12.914547+0000","last_became_peered":"2024-05-10T03:32:12.914547+0000","last_unstale":"2024-05-10T03:32:14.174969+0000","last_undegraded":"2024-05-10T03:32:14.174969+0000","last_fullsized":"2024-05-10T03:32:14.174969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175394+0000","last_change":"2024-05-10T03:32:12.925418+0000","last_active":"2024-05-10T03:32:14.175394+0000","last_peered":"2024-05-10T03:32:14.175394+0000","last_clean":"2024-05-10T03:32:14.175394+0000","last_became_active":"2024-05-10T03:32:12.924225+0000","last_became_peered":"2024-05-10T03:32:12.924225+0000","last_unstale":"2024-05-10T03:32:14.175394+0000","last_undegraded":"2024-05-10T03:32:14.175394+0000","last_fullsized":"2024-05-10T03:32:14.175394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:26.673168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178116+0000","last_change":"2024-05-10T03:32:12.910349+0000","last_active":"2024-05-10T03:32:14.178116+0000","last_peered":"2024-05-10T03:32:14.178116+0000","last_clean":"2024-05-10T03:32:14.178116+0000","last_became_active":"2024-05-10T03:32:12.910056+0000","last_became_peered":"2024-05-10T03:32:12.910056+0000","last_unstale":"2024-05-10T03:32:14.178116+0000","last_undegraded":"2024-05-10T03:32:14.178116+0000","last_fullsized":"2024-05-10T03:32:14.178116+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:08.718135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899066+0000","last_change":"2024-05-10T03:32:12.919859+0000","last_active":"2024-05-10T03:32:13.899066+0000","last_peered":"2024-05-10T03:32:13.899066+0000","last_clean":"2024-05-10T03:32:13.899066+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.899066+0000","last_undegraded":"2024-05-10T03:32:13.899066+0000","last_fullsized":"2024-05-10T03:32:13.899066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178414+0000","last_change":"2024-05-10T03:32:12.927253+0000","last_active":"2024-05-10T03:32:14.178414+0000","last_peered":"2024-05-10T03:32:14.178414+0000","last_clean":"2024-05-10T03:32:14.178414+0000","last_became_active":"2024-05-10T03:32:12.926930+0000","last_became_peered":"2024-05-10T03:32:12.926930+0000","last_unstale":"2024-05-10T03:32:14.178414+0000","last_undegraded":"2024-05-10T03:32:14.178414+0000","last_fullsized":"2024-05-10T03:32:14.178414+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:44.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176263+0000","last_change":"2024-05-10T03:32:12.921104+0000","last_active":"2024-05-10T03:32:14.176263+0000","last_peered":"2024-05-10T03:32:14.176263+0000","last_clean":"2024-05-10T03:32:14.176263+0000","last_became_active":"2024-05-10T03:32:12.920745+0000","last_became_peered":"2024-05-10T03:32:12.920745+0000","last_unstale":"2024-05-10T03:32:14.176263+0000","last_undegraded":"2024-05-10T03:32:14.176263+0000","last_fullsized":"2024-05-10T03:32:14.176263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:02.965291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177679+0000","last_change":"2024-05-10T03:32:12.927250+0000","last_active":"2024-05-10T03:32:14.177679+0000","last_peered":"2024-05-10T03:32:14.177679+0000","last_clean":"2024-05-10T03:32:14.177679+0000","last_became_active":"2024-05-10T03:32:12.926910+0000","last_became_peered":"2024-05-10T03:32:12.926910+0000","last_unstale":"2024-05-10T03:32:14.177679+0000","last_undegraded":"2024-05-10T03:32:14.177679+0000","last_fullsized":"2024-05-10T03:32:14.177679+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:57.943475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.174982+0000","last_change":"2024-05-10T03:32:12.917812+0000","last_active":"2024-05-10T03:32:14.174982+0000","last_peered":"2024-05-10T03:32:14.174982+0000","last_clean":"2024-05-10T03:32:14.174982+0000","last_became_active":"2024-05-10T03:32:12.913954+0000","last_became_peered":"2024-05-10T03:32:12.913954+0000","last_unstale":"2024-05-10T03:32:14.174982+0000","last_undegraded":"2024-05-10T03:32:14.174982+0000","last_fullsized":"2024-05-10T03:32:14.174982+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899746+0000","last_change":"2024-05-10T03:32:12.902462+0000","last_active":"2024-05-10T03:32:13.899746+0000","last_peered":"2024-05-10T03:32:13.899746+0000","last_clean":"2024-05-10T03:32:13.899746+0000","last_became_active":"2024-05-10T03:32:12.901977+0000","last_became_peered":"2024-05-10T03:32:12.901977+0000","last_unstale":"2024-05-10T03:32:13.899746+0000","last_undegraded":"2024-05-10T03:32:13.899746+0000","last_fullsized":"2024-05-10T03:32:13.899746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:05.823783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176963+0000","last_change":"2024-05-10T03:32:12.923673+0000","last_active":"2024-05-10T03:32:14.176963+0000","last_peered":"2024-05-10T03:32:14.176963+0000","last_clean":"2024-05-10T03:32:14.176963+0000","last_became_active":"2024-05-10T03:32:12.923367+0000","last_became_peered":"2024-05-10T03:32:12.923367+0000","last_unstale":"2024-05-10T03:32:14.176963+0000","last_undegraded":"2024-05-10T03:32:14.176963+0000","last_fullsized":"2024-05-10T03:32:14.176963+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:43.139918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927108+0000","last_change":"2024-05-10T03:32:12.321598+0000","last_active":"2024-05-10T03:32:12.927108+0000","last_peered":"2024-05-10T03:32:12.927108+0000","last_clean":"2024-05-10T03:32:12.927108+0000","last_became_active":"2024-05-10T03:32:12.320780+0000","last_became_peered":"2024-05-10T03:32:12.320780+0000","last_unstale":"2024-05-10T03:32:12.927108+0000","last_undegraded":"2024-05-10T03:32:12.927108+0000","last_fullsized":"2024-05-10T03:32:12.927108+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:09:45.664808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900214+0000","last_change":"2024-05-10T03:32:12.912792+0000","last_active":"2024-05-10T03:32:13.900214+0000","last_peered":"2024-05-10T03:32:13.900214+0000","last_clean":"2024-05-10T03:32:13.900214+0000","last_became_active":"2024-05-10T03:32:12.907370+0000","last_became_peered":"2024-05-10T03:32:12.907370+0000","last_unstale":"2024-05-10T03:32:13.900214+0000","last_undegraded":"2024-05-10T03:32:13.900214+0000","last_fullsized":"2024-05-10T03:32:13.900214+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:20.298764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.929409+0000","last_change":"2024-05-10T03:32:12.323974+0000","last_active":"2024-05-10T03:32:12.929409+0000","last_peered":"2024-05-10T03:32:12.929409+0000","last_clean":"2024-05-10T03:32:12.929409+0000","last_became_active":"2024-05-10T03:32:12.323651+0000","last_became_peered":"2024-05-10T03:32:12.323651+0000","last_unstale":"2024-05-10T03:32:12.929409+0000","last_undegraded":"2024-05-10T03:32:12.929409+0000","last_fullsized":"2024-05-10T03:32:12.929409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177373+0000","last_change":"2024-05-10T03:32:12.926741+0000","last_active":"2024-05-10T03:32:14.177373+0000","last_peered":"2024-05-10T03:32:14.177373+0000","last_clean":"2024-05-10T03:32:14.177373+0000","last_became_active":"2024-05-10T03:32:12.926261+0000","last_became_peered":"2024-05-10T03:32:12.926261+0000","last_unstale":"2024-05-10T03:32:14.177373+0000","last_undegraded":"2024-05-10T03:32:14.177373+0000","last_fullsized":"2024-05-10T03:32:14.177373+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.922236+0000","last_change":"2024-05-10T03:32:12.327381+0000","last_active":"2024-05-10T03:32:12.922236+0000","last_peered":"2024-05-10T03:32:12.922236+0000","last_clean":"2024-05-10T03:32:12.922236+0000","last_became_active":"2024-05-10T03:32:12.327048+0000","last_became_peered":"2024-05-10T03:32:12.327048+0000","last_unstale":"2024-05-10T03:32:12.922236+0000","last_undegraded":"2024-05-10T03:32:12.922236+0000","last_fullsized":"2024-05-10T03:32:12.922236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.177770+0000","last_change":"2024-05-10T03:32:12.910467+0000","last_active":"2024-05-10T03:32:14.177770+0000","last_peered":"2024-05-10T03:32:14.177770+0000","last_clean":"2024-05-10T03:32:14.177770+0000","last_became_active":"2024-05-10T03:32:12.910188+0000","last_became_peered":"2024-05-10T03:32:12.910188+0000","last_unstale":"2024-05-10T03:32:14.177770+0000","last_undegraded":"2024-05-10T03:32:14.177770+0000","last_fullsized":"2024-05-10T03:32:14.177770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:35:22.601184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923121+0000","last_change":"2024-05-10T03:32:12.325376+0000","last_active":"2024-05-10T03:32:12.923121+0000","last_peered":"2024-05-10T03:32:12.923121+0000","last_clean":"2024-05-10T03:32:12.923121+0000","last_became_active":"2024-05-10T03:32:12.324035+0000","last_became_peered":"2024-05-10T03:32:12.324035+0000","last_unstale":"2024-05-10T03:32:12.923121+0000","last_undegraded":"2024-05-10T03:32:12.923121+0000","last_fullsized":"2024-05-10T03:32:12.923121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:43:43.411173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.903007+0000","last_change":"2024-05-10T03:32:12.924458+0000","last_active":"2024-05-10T03:32:13.903007+0000","last_peered":"2024-05-10T03:32:13.903007+0000","last_clean":"2024-05-10T03:32:13.903007+0000","last_became_active":"2024-05-10T03:32:12.923064+0000","last_became_peered":"2024-05-10T03:32:12.923064+0000","last_unstale":"2024-05-10T03:32:13.903007+0000","last_undegraded":"2024-05-10T03:32:13.903007+0000","last_fullsized":"2024-05-10T03:32:13.903007+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:45:36.843548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930846+0000","last_change":"2024-05-10T03:32:12.328698+0000","last_active":"2024-05-10T03:32:12.930846+0000","last_peered":"2024-05-10T03:32:12.930846+0000","last_clean":"2024-05-10T03:32:12.930846+0000","last_became_active":"2024-05-10T03:32:12.328069+0000","last_became_peered":"2024-05-10T03:32:12.328069+0000","last_unstale":"2024-05-10T03:32:12.930846+0000","last_undegraded":"2024-05-10T03:32:12.930846+0000","last_fullsized":"2024-05-10T03:32:12.930846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178392+0000","last_change":"2024-05-10T03:32:12.929164+0000","last_active":"2024-05-10T03:32:14.178392+0000","last_peered":"2024-05-10T03:32:14.178392+0000","last_clean":"2024-05-10T03:32:14.178392+0000","last_became_active":"2024-05-10T03:32:12.928657+0000","last_became_peered":"2024-05-10T03:32:12.928657+0000","last_unstale":"2024-05-10T03:32:14.178392+0000","last_undegraded":"2024-05-10T03:32:14.178392+0000","last_fullsized":"2024-05-10T03:32:14.178392+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.923421+0000","last_change":"2024-05-10T03:32:12.317756+0000","last_active":"2024-05-10T03:32:12.923421+0000","last_peered":"2024-05-10T03:32:12.923421+0000","last_clean":"2024-05-10T03:32:12.923421+0000","last_became_active":"2024-05-10T03:32:12.317442+0000","last_became_peered":"2024-05-10T03:32:12.317442+0000","last_unstale":"2024-05-10T03:32:12.923421+0000","last_undegraded":"2024-05-10T03:32:12.923421+0000","last_fullsized":"2024-05-10T03:32:12.923421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:26.565850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178416+0000","last_change":"2024-05-10T03:32:12.904312+0000","last_active":"2024-05-10T03:32:14.178416+0000","last_peered":"2024-05-10T03:32:14.178416+0000","last_clean":"2024-05-10T03:32:14.178416+0000","last_became_active":"2024-05-10T03:32:12.904017+0000","last_became_peered":"2024-05-10T03:32:12.904017+0000","last_unstale":"2024-05-10T03:32:14.178416+0000","last_undegraded":"2024-05-10T03:32:14.178416+0000","last_fullsized":"2024-05-10T03:32:14.178416+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:00:06.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906708+0000","last_change":"2024-05-10T03:32:12.324225+0000","last_active":"2024-05-10T03:32:12.906708+0000","last_peered":"2024-05-10T03:32:12.906708+0000","last_clean":"2024-05-10T03:32:12.906708+0000","last_became_active":"2024-05-10T03:32:12.323907+0000","last_became_peered":"2024-05-10T03:32:12.323907+0000","last_unstale":"2024-05-10T03:32:12.906708+0000","last_undegraded":"2024-05-10T03:32:12.906708+0000","last_fullsized":"2024-05-10T03:32:12.906708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:34.518049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178666+0000","last_change":"2024-05-10T03:32:12.931385+0000","last_active":"2024-05-10T03:32:14.178666+0000","last_peered":"2024-05-10T03:32:14.178666+0000","last_clean":"2024-05-10T03:32:14.178666+0000","last_became_active":"2024-05-10T03:32:12.931015+0000","last_became_peered":"2024-05-10T03:32:12.931015+0000","last_unstale":"2024-05-10T03:32:14.178666+0000","last_undegraded":"2024-05-10T03:32:14.178666+0000","last_fullsized":"2024-05-10T03:32:14.178666+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907110+0000","last_change":"2024-05-10T03:32:12.335472+0000","last_active":"2024-05-10T03:32:12.907110+0000","last_peered":"2024-05-10T03:32:12.907110+0000","last_clean":"2024-05-10T03:32:12.907110+0000","last_became_active":"2024-05-10T03:32:12.335158+0000","last_became_peered":"2024-05-10T03:32:12.335158+0000","last_unstale":"2024-05-10T03:32:12.907110+0000","last_undegraded":"2024-05-10T03:32:12.907110+0000","last_fullsized":"2024-05-10T03:32:12.907110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902078+0000","last_change":"2024-05-10T03:32:12.907425+0000","last_active":"2024-05-10T03:32:13.902078+0000","last_peered":"2024-05-10T03:32:13.902078+0000","last_clean":"2024-05-10T03:32:13.902078+0000","last_became_active":"2024-05-10T03:32:12.904255+0000","last_became_peered":"2024-05-10T03:32:12.904255+0000","last_unstale":"2024-05-10T03:32:13.902078+0000","last_undegraded":"2024-05-10T03:32:13.902078+0000","last_fullsized":"2024-05-10T03:32:13.902078+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:17:59.106698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910427+0000","last_change":"2024-05-10T03:32:12.326744+0000","last_active":"2024-05-10T03:32:12.910427+0000","last_peered":"2024-05-10T03:32:12.910427+0000","last_clean":"2024-05-10T03:32:12.910427+0000","last_became_active":"2024-05-10T03:32:12.326336+0000","last_became_peered":"2024-05-10T03:32:12.326336+0000","last_unstale":"2024-05-10T03:32:12.910427+0000","last_undegraded":"2024-05-10T03:32:12.910427+0000","last_fullsized":"2024-05-10T03:32:12.910427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899885+0000","last_change":"2024-05-10T03:32:12.904290+0000","last_active":"2024-05-10T03:32:13.899885+0000","last_peered":"2024-05-10T03:32:13.899885+0000","last_clean":"2024-05-10T03:32:13.899885+0000","last_became_active":"2024-05-10T03:32:12.903859+0000","last_became_peered":"2024-05-10T03:32:12.903859+0000","last_unstale":"2024-05-10T03:32:13.899885+0000","last_undegraded":"2024-05-10T03:32:13.899885+0000","last_fullsized":"2024-05-10T03:32:13.899885+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:38.939944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895544+0000","last_change":"2024-05-10T03:32:12.326240+0000","last_active":"2024-05-10T03:32:12.895544+0000","last_peered":"2024-05-10T03:32:12.895544+0000","last_clean":"2024-05-10T03:32:12.895544+0000","last_became_active":"2024-05-10T03:32:12.325869+0000","last_became_peered":"2024-05-10T03:32:12.325869+0000","last_unstale":"2024-05-10T03:32:12.895544+0000","last_undegraded":"2024-05-10T03:32:12.895544+0000","last_fullsized":"2024-05-10T03:32:12.895544+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175662+0000","last_change":"2024-05-10T03:32:12.924590+0000","last_active":"2024-05-10T03:32:14.175662+0000","last_peered":"2024-05-10T03:32:14.175662+0000","last_clean":"2024-05-10T03:32:14.175662+0000","last_became_active":"2024-05-10T03:32:12.924100+0000","last_became_peered":"2024-05-10T03:32:12.924100+0000","last_unstale":"2024-05-10T03:32:14.175662+0000","last_undegraded":"2024-05-10T03:32:14.175662+0000","last_fullsized":"2024-05-10T03:32:14.175662+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:10:07.133902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.928529+0000","last_change":"2024-05-10T03:32:12.331932+0000","last_active":"2024-05-10T03:32:12.928529+0000","last_peered":"2024-05-10T03:32:12.928529+0000","last_clean":"2024-05-10T03:32:12.928529+0000","last_became_active":"2024-05-10T03:32:12.331589+0000","last_became_peered":"2024-05-10T03:32:12.331589+0000","last_unstale":"2024-05-10T03:32:12.928529+0000","last_undegraded":"2024-05-10T03:32:12.928529+0000","last_fullsized":"2024-05-10T03:32:12.928529+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902755+0000","last_change":"2024-05-10T03:32:12.904583+0000","last_active":"2024-05-10T03:32:13.902755+0000","last_peered":"2024-05-10T03:32:13.902755+0000","last_clean":"2024-05-10T03:32:13.902755+0000","last_became_active":"2024-05-10T03:32:12.903287+0000","last_became_peered":"2024-05-10T03:32:12.903287+0000","last_unstale":"2024-05-10T03:32:13.902755+0000","last_undegraded":"2024-05-10T03:32:13.902755+0000","last_fullsized":"2024-05-10T03:32:13.902755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934219+0000","last_change":"2024-05-10T03:32:12.322783+0000","last_active":"2024-05-10T03:32:12.934219+0000","last_peered":"2024-05-10T03:32:12.934219+0000","last_clean":"2024-05-10T03:32:12.934219+0000","last_became_active":"2024-05-10T03:32:12.320656+0000","last_became_peered":"2024-05-10T03:32:12.320656+0000","last_unstale":"2024-05-10T03:32:12.934219+0000","last_undegraded":"2024-05-10T03:32:12.934219+0000","last_fullsized":"2024-05-10T03:32:12.934219+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:46.848796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900290+0000","last_change":"2024-05-10T03:32:12.920087+0000","last_active":"2024-05-10T03:32:13.900290+0000","last_peered":"2024-05-10T03:32:13.900290+0000","last_clean":"2024-05-10T03:32:13.900290+0000","last_became_active":"2024-05-10T03:32:12.919738+0000","last_became_peered":"2024-05-10T03:32:12.919738+0000","last_unstale":"2024-05-10T03:32:13.900290+0000","last_undegraded":"2024-05-10T03:32:13.900290+0000","last_fullsized":"2024-05-10T03:32:13.900290+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:36.708699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.930884+0000","last_change":"2024-05-10T03:32:12.329191+0000","last_active":"2024-05-10T03:32:12.930884+0000","last_peered":"2024-05-10T03:32:12.930884+0000","last_clean":"2024-05-10T03:32:12.930884+0000","last_became_active":"2024-05-10T03:32:12.326891+0000","last_became_peered":"2024-05-10T03:32:12.326891+0000","last_unstale":"2024-05-10T03:32:12.930884+0000","last_undegraded":"2024-05-10T03:32:12.930884+0000","last_fullsized":"2024-05-10T03:32:12.930884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:48:20.732699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175971+0000","last_change":"2024-05-10T03:32:12.915905+0000","last_active":"2024-05-10T03:32:14.175971+0000","last_peered":"2024-05-10T03:32:14.175971+0000","last_clean":"2024-05-10T03:32:14.175971+0000","last_became_active":"2024-05-10T03:32:12.915314+0000","last_became_peered":"2024-05-10T03:32:12.915314+0000","last_unstale":"2024-05-10T03:32:14.175971+0000","last_undegraded":"2024-05-10T03:32:14.175971+0000","last_fullsized":"2024-05-10T03:32:14.175971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:38:53.287475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909280+0000","last_change":"2024-05-10T03:32:12.340972+0000","last_active":"2024-05-10T03:32:12.909280+0000","last_peered":"2024-05-10T03:32:12.909280+0000","last_clean":"2024-05-10T03:32:12.909280+0000","last_became_active":"2024-05-10T03:32:12.340465+0000","last_became_peered":"2024-05-10T03:32:12.340465+0000","last_unstale":"2024-05-10T03:32:12.909280+0000","last_undegraded":"2024-05-10T03:32:12.909280+0000","last_fullsized":"2024-05-10T03:32:12.909280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178707+0000","last_change":"2024-05-10T03:32:12.917579+0000","last_active":"2024-05-10T03:32:14.178707+0000","last_peered":"2024-05-10T03:32:14.178707+0000","last_clean":"2024-05-10T03:32:14.178707+0000","last_became_active":"2024-05-10T03:32:12.917261+0000","last_became_peered":"2024-05-10T03:32:12.917261+0000","last_unstale":"2024-05-10T03:32:14.178707+0000","last_undegraded":"2024-05-10T03:32:14.178707+0000","last_fullsized":"2024-05-10T03:32:14.178707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:16.513183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907440+0000","last_change":"2024-05-10T03:32:12.328850+0000","last_active":"2024-05-10T03:32:12.907440+0000","last_peered":"2024-05-10T03:32:12.907440+0000","last_clean":"2024-05-10T03:32:12.907440+0000","last_became_active":"2024-05-10T03:32:12.328541+0000","last_became_peered":"2024-05-10T03:32:12.328541+0000","last_unstale":"2024-05-10T03:32:12.907440+0000","last_undegraded":"2024-05-10T03:32:12.907440+0000","last_fullsized":"2024-05-10T03:32:12.907440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:27.898137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175276+0000","last_change":"2024-05-10T03:32:12.925092+0000","last_active":"2024-05-10T03:32:14.175276+0000","last_peered":"2024-05-10T03:32:14.175276+0000","last_clean":"2024-05-10T03:32:14.175276+0000","last_became_active":"2024-05-10T03:32:12.920412+0000","last_became_peered":"2024-05-10T03:32:12.920412+0000","last_unstale":"2024-05-10T03:32:14.175276+0000","last_undegraded":"2024-05-10T03:32:14.175276+0000","last_fullsized":"2024-05-10T03:32:14.175276+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:57:10.962423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.926806+0000","last_change":"2024-05-10T03:32:12.330153+0000","last_active":"2024-05-10T03:32:12.926806+0000","last_peered":"2024-05-10T03:32:12.926806+0000","last_clean":"2024-05-10T03:32:12.926806+0000","last_became_active":"2024-05-10T03:32:12.328275+0000","last_became_peered":"2024-05-10T03:32:12.328275+0000","last_unstale":"2024-05-10T03:32:12.926806+0000","last_undegraded":"2024-05-10T03:32:12.926806+0000","last_fullsized":"2024-05-10T03:32:12.926806+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:56.897434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178945+0000","last_change":"2024-05-10T03:32:12.912037+0000","last_active":"2024-05-10T03:32:14.178945+0000","last_peered":"2024-05-10T03:32:14.178945+0000","last_clean":"2024-05-10T03:32:14.178945+0000","last_became_active":"2024-05-10T03:32:12.911220+0000","last_became_peered":"2024-05-10T03:32:12.911220+0000","last_unstale":"2024-05-10T03:32:14.178945+0000","last_undegraded":"2024-05-10T03:32:14.178945+0000","last_fullsized":"2024-05-10T03:32:14.178945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:21:22.435460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.934217+0000","last_change":"2024-05-10T03:32:12.332211+0000","last_active":"2024-05-10T03:32:12.934217+0000","last_peered":"2024-05-10T03:32:12.934217+0000","last_clean":"2024-05-10T03:32:12.934217+0000","last_became_active":"2024-05-10T03:32:12.331391+0000","last_became_peered":"2024-05-10T03:32:12.331391+0000","last_unstale":"2024-05-10T03:32:12.934217+0000","last_undegraded":"2024-05-10T03:32:12.934217+0000","last_fullsized":"2024-05-10T03:32:12.934217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:25:47.556396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898530+0000","last_change":"2024-05-10T03:32:12.919886+0000","last_active":"2024-05-10T03:32:13.898530+0000","last_peered":"2024-05-10T03:32:13.898530+0000","last_clean":"2024-05-10T03:32:13.898530+0000","last_became_active":"2024-05-10T03:32:12.919559+0000","last_became_peered":"2024-05-10T03:32:12.919559+0000","last_unstale":"2024-05-10T03:32:13.898530+0000","last_undegraded":"2024-05-10T03:32:13.898530+0000","last_fullsized":"2024-05-10T03:32:13.898530+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.907502+0000","last_change":"2024-05-10T03:32:12.320826+0000","last_active":"2024-05-10T03:32:12.907502+0000","last_peered":"2024-05-10T03:32:12.907502+0000","last_clean":"2024-05-10T03:32:12.907502+0000","last_became_active":"2024-05-10T03:32:12.320490+0000","last_became_peered":"2024-05-10T03:32:12.320490+0000","last_unstale":"2024-05-10T03:32:12.907502+0000","last_undegraded":"2024-05-10T03:32:12.907502+0000","last_fullsized":"2024-05-10T03:32:12.907502+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:42:58.799666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.902476+0000","last_change":"2024-05-10T03:32:12.918118+0000","last_active":"2024-05-10T03:32:13.902476+0000","last_peered":"2024-05-10T03:32:13.902476+0000","last_clean":"2024-05-10T03:32:13.902476+0000","last_became_active":"2024-05-10T03:32:12.917768+0000","last_became_peered":"2024-05-10T03:32:12.917768+0000","last_unstale":"2024-05-10T03:32:13.902476+0000","last_undegraded":"2024-05-10T03:32:13.902476+0000","last_fullsized":"2024-05-10T03:32:13.902476+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.924200+0000","last_change":"2024-05-10T03:32:12.321740+0000","last_active":"2024-05-10T03:32:12.924200+0000","last_peered":"2024-05-10T03:32:12.924200+0000","last_clean":"2024-05-10T03:32:12.924200+0000","last_became_active":"2024-05-10T03:32:12.321433+0000","last_became_peered":"2024-05-10T03:32:12.321433+0000","last_unstale":"2024-05-10T03:32:12.924200+0000","last_undegraded":"2024-05-10T03:32:12.924200+0000","last_fullsized":"2024-05-10T03:32:12.924200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:19:55.932711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898874+0000","last_change":"2024-05-10T03:32:12.917048+0000","last_active":"2024-05-10T03:32:13.898874+0000","last_peered":"2024-05-10T03:32:13.898874+0000","last_clean":"2024-05-10T03:32:13.898874+0000","last_became_active":"2024-05-10T03:32:12.916721+0000","last_became_peered":"2024-05-10T03:32:12.916721+0000","last_unstale":"2024-05-10T03:32:13.898874+0000","last_undegraded":"2024-05-10T03:32:13.898874+0000","last_fullsized":"2024-05-10T03:32:13.898874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:56:41.495565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.910839+0000","last_change":"2024-05-10T03:32:12.326718+0000","last_active":"2024-05-10T03:32:12.910839+0000","last_peered":"2024-05-10T03:32:12.910839+0000","last_clean":"2024-05-10T03:32:12.910839+0000","last_became_active":"2024-05-10T03:32:12.325934+0000","last_became_peered":"2024-05-10T03:32:12.325934+0000","last_unstale":"2024-05-10T03:32:12.910839+0000","last_undegraded":"2024-05-10T03:32:12.910839+0000","last_fullsized":"2024-05-10T03:32:12.910839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:02:11.209184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900458+0000","last_change":"2024-05-10T03:32:12.908080+0000","last_active":"2024-05-10T03:32:13.900458+0000","last_peered":"2024-05-10T03:32:13.900458+0000","last_clean":"2024-05-10T03:32:13.900458+0000","last_became_active":"2024-05-10T03:32:12.907698+0000","last_became_peered":"2024-05-10T03:32:12.907698+0000","last_unstale":"2024-05-10T03:32:13.900458+0000","last_undegraded":"2024-05-10T03:32:13.900458+0000","last_fullsized":"2024-05-10T03:32:13.900458+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:47.862762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.927874+0000","last_change":"2024-05-10T03:32:12.321882+0000","last_active":"2024-05-10T03:32:12.927874+0000","last_peered":"2024-05-10T03:32:12.927874+0000","last_clean":"2024-05-10T03:32:12.927874+0000","last_became_active":"2024-05-10T03:32:12.321451+0000","last_became_peered":"2024-05-10T03:32:12.321451+0000","last_unstale":"2024-05-10T03:32:12.927874+0000","last_undegraded":"2024-05-10T03:32:12.927874+0000","last_fullsized":"2024-05-10T03:32:12.927874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.388284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.179016+0000","last_change":"2024-05-10T03:32:12.918914+0000","last_active":"2024-05-10T03:32:14.179016+0000","last_peered":"2024-05-10T03:32:14.179016+0000","last_clean":"2024-05-10T03:32:14.179016+0000","last_became_active":"2024-05-10T03:32:12.918226+0000","last_became_peered":"2024-05-10T03:32:12.918226+0000","last_unstale":"2024-05-10T03:32:14.179016+0000","last_undegraded":"2024-05-10T03:32:14.179016+0000","last_fullsized":"2024-05-10T03:32:14.179016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:09:58.395210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894269+0000","last_change":"2024-05-10T03:32:12.323189+0000","last_active":"2024-05-10T03:32:12.894269+0000","last_peered":"2024-05-10T03:32:12.894269+0000","last_clean":"2024-05-10T03:32:12.894269+0000","last_became_active":"2024-05-10T03:32:12.322864+0000","last_became_peered":"2024-05-10T03:32:12.322864+0000","last_unstale":"2024-05-10T03:32:12.894269+0000","last_undegraded":"2024-05-10T03:32:12.894269+0000","last_fullsized":"2024-05-10T03:32:12.894269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:04.809817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175964+0000","last_change":"2024-05-10T03:32:12.925481+0000","last_active":"2024-05-10T03:32:14.175964+0000","last_peered":"2024-05-10T03:32:14.175964+0000","last_clean":"2024-05-10T03:32:14.175964+0000","last_became_active":"2024-05-10T03:32:12.925134+0000","last_became_peered":"2024-05-10T03:32:12.925134+0000","last_unstale":"2024-05-10T03:32:14.175964+0000","last_undegraded":"2024-05-10T03:32:14.175964+0000","last_fullsized":"2024-05-10T03:32:14.175964+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:19:13.132758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.906033+0000","last_change":"2024-05-10T03:32:12.327618+0000","last_active":"2024-05-10T03:32:12.906033+0000","last_peered":"2024-05-10T03:32:12.906033+0000","last_clean":"2024-05-10T03:32:12.906033+0000","last_became_active":"2024-05-10T03:32:12.327215+0000","last_became_peered":"2024-05-10T03:32:12.327215+0000","last_unstale":"2024-05-10T03:32:12.906033+0000","last_undegraded":"2024-05-10T03:32:12.906033+0000","last_fullsized":"2024-05-10T03:32:12.906033+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:32:39.030457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899016+0000","last_change":"2024-05-10T03:32:12.911714+0000","last_active":"2024-05-10T03:32:13.899016+0000","last_peered":"2024-05-10T03:32:13.899016+0000","last_clean":"2024-05-10T03:32:13.899016+0000","last_became_active":"2024-05-10T03:32:12.908258+0000","last_became_peered":"2024-05-10T03:32:12.908258+0000","last_unstale":"2024-05-10T03:32:13.899016+0000","last_undegraded":"2024-05-10T03:32:13.899016+0000","last_fullsized":"2024-05-10T03:32:13.899016+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.882028+0000","last_change":"2024-05-10T03:32:12.325007+0000","last_active":"2024-05-10T03:32:12.882028+0000","last_peered":"2024-05-10T03:32:12.882028+0000","last_clean":"2024-05-10T03:32:12.882028+0000","last_became_active":"2024-05-10T03:32:12.324692+0000","last_became_peered":"2024-05-10T03:32:12.324692+0000","last_unstale":"2024-05-10T03:32:12.882028+0000","last_undegraded":"2024-05-10T03:32:12.882028+0000","last_fullsized":"2024-05-10T03:32:12.882028+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:35:55.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.899740+0000","last_change":"2024-05-10T03:32:12.899378+0000","last_active":"2024-05-10T03:32:13.899740+0000","last_peered":"2024-05-10T03:32:13.899740+0000","last_clean":"2024-05-10T03:32:13.899740+0000","last_became_active":"2024-05-10T03:32:12.898752+0000","last_became_peered":"2024-05-10T03:32:12.898752+0000","last_unstale":"2024-05-10T03:32:13.899740+0000","last_undegraded":"2024-05-10T03:32:13.899740+0000","last_fullsized":"2024-05-10T03:32:13.899740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:58:52.141008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.931779+0000","last_change":"2024-05-10T03:32:12.332881+0000","last_active":"2024-05-10T03:32:12.931779+0000","last_peered":"2024-05-10T03:32:12.931779+0000","last_clean":"2024-05-10T03:32:12.931779+0000","last_became_active":"2024-05-10T03:32:12.332577+0000","last_became_peered":"2024-05-10T03:32:12.332577+0000","last_unstale":"2024-05-10T03:32:12.931779+0000","last_undegraded":"2024-05-10T03:32:12.931779+0000","last_fullsized":"2024-05-10T03:32:12.931779+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.898806+0000","last_change":"2024-05-10T03:32:12.901260+0000","last_active":"2024-05-10T03:32:13.898806+0000","last_peered":"2024-05-10T03:32:13.898806+0000","last_clean":"2024-05-10T03:32:13.898806+0000","last_became_active":"2024-05-10T03:32:12.900971+0000","last_became_peered":"2024-05-10T03:32:12.900971+0000","last_unstale":"2024-05-10T03:32:13.898806+0000","last_undegraded":"2024-05-10T03:32:13.898806+0000","last_fullsized":"2024-05-10T03:32:13.898806+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:08:49.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.909685+0000","last_change":"2024-05-10T03:32:12.340985+0000","last_active":"2024-05-10T03:32:12.909685+0000","last_peered":"2024-05-10T03:32:12.909685+0000","last_clean":"2024-05-10T03:32:12.909685+0000","last_became_active":"2024-05-10T03:32:12.340613+0000","last_became_peered":"2024-05-10T03:32:12.340613+0000","last_unstale":"2024-05-10T03:32:12.909685+0000","last_undegraded":"2024-05-10T03:32:12.909685+0000","last_fullsized":"2024-05-10T03:32:12.909685+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:10.648273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178003+0000","last_change":"2024-05-10T03:32:12.905988+0000","last_active":"2024-05-10T03:32:14.178003+0000","last_peered":"2024-05-10T03:32:14.178003+0000","last_clean":"2024-05-10T03:32:14.178003+0000","last_became_active":"2024-05-10T03:32:12.905703+0000","last_became_peered":"2024-05-10T03:32:12.905703+0000","last_unstale":"2024-05-10T03:32:14.178003+0000","last_undegraded":"2024-05-10T03:32:14.178003+0000","last_fullsized":"2024-05-10T03:32:14.178003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:25:53.297139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.911269+0000","last_change":"2024-05-10T03:32:12.329354+0000","last_active":"2024-05-10T03:32:12.911269+0000","last_peered":"2024-05-10T03:32:12.911269+0000","last_clean":"2024-05-10T03:32:12.911269+0000","last_became_active":"2024-05-10T03:32:12.328206+0000","last_became_peered":"2024-05-10T03:32:12.328206+0000","last_unstale":"2024-05-10T03:32:12.911269+0000","last_undegraded":"2024-05-10T03:32:12.911269+0000","last_fullsized":"2024-05-10T03:32:12.911269+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:50:28.939240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176002+0000","last_change":"2024-05-10T03:32:12.917543+0000","last_active":"2024-05-10T03:32:14.176002+0000","last_peered":"2024-05-10T03:32:14.176002+0000","last_clean":"2024-05-10T03:32:14.176002+0000","last_became_active":"2024-05-10T03:32:12.917311+0000","last_became_peered":"2024-05-10T03:32:12.917311+0000","last_unstale":"2024-05-10T03:32:14.176002+0000","last_undegraded":"2024-05-10T03:32:14.176002+0000","last_fullsized":"2024-05-10T03:32:14.176002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:09:39.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894845+0000","last_change":"2024-05-10T03:32:12.323909+0000","last_active":"2024-05-10T03:32:12.894845+0000","last_peered":"2024-05-10T03:32:12.894845+0000","last_clean":"2024-05-10T03:32:12.894845+0000","last_became_active":"2024-05-10T03:32:12.321976+0000","last_became_peered":"2024-05-10T03:32:12.321976+0000","last_unstale":"2024-05-10T03:32:12.894845+0000","last_undegraded":"2024-05-10T03:32:12.894845+0000","last_fullsized":"2024-05-10T03:32:12.894845+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:43:37.925978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:13.900409+0000","last_change":"2024-05-10T03:32:12.911863+0000","last_active":"2024-05-10T03:32:13.900409+0000","last_peered":"2024-05-10T03:32:13.900409+0000","last_clean":"2024-05-10T03:32:13.900409+0000","last_became_active":"2024-05-10T03:32:12.911565+0000","last_became_peered":"2024-05-10T03:32:12.911565+0000","last_unstale":"2024-05-10T03:32:13.900409+0000","last_undegraded":"2024-05-10T03:32:13.900409+0000","last_fullsized":"2024-05-10T03:32:13.900409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:57.911400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.894283+0000","last_change":"2024-05-10T03:32:12.323028+0000","last_active":"2024-05-10T03:32:12.894283+0000","last_peered":"2024-05-10T03:32:12.894283+0000","last_clean":"2024-05-10T03:32:12.894283+0000","last_became_active":"2024-05-10T03:32:12.322679+0000","last_became_peered":"2024-05-10T03:32:12.322679+0000","last_unstale":"2024-05-10T03:32:12.894283+0000","last_undegraded":"2024-05-10T03:32:12.894283+0000","last_fullsized":"2024-05-10T03:32:12.894283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:12.822129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.175892+0000","last_change":"2024-05-10T03:32:12.912422+0000","last_active":"2024-05-10T03:32:14.175892+0000","last_peered":"2024-05-10T03:32:14.175892+0000","last_clean":"2024-05-10T03:32:14.175892+0000","last_became_active":"2024-05-10T03:32:12.911930+0000","last_became_peered":"2024-05-10T03:32:12.911930+0000","last_unstale":"2024-05-10T03:32:14.175892+0000","last_undegraded":"2024-05-10T03:32:14.175892+0000","last_fullsized":"2024-05-10T03:32:14.175892+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.895035+0000","last_change":"2024-05-10T03:32:12.326371+0000","last_active":"2024-05-10T03:32:12.895035+0000","last_peered":"2024-05-10T03:32:12.895035+0000","last_clean":"2024-05-10T03:32:12.895035+0000","last_became_active":"2024-05-10T03:32:12.325858+0000","last_became_peered":"2024-05-10T03:32:12.325858+0000","last_unstale":"2024-05-10T03:32:12.895035+0000","last_undegraded":"2024-05-10T03:32:12.895035+0000","last_fullsized":"2024-05-10T03:32:12.895035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:54:13.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.178757+0000","last_change":"2024-05-10T03:32:12.912144+0000","last_active":"2024-05-10T03:32:14.178757+0000","last_peered":"2024-05-10T03:32:14.178757+0000","last_clean":"2024-05-10T03:32:14.178757+0000","last_became_active":"2024-05-10T03:32:12.911805+0000","last_became_peered":"2024-05-10T03:32:12.911805+0000","last_unstale":"2024-05-10T03:32:14.178757+0000","last_undegraded":"2024-05-10T03:32:14.178757+0000","last_fullsized":"2024-05-10T03:32:14.178757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:48:34.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.886727+0000","last_change":"2024-05-10T03:32:12.319651+0000","last_active":"2024-05-10T03:32:12.886727+0000","last_peered":"2024-05-10T03:32:12.886727+0000","last_clean":"2024-05-10T03:32:12.886727+0000","last_became_active":"2024-05-10T03:32:12.319302+0000","last_became_peered":"2024-05-10T03:32:12.319302+0000","last_unstale":"2024-05-10T03:32:12.886727+0000","last_undegraded":"2024-05-10T03:32:12.886727+0000","last_fullsized":"2024-05-10T03:32:12.886727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:36:42.125952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176756+0000","last_change":"2024-05-10T03:32:12.923331+0000","last_active":"2024-05-10T03:32:14.176756+0000","last_peered":"2024-05-10T03:32:14.176756+0000","last_clean":"2024-05-10T03:32:14.176756+0000","last_became_active":"2024-05-10T03:32:12.922968+0000","last_became_peered":"2024-05-10T03:32:12.922968+0000","last_unstale":"2024-05-10T03:32:14.176756+0000","last_undegraded":"2024-05-10T03:32:14.176756+0000","last_fullsized":"2024-05-10T03:32:14.176756+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:12.278239+0000","last_clean_scrub_stamp":"2024-05-10T03:32:12.278239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:35.494303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-10T03:32:14.176160+0000","last_change":"2024-05-10T03:32:12.327328+0000","last_active":"2024-05-10T03:32:14.176160+0000","last_peered":"2024-05-10T03:32:14.176160+0000","last_clean":"2024-05-10T03:32:14.176160+0000","last_became_active":"2024-05-10T03:32:12.326415+0000","last_became_peered":"2024-05-10T03:32:12.326415+0000","last_unstale":"2024-05-10T03:32:14.176160+0000","last_undegraded":"2024-05-10T03:32:14.176160+0000","last_fullsized":"2024-05-10T03:32:14.176160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:11.264273+0000","last_clean_scrub_stamp":"2024-05-10T03:32:11.264273+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:52:42.441359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-10T03:32:12.905988+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:12.905988+0000","last_peered":"2024-05-10T03:32:12.905988+0000","last_clean":"2024-05-10T03:32:12.905988+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:12.905988+0000","last_undegraded":"2024-05-10T03:32:12.905988+0000","last_fullsized":"2024-05-10T03:32:12.905988+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2230000000000001,"15min":1.2270000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.077},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.367,"15min":1.365},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.8100000000000001,"5min":1.877,"15min":1.877},"last":1.2430000000000001}]},{"osd":1,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1379999999999999,"15min":1.1399999999999999},"min":{"1min":0.72399999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.266,"15min":1.274},"min":{"1min":0.84999999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.667,"5min":1.925,"15min":1.925},"last":1.139}]},{"osd":2,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3700000000000001,"15min":1.385},"min":{"1min":0.81200000000000006,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1349999999999998,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4410000000000001,"15min":1.4490000000000001},"min":{"1min":0.81200000000000006,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.819,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.173}]},{"osd":3,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3899999999999999,"15min":1.3919999999999999},"min":{"1min":1.0489999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.3520000000000001,"15min":1.3340000000000001},"min":{"1min":1.0109999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.1120000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.5349999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.49,"15min":1.484},"min":{"1min":1.071,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.1339999999999999,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.415},{"interface":"front","average":{"1min":1.583,"5min":1.5169999999999999,"15min":1.506},"min":{"1min":1.208,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.2050000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.546,"15min":1.516},"min":{"1min":1.2030000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1779999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.76,"5min":1.712,"15min":1.704},"min":{"1min":1.383,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.5780000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:39 2024","interfaces":[{"interface":"back","average":{"1min":1.762,"5min":1.7370000000000001,"15min":1.7330000000000001},"min":{"1min":1.3939999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.74},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.653,"15min":1.641},"min":{"1min":1.0600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.7989999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":272133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.073,"15min":1.071},"min":{"1min":0.69799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.329,"15min":1.351},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8640000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.7230000000000001}]},{"osd":1,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84699999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4590000000000001,"5min":3.113,"15min":3.113},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4750000000000001,"15min":1.4970000000000001},"min":{"1min":1.046,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.944,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.7589999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.246,"15min":1.232},"min":{"1min":0.93000000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.482,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5589999999999999,"5min":3.569,"15min":3.569},"last":1.3280000000000001}]},{"osd":3,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.591,"15min":1.6279999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.613},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.306,"15min":1.304},"min":{"1min":1.0409999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.744,"5min":1.931,"15min":1.931},"last":1.3799999999999999}]},{"osd":4,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4359999999999999,"15min":1.4299999999999999},"min":{"1min":1.069,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.8759999999999999,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5640000000000001,"15min":1.585},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.819,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.111}]},{"osd":5,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.397,"15min":1.389},"min":{"1min":1.1259999999999999,"5min":1,"15min":1},"max":{"1min":1.857,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.655},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.0029999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.004,"5min":2.472,"15min":2.472},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.665,"15min":1.671},"min":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"max":{"1min":1.9199999999999999,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.514},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.651,"15min":1.655},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0070000000000001,"5min":2.359,"15min":2.359},"last":1.4790000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":271481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0529999999999999,"15min":1.0580000000000001},"min":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.2130000000000001,"15min":1.2070000000000001},"min":{"1min":0.877,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.379}]},{"osd":1,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.141},"min":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7649999999999999,"5min":1.855,"15min":1.855},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.2270000000000001},"min":{"1min":0.77700000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":1.954,"15min":1.954},"last":1.2989999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2270000000000001,"15min":1.2210000000000001},"min":{"1min":0.81599999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.633,"5min":2.359,"15min":2.359},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":0.93799999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3680000000000001,"15min":1.359},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7769999999999999,"5min":1.881,"15min":1.881},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3100000000000001,"15min":1.2889999999999999},"min":{"1min":0.96399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.5980000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4339999999999999,"15min":1.4259999999999999},"min":{"1min":0.94199999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9259999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.6120000000000001},"min":{"1min":1.24,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.0289999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.5249999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.514},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.9019999999999999},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5609999999999999,"15min":1.5549999999999999},"min":{"1min":1.0760000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.675}]},{"osd":7,"last update":"Fri May 10 03:31:31 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.831,"15min":1.823},"min":{"1min":1.4039999999999999,"5min":0.753,"15min":0.753},"max":{"1min":2.4510000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.635,"15min":1.6180000000000001},"min":{"1min":1.1950000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1110000000000002,"5min":2.194,"15min":2.194},"last":1.827}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":622592,"data_stored":272063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.173,"15min":1.177},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.556},{"interface":"front","average":{"1min":1.117,"5min":1.1359999999999999,"15min":1.139},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5629999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.591}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.165,"15min":1.1559999999999999},"min":{"1min":0.79100000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.734,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.466},{"interface":"front","average":{"1min":1.246,"5min":1.278,"15min":1.2829999999999999},"min":{"1min":0.85699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.994,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.4279999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2829999999999999,"15min":1.2849999999999999},"min":{"1min":0.88,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8280000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.361},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.9179999999999999,"5min":2.29,"15min":2.29},"last":1.3089999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.458,"15min":1.476},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.76,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.518},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5900000000000001,"15min":1.589},"min":{"1min":1.2729999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":1.9530000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.7030000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5009999999999999,"15min":1.5049999999999999},"min":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.8129999999999999,"5min":2.512,"15min":2.512},"last":1.665},{"interface":"front","average":{"1min":1.456,"5min":1.552,"15min":1.5680000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.982,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.3999999999999999}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5900000000000001,"15min":1.5940000000000001},"min":{"1min":1.153,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1139999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.702,"5min":1.7829999999999999,"15min":1.7969999999999999},"min":{"1min":1.1040000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0409999999999999,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.0549999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6930000000000001,"15min":1.7030000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.069,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.7609999999999999,"15min":1.7509999999999999},"min":{"1min":1.4730000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.282,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.6279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":634880,"data_stored":273067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.754},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.95399999999999996,"15min":0.97599999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4850000000000001,"5min":1.708,"15min":1.708},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"min":{"1min":0.55800000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4370000000000001,"5min":1.603,"15min":1.603},"last":0.874},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.24,"15min":1.254},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.925,"5min":1.964,"15min":1.964},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1779999999999999,"15min":1.1839999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.897,"5min":1.99,"15min":1.99},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.343,"15min":1.369},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.919,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.627}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.95099999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.8080000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.3879999999999999,"15min":1.3779999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.1349999999999998,"5min":2.464,"15min":2.464},"last":1.544}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.6479999999999999,"15min":1.6859999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.109,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.9670000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.462,"15min":1.484},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0499999999999998,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.597}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.2559999999999998,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.774,"15min":1.804},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4729999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":2.4470000000000001}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.5629999999999999,"15min":1.5900000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.1379999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.6479999999999999,"15min":1.661},"min":{"1min":1.052,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.2690000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.786}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":630784,"data_stored":273045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.494,"5min":1.772,"15min":1.772},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.431,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.879}]},{"osd":1,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5600000000000001,"5min":1.621,"15min":1.621},"last":1.746},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1930000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6930000000000001,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1040000000000001,"15min":1.1240000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5009999999999999,"5min":2.169,"15min":2.169},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.25,"15min":1.2629999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9650000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.526,"15min":1.548},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.133,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.516,"15min":1.5569999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.821,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.3120000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.036,"5min":2.508,"15min":2.508},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.6140000000000001,"15min":1.625},"min":{"1min":1.069,"5min":1.004,"15min":1.004},"max":{"1min":2.1190000000000002,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.278}]},{"osd":6,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5409999999999999,"15min":1.5549999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1139999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.681},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.5820000000000001,"15min":1.5980000000000001},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.982,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":2.3799999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:32 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.641,"15min":1.6599999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.1360000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.597},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.659,"15min":1.679},"min":{"1min":1.052,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.169,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":2.468}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":626688,"data_stored":271641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0669999999999999,"15min":1.0920000000000001},"min":{"1min":0.54600000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.256,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.488,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.996},{"interface":"front","average":{"1min":1.044,"5min":1.1699999999999999,"15min":1.1910000000000001},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4330000000000001,"5min":2.444,"15min":2.444},"last":1.2589999999999999}]},{"osd":3,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.3,"15min":1.3280000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.556,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3009999999999999,"15min":1.3100000000000001},"min":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7789999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.3200000000000001}]},{"osd":4,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.673,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.446},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5669999999999999,"15min":1.597},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":1.8220000000000001,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.3600000000000001}]},{"osd":5,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.5600000000000001,"15min":1.5980000000000001},"min":{"1min":1.069,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.6200000000000001,"15min":1.647},"min":{"1min":1.1080000000000001,"5min":1.077,"15min":1.077},"max":{"1min":1.9770000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.407}]},{"osd":6,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.6990000000000001,"15min":1.7370000000000001},"min":{"1min":0.94999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.0550000000000002,"5min":3.48,"15min":3.48},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.716,"15min":1.748},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.0800000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.7270000000000001,"15min":1.7470000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.0910000000000002,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.657},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.762,"15min":1.796},"min":{"1min":1.173,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.7909999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.2070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1220608,"data_stored":861905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.999,"15min":1.036},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5629999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":1.1599999999999999,"15min":1.2110000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.179}]},{"osd":2,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5389999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4550000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.5680000000000001}]},{"osd":3,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.089,"15min":1.1140000000000001},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.448,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.266},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.929,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.389}]},{"osd":4,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.6040000000000001,"15min":1.6539999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0209999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.431,"15min":1.446},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.661,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.637}]},{"osd":5,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.464,"15min":1.486},"min":{"1min":0.85799999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9710000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.6080000000000001,"15min":1.647},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8999999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.6299999999999999,"15min":1.6539999999999999},"min":{"1min":1.028,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.206,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.835},{"interface":"front","average":{"1min":1.427,"5min":1.7150000000000001,"15min":1.7629999999999999},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.972,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.8089999999999999}]},{"osd":7,"last update":"Fri May 10 03:31:36 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.77,"15min":1.8200000000000001},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":1.946,"5min":3.149,"15min":3.149},"last":1.75},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.5609999999999999,"15min":1.581},"min":{"1min":0.91500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.99,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.768}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:32:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:32:24.103 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:32:24.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.432+0000 7fdb75800640 1 Processor -- start 2024-05-10T03:32:24.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.432+0000 7fdb75800640 1 -- start start 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.435+0000 7fdb75800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdb700b3c50 0x7fdb700b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.435+0000 7fdb75800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb7015e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.436+0000 7fdb75800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7015f2c0 con 0x7fdb70161450 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.436+0000 7fdb75800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700cf000 con 0x7fdb700b3c50 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.436+0000 7fdb75800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70161c30 con 0x7fdb7015de60 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.436+0000 7fdb6fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdb70161450 0x7fdb700ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45444/0 (socket says 172.21.15.55:45444) 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.436+0000 7fdb6fe00640 1 -- 172.21.15.55:0/775212857 learned_addr learned my addr 172.21.15.55:0/775212857 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.437+0000 7fdb74e00640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb7015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.437+0000 7fdb67e00640 1 --2- 172.21.15.55:0/775212857 >> v2:172.21.15.162:3300/0 conn(0x7fdb700b3c50 0x7fdb700b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.437+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086720332 0 0) 0x7fdb7015f2c0 con 0x7fdb70161450 2024-05-10T03:32:24.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.437+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb54003520 con 0x7fdb70161450 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb67e00640 1 -- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 msgr2=0x7fdb7015e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb67e00640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb7015e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb67e00640 1 -- 172.21.15.55:0/775212857 >> v1:172.21.15.55:6789/0 conn(0x7fdb70161450 legacy=0x7fdb700ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3489723630 0 0) 0x7fdb54003520 con 0x7fdb70161450 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb67e00640 1 -- 172.21.15.55:0/775212857 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70161f10 con 0x7fdb700b3c50 2024-05-10T03:32:24.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.438+0000 7fdb74e00640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb7015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:32:24.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.440+0000 7fdb67e00640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb700b3c50 0x7fdb700b4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb580097a0 tx=0x7fdb5802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3451cce8fbab3db server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:24.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.443+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60001670 con 0x7fdb700b3c50 2024-05-10T03:32:24.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.443+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5802ab10 con 0x7fdb700b3c50 2024-05-10T03:32:24.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.443+0000 7fdb6f400640 1 -- 172.21.15.55:0/775212857 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5802aea0 con 0x7fdb700b3c50 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 -- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb700b3c50 msgr2=0x7fdb700b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb700b3c50 0x7fdb700b4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb580097a0 tx=0x7fdb5802a080 comp rx=0 tx=0).stop 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 -- 172.21.15.55:0/775212857 shutdown_connections 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb7015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 --2- 172.21.15.55:0/775212857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb700b3c50 0x7fdb700b4060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 -- 172.21.15.55:0/775212857 >> 172.21.15.55:0/775212857 conn(0x7fdb700b5a40 msgr2=0x7fdb700d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 -- 172.21.15.55:0/775212857 shutdown_connections 2024-05-10T03:32:24.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.444+0000 7fdb75800640 1 -- 172.21.15.55:0/775212857 wait complete. 2024-05-10T03:32:24.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.447+0000 7fdb75800640 1 Processor -- start 2024-05-10T03:32:24.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.447+0000 7fdb75800640 1 -- start start 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.448+0000 7fdb75800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.448+0000 7fdb75800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb700c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb75800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb70161450 0x7fdb7014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb75800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701f3df0 con 0x7fdb700b3c50 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb75800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700c1330 con 0x7fdb70161450 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb75800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70155120 con 0x7fdb7015de60 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52068/0 (socket says 172.21.15.55:52068) 2024-05-10T03:32:24.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 -- 172.21.15.55:0/1680984319 learned_addr learned my addr 172.21.15.55:0/1680984319 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb74e00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb70161450 0x7fdb7014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb67e00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb700c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 -- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 msgr2=0x7fdb700c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb700c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.449+0000 7fdb6fe00640 1 -- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb70161450 msgr2=0x7fdb7014ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6fe00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb70161450 0x7fdb7014ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6fe00640 1 -- 172.21.15.55:0/1680984319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68009a70 con 0x7fdb700b3c50 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6fe00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb600011e0 tx=0x7fdb6000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60005ad0 con 0x7fdb700b3c50 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60005c30 con 0x7fdb700b3c50 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60007070 con 0x7fdb700b3c50 2024-05-10T03:32:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.450+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb58009400 con 0x7fdb700b3c50 2024-05-10T03:32:24.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.451+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb70155650 con 0x7fdb700b3c50 2024-05-10T03:32:24.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.452+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70067220 con 0x7fdb700b3c50 2024-05-10T03:32:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.454+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdb600071d0 con 0x7fdb700b3c50 2024-05-10T03:32:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.455+0000 7fdb6cc00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 0x7fdb4c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.455+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fdb60088bc0 con 0x7fdb700b3c50 2024-05-10T03:32:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.456+0000 7fdb67e00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 0x7fdb4c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:24.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.457+0000 7fdb67e00640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 0x7fdb4c06d110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdb58034000 tx=0x7fdb5802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.460+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdb60053ba0 con 0x7fdb700b3c50 2024-05-10T03:32:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.731+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdb700d1890 con 0x7fdb700b3c50 2024-05-10T03:32:24.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.740+0000 7fdb6cc00640 1 -- 172.21.15.55:0/1680984319 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdb60017070 con 0x7fdb700b3c50 2024-05-10T03:32:24.743 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:32:24.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 msgr2=0x7fdb4c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 0x7fdb4c06d110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdb58034000 tx=0x7fdb5802e000 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 msgr2=0x7fdb70156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb600011e0 tx=0x7fdb6000cbe0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 shutdown_connections 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdb4c06ac00 0x7fdb4c06d110 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb70161450 0x7fdb7014ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb7015de60 0x7fdb700c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 --2- 172.21.15.55:0/1680984319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb700b3c50 0x7fdb70156410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 >> 172.21.15.55:0/1680984319 conn(0x7fdb700b5a40 msgr2=0x7fdb700ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.743+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 shutdown_connections 2024-05-10T03:32:24.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:24.744+0000 7fdb75800640 1 -- 172.21.15.55:0/1680984319 wait complete. 2024-05-10T03:32:24.773 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.060+0000 7fed27e00640 1 Processor -- start 2024-05-10T03:32:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.060+0000 7fed27e00640 1 -- start start 2024-05-10T03:32:25.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed280b2170 0x7fed2815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 0x7fed2806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280d25a0 con 0x7fed280b6680 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed280d2b20 con 0x7fed280b2170 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2815db20 con 0x7fed281618b0 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed26000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed280b2170 0x7fed2815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed26000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed280b2170 0x7fed2815d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52072/0 (socket says 172.21.15.55:52072) 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed26000640 1 -- 172.21.15.55:0/2496154745 learned_addr learned my addr 172.21.15.55:0/2496154745 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.063+0000 7fed27400640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 0x7fed2806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.064+0000 7fed26000640 1 -- 172.21.15.55:0/2496154745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 msgr2=0x7fed2806aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.064+0000 7fed26000640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 0x7fed2806aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.064+0000 7fed26000640 1 -- 172.21.15.55:0/2496154745 >> v1:172.21.15.162:6789/0 conn(0x7fed280b6680 legacy=0x7fed281611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.064+0000 7fed26000640 1 -- 172.21.15.55:0/2496154745 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed280b28d0 con 0x7fed280b2170 2024-05-10T03:32:25.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.064+0000 7fed27400640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 0x7fed2806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:25.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.065+0000 7fed26000640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed280b2170 0x7fed2815d5e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c0a1b5a67146ca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:25.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.065+0000 7fed25600640 1 -- 172.21.15.55:0/2496154745 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18032440 con 0x7fed280b2170 2024-05-10T03:32:25.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.065+0000 7fed25600640 1 -- 172.21.15.55:0/2496154745 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed180325a0 con 0x7fed280b2170 2024-05-10T03:32:25.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.065+0000 7fed25600640 1 -- 172.21.15.55:0/2496154745 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18032930 con 0x7fed280b2170 2024-05-10T03:32:25.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 -- 172.21.15.55:0/2496154745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed280b2170 msgr2=0x7fed2815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:25.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed280b2170 0x7fed2815d5e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1802a210 comp rx=0 tx=0).stop 2024-05-10T03:32:25.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 -- 172.21.15.55:0/2496154745 shutdown_connections 2024-05-10T03:32:25.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed281618b0 0x7fed2806aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 --2- 172.21.15.55:0/2496154745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed280b2170 0x7fed2815d5e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 -- 172.21.15.55:0/2496154745 >> 172.21.15.55:0/2496154745 conn(0x7fed280b5730 msgr2=0x7fed280d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 -- 172.21.15.55:0/2496154745 shutdown_connections 2024-05-10T03:32:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.066+0000 7fed27e00640 1 -- 172.21.15.55:0/2496154745 wait complete. 2024-05-10T03:32:25.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.068+0000 7fed27e00640 1 Processor -- start 2024-05-10T03:32:25.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.069+0000 7fed27e00640 1 -- start start 2024-05-10T03:32:25.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed280b2170 0x7fed281e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 0x7fed2814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed281e9fe0 con 0x7fed281618b0 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2814a0d0 con 0x7fed280b6680 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.070+0000 7fed27e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed281e7f60 con 0x7fed280b2170 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52076/0 (socket says 172.21.15.55:52076) 2024-05-10T03:32:25.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 -- 172.21.15.55:0/2314064040 learned_addr learned my addr 172.21.15.55:0/2314064040 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:25.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed26a00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed280b2170 0x7fed281e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed26000640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 0x7fed2814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 -- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed280b2170 msgr2=0x7fed281e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed280b2170 0x7fed281e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 -- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 msgr2=0x7fed2814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 0x7fed2814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed27400640 1 -- 172.21.15.55:0/2314064040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180094d0 con 0x7fed281618b0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.071+0000 7fed26000640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 0x7fed2814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed27400640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed2000cd80 tx=0x7fed2000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000f680 con 0x7fed281618b0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2000f800 con 0x7fed281618b0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000fbb0 con 0x7fed281618b0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed281e8dc0 con 0x7fed281618b0 2024-05-10T03:32:25.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.072+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed281e8490 con 0x7fed281618b0 2024-05-10T03:32:25.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.073+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed28067740 con 0x7fed281618b0 2024-05-10T03:32:25.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.074+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed2000b040 con 0x7fed281618b0 2024-05-10T03:32:25.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.075+0000 7fed1ea00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 0x7fed0406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:25.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.075+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fed20089e40 con 0x7fed281618b0 2024-05-10T03:32:25.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.076+0000 7fed26a00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 0x7fed0406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:25.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.077+0000 7fed26a00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 0x7fed0406d110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed10006fd0 tx=0x7fed10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:25.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.080+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed20054ea0 con 0x7fed281618b0 2024-05-10T03:32:25.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.315+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed281e8f50 con 0x7fed281618b0 2024-05-10T03:32:25.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.316+0000 7fed1ea00640 1 -- 172.21.15.55:0/2314064040 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fed200584c0 con 0x7fed281618b0 2024-05-10T03:32:25.320 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:25.320 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:25.320 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 msgr2=0x7fed0406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 0x7fed0406d110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fed10006fd0 tx=0x7fed10008040 comp rx=0 tx=0).stop 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 msgr2=0x7fed28148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fed2000cd80 tx=0x7fed2000a650 comp rx=0 tx=0).stop 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 shutdown_connections 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed281618b0 0x7fed28148db0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed0406ac00 0x7fed0406d110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed280b6680 0x7fed2814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 --2- 172.21.15.55:0/2314064040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed280b2170 0x7fed281e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.319+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 >> 172.21.15.55:0/2314064040 conn(0x7fed280b5730 msgr2=0x7fed2815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.320+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 shutdown_connections 2024-05-10T03:32:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:25.320+0000 7fed27e00640 1 -- 172.21.15.55:0/2314064040 wait complete. 2024-05-10T03:32:25.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:32:12.869975+0000', 'modified': '2024-05-10T03:32:16.337855+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6834/3706115975', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 3706115975}, {'type': 'v1', 'addr': '172.21.15.55:6834', 'nonce': 3706115975}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:32:25.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:32:25.755 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T03:32:26.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.044+0000 7f8e55800640 1 Processor -- start 2024-05-10T03:32:26.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.044+0000 7f8e55800640 1 -- start start 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e55800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e5006a4b0 0x7f8e50158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e55800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e500b3860 0x7f8e500b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e55800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e500708b0 con 0x7f8e50071db0 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e55800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e500b2490 con 0x7f8e5006a4b0 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e55800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e50070050 con 0x7f8e500b3860 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e46000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e5006a4b0 0x7f8e50158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e46000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e5006a4b0 0x7f8e50158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52080/0 (socket says 172.21.15.55:52080) 2024-05-10T03:32:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e46000640 1 -- 172.21.15.55:0/3667234136 learned_addr learned my addr 172.21.15.55:0/3667234136 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.047+0000 7f8e4f400640 1 --2- 172.21.15.55:0/3667234136 >> v2:172.21.15.162:3300/0 conn(0x7f8e500b3860 0x7f8e500b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.048+0000 7f8e46000640 1 -- 172.21.15.55:0/3667234136 >> v2:172.21.15.162:3300/0 conn(0x7f8e500b3860 msgr2=0x7f8e500b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.048+0000 7f8e46000640 1 --2- 172.21.15.55:0/3667234136 >> v2:172.21.15.162:3300/0 conn(0x7f8e500b3860 0x7f8e500b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.048+0000 7f8e46000640 1 -- 172.21.15.55:0/3667234136 >> v1:172.21.15.162:6789/0 conn(0x7f8e50071db0 legacy=0x7f8e500ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.048+0000 7f8e46000640 1 -- 172.21.15.55:0/3667234136 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5015b940 con 0x7f8e5006a4b0 2024-05-10T03:32:26.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.049+0000 7f8e46000640 1 --2- 172.21.15.55:0/3667234136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e50158150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8e5015ab50 tx=0x7f8e3802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604242383b0b772b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.050+0000 7f8e4e000640 1 -- 172.21.15.55:0/3667234136 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3802ae20 con 0x7f8e5006a4b0 2024-05-10T03:32:26.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.050+0000 7f8e4e000640 1 -- 172.21.15.55:0/3667234136 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38032440 con 0x7f8e5006a4b0 2024-05-10T03:32:26.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.050+0000 7f8e4e000640 1 -- 172.21.15.55:0/3667234136 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380327d0 con 0x7f8e5006a4b0 2024-05-10T03:32:26.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 -- 172.21.15.55:0/3667234136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 msgr2=0x7f8e50158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 --2- 172.21.15.55:0/3667234136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e50158150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8e5015ab50 tx=0x7f8e3802a110 comp rx=0 tx=0).stop 2024-05-10T03:32:26.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 -- 172.21.15.55:0/3667234136 shutdown_connections 2024-05-10T03:32:26.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 --2- 172.21.15.55:0/3667234136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e50158150 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 --2- 172.21.15.55:0/3667234136 >> v2:172.21.15.162:3300/0 conn(0x7f8e500b3860 0x7f8e500b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 -- 172.21.15.55:0/3667234136 >> 172.21.15.55:0/3667234136 conn(0x7f8e500cbb90 msgr2=0x7f8e500cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 -- 172.21.15.55:0/3667234136 shutdown_connections 2024-05-10T03:32:26.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.051+0000 7f8e55800640 1 -- 172.21.15.55:0/3667234136 wait complete. 2024-05-10T03:32:26.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.053+0000 7f8e55800640 1 Processor -- start 2024-05-10T03:32:26.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.054+0000 7f8e55800640 1 -- start start 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e50071db0 0x7f8e5013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 0x7f8e500afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e501401d0 con 0x7f8e5006a4b0 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50139040 con 0x7f8e500b3860 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e55800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e500c4690 con 0x7f8e50071db0 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.055+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52086/0 (socket says 172.21.15.55:52086) 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 -- 172.21.15.55:0/2103013228 learned_addr learned my addr 172.21.15.55:0/2103013228 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e46000640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e50071db0 0x7f8e5013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4f400640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 0x7f8e500afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 -- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e50071db0 msgr2=0x7f8e5013fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e50071db0 0x7f8e5013fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 -- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 msgr2=0x7f8e500afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 0x7f8e500afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 -- 172.21.15.55:0/2103013228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e380094d0 con 0x7f8e5006a4b0 2024-05-10T03:32:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4f400640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 0x7f8e500afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.056+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b3e0 tx=0x7f8e4000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.057+0000 7f8e47400640 1 -- 172.21.15.55:0/2103013228 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011830 con 0x7f8e5006a4b0 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.057+0000 7f8e47400640 1 -- 172.21.15.55:0/2103013228 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40011990 con 0x7f8e5006a4b0 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.057+0000 7f8e47400640 1 -- 172.21.15.55:0/2103013228 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011d40 con 0x7f8e5006a4b0 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.057+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e500c54f0 con 0x7f8e5006a4b0 2024-05-10T03:32:26.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.057+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e500c4bc0 con 0x7f8e5006a4b0 2024-05-10T03:32:26.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.059+0000 7f8e47400640 1 -- 172.21.15.55:0/2103013228 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e4001a400 con 0x7f8e5006a4b0 2024-05-10T03:32:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.060+0000 7f8e47400640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 0x7f8e2c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.060+0000 7f8e47400640 1 -- 172.21.15.55:0/2103013228 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8e40089690 con 0x7f8e5006a4b0 2024-05-10T03:32:26.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.061+0000 7f8e46000640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 0x7f8e2c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.061+0000 7f8e5c822740 1 Processor -- start 2024-05-10T03:32:26.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.062+0000 7f8e5c822740 1 -- start start 2024-05-10T03:32:26.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.062+0000 7f8e46000640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 0x7f8e2c06d160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e38034000 tx=0x7f8e3802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30852ba70 con 0x55a3081fa3b0 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a308529200 con 0x55a308202590 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a308524af0 con 0x55a308223270 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e46000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e46000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52098/0 (socket says 172.21.15.55:52098) 2024-05-10T03:32:26.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.063+0000 7f8e46000640 1 -- 172.21.15.55:0/673194724 learned_addr learned my addr 172.21.15.55:0/673194724 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e4f400640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 -- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 msgr2=0x55a3085ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 -- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 msgr2=0x55a308607df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 -- 172.21.15.55:0/673194724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3084efb90 con 0x55a3081fa3b0 2024-05-10T03:32:26.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e4f400640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:26.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.064+0000 7f8e46000640 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009930 tx=0x7f8e3802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89c34cc2c2f67b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.065+0000 7f8e3f400640 1 -- 172.21.15.55:0/673194724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3803c890 con 0x55a3081fa3b0 2024-05-10T03:32:26.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.065+0000 7f8e3f400640 1 -- 172.21.15.55:0/673194724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38042070 con 0x55a3081fa3b0 2024-05-10T03:32:26.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.065+0000 7f8e3f400640 1 -- 172.21.15.55:0/673194724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3803cbf0 con 0x55a3081fa3b0 2024-05-10T03:32:26.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 -- 172.21.15.55:0/673194724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 msgr2=0x55a3086078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009930 tx=0x7f8e3802c000 comp rx=0 tx=0).stop 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 -- 172.21.15.55:0/673194724 shutdown_connections 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 --2- 172.21.15.55:0/673194724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 -- 172.21.15.55:0/673194724 >> 172.21.15.55:0/673194724 conn(0x55a3082173c0 msgr2=0x55a3085c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 -- 172.21.15.55:0/673194724 shutdown_connections 2024-05-10T03:32:26.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.066+0000 7f8e5c822740 1 -- 172.21.15.55:0/673194724 wait complete. 2024-05-10T03:32:26.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.067+0000 7f8e5c822740 1 -- start start 2024-05-10T03:32:26.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.067+0000 7f8e5c822740 1 Processor -- start 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3081fa3b0 0x55a3085e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308202590 0x55a308607dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3084c6220 con 0x55a308223270 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3084c5ec0 con 0x55a3081fa3b0 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a30828bf20 con 0x55a308202590 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e5c822740 10 client.0 _ll_register_callbacks cb 0x55a30851e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52102/0 (socket says 172.21.15.55:52102) 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e4f400640 1 -- 172.21.15.55:0/1609629102 learned_addr learned my addr 172.21.15.55:0/1609629102 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.069+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3081fa3b0 0x55a3085e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e46000640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308202590 0x55a308607dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 -- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308202590 msgr2=0x55a308607dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308202590 0x55a308607dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 -- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3081fa3b0 msgr2=0x55a3085e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3081fa3b0 0x55a3085e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 -- 172.21.15.55:0/1609629102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5015b940 con 0x55a308223270 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e4f400640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8e480079a0 tx=0x7f8e48007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319ae53a1a13a6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e33e00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48003da0 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8e480107a0 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.070+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a30854ec90 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e48010960 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 10 client.5889 fetch_fsmap learned FSMap version 24 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a308523170 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a30854ec90 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8e48010ac0 con 0x55a308223270 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 10 client.5889 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T03:32:26.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 10 client.5889 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 20 client.5889 populate_metadata read hostname 'smithi055' 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a3081e1e40 0x55a3086093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 4 client.5889 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.071+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 --> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3082183c0 con 0x55a3081e1e40 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.072+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a3081e1e40 0x55a3086093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.072+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a3081e1e40 0x55a3086093f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d66da374165eac4 server_cookie=ec47a70a77346d09 in_seq=0 out_seq=0 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.072+0000 7f8e33e00640 10 client.5889 ms_handle_connect on v2:172.21.15.55:6832/3706115975 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e33e00640 1 -- 172.21.15.55:0/1609629102 <== mds.0 v2:172.21.15.55:6832/3706115975 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3082183c0 con 0x55a3081e1e40 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e33e00640 10 client.5889 handle_command_reply: tid=0 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 client.5889 shutdown 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 msgr2=0x55a3085bc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8e480079a0 tx=0x7f8e48007e60 comp rx=0 tx=0).stop 2024-05-10T03:32:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 shutdown_connections 2024-05-10T03:32:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a3081e1e40 0x55a3086093f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a3085bc020 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308202590 0x55a308607dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e33e00640 0 client.5889 ms_handle_reset on v2:172.21.15.55:6832/3706115975 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1609629102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3081fa3b0 0x55a3085e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.073+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 >> 172.21.15.55:0/1609629102 conn(0x55a3082173c0 msgr2=0x55a3085c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.074+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 shutdown_connections 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.074+0000 7f8e5c822740 1 -- 172.21.15.55:0/1609629102 wait complete. 2024-05-10T03:32:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.074+0000 7f8e5c822740 20 client.5889 trim_cache size 0 max 16384 2024-05-10T03:32:26.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.092+0000 7f8e5c822740 1 Processor -- start 2024-05-10T03:32:26.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.092+0000 7f8e5c822740 1 -- start start 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a3081fa3b0 0x55a3085c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a3086078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3085ed700 con 0x55a308223270 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3085ed200 con 0x55a308202590 2024-05-10T03:32:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.093+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3085ff190 con 0x55a3081fa3b0 2024-05-10T03:32:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52118/0 (socket says 172.21.15.55:52118) 2024-05-10T03:32:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 -- 172.21.15.55:0/2579987049 learned_addr learned my addr 172.21.15.55:0/2579987049 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a3081fa3b0 0x55a3085c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e46000640 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a3086078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 -- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a3081fa3b0 msgr2=0x55a3085c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a3081fa3b0 0x55a3085c0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 -- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 msgr2=0x55a3086078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a3086078b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 -- 172.21.15.55:0/2579987049 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a308558130 con 0x55a308223270 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.094+0000 7f8e4f400640 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8e48007080 tx=0x7f8e48007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4675de46aed1a2c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.095+0000 7f8e3fe00640 1 -- 172.21.15.55:0/2579987049 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e480109a0 con 0x55a308223270 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.095+0000 7f8e3fe00640 1 -- 172.21.15.55:0/2579987049 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4800f040 con 0x55a308223270 2024-05-10T03:32:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.095+0000 7f8e3fe00640 1 -- 172.21.15.55:0/2579987049 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48015460 con 0x55a308223270 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.095+0000 7f8e5c822740 1 -- 172.21.15.55:0/2579987049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 msgr2=0x55a308607df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.095+0000 7f8e5c822740 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8e48007080 tx=0x7f8e48007f90 comp rx=0 tx=0).stop 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 -- 172.21.15.55:0/2579987049 shutdown_connections 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a308223270 0x55a308607df0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a3086078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 --2- 172.21.15.55:0/2579987049 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a3081fa3b0 0x55a3085c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 -- 172.21.15.55:0/2579987049 >> 172.21.15.55:0/2579987049 conn(0x55a3082173c0 msgr2=0x55a308616e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 -- 172.21.15.55:0/2579987049 shutdown_connections 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.096+0000 7f8e5c822740 1 -- 172.21.15.55:0/2579987049 wait complete. 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.097+0000 7f8e5c822740 1 -- start start 2024-05-10T03:32:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.097+0000 7f8e5c822740 1 Processor -- start 2024-05-10T03:32:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3085f0b40 con 0x55a3081fa3b0 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a308671fa0 con 0x55a308202590 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3085c0820 con 0x55a308223270 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e5c822740 10 client.0 _ll_register_callbacks cb 0x7f8e48009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52132/0 (socket says 172.21.15.55:52132) 2024-05-10T03:32:26.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 -- 172.21.15.55:0/1804280268 learned_addr learned my addr 172.21.15.55:0/1804280268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 -- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 msgr2=0x55a3085e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 -- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 msgr2=0x55a308607df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.099+0000 7f8e4ea00640 1 -- 172.21.15.55:0/1804280268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a308523170 con 0x55a3081fa3b0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e4ea00640 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000bda0 tx=0x7f8e40011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca2d3b4f96a33b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e33e00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40056500 con 0x55a3081fa3b0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8e40086070 con 0x55a3081fa3b0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e38042070 con 0x55a3081fa3b0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e400566a0 con 0x55a3081fa3b0 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e5c822740 10 client.5895 fetch_fsmap learned FSMap version 24 2024-05-10T03:32:26.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55a3084c5430 con 0x55a3081fa3b0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.100+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38042070 con 0x55a3081fa3b0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8e40054a10 con 0x55a3081fa3b0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 10 client.5895 fetch_fsmap finished waiting for FSMap version 24 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 10 client.5895 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 20 client.5895 populate_metadata read hostname 'smithi055' 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a30861a760 0x55a3085e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 4 client.5895 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 --> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3082296c0 con 0x55a30861a760 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.101+0000 7f8e46000640 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a30861a760 0x55a3085e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e46000640 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a30861a760 0x55a3085e7110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6521b1ea79ca1d44 server_cookie=cbf4275864b90ac7 in_seq=0 out_seq=0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e33e00640 10 client.5895 ms_handle_connect on v2:172.21.15.55:6832/3706115975 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e33e00640 1 -- 172.21.15.55:0/1804280268 <== mds.0 v2:172.21.15.55:6832/3706115975 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55a3082296c0 con 0x55a30861a760 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e33e00640 10 client.5895 handle_command_reply: tid=0 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 client.5895 shutdown 2024-05-10T03:32:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 msgr2=0x55a3086078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000bda0 tx=0x7f8e40011f90 comp rx=0 tx=0).stop 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 shutdown_connections 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:6832/3706115975,v1:172.21.15.55:6834/3706115975] conn(0x55a30861a760 0x55a3085e7110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a308223270 0x55a3085e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a308202590 0x55a308607df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 --2- 172.21.15.55:0/1804280268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3081fa3b0 0x55a3086078b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e33e00640 0 client.5895 ms_handle_reset on v2:172.21.15.55:6832/3706115975 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 >> 172.21.15.55:0/1804280268 conn(0x55a3082173c0 msgr2=0x55a308616e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 shutdown_connections 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.102+0000 7f8e5c822740 1 -- 172.21.15.55:0/1804280268 wait complete. 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e5c822740 20 client.5895 trim_cache size 0 max 16384 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 5739, 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:32:26.111 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 13.226804881, 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 24, 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 54, 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 52, 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 18.939295924 2024-05-10T03:32:26.112 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 msgr2=0x7f8e2c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 0x7f8e2c06d160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e38034000 tx=0x7f8e3802e000 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 msgr2=0x7f8e500c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b3e0 tx=0x7f8e4000b8a0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 shutdown_connections 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e2c06ac50 0x7f8e2c06d160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e500b3860 0x7f8e500afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e50071db0 0x7f8e5013fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 --2- 172.21.15.55:0/2103013228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e5006a4b0 0x7f8e500c40b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 >> 172.21.15.55:0/2103013228 conn(0x7f8e500cbb90 msgr2=0x7f8e5006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.103+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 shutdown_connections 2024-05-10T03:32:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.104+0000 7f8e55800640 1 -- 172.21.15.55:0/2103013228 wait complete. 2024-05-10T03:32:26.136 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:32:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.415+0000 7fd91f000640 1 Processor -- start 2024-05-10T03:32:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.416+0000 7fd91f000640 1 -- start start 2024-05-10T03:32:26.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.418+0000 7fd91f000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd91815fe50 0x7fd918141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.418+0000 7fd91f000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9180b2140 0x7fd918142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.418+0000 7fd91f000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91806ad20 con 0x7fd9180b2550 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.418+0000 7fd91f000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd918141f50 con 0x7fd9180b2140 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.418+0000 7fd91f000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91806a110 con 0x7fd91815fe50 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9180b2140 0x7fd918142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9180b2140 0x7fd918142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52138/0 (socket says 172.21.15.55:52138) 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 -- 172.21.15.55:0/2031930031 learned_addr learned my addr 172.21.15.55:0/2031930031 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917e00640 1 --2- 172.21.15.55:0/2031930031 >> v2:172.21.15.162:3300/0 conn(0x7fd91815fe50 0x7fd918141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 -- 172.21.15.55:0/2031930031 >> v2:172.21.15.162:3300/0 conn(0x7fd91815fe50 msgr2=0x7fd918141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 --2- 172.21.15.55:0/2031930031 >> v2:172.21.15.162:3300/0 conn(0x7fd91815fe50 0x7fd918141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 -- 172.21.15.55:0/2031930031 >> v1:172.21.15.162:6789/0 conn(0x7fd9180b2550 legacy=0x7fd91806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.419+0000 7fd917400640 1 -- 172.21.15.55:0/2031930031 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91806a3f0 con 0x7fd9180b2140 2024-05-10T03:32:26.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.420+0000 7fd917400640 1 --2- 172.21.15.55:0/2031930031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9180b2140 0x7fd918142440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd91806e520 tx=0x7fd90402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf878ec7bb4baad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.421+0000 7fd916a00640 1 -- 172.21.15.55:0/2031930031 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904032440 con 0x7fd9180b2140 2024-05-10T03:32:26.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.421+0000 7fd916a00640 1 -- 172.21.15.55:0/2031930031 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040325a0 con 0x7fd9180b2140 2024-05-10T03:32:26.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.421+0000 7fd916a00640 1 -- 172.21.15.55:0/2031930031 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904032930 con 0x7fd9180b2140 2024-05-10T03:32:26.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 -- 172.21.15.55:0/2031930031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9180b2140 msgr2=0x7fd918142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 --2- 172.21.15.55:0/2031930031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9180b2140 0x7fd918142440 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd91806e520 tx=0x7fd90402a2c0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 -- 172.21.15.55:0/2031930031 shutdown_connections 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 --2- 172.21.15.55:0/2031930031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9180b2140 0x7fd918142440 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 --2- 172.21.15.55:0/2031930031 >> v2:172.21.15.162:3300/0 conn(0x7fd91815fe50 0x7fd918141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 -- 172.21.15.55:0/2031930031 >> 172.21.15.55:0/2031930031 conn(0x7fd9180cbb90 msgr2=0x7fd9180cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 -- 172.21.15.55:0/2031930031 shutdown_connections 2024-05-10T03:32:26.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.422+0000 7fd91f000640 1 -- 172.21.15.55:0/2031930031 wait complete. 2024-05-10T03:32:26.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.425+0000 7fd91f000640 1 Processor -- start 2024-05-10T03:32:26.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.425+0000 7fd91f000640 1 -- start start 2024-05-10T03:32:26.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9180b2140 0x7fd918144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9180b2550 0x7fd9181436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180acc70 con 0x7fd91815fe50 2024-05-10T03:32:26.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9181423f0 con 0x7fd9180b2140 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.427+0000 7fd91f000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918143250 con 0x7fd9180b2550 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52150/0 (socket says 172.21.15.55:52150) 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 -- 172.21.15.55:0/992742186 learned_addr learned my addr 172.21.15.55:0/992742186 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd917e00640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9180b2140 0x7fd918144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd917400640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9180b2550 0x7fd9181436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 -- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9180b2550 msgr2=0x7fd9181436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9180b2550 0x7fd9181436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 -- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9180b2140 msgr2=0x7fd918144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9180b2140 0x7fd918144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.428+0000 7fd91d000640 1 -- 172.21.15.55:0/992742186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040094d0 con 0x7fd91815fe50 2024-05-10T03:32:26.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd91d000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b9a0 tx=0x7fd91000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000cba0 con 0x7fd91815fe50 2024-05-10T03:32:26.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91000cd00 con 0x7fd91815fe50 2024-05-10T03:32:26.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910014680 con 0x7fd91815fe50 2024-05-10T03:32:26.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918142640 con 0x7fd91815fe50 2024-05-10T03:32:26.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.429+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918142c00 con 0x7fd91815fe50 2024-05-10T03:32:26.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.431+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918066f60 con 0x7fd91815fe50 2024-05-10T03:32:26.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.432+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd91001d3b0 con 0x7fd91815fe50 2024-05-10T03:32:26.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.434+0000 7fd90fe00640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 0x7fd8f806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:26.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.434+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd910089450 con 0x7fd91815fe50 2024-05-10T03:32:26.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.434+0000 7fd917e00640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 0x7fd8f806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:26.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.436+0000 7fd917e00640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 0x7fd8f806d110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd908006fd0 tx=0x7fd908008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:26.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.438+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd910054380 con 0x7fd91815fe50 2024-05-10T03:32:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.678+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd918067220 con 0x7fd91815fe50 2024-05-10T03:32:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.679+0000 7fd90fe00640 1 -- 172.21.15.55:0/992742186 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd9100579a0 con 0x7fd91815fe50 2024-05-10T03:32:26.682 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.681+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 msgr2=0x7fd8f806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.681+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 0x7fd8f806d110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd908006fd0 tx=0x7fd908008040 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.681+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 msgr2=0x7fd91815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.681+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b9a0 tx=0x7fd91000be60 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 shutdown_connections 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd91815fe50 0x7fd91815ae60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9180b2550 0x7fd9181436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8f806ac00 0x7fd8f806d110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 --2- 172.21.15.55:0/992742186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9180b2140 0x7fd918144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 >> 172.21.15.55:0/992742186 conn(0x7fd9180cbb90 msgr2=0x7fd91813f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:26.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 shutdown_connections 2024-05-10T03:32:26.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:26.682+0000 7fd91f000640 1 -- 172.21.15.55:0/992742186 wait complete. 2024-05-10T03:32:26.711 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:32:26.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:32:26.711 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:32:26.711 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:32:26.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:32:26.712 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:32:26.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:32:26.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:32:26.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:32:26.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:26.712 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:32:26.769 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:32:26.769 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:32:26.769 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:26.770 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42305sec preferred_lft 42305sec 2024-05-10T03:32:26.771 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:32:26.772 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:26.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:26.773 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:32:26.856 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:32:26.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:26.860 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:32:26.919 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:32:26.919 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:32:26.919 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:32:26.919 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:32:26.919 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:32:26.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:26.920 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:32:26.920 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:32:26.920 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:32:26.920 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:32:26.920 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:32:26.921 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:32:27.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:27.084 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:32:27.138 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:32:27.138 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:32:27.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:32:27.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:27.139 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:32:27.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:32:27.226 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:32:27.282 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:32:27.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:32:27.282 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:32:27.337 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:32:27.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:27.337 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:32:27.555 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:32:27.555 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:32:27.555 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:32:27.584 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:32:27.584 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:32:27.584 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:32:27.584 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:32:27.584 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:32:27.617 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:32:27.617 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:32:27.617 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:32:27.617 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:32:27.618 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:32:27.618 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:32:27.618 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:32:27.618 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:32:27.619 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:32:27.620 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:32:27.620 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:32:27.620 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:32:27.620 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:32:27.623 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:32:27.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:27.623 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:32:27.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:27.695 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:32:27.759 INFO:teuthology.orchestra.run.smithi055.stdout:client5030 2024-05-10T03:32:27.759 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:27.759 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:27.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:32:28.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.140+0000 7f40a3e00640 1 Processor -- start 2024-05-10T03:32:28.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.140+0000 7f40a3e00640 1 -- start start 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.142+0000 7f40a3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40a4144710 0x7f40a4144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a3e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 0x7f40a40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a406d350 con 0x7f40a406d7e0 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a406c740 con 0x7f40a4144710 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a3e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a406ca20 con 0x7f40a40b3190 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40a4144710 0x7f40a4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40a4144710 0x7f40a4144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52166/0 (socket says 172.21.15.55:52166) 2024-05-10T03:32:28.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a2a00640 1 -- 172.21.15.55:0/3346646233 learned_addr learned my addr 172.21.15.55:0/3346646233 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a3400640 1 --2- 172.21.15.55:0/3346646233 >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 0x7f40a40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.143+0000 7f40a2a00640 1 -- 172.21.15.55:0/3346646233 >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 msgr2=0x7f40a40cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.144+0000 7f40a2a00640 1 --2- 172.21.15.55:0/3346646233 >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 0x7f40a40cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.144+0000 7f40a2a00640 1 -- 172.21.15.55:0/3346646233 >> v1:172.21.15.162:6789/0 conn(0x7f40a406d7e0 legacy=0x7f40a40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.144+0000 7f40a2a00640 1 -- 172.21.15.55:0/3346646233 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a406cd00 con 0x7f40a4144710 2024-05-10T03:32:28.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.144+0000 7f40a3400640 1 --2- 172.21.15.55:0/3346646233 >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 0x7f40a40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:28.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.145+0000 7f40a2a00640 1 --2- 172.21.15.55:0/3346646233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a4144710 0x7f40a4144120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f40a40b5200 tx=0x7f409402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e62af47483aaf2d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:28.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.146+0000 7f40a1600640 1 -- 172.21.15.55:0/3346646233 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402ae10 con 0x7f40a4144710 2024-05-10T03:32:28.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.146+0000 7f40a1600640 1 -- 172.21.15.55:0/3346646233 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094032440 con 0x7f40a4144710 2024-05-10T03:32:28.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.146+0000 7f40a1600640 1 -- 172.21.15.55:0/3346646233 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940327d0 con 0x7f40a4144710 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.146+0000 7f40a3e00640 1 -- 172.21.15.55:0/3346646233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a4144710 msgr2=0x7f40a4144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.146+0000 7f40a3e00640 1 --2- 172.21.15.55:0/3346646233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a4144710 0x7f40a4144120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f40a40b5200 tx=0x7f409402a100 comp rx=0 tx=0).stop 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 -- 172.21.15.55:0/3346646233 shutdown_connections 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 --2- 172.21.15.55:0/3346646233 >> v2:172.21.15.162:3300/0 conn(0x7f40a40b3190 0x7f40a40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 --2- 172.21.15.55:0/3346646233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a4144710 0x7f40a4144120 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 -- 172.21.15.55:0/3346646233 >> 172.21.15.55:0/3346646233 conn(0x7f40a406a110 msgr2=0x7f40a406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 -- 172.21.15.55:0/3346646233 shutdown_connections 2024-05-10T03:32:28.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.147+0000 7f40a3e00640 1 -- 172.21.15.55:0/3346646233 wait complete. 2024-05-10T03:32:28.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.149+0000 7f40a3e00640 1 Processor -- start 2024-05-10T03:32:28.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.149+0000 7f40a3e00640 1 -- start start 2024-05-10T03:32:28.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40a40b3190 0x7f40a4070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40a4144710 0x7f40a4071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4138af0 con 0x7f40a406d7e0 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41f7b30 con 0x7f40a4144710 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415e9d0 con 0x7f40a40b3190 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52168/0 (socket says 172.21.15.55:52168) 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.151+0000 7f40a2a00640 1 -- 172.21.15.55:0/4014767900 learned_addr learned my addr 172.21.15.55:0/4014767900 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2000640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40a40b3190 0x7f40a4070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a3400640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40a4144710 0x7f40a4071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 -- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40a40b3190 msgr2=0x7f40a4070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:28.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40a40b3190 0x7f40a4070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 -- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40a4144710 msgr2=0x7f40a4071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40a4144710 0x7f40a4071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 -- 172.21.15.55:0/4014767900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940094d0 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f40a2a00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4094009960 tx=0x7f40940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.152+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094030990 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.153+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094030b10 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.153+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409403b400 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.153+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a41f7910 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.153+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a415e6f0 con 0x7f40a406d7e0 2024-05-10T03:32:28.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.154+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4066ce0 con 0x7f40a406d7e0 2024-05-10T03:32:28.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.155+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4094039040 con 0x7f40a406d7e0 2024-05-10T03:32:28.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.156+0000 7f409aa00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 0x7f408406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:28.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.156+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f40940a9dc0 con 0x7f40a406d7e0 2024-05-10T03:32:28.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.157+0000 7f40a2000640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 0x7f408406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:28.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.158+0000 7f40a2000640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 0x7f408406d110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4090006fd0 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:28.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.162+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4094074d80 con 0x7f40a406d7e0 2024-05-10T03:32:28.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.402+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f40a404e290 con 0x7f40a406d7e0 2024-05-10T03:32:28.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.403+0000 7f409aa00640 1 -- 172.21.15.55:0/4014767900 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4094042070 con 0x7f40a406d7e0 2024-05-10T03:32:28.406 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:32:28.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 msgr2=0x7f408406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:28.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 0x7f408406d110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4090006fd0 tx=0x7f4090008040 comp rx=0 tx=0).stop 2024-05-10T03:32:28.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 msgr2=0x7f40a4139120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:28.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4094009960 tx=0x7f40940046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 shutdown_connections 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f408406ac00 0x7f408406d110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40a4144710 0x7f40a4071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40a40b3190 0x7f40a4070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 --2- 172.21.15.55:0/4014767900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40a406d7e0 0x7f40a4139120 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.406+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 >> 172.21.15.55:0/4014767900 conn(0x7f40a406a110 msgr2=0x7f40a40b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:28.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.407+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 shutdown_connections 2024-05-10T03:32:28.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:28.407+0000 7f40a3e00640 1 -- 172.21.15.55:0/4014767900 wait complete. 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:32:28.437 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:32:28.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:32:28.438 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:32:28.438 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:32:28.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.438 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:32:28.495 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:32:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42328sec preferred_lft 42328sec 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:32:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:32:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.498 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:32:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:32:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:32:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:32:28.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.499 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:32:28.583 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:32:28.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.587 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:32:28.640 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:32:28.641 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:32:28.641 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:32:28.641 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:32:28.641 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:32:28.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.642 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:32:28.642 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:32:28.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:32:28.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:32:28.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:32:28.643 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:32:28.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.791 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:32:28.837 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:32:28.837 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:32:28.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:32:28.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:28.837 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:32:28.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:32:28.971 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:32:29.022 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:32:29.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:32:29.022 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:32:29.075 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:32:29.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:29.075 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:32:29.273 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:32:29.273 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:32:29.273 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:32:29.298 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:32:29.298 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:32:29.298 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:32:29.298 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:32:29.298 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:32:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:32:29.336 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:32:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:32:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:32:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:32:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:32:29.340 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:32:29.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:29.340 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:32:29.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:29.416 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:32:29.469 INFO:teuthology.orchestra.run.smithi162.stdout:client5036 2024-05-10T03:32:29.469 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:29.469 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:29.516 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:32:29.516 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:29.630 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:29.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:32:29.744 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:29.744 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:29.806 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:29.807 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:29.860 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:30.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.135+0000 7f8d8ec00640 1 Processor -- start 2024-05-10T03:32:30.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.135+0000 7f8d8ec00640 1 -- start start 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.137+0000 7f8d8ec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d8806d430 0x7f8d88141b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.137+0000 7f8d8ec00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d880b5020 0x7f8d880b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.137+0000 7f8d8ec00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880b4b90 con 0x7f8d880b5970 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.137+0000 7f8d8ec00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d880d0e80 con 0x7f8d880b5020 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.137+0000 7f8d8ec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8806d840 con 0x7f8d8806d430 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d880b5020 0x7f8d880b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d880b5020 0x7f8d880b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52174/0 (socket says 172.21.15.55:52174) 2024-05-10T03:32:30.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 -- 172.21.15.55:0/1375561619 learned_addr learned my addr 172.21.15.55:0/1375561619 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8d800640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d8806d430 0x7f8d88141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 -- 172.21.15.55:0/1375561619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d8806d430 msgr2=0x7f8d88141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d8806d430 0x7f8d88141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 -- 172.21.15.55:0/1375561619 >> v1:172.21.15.162:6789/0 conn(0x7f8d880b5970 legacy=0x7f8d8815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.138+0000 7f8d8ce00640 1 -- 172.21.15.55:0/1375561619 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880b5d80 con 0x7f8d880b5020 2024-05-10T03:32:30.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.140+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d880b5020 0x7f8d880b5430 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009870 tx=0x7f8d7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f687c520bd28938 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.140+0000 7f8d87e00640 1 -- 172.21.15.55:0/1375561619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74032440 con 0x7f8d880b5020 2024-05-10T03:32:30.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.140+0000 7f8d87e00640 1 -- 172.21.15.55:0/1375561619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d740325a0 con 0x7f8d880b5020 2024-05-10T03:32:30.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.140+0000 7f8d87e00640 1 -- 172.21.15.55:0/1375561619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74032930 con 0x7f8d880b5020 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1375561619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d880b5020 msgr2=0x7f8d880b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d880b5020 0x7f8d880b5430 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009870 tx=0x7f8d7402a210 comp rx=0 tx=0).stop 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1375561619 shutdown_connections 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d880b5020 0x7f8d880b5430 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1375561619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d8806d430 0x7f8d88141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1375561619 >> 172.21.15.55:0/1375561619 conn(0x7f8d880b6680 msgr2=0x7f8d880b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1375561619 shutdown_connections 2024-05-10T03:32:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.141+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1375561619 wait complete. 2024-05-10T03:32:30.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.143+0000 7f8d8ec00640 1 Processor -- start 2024-05-10T03:32:30.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.144+0000 7f8d8ec00640 1 -- start start 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 0x7f8d880c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 0x7f8d880afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880b01b0 con 0x7f8d8806d430 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88137e40 con 0x7f8d880b5970 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8ec00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880ae980 con 0x7f8d880b5020 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52188/0 (socket says 172.21.15.55:52188) 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.145+0000 7f8d8d800640 1 -- 172.21.15.55:0/1600180324 learned_addr learned my addr 172.21.15.55:0/1600180324 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:30.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 0x7f8d880c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8e200640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 0x7f8d880afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 -- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 msgr2=0x7f8d880c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 0x7f8d880c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 -- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 msgr2=0x7f8d880afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 0x7f8d880afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 -- 172.21.15.55:0/1600180324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740094d0 con 0x7f8d8806d430 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8e200640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 0x7f8d880afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:30.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 0x7f8d880c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.146+0000 7f8d8d800640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00d600 tx=0x7f8d7c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.147+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004240 con 0x7f8d8806d430 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.147+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0043a0 con 0x7f8d8806d430 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.147+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c003830 con 0x7f8d8806d430 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.147+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d880aebd0 con 0x7f8d8806d430 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.147+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d880af190 con 0x7f8d8806d430 2024-05-10T03:32:30.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.148+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880ced70 con 0x7f8d8806d430 2024-05-10T03:32:30.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.149+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d7c003990 con 0x7f8d8806d430 2024-05-10T03:32:30.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.150+0000 7f8d85600640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 0x7f8d6806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.150+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8d7c089740 con 0x7f8d8806d430 2024-05-10T03:32:30.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.151+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 0x7f8d6806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.152+0000 7f8d8ce00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 0x7f8d6806d110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d74034000 tx=0x7f8d7402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.155+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8d7c054710 con 0x7f8d8806d430 2024-05-10T03:32:30.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.355+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d8804e290 con 0x7f8d8806d430 2024-05-10T03:32:30.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.358+0000 7f8d85600640 1 -- 172.21.15.55:0/1600180324 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16345 (secure 0 0 0) 0x7f8d7c057d30 con 0x7f8d8806d430 2024-05-10T03:32:30.363 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:30.363 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:24.417028+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:30.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 msgr2=0x7f8d6806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 0x7f8d6806d110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d74034000 tx=0x7f8d7402e000 comp rx=0 tx=0).stop 2024-05-10T03:32:30.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 msgr2=0x7f8d880ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00d600 tx=0x7f8d7c00dac0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 shutdown_connections 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d6806ac00 0x7f8d6806d110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d880b5970 0x7f8d880afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d880b5020 0x7f8d880c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 --2- 172.21.15.55:0/1600180324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d8806d430 0x7f8d880ae410 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 >> 172.21.15.55:0/1600180324 conn(0x7f8d880b6680 msgr2=0x7f8d8815d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 shutdown_connections 2024-05-10T03:32:30.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.361+0000 7f8d8ec00640 1 -- 172.21.15.55:0/1600180324 wait complete. 2024-05-10T03:32:30.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:11.083041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:11.883212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:30.395 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:30.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.681+0000 7f97cf200640 1 Processor -- start 2024-05-10T03:32:30.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.681+0000 7f97cf200640 1 -- start start 2024-05-10T03:32:30.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cf200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f97c806b830 0x7f97c80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cf200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c80b2a30 con 0x7f97c815e920 2024-05-10T03:32:30.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cf200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c815eec0 con 0x7f97c80cdb70 2024-05-10T03:32:30.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cf200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c815a900 con 0x7f97c806b830 2024-05-10T03:32:30.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cde00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f97c815e920 0x7f97c806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45460/0 (socket says 172.21.15.55:45460) 2024-05-10T03:32:30.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.684+0000 7f97cde00640 1 -- 172.21.15.55:0/1664406693 learned_addr learned my addr 172.21.15.55:0/1664406693 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:30.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.685+0000 7f97cd400640 1 --2- 172.21.15.55:0/1664406693 >> v2:172.21.15.162:3300/0 conn(0x7f97c806b830 0x7f97c80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.685+0000 7f97cca00640 1 -- 172.21.15.55:0/1664406693 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255178649 0 0) 0x7f97c80b2a30 con 0x7f97c815e920 2024-05-10T03:32:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.685+0000 7f97cca00640 1 -- 172.21.15.55:0/1664406693 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b0005730 con 0x7f97c815e920 2024-05-10T03:32:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.685+0000 7f97cd400640 1 -- 172.21.15.55:0/1664406693 >> v1:172.21.15.162:6789/0 conn(0x7f97c80cdb70 legacy=0x7f97c80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.685+0000 7f97cd400640 1 -- 172.21.15.55:0/1664406693 >> v1:172.21.15.55:6789/0 conn(0x7f97c815e920 legacy=0x7f97c806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.686+0000 7f97cd400640 1 -- 172.21.15.55:0/1664406693 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c815b8b0 con 0x7f97c806b830 2024-05-10T03:32:30.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.687+0000 7f97cd400640 1 --2- 172.21.15.55:0/1664406693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c806b830 0x7f97c80cc240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b90353ea30b241 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.687+0000 7f97cca00640 1 -- 172.21.15.55:0/1664406693 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8032440 con 0x7f97c806b830 2024-05-10T03:32:30.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.687+0000 7f97cca00640 1 -- 172.21.15.55:0/1664406693 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b80325a0 con 0x7f97c806b830 2024-05-10T03:32:30.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.687+0000 7f97cca00640 1 -- 172.21.15.55:0/1664406693 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8032930 con 0x7f97c806b830 2024-05-10T03:32:30.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 -- 172.21.15.55:0/1664406693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c806b830 msgr2=0x7f97c80cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 --2- 172.21.15.55:0/1664406693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c806b830 0x7f97c80cc240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b802a230 comp rx=0 tx=0).stop 2024-05-10T03:32:30.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 -- 172.21.15.55:0/1664406693 shutdown_connections 2024-05-10T03:32:30.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 --2- 172.21.15.55:0/1664406693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c806b830 0x7f97c80cc240 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 -- 172.21.15.55:0/1664406693 >> 172.21.15.55:0/1664406693 conn(0x7f97c806c750 msgr2=0x7f97c80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:30.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 -- 172.21.15.55:0/1664406693 shutdown_connections 2024-05-10T03:32:30.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.688+0000 7f97cf200640 1 -- 172.21.15.55:0/1664406693 wait complete. 2024-05-10T03:32:30.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.691+0000 7f97cf200640 1 Processor -- start 2024-05-10T03:32:30.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.691+0000 7f97cf200640 1 -- start start 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 0x7f97c80c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c81596e0 0x7f97c8070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c9420 con 0x7f97c80cdb70 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80cab80 con 0x7f97c81596e0 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cf200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8142af0 con 0x7f97c815e920 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cde00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cde00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52192/0 (socket says 172.21.15.55:52192) 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cde00640 1 -- 172.21.15.55:0/1562051382 learned_addr learned my addr 172.21.15.55:0/1562051382 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97cd400640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 0x7f97c80c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.693+0000 7f97ce800640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c81596e0 0x7f97c8070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 -- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 msgr2=0x7f97c80c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 0x7f97c80c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 -- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c81596e0 msgr2=0x7f97c8070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c81596e0 0x7f97c8070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 -- 172.21.15.55:0/1562051382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009400 con 0x7f97c80cdb70 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cd400640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 0x7f97c80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cde00640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c440 tx=0x7f97bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc005ce0 con 0x7f97c80cdb70 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc005e40 con 0x7f97c80cdb70 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0072b0 con 0x7f97c80cdb70 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.694+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c812f770 con 0x7f97c80cdb70 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.695+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c81e3d60 con 0x7f97c80cdb70 2024-05-10T03:32:30.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.696+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c80c9690 con 0x7f97c80cdb70 2024-05-10T03:32:30.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.697+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f97bc007410 con 0x7f97c80cdb70 2024-05-10T03:32:30.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.698+0000 7f97c2000640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 0x7f97ac06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:30.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.698+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f97bc088aa0 con 0x7f97c80cdb70 2024-05-10T03:32:30.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.699+0000 7f97cd400640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 0x7f97ac06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:30.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.700+0000 7f97cd400640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 0x7f97ac06d140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f97b8034000 tx=0x7f97b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:30.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.703+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f97bc053b20 con 0x7f97c80cdb70 2024-05-10T03:32:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.902+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97c804e290 con 0x7f97c80cdb70 2024-05-10T03:32:30.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.903+0000 7f97c2000640 1 -- 172.21.15.55:0/1562051382 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97bc017020 con 0x7f97c80cdb70 2024-05-10T03:32:30.907 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:30.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 msgr2=0x7f97ac06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 0x7f97ac06d140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f97b8034000 tx=0x7f97b802e000 comp rx=0 tx=0).stop 2024-05-10T03:32:30.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 msgr2=0x7f97c8142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:30.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c440 tx=0x7f97bc00c900 comp rx=0 tx=0).stop 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 shutdown_connections 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f97ac06ac30 0x7f97ac06d140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97c81596e0 0x7f97c8070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97c815e920 0x7f97c80c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 --2- 172.21.15.55:0/1562051382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97c80cdb70 0x7f97c8142580 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 >> 172.21.15.55:0/1562051382 conn(0x7f97c806c750 msgr2=0x7f97c815ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 shutdown_connections 2024-05-10T03:32:30.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:30.906+0000 7f97cf200640 1 -- 172.21.15.55:0/1562051382 wait complete. 2024-05-10T03:32:30.937 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:32:31.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.223+0000 7f71b6a00640 1 Processor -- start 2024-05-10T03:32:31.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.223+0000 7f71b6a00640 1 -- start start 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.225+0000 7f71b6a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71b00719f0 0x7f71b006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b6a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 0x7f71b0141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b6a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b015fdf0 con 0x7f71b006b840 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b6a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b015f1e0 con 0x7f71b00719f0 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b6a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b015f4c0 con 0x7f71b0071e00 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b5600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71b00719f0 0x7f71b006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b5600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71b00719f0 0x7f71b006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52204/0 (socket says 172.21.15.55:52204) 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b5600640 1 -- 172.21.15.55:0/1714140084 learned_addr learned my addr 172.21.15.55:0/1714140084 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.226+0000 7f71b6000640 1 --2- 172.21.15.55:0/1714140084 >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 0x7f71b0141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.227+0000 7f71b5600640 1 -- 172.21.15.55:0/1714140084 >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 msgr2=0x7f71b0141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.227+0000 7f71b5600640 1 --2- 172.21.15.55:0/1714140084 >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 0x7f71b0141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.227+0000 7f71b5600640 1 -- 172.21.15.55:0/1714140084 >> v1:172.21.15.162:6789/0 conn(0x7f71b006b840 legacy=0x7f71b00700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.227+0000 7f71b5600640 1 -- 172.21.15.55:0/1714140084 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b015f7a0 con 0x7f71b00719f0 2024-05-10T03:32:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.227+0000 7f71b6000640 1 --2- 172.21.15.55:0/1714140084 >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 0x7f71b0141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.228+0000 7f71b5600640 1 --2- 172.21.15.55:0/1714140084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b006b300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ced30 tx=0x7f71a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dc8f8a85391a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.228+0000 7f71afe00640 1 -- 172.21.15.55:0/1714140084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4032440 con 0x7f71b00719f0 2024-05-10T03:32:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.228+0000 7f71afe00640 1 -- 172.21.15.55:0/1714140084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a40325a0 con 0x7f71b00719f0 2024-05-10T03:32:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.228+0000 7f71afe00640 1 -- 172.21.15.55:0/1714140084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4032930 con 0x7f71b00719f0 2024-05-10T03:32:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 -- 172.21.15.55:0/1714140084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 msgr2=0x7f71b006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 --2- 172.21.15.55:0/1714140084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b006b300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ced30 tx=0x7f71a402a1a0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 -- 172.21.15.55:0/1714140084 shutdown_connections 2024-05-10T03:32:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 --2- 172.21.15.55:0/1714140084 >> v2:172.21.15.162:3300/0 conn(0x7f71b0071e00 0x7f71b0141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 --2- 172.21.15.55:0/1714140084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b006b300 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 -- 172.21.15.55:0/1714140084 >> 172.21.15.55:0/1714140084 conn(0x7f71b006c750 msgr2=0x7f71b00b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:31.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 -- 172.21.15.55:0/1714140084 shutdown_connections 2024-05-10T03:32:31.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.229+0000 7f71b6a00640 1 -- 172.21.15.55:0/1714140084 wait complete. 2024-05-10T03:32:31.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.232+0000 7f71b6a00640 1 Processor -- start 2024-05-10T03:32:31.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.232+0000 7f71b6a00640 1 -- start start 2024-05-10T03:32:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.233+0000 7f71b6a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71b006b840 0x7f71b013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71b0071e00 0x7f71b00b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0155880 con 0x7f71b00719f0 2024-05-10T03:32:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00ac8e0 con 0x7f71b0071e00 2024-05-10T03:32:31.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0159540 con 0x7f71b006b840 2024-05-10T03:32:31.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52216/0 (socket says 172.21.15.55:52216) 2024-05-10T03:32:31.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 -- 172.21.15.55:0/4121452451 learned_addr learned my addr 172.21.15.55:0/4121452451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:31.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b5600640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71b006b840 0x7f71b013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b6000640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71b0071e00 0x7f71b00b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 -- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71b006b840 msgr2=0x7f71b013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71b006b840 0x7f71b013f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 -- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71b0071e00 msgr2=0x7f71b00b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71b0071e00 0x7f71b00b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.234+0000 7f71b4c00640 1 -- 172.21.15.55:0/4121452451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a40094d0 con 0x7f71b00719f0 2024-05-10T03:32:31.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71b4c00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f719c00b3c0 tx=0x7f719c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c011830 con 0x7f71b00719f0 2024-05-10T03:32:31.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c011990 con 0x7f71b00719f0 2024-05-10T03:32:31.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c011d80 con 0x7f71b00719f0 2024-05-10T03:32:31.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00acb30 con 0x7f71b00719f0 2024-05-10T03:32:31.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.235+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b00ad0f0 con 0x7f71b00719f0 2024-05-10T03:32:31.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.237+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b004e290 con 0x7f71b00719f0 2024-05-10T03:32:31.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.238+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f719c01b460 con 0x7f71b00719f0 2024-05-10T03:32:31.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.239+0000 7f71ad600640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 0x7f719006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.239+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f719c089140 con 0x7f71b00719f0 2024-05-10T03:32:31.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.240+0000 7f71b5600640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 0x7f719006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.241+0000 7f71b5600640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 0x7f719006d160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71a4034000 tx=0x7f71a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.243+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f719c0540f0 con 0x7f71b00719f0 2024-05-10T03:32:31.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.439+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71b0140e00 con 0x7f71b00719f0 2024-05-10T03:32:31.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.440+0000 7f71ad600640 1 -- 172.21.15.55:0/4121452451 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f719c057710 con 0x7f71b00719f0 2024-05-10T03:32:31.445 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:32:31.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 msgr2=0x7f719006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 0x7f719006d160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71a4034000 tx=0x7f71a402e000 comp rx=0 tx=0).stop 2024-05-10T03:32:31.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 msgr2=0x7f71b00adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f719c00b3c0 tx=0x7f719c00b880 comp rx=0 tx=0).stop 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 shutdown_connections 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f719006ac50 0x7f719006d160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71b0071e00 0x7f71b00b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71b00719f0 0x7f71b00adbd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 --2- 172.21.15.55:0/4121452451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71b006b840 0x7f71b013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 >> 172.21.15.55:0/4121452451 conn(0x7f71b006c750 msgr2=0x7f71b015e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 shutdown_connections 2024-05-10T03:32:31.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.443+0000 7f71b6a00640 1 -- 172.21.15.55:0/4121452451 wait complete. 2024-05-10T03:32:31.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:32:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.762+0000 7f1833200640 1 Processor -- start 2024-05-10T03:32:31.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.762+0000 7f1833200640 1 -- start start 2024-05-10T03:32:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.765+0000 7f1833200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1834162bf0 0x7f18340ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.765+0000 7f1833200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.765+0000 7f1833200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340cec70 con 0x7f183406b120 2024-05-10T03:32:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.765+0000 7f1833200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340ce990 con 0x7f1834162bf0 2024-05-10T03:32:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.765+0000 7f1833200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1834162110 con 0x7f183415f490 2024-05-10T03:32:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.766+0000 7f1831400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.766+0000 7f1831e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1834162bf0 0x7f18340ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.766+0000 7f1831400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:34742/0 (socket says 172.21.15.55:34742) 2024-05-10T03:32:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.766+0000 7f1831400640 1 -- 172.21.15.55:0/329038656 learned_addr learned my addr 172.21.15.55:0/329038656 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:31.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.767+0000 7f1831400640 1 -- 172.21.15.55:0/329038656 >> v2:172.21.15.162:3300/0 conn(0x7f1834162bf0 msgr2=0x7f18340ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:31.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.767+0000 7f1831400640 1 --2- 172.21.15.55:0/329038656 >> v2:172.21.15.162:3300/0 conn(0x7f1834162bf0 0x7f18340ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.767+0000 7f1831400640 1 -- 172.21.15.55:0/329038656 >> v1:172.21.15.162:6789/0 conn(0x7f183406b120 legacy=0x7f183406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:31.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.767+0000 7f1831400640 1 -- 172.21.15.55:0/329038656 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18341623f0 con 0x7f183415f490 2024-05-10T03:32:31.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.768+0000 7f1831400640 1 --2- 172.21.15.55:0/329038656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1820009850 tx=0x7f182002a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b261a96f8dc294 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.777+0000 7f1830a00640 1 -- 172.21.15.55:0/329038656 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182002ade0 con 0x7f183415f490 2024-05-10T03:32:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.777+0000 7f1830a00640 1 -- 172.21.15.55:0/329038656 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820004030 con 0x7f183415f490 2024-05-10T03:32:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.777+0000 7f1830a00640 1 -- 172.21.15.55:0/329038656 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820004420 con 0x7f183415f490 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 -- 172.21.15.55:0/329038656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 msgr2=0x7f18340b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 --2- 172.21.15.55:0/329038656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1820009850 tx=0x7f182002a0d0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 -- 172.21.15.55:0/329038656 shutdown_connections 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 --2- 172.21.15.55:0/329038656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f18340b3d70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 --2- 172.21.15.55:0/329038656 >> v2:172.21.15.162:3300/0 conn(0x7f1834162bf0 0x7f18340ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 -- 172.21.15.55:0/329038656 >> 172.21.15.55:0/329038656 conn(0x7f183406c730 msgr2=0x7f18340b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 -- 172.21.15.55:0/329038656 shutdown_connections 2024-05-10T03:32:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.778+0000 7f1833200640 1 -- 172.21.15.55:0/329038656 wait complete. 2024-05-10T03:32:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.781+0000 7f1833200640 1 Processor -- start 2024-05-10T03:32:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.781+0000 7f1833200640 1 -- start start 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f1834158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1834162bf0 0x7f1834158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340afd30 con 0x7f183406b120 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834163380 con 0x7f1834162bf0 2024-05-10T03:32:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.783+0000 7f1833200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340aff80 con 0x7f183415f490 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52226/0 (socket says 172.21.15.55:52226) 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 -- 172.21.15.55:0/4039530141 learned_addr learned my addr 172.21.15.55:0/4039530141 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831400640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f1834158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1832800640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1834162bf0 0x7f1834158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 -- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 msgr2=0x7f1834158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f1834158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 -- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1834162bf0 msgr2=0x7f1834158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1834162bf0 0x7f1834158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.784+0000 7f1831e00640 1 -- 172.21.15.55:0/4039530141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820009460 con 0x7f183406b120 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f1831e00640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f182400e840 tx=0x7f182400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240094d0 con 0x7f183406b120 2024-05-10T03:32:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824004240 con 0x7f183406b120 2024-05-10T03:32:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1834163640 con 0x7f183406b120 2024-05-10T03:32:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400c730 con 0x7f183406b120 2024-05-10T03:32:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.785+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1834163c00 con 0x7f183406b120 2024-05-10T03:32:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.786+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18340b44b0 con 0x7f183406b120 2024-05-10T03:32:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.788+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f182400c890 con 0x7f183406b120 2024-05-10T03:32:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.789+0000 7f182a000640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 0x7f181406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.789+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f18240898e0 con 0x7f183406b120 2024-05-10T03:32:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.790+0000 7f1831400640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 0x7f181406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.791+0000 7f1831400640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 0x7f181406d110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1820033000 tx=0x7f182002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:31.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.794+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1824054810 con 0x7f183406b120 2024-05-10T03:32:31.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.993+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f183404e290 con 0x7f183406b120 2024-05-10T03:32:31.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.994+0000 7f182a000640 1 -- 172.21.15.55:0/4039530141 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1824017070 con 0x7f183406b120 2024-05-10T03:32:31.999 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.997+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 msgr2=0x7f181406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.997+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 0x7f181406d110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1820033000 tx=0x7f182002e000 comp rx=0 tx=0).stop 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.997+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 msgr2=0x7f1834149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.997+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f182400e840 tx=0x7f182400ed00 comp rx=0 tx=0).stop 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 shutdown_connections 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f181406ac00 0x7f181406d110 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1834162bf0 0x7f1834158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415f490 0x7f1834158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 --2- 172.21.15.55:0/4039530141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f183406b120 0x7f1834149310 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 >> 172.21.15.55:0/4039530141 conn(0x7f183406c730 msgr2=0x7f18340b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:32.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 shutdown_connections 2024-05-10T03:32:32.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:31.998+0000 7f1833200640 1 -- 172.21.15.55:0/4039530141 wait complete. 2024-05-10T03:32:32.029 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:32:32.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.313+0000 7fbb2a400640 1 Processor -- start 2024-05-10T03:32:32.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.314+0000 7fbb2a400640 1 -- start start 2024-05-10T03:32:32.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.316+0000 7fbb2a400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb2415b9d0 0x7fbb2406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:32.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.316+0000 7fbb2a400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2415bde0 con 0x7fbb2406b660 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.316+0000 7fbb2a400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2413eae0 con 0x7fbb2413e390 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.316+0000 7fbb2a400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2413e7a0 con 0x7fbb2415b9d0 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb2415b9d0 0x7fbb2406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb2415b9d0 0x7fbb2406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52234/0 (socket says 172.21.15.55:52234) 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 -- 172.21.15.55:0/1736346914 learned_addr learned my addr 172.21.15.55:0/1736346914 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:32.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 -- 172.21.15.55:0/1736346914 >> v1:172.21.15.162:6789/0 conn(0x7fbb2413e390 legacy=0x7fbb241414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 -- 172.21.15.55:0/1736346914 >> v1:172.21.15.55:6789/0 conn(0x7fbb2406b660 legacy=0x7fbb2413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.317+0000 7fbb23e00640 1 -- 172.21.15.55:0/1736346914 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2406ba70 con 0x7fbb2415b9d0 2024-05-10T03:32:32.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.318+0000 7fbb23e00640 1 --2- 172.21.15.55:0/1736346914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2415b9d0 0x7fbb2406b120 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009850 tx=0x7fbb1002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f09b444d03227 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:32.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.319+0000 7fbb23400640 1 -- 172.21.15.55:0/1736346914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004030 con 0x7fbb2415b9d0 2024-05-10T03:32:32.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.319+0000 7fbb23400640 1 -- 172.21.15.55:0/1736346914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10004190 con 0x7fbb2415b9d0 2024-05-10T03:32:32.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.319+0000 7fbb23400640 1 -- 172.21.15.55:0/1736346914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10003610 con 0x7fbb2415b9d0 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.319+0000 7fbb2a400640 1 -- 172.21.15.55:0/1736346914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2415b9d0 msgr2=0x7fbb2406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.319+0000 7fbb2a400640 1 --2- 172.21.15.55:0/1736346914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2415b9d0 0x7fbb2406b120 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009850 tx=0x7fbb1002a250 comp rx=0 tx=0).stop 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.320+0000 7fbb2a400640 1 -- 172.21.15.55:0/1736346914 shutdown_connections 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.320+0000 7fbb2a400640 1 --2- 172.21.15.55:0/1736346914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2415b9d0 0x7fbb2406b120 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.320+0000 7fbb2a400640 1 -- 172.21.15.55:0/1736346914 >> 172.21.15.55:0/1736346914 conn(0x7fbb2406c730 msgr2=0x7fbb240b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.320+0000 7fbb2a400640 1 -- 172.21.15.55:0/1736346914 shutdown_connections 2024-05-10T03:32:32.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.320+0000 7fbb2a400640 1 -- 172.21.15.55:0/1736346914 wait complete. 2024-05-10T03:32:32.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.322+0000 7fbb2a400640 1 Processor -- start 2024-05-10T03:32:32.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.322+0000 7fbb2a400640 1 -- start start 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 0x7fbb24152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 0x7fbb24138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24143bd0 con 0x7fbb2413e390 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24153160 con 0x7fbb2415b9d0 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb2a400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24153fc0 con 0x7fbb2406b660 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52236/0 (socket says 172.21.15.55:52236) 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.324+0000 7fbb23e00640 1 -- 172.21.15.55:0/3901649528 learned_addr learned my addr 172.21.15.55:0/3901649528 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:32.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb29000640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 0x7fbb24152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb29a00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 0x7fbb24138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 -- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 msgr2=0x7fbb24152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 0x7fbb24152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 -- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 msgr2=0x7fbb24138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 0x7fbb24138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 -- 172.21.15.55:0/3901649528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb10009460 con 0x7fbb2413e390 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb29000640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 0x7fbb24152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb29a00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 0x7fbb24138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:32.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb23e00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034040 tx=0x7fbb100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.325+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004030 con 0x7fbb2413e390 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.326+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10003bd0 con 0x7fbb2413e390 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.326+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10003d60 con 0x7fbb2413e390 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.326+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb24143700 con 0x7fbb2413e390 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.326+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb24153690 con 0x7fbb2413e390 2024-05-10T03:32:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.327+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2404e290 con 0x7fbb2413e390 2024-05-10T03:32:32.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.328+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb10006040 con 0x7fbb2413e390 2024-05-10T03:32:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.329+0000 7fbb20c00640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 0x7fbb0406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.329+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fbb100aa6b0 con 0x7fbb2413e390 2024-05-10T03:32:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.330+0000 7fbb29000640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 0x7fbb0406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.331+0000 7fbb29000640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 0x7fbb0406d110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:32.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.334+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb10079070 con 0x7fbb2413e390 2024-05-10T03:32:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.551+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb240712e0 con 0x7fbb2413e390 2024-05-10T03:32:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.552+0000 7fbb20c00640 1 -- 172.21.15.55:0/3901649528 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb10041080 con 0x7fbb2413e390 2024-05-10T03:32:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:32:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 msgr2=0x7fbb0406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 0x7fbb0406d110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2024-05-10T03:32:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 msgr2=0x7fbb241431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034040 tx=0x7fbb100046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 shutdown_connections 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb0406ac00 0x7fbb0406d110 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb2415b9d0 0x7fbb24138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb2413e390 0x7fbb241431c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 --2- 172.21.15.55:0/3901649528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb2406b660 0x7fbb24152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.555+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 >> 172.21.15.55:0/3901649528 conn(0x7fbb2406c730 msgr2=0x7fbb2406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.556+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 shutdown_connections 2024-05-10T03:32:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:32.556+0000 7fbb2a400640 1 -- 172.21.15.55:0/3901649528 wait complete. 2024-05-10T03:32:32.587 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:32:32.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:32.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:32:32.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:32:32.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:32.685 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:32:32.738 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:32.738 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:32.794 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:32.794 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:32.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:33.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.133+0000 7f9e45000640 1 Processor -- start 2024-05-10T03:32:33.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.133+0000 7f9e45000640 1 -- start start 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.135+0000 7f9e45000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e40144710 0x7f9e40144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e45000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e400b3190 0x7f9e400cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e45000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4006d350 con 0x7f9e4006d7e0 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e45000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4006c740 con 0x7f9e400b3190 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e45000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4006ca20 con 0x7f9e40144710 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e3fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e400b3190 0x7f9e400cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e3fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e400b3190 0x7f9e400cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43694/0 (socket says 172.21.15.55:43694) 2024-05-10T03:32:33.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2530811558 learned_addr learned my addr 172.21.15.55:0/2530811558 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:33.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.136+0000 7f9e3f400640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e40144710 0x7f9e40144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.137+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2530811558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e40144710 msgr2=0x7f9e40144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:33.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.137+0000 7f9e3fe00640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e40144710 0x7f9e40144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.137+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2530811558 >> v1:172.21.15.162:6789/0 conn(0x7f9e4006d7e0 legacy=0x7f9e400b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:33.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.137+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2530811558 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4006cd00 con 0x7f9e400b3190 2024-05-10T03:32:33.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.138+0000 7f9e3fe00640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e400b3190 0x7f9e400cd9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009e50 tx=0x7f9e3802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d1fe8f81f85546 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.138+0000 7f9e3ea00640 1 -- 172.21.15.55:0/2530811558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38031600 con 0x7f9e400b3190 2024-05-10T03:32:33.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.138+0000 7f9e3ea00640 1 -- 172.21.15.55:0/2530811558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3802d050 con 0x7f9e400b3190 2024-05-10T03:32:33.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.138+0000 7f9e3ea00640 1 -- 172.21.15.55:0/2530811558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38031980 con 0x7f9e400b3190 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 -- 172.21.15.55:0/2530811558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e400b3190 msgr2=0x7f9e400cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e400b3190 0x7f9e400cd9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009e50 tx=0x7f9e3802a5b0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 -- 172.21.15.55:0/2530811558 shutdown_connections 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e400b3190 0x7f9e400cd9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 --2- 172.21.15.55:0/2530811558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e40144710 0x7f9e40144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.139+0000 7f9e45000640 1 -- 172.21.15.55:0/2530811558 >> 172.21.15.55:0/2530811558 conn(0x7f9e4006a110 msgr2=0x7f9e4006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.140+0000 7f9e45000640 1 -- 172.21.15.55:0/2530811558 shutdown_connections 2024-05-10T03:32:33.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.140+0000 7f9e45000640 1 -- 172.21.15.55:0/2530811558 wait complete. 2024-05-10T03:32:33.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.142+0000 7f9e45000640 1 Processor -- start 2024-05-10T03:32:33.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.142+0000 7f9e45000640 1 -- start start 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 0x7f9e40158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e400b3190 0x7f9e40157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4015d870 con 0x7f9e40144710 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401591f0 con 0x7f9e400b3190 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e45000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4015c0a0 con 0x7f9e4006d7e0 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43698/0 (socket says 172.21.15.55:43698) 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.144+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2451246933 learned_addr learned my addr 172.21.15.55:0/2451246933 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e37e00640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e400b3190 0x7f9e40157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3f400640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 0x7f9e40158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 msgr2=0x7f9e40158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 0x7f9e40158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e400b3190 msgr2=0x7f9e40157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e400b3190 0x7f9e40157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 -- 172.21.15.55:0/2451246933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e38009b30 con 0x7f9e40144710 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3f400640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 0x7f9e40158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e3fe00640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9e38034040 tx=0x7f9e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3803b820 con 0x7f9e40144710 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3803b9a0 con 0x7f9e40144710 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.145+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3803bd50 con 0x7f9e40144710 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.146+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4013c290 con 0x7f9e40144710 2024-05-10T03:32:33.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.146+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e40159440 con 0x7f9e40144710 2024-05-10T03:32:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.147+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4015c250 con 0x7f9e40144710 2024-05-10T03:32:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.148+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9e38038050 con 0x7f9e40144710 2024-05-10T03:32:33.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.149+0000 7f9e37400640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 0x7f9e1c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.149+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f9e380aa0b0 con 0x7f9e40144710 2024-05-10T03:32:33.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.150+0000 7f9e3f400640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 0x7f9e1c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.151+0000 7f9e3f400640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 0x7f9e1c06d110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e30006fd0 tx=0x7f9e30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.154+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9e4015c250 con 0x7f9e40144710 2024-05-10T03:32:33.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.353+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e40066ce0 con 0x7f9e40144710 2024-05-10T03:32:33.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.356+0000 7f9e37400640 1 -- 172.21.15.55:0/2451246933 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16345 (secure 0 0 0) 0x7f9e3804b090 con 0x7f9e40144710 2024-05-10T03:32:33.361 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:33.361 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:24.417028+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 msgr2=0x7f9e1c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 0x7f9e1c06d110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e30006fd0 tx=0x7f9e30008040 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 msgr2=0x7f9e4013bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9e38034040 tx=0x7f9e38004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 shutdown_connections 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e40144710 0x7f9e4013bd50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9e1c06ac00 0x7f9e1c06d110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e400b3190 0x7f9e40157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 --2- 172.21.15.55:0/2451246933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e4006d7e0 0x7f9e40158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 >> 172.21.15.55:0/2451246933 conn(0x7f9e4006a110 msgr2=0x7f9e4006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 shutdown_connections 2024-05-10T03:32:33.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.359+0000 7f9e45000640 1 -- 172.21.15.55:0/2451246933 wait complete. 2024-05-10T03:32:33.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:11.083041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:11.883212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:33.391 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:33.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.681+0000 7f5d7d800640 1 Processor -- start 2024-05-10T03:32:33.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.681+0000 7f5d7d800640 1 -- start start 2024-05-10T03:32:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.683+0000 7f5d7d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.683+0000 7f5d7d800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 0x7f5d7806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.683+0000 7f5d7d800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780b6780 con 0x7f5d7806b060 2024-05-10T03:32:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.683+0000 7f5d7d800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7806baf0 con 0x7f5d780ce6b0 2024-05-10T03:32:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.683+0000 7f5d7d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780ceac0 con 0x7f5d780b2260 2024-05-10T03:32:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.684+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.684+0000 7f5d6fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 0x7f5d7806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.684+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:37974/0 (socket says 172.21.15.55:37974) 2024-05-10T03:32:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.684+0000 7f5d77e00640 1 -- 172.21.15.55:0/1469181929 learned_addr learned my addr 172.21.15.55:0/1469181929 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.685+0000 7f5d77e00640 1 -- 172.21.15.55:0/1469181929 >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 msgr2=0x7f5d7806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.685+0000 7f5d77e00640 1 --2- 172.21.15.55:0/1469181929 >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 0x7f5d7806ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.685+0000 7f5d77e00640 1 -- 172.21.15.55:0/1469181929 >> v1:172.21.15.162:6789/0 conn(0x7f5d7806b060 legacy=0x7f5d780d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.685+0000 7f5d77e00640 1 -- 172.21.15.55:0/1469181929 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7806b810 con 0x7f5d780b2260 2024-05-10T03:32:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.685+0000 7f5d6fe00640 1 --2- 172.21.15.55:0/1469181929 >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 0x7f5d7806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:32:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.687+0000 7f5d77e00640 1 --2- 172.21.15.55:0/1469181929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009870 tx=0x7f5d6802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0be280ac577a49b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.687+0000 7f5d77400640 1 -- 172.21.15.55:0/1469181929 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68032440 con 0x7f5d780b2260 2024-05-10T03:32:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.687+0000 7f5d77400640 1 -- 172.21.15.55:0/1469181929 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d680325a0 con 0x7f5d780b2260 2024-05-10T03:32:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.687+0000 7f5d77400640 1 -- 172.21.15.55:0/1469181929 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68032930 con 0x7f5d780b2260 2024-05-10T03:32:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 -- 172.21.15.55:0/1469181929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 msgr2=0x7f5d780ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1469181929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009870 tx=0x7f5d6802a250 comp rx=0 tx=0).stop 2024-05-10T03:32:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 -- 172.21.15.55:0/1469181929 shutdown_connections 2024-05-10T03:32:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1469181929 >> v2:172.21.15.162:3300/0 conn(0x7f5d780ce6b0 0x7f5d7806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1469181929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d780ce170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 -- 172.21.15.55:0/1469181929 >> 172.21.15.55:0/1469181929 conn(0x7f5d780d3130 msgr2=0x7f5d780d3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 -- 172.21.15.55:0/1469181929 shutdown_connections 2024-05-10T03:32:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.688+0000 7f5d7d800640 1 -- 172.21.15.55:0/1469181929 wait complete. 2024-05-10T03:32:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.691+0000 7f5d7d800640 1 Processor -- start 2024-05-10T03:32:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.691+0000 7f5d7d800640 1 -- start start 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d78156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d780ce6b0 0x7f5d78145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78158160 con 0x7f5d7806b060 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781585c0 con 0x7f5d780ce6b0 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.692+0000 7f5d7d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78144ad0 con 0x7f5d780b2260 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43708/0 (socket says 172.21.15.55:43708) 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 -- 172.21.15.55:0/1944279209 learned_addr learned my addr 172.21.15.55:0/1944279209 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d6fe00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d78156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d7ce00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d780ce6b0 0x7f5d78145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 -- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 msgr2=0x7f5d78156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d78156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 -- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d780ce6b0 msgr2=0x7f5d78145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d780ce6b0 0x7f5d78145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.693+0000 7f5d77e00640 1 -- 172.21.15.55:0/1944279209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d60009400 con 0x7f5d7806b060 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d77e00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009990 tx=0x7f5d680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68032440 con 0x7f5d7806b060 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68030980 con 0x7f5d7806b060 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68030d10 con 0x7f5d7806b060 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680094d0 con 0x7f5d7806b060 2024-05-10T03:32:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.694+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d78145930 con 0x7f5d7806b060 2024-05-10T03:32:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.696+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78145ac0 con 0x7f5d7806b060 2024-05-10T03:32:33.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.697+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d68039040 con 0x7f5d7806b060 2024-05-10T03:32:33.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.698+0000 7f5d74c00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 0x7f5d5406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:33.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.698+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f5d680a9f30 con 0x7f5d7806b060 2024-05-10T03:32:33.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.699+0000 7f5d6fe00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 0x7f5d5406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:33.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.700+0000 7f5d6fe00640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 0x7f5d5406d110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d7814ae50 tx=0x7f5d60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:33.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.705+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d68074e60 con 0x7f5d7806b060 2024-05-10T03:32:33.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.910+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d78157020 con 0x7f5d7806b060 2024-05-10T03:32:33.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.910+0000 7f5d74c00640 1 -- 172.21.15.55:0/1944279209 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d680370a0 con 0x7f5d7806b060 2024-05-10T03:32:33.914 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:33.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 msgr2=0x7f5d5406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 0x7f5d5406d110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d7814ae50 tx=0x7f5d60008040 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 msgr2=0x7f5d78157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009990 tx=0x7f5d680046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 shutdown_connections 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d5406ac00 0x7f5d5406d110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d780ce6b0 0x7f5d78145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d780b2260 0x7f5d78156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 --2- 172.21.15.55:0/1944279209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7806b060 0x7f5d78157c20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 >> 172.21.15.55:0/1944279209 conn(0x7f5d780d3130 msgr2=0x7f5d7815dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 shutdown_connections 2024-05-10T03:32:33.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:33.913+0000 7f5d7d800640 1 -- 172.21.15.55:0/1944279209 wait complete. 2024-05-10T03:32:33.944 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:32:34.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.228+0000 7f099b800640 1 Processor -- start 2024-05-10T03:32:34.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.228+0000 7f099b800640 1 -- start start 2024-05-10T03:32:34.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.230+0000 7f099b800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f0994141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.230+0000 7f099b800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0994071ce0 con 0x7f09940cd750 2024-05-10T03:32:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.230+0000 7f099b800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0994070460 con 0x7f099415bce0 2024-05-10T03:32:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.230+0000 7f099b800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0994070f40 con 0x7f0994070050 2024-05-10T03:32:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.231+0000 7f0993e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f09940cd750 0x7f099415f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43500/0 (socket says 172.21.15.55:43500) 2024-05-10T03:32:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.231+0000 7f0993e00640 1 -- 172.21.15.55:0/513067406 learned_addr learned my addr 172.21.15.55:0/513067406 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.231+0000 7f0998e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f0994141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.231+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148999576 0 0) 0x7f0980001110 con 0x7f09940cd750 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.231+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097c003520 con 0x7f09940cd750 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26512643 0 0) 0x7f097c003520 con 0x7f09940cd750 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 msgr2=0x7f0994141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f0994141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 >> v1:172.21.15.162:6789/0 conn(0x7f099415bce0 legacy=0x7f099406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940cdb60 con 0x7f09940cd750 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0980001800 con 0x7f09940cd750 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 >> v1:172.21.15.55:6789/0 conn(0x7f09940cd750 legacy=0x7f099415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0039a0 0x7f097c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f097c0047c0 0x7f097c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f097c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0994070f40 con 0x7f097c0047c0 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00bab0 con 0x7f097c0039a0 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00bc70 con 0x7f0994070050 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.232+0000 7f0993e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f097c0047c0 0x7f097c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0998e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0039a0 0x7f097c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0999800640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f097c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0993e00640 1 -- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 msgr2=0x7f097c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0993e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f097c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0993e00640 1 -- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0039a0 msgr2=0x7f097c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0993e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0039a0 0x7f097c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0993e00640 1 -- 172.21.15.55:0/513067406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c003520 con 0x7f097c0047c0 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0998e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0039a0 0x7f097c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.233+0000 7f0999800640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0994070050 0x7f097c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.234+0000 7f0993e00640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f097c0047c0 0x7f097c004bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f098000c210 tx=0x7f097c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b82b99c38c7177 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.234+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980005ad0 con 0x7f097c0047c0 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.234+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098002d070 con 0x7f097c0047c0 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.234+0000 7f0993400640 1 -- 172.21.15.55:0/513067406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980005e90 con 0x7f097c0047c0 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 -- 172.21.15.55:0/513067406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f097c0047c0 msgr2=0x7f097c004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 --2- 172.21.15.55:0/513067406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f097c0047c0 0x7f097c004bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f098000c210 tx=0x7f097c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f0999800640 1 -- 172.21.15.55:0/513067406 reap_dead start 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 -- 172.21.15.55:0/513067406 shutdown_connections 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 -- 172.21.15.55:0/513067406 >> 172.21.15.55:0/513067406 conn(0x7f099406c120 msgr2=0x7f099406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 -- 172.21.15.55:0/513067406 shutdown_connections 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.235+0000 7f099b800640 1 -- 172.21.15.55:0/513067406 wait complete. 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.238+0000 7f099b800640 1 Processor -- start 2024-05-10T03:32:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.238+0000 7f099b800640 1 -- start start 2024-05-10T03:32:34.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0047c0 0x7f09941e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09940b36c0 0x7f0994158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941e5980 con 0x7f09940acb40 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940ad860 con 0x7f097c0047c0 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.239+0000 7f099b800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09941e5af0 con 0x7f09940b36c0 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43724/0 (socket says 172.21.15.55:43724) 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 -- 172.21.15.55:0/1852837313 learned_addr learned my addr 172.21.15.55:0/1852837313 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0998e00640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0047c0 0x7f09941e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0993e00640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09940b36c0 0x7f0994158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 -- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09940b36c0 msgr2=0x7f0994158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09940b36c0 0x7f0994158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 -- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0047c0 msgr2=0x7f09941e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0047c0 0x7f09941e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.240+0000 7f0999800640 1 -- 172.21.15.55:0/1852837313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940b3230 con 0x7f09940acb40 2024-05-10T03:32:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f0999800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f09940a84d0 tx=0x7f098c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c006d50 con 0x7f09940acb40 2024-05-10T03:32:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c003970 con 0x7f09940acb40 2024-05-10T03:32:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c003d20 con 0x7f09940acb40 2024-05-10T03:32:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09940b3ad0 con 0x7f09940acb40 2024-05-10T03:32:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.241+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941e6020 con 0x7f09940acb40 2024-05-10T03:32:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.242+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09941e6240 con 0x7f09940acb40 2024-05-10T03:32:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.243+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f098c005070 con 0x7f09940acb40 2024-05-10T03:32:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.244+0000 7f0990c00640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 0x7f097406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.244+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f098c0890d0 con 0x7f09940acb40 2024-05-10T03:32:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.245+0000 7f0998e00640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 0x7f097406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.246+0000 7f0998e00640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 0x7f097406d110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f099415ce50 tx=0x7f0988008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.249+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f098c054090 con 0x7f09940acb40 2024-05-10T03:32:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.448+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0994070460 con 0x7f09940acb40 2024-05-10T03:32:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.449+0000 7f0990c00640 1 -- 172.21.15.55:0/1852837313 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0994070460 con 0x7f09940acb40 2024-05-10T03:32:34.453 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:32:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 msgr2=0x7f097406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 0x7f097406d110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f099415ce50 tx=0x7f0988008040 comp rx=0 tx=0).stop 2024-05-10T03:32:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 msgr2=0x7f09940acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f09940a84d0 tx=0x7f098c00dc00 comp rx=0 tx=0).stop 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 shutdown_connections 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09940acb40 0x7f09940acf50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09940b36c0 0x7f0994158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f097406ac00 0x7f097406d110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 --2- 172.21.15.55:0/1852837313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f097c0047c0 0x7f09941e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 >> 172.21.15.55:0/1852837313 conn(0x7f099406c120 msgr2=0x7f09941410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 shutdown_connections 2024-05-10T03:32:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.452+0000 7f099b800640 1 -- 172.21.15.55:0/1852837313 wait complete. 2024-05-10T03:32:34.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:32:34.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.760+0000 7f6ec1600640 1 Processor -- start 2024-05-10T03:32:34.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.760+0000 7f6ec1600640 1 -- start start 2024-05-10T03:32:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.762+0000 7f6ec1600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ebc13e390 0x7f6ebc15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.762+0000 7f6ec1600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc13eae0 con 0x7f6ebc06b830 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.762+0000 7f6ec1600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc15c020 con 0x7f6ebc1414c0 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.762+0000 7f6ec1600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc13e7a0 con 0x7f6ebc13e390 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ebbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6ebc13e390 0x7f6ebc15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ec0c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6ebc06b830 0x7f6ebc13ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43512/0 (socket says 172.21.15.55:43512) 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ec0c00640 1 -- 172.21.15.55:0/503070102 learned_addr learned my addr 172.21.15.55:0/503070102 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ebbe00640 1 -- 172.21.15.55:0/503070102 >> v1:172.21.15.162:6789/0 conn(0x7f6ebc1414c0 legacy=0x7f6ebc06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ebbe00640 1 -- 172.21.15.55:0/503070102 >> v1:172.21.15.55:6789/0 conn(0x7f6ebc06b830 legacy=0x7f6ebc13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.763+0000 7f6ebbe00640 1 -- 172.21.15.55:0/503070102 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc15c300 con 0x7f6ebc13e390 2024-05-10T03:32:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.764+0000 7f6ebbe00640 1 --2- 172.21.15.55:0/503070102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc15b9d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097a0 tx=0x7f6eac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5998ec1a270731aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.765+0000 7f6ebaa00640 1 -- 172.21.15.55:0/503070102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac032440 con 0x7f6ebc13e390 2024-05-10T03:32:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.765+0000 7f6ebaa00640 1 -- 172.21.15.55:0/503070102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac0325a0 con 0x7f6ebc13e390 2024-05-10T03:32:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.765+0000 7f6ebaa00640 1 -- 172.21.15.55:0/503070102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac032930 con 0x7f6ebc13e390 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 -- 172.21.15.55:0/503070102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 msgr2=0x7f6ebc15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 --2- 172.21.15.55:0/503070102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc15b9d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0097a0 tx=0x7f6eac02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 -- 172.21.15.55:0/503070102 shutdown_connections 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 --2- 172.21.15.55:0/503070102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc15b9d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 -- 172.21.15.55:0/503070102 >> 172.21.15.55:0/503070102 conn(0x7f6ebc06c730 msgr2=0x7f6ebc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 -- 172.21.15.55:0/503070102 shutdown_connections 2024-05-10T03:32:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.766+0000 7f6ec1600640 1 -- 172.21.15.55:0/503070102 wait complete. 2024-05-10T03:32:34.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.768+0000 7f6ec1600640 1 Processor -- start 2024-05-10T03:32:34.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.768+0000 7f6ec1600640 1 -- start start 2024-05-10T03:32:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 0x7f6ebc13d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 0x7f6ebc154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0ac9b0 con 0x7f6ebc13e390 2024-05-10T03:32:34.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc152ba0 con 0x7f6ebc153e90 2024-05-10T03:32:34.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ec1600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc13dca0 con 0x7f6ebc1414c0 2024-05-10T03:32:34.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43742/0 (socket says 172.21.15.55:43742) 2024-05-10T03:32:34.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.770+0000 7f6ebbe00640 1 -- 172.21.15.55:0/3813879112 learned_addr learned my addr 172.21.15.55:0/3813879112 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ec0c00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 0x7f6ebc154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 -- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 msgr2=0x7f6ebc13d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebb400640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 0x7f6ebc13d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 0x7f6ebc13d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 -- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 msgr2=0x7f6ebc154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 0x7f6ebc154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 -- 172.21.15.55:0/3813879112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009400 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ec0c00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 0x7f6ebc154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebb400640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 0x7f6ebc13d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.771+0000 7f6ebbe00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.772+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac030b80 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.772+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac030ce0 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.772+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac03b640 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.772+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc141a50 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.772+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc153a00 con 0x7f6ebc13e390 2024-05-10T03:32:34.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.773+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc153b90 con 0x7f6ebc13e390 2024-05-10T03:32:34.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.774+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6eac039040 con 0x7f6ebc13e390 2024-05-10T03:32:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.775+0000 7f6eb3e00640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 0x7f6e9c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.775+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f6eac0a9ec0 con 0x7f6ebc13e390 2024-05-10T03:32:34.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.776+0000 7f6ebb400640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 0x7f6e9c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:34.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.777+0000 7f6ebb400640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 0x7f6e9c06d110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc1f7050 tx=0x7f6ea8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:34.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.780+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6eac074e90 con 0x7f6ebc13e390 2024-05-10T03:32:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.980+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ebc04e290 con 0x7f6ebc13e390 2024-05-10T03:32:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.981+0000 7f6eb3e00640 1 -- 172.21.15.55:0/3813879112 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6eac037070 con 0x7f6ebc13e390 2024-05-10T03:32:34.986 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:32:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 msgr2=0x7f6e9c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 0x7f6e9c06d110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc1f7050 tx=0x7f6ea8008040 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 msgr2=0x7f6ebc152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 shutdown_connections 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e9c06ac00 0x7f6e9c06d110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ebc153e90 0x7f6ebc154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ebc1414c0 0x7f6ebc13d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 --2- 172.21.15.55:0/3813879112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ebc13e390 0x7f6ebc152040 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 >> 172.21.15.55:0/3813879112 conn(0x7f6ebc06c730 msgr2=0x7f6ebc15e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 shutdown_connections 2024-05-10T03:32:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:34.984+0000 7f6ec1600640 1 -- 172.21.15.55:0/3813879112 wait complete. 2024-05-10T03:32:35.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:32:35.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.288+0000 7f8ce9600640 1 Processor -- start 2024-05-10T03:32:35.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.288+0000 7f8ce9600640 1 -- start start 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce9600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ce413e390 0x7f8ce415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce9600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ce41414c0 0x7f8ce406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce9600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce413eae0 con 0x7f8ce406b660 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce9600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce413e7a0 con 0x7f8ce413e390 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce9600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce406ba70 con 0x7f8ce41414c0 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ce413e390 0x7f8ce415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ce413e390 0x7f8ce415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43746/0 (socket says 172.21.15.55:43746) 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.291+0000 7f8ce3e00640 1 -- 172.21.15.55:0/3850262851 learned_addr learned my addr 172.21.15.55:0/3850262851 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.292+0000 7f8ce3400640 1 --2- 172.21.15.55:0/3850262851 >> v2:172.21.15.162:3300/0 conn(0x7f8ce41414c0 0x7f8ce406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.292+0000 7f8ce3e00640 1 -- 172.21.15.55:0/3850262851 >> v2:172.21.15.162:3300/0 conn(0x7f8ce41414c0 msgr2=0x7f8ce406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:35.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.292+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3850262851 >> v2:172.21.15.162:3300/0 conn(0x7f8ce41414c0 0x7f8ce406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.292+0000 7f8ce3e00640 1 -- 172.21.15.55:0/3850262851 >> v1:172.21.15.162:6789/0 conn(0x7f8ce406b660 legacy=0x7f8ce415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:35.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.292+0000 7f8ce3e00640 1 -- 172.21.15.55:0/3850262851 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce415c020 con 0x7f8ce413e390 2024-05-10T03:32:35.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.293+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3850262851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce415b9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40ae8f0 tx=0x7f8ccc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625585f2b0457cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:35.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.293+0000 7f8ce2a00640 1 -- 172.21.15.55:0/3850262851 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc032440 con 0x7f8ce413e390 2024-05-10T03:32:35.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.293+0000 7f8ce2a00640 1 -- 172.21.15.55:0/3850262851 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc0325a0 con 0x7f8ce413e390 2024-05-10T03:32:35.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.293+0000 7f8ce2a00640 1 -- 172.21.15.55:0/3850262851 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc032930 con 0x7f8ce413e390 2024-05-10T03:32:35.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.294+0000 7f8ce9600640 1 -- 172.21.15.55:0/3850262851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 msgr2=0x7f8ce415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.294+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3850262851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce415b9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40ae8f0 tx=0x7f8ccc02a230 comp rx=0 tx=0).stop 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 -- 172.21.15.55:0/3850262851 shutdown_connections 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3850262851 >> v2:172.21.15.162:3300/0 conn(0x7f8ce41414c0 0x7f8ce406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3850262851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce415b9d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 -- 172.21.15.55:0/3850262851 >> 172.21.15.55:0/3850262851 conn(0x7f8ce406c730 msgr2=0x7f8ce40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 -- 172.21.15.55:0/3850262851 shutdown_connections 2024-05-10T03:32:35.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.295+0000 7f8ce9600640 1 -- 172.21.15.55:0/3850262851 wait complete. 2024-05-10T03:32:35.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.297+0000 7f8ce9600640 1 Processor -- start 2024-05-10T03:32:35.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.297+0000 7f8ce9600640 1 -- start start 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 0x7f8ce412c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 0x7f8ce415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4152880 con 0x7f8ce413e390 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce40b4100 con 0x7f8ce41414c0 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce9600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce412cf20 con 0x7f8ce406b660 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43750/0 (socket says 172.21.15.55:43750) 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.299+0000 7f8ce3400640 1 -- 172.21.15.55:0/3669194564 learned_addr learned my addr 172.21.15.55:0/3669194564 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce8c00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 0x7f8ce415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 0x7f8ce412c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 -- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 msgr2=0x7f8ce412c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 0x7f8ce412c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 -- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 msgr2=0x7f8ce415a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 0x7f8ce415a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 -- 172.21.15.55:0/3669194564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0094d0 con 0x7f8ce413e390 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 0x7f8ce412c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce8c00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 0x7f8ce415a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8ce3400640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4007ef0 tx=0x7f8cd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.300+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4003b90 con 0x7f8ce413e390 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.301+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4003cf0 con 0x7f8ce413e390 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.301+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce4152a90 con 0x7f8ce413e390 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.301+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004270 con 0x7f8ce413e390 2024-05-10T03:32:35.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.301+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce4153050 con 0x7f8ce413e390 2024-05-10T03:32:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.302+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce404e290 con 0x7f8ce413e390 2024-05-10T03:32:35.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.303+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8cd40043d0 con 0x7f8ce413e390 2024-05-10T03:32:35.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.304+0000 7f8cdbe00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 0x7f8cc006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:35.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.304+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f8cd40899e0 con 0x7f8ce413e390 2024-05-10T03:32:35.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.305+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 0x7f8cc006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:35.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.306+0000 7f8ce3e00640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 0x7f8cc006d110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc034000 tx=0x7f8ccc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:35.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.309+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8cd4054a50 con 0x7f8ce413e390 2024-05-10T03:32:35.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.519+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ce40cdf70 con 0x7f8ce413e390 2024-05-10T03:32:35.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.520+0000 7f8cdbe00640 1 -- 172.21.15.55:0/3669194564 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8cd401a020 con 0x7f8ce413e390 2024-05-10T03:32:35.524 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 msgr2=0x7f8cc006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 0x7f8cc006d110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc034000 tx=0x7f8ccc02e000 comp rx=0 tx=0).stop 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 msgr2=0x7f8ce4151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4007ef0 tx=0x7f8cd400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 shutdown_connections 2024-05-10T03:32:35.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8cc006ac00 0x7f8cc006d110 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce41414c0 0x7f8ce415a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ce413e390 0x7f8ce4151470 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 --2- 172.21.15.55:0/3669194564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce406b660 0x7f8ce412c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 >> 172.21.15.55:0/3669194564 conn(0x7f8ce406c730 msgr2=0x7f8ce413eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 shutdown_connections 2024-05-10T03:32:35.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:35.523+0000 7f8ce9600640 1 -- 172.21.15.55:0/3669194564 wait complete. 2024-05-10T03:32:35.552 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:32:35.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:35.553 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:32:35.652 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:32:35.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:32:35.653 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:32:35.698 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:35.698 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:35.758 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:32:35.759 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:35.759 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:35.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:32:35.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.107+0000 7f2025400640 1 Processor -- start 2024-05-10T03:32:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.107+0000 7f2025400640 1 -- start start 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2025400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2020144120 0x7f202006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2025400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20200b3190 0x7f20200cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2025400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202006d350 con 0x7f2020144710 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2025400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2020144530 con 0x7f20200b3190 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2025400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202006dd20 con 0x7f2020144120 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2024a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20200b3190 0x7f20200cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2024a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20200b3190 0x7f20200cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43756/0 (socket says 172.21.15.55:43756) 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f2024a00640 1 -- 172.21.15.55:0/125562386 learned_addr learned my addr 172.21.15.55:0/125562386 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.110+0000 7f201fe00640 1 --1- 172.21.15.55:0/125562386 >> v1:172.21.15.55:6789/0 conn(0x7f2020144710 0x7f20200b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43520/0 (socket says 172.21.15.55:43520) 2024-05-10T03:32:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.111+0000 7f201f400640 1 --2- 172.21.15.55:0/125562386 >> v2:172.21.15.162:3300/0 conn(0x7f2020144120 0x7f202006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.111+0000 7f2024a00640 1 -- 172.21.15.55:0/125562386 >> v2:172.21.15.162:3300/0 conn(0x7f2020144120 msgr2=0x7f202006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:36.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.111+0000 7f2024a00640 1 --2- 172.21.15.55:0/125562386 >> v2:172.21.15.162:3300/0 conn(0x7f2020144120 0x7f202006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.111+0000 7f2024a00640 1 -- 172.21.15.55:0/125562386 >> v1:172.21.15.55:6789/0 conn(0x7f2020144710 legacy=0x7f20200b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.111+0000 7f2024a00640 1 -- 172.21.15.55:0/125562386 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202006c740 con 0x7f20200b3190 2024-05-10T03:32:36.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.112+0000 7f2024a00640 1 --2- 172.21.15.55:0/125562386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200cd9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f20200ae870 tx=0x7f201802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb02562c40ef345 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.112+0000 7f201ea00640 1 -- 172.21.15.55:0/125562386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018031600 con 0x7f20200b3190 2024-05-10T03:32:36.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.112+0000 7f201ea00640 1 -- 172.21.15.55:0/125562386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201802d050 con 0x7f20200b3190 2024-05-10T03:32:36.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f201ea00640 1 -- 172.21.15.55:0/125562386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018031980 con 0x7f20200b3190 2024-05-10T03:32:36.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f2025400640 1 -- 172.21.15.55:0/125562386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 msgr2=0x7f20200cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f2025400640 1 --2- 172.21.15.55:0/125562386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200cd9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f20200ae870 tx=0x7f201802a610 comp rx=0 tx=0).stop 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f2025400640 1 -- 172.21.15.55:0/125562386 shutdown_connections 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f2025400640 1 --2- 172.21.15.55:0/125562386 >> v2:172.21.15.162:3300/0 conn(0x7f2020144120 0x7f202006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.113+0000 7f2025400640 1 --2- 172.21.15.55:0/125562386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200cd9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.114+0000 7f2025400640 1 -- 172.21.15.55:0/125562386 >> 172.21.15.55:0/125562386 conn(0x7f202006a110 msgr2=0x7f202006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.114+0000 7f2025400640 1 -- 172.21.15.55:0/125562386 shutdown_connections 2024-05-10T03:32:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.114+0000 7f2025400640 1 -- 172.21.15.55:0/125562386 wait complete. 2024-05-10T03:32:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.116+0000 7f2025400640 1 Processor -- start 2024-05-10T03:32:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.116+0000 7f2025400640 1 -- start start 2024-05-10T03:32:36.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2020144120 0x7f2020150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2020144710 0x7f202015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202015b010 con 0x7f20200b3190 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20201f24a0 con 0x7f2020144120 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f2025400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2020141b40 con 0x7f2020144710 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43758/0 (socket says 172.21.15.55:43758) 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 -- 172.21.15.55:0/705790327 learned_addr learned my addr 172.21.15.55:0/705790327 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201f400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2020144120 0x7f2020150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 -- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2020144710 msgr2=0x7f202015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2020144710 0x7f202015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 -- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2020144120 msgr2=0x7f2020150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.118+0000 7f201fe00640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2020144120 0x7f2020150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f201fe00640 1 -- 172.21.15.55:0/705790327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2018009b30 con 0x7f20200b3190 2024-05-10T03:32:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f201fe00640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f201000c3c0 tx=0x7f201000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100074e0 con 0x7f20200b3190 2024-05-10T03:32:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010007640 con 0x7f20200b3190 2024-05-10T03:32:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010004a00 con 0x7f20200b3190 2024-05-10T03:32:36.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2020141d90 con 0x7f20200b3190 2024-05-10T03:32:36.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.119+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020142350 con 0x7f20200b3190 2024-05-10T03:32:36.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.121+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020066ce0 con 0x7f20200b3190 2024-05-10T03:32:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.122+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2010004b60 con 0x7f20200b3190 2024-05-10T03:32:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.123+0000 7f2017e00640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 0x7f200006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.123+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2010088f10 con 0x7f20200b3190 2024-05-10T03:32:36.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.124+0000 7f201f400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 0x7f200006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.125+0000 7f201f400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 0x7f200006d110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.128+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2010053ed0 con 0x7f20200b3190 2024-05-10T03:32:36.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.371+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f202004e290 con 0x7f20200b3190 2024-05-10T03:32:36.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.372+0000 7f2017e00640 1 -- 172.21.15.55:0/705790327 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f20100574f0 con 0x7f20200b3190 2024-05-10T03:32:36.377 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:36.377 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:36.377 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:36.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 msgr2=0x7f200006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 0x7f200006d110 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c008040 comp rx=0 tx=0).stop 2024-05-10T03:32:36.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 msgr2=0x7f20200a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f201000c3c0 tx=0x7f201000c880 comp rx=0 tx=0).stop 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 shutdown_connections 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2020144710 0x7f202015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f200006ac00 0x7f200006d110 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2020144120 0x7f2020150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 --2- 172.21.15.55:0/705790327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20200b3190 0x7f20200a84d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.375+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 >> 172.21.15.55:0/705790327 conn(0x7f202006a110 msgr2=0x7f20201432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.376+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 shutdown_connections 2024-05-10T03:32:36.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.376+0000 7f2025400640 1 -- 172.21.15.55:0/705790327 wait complete. 2024-05-10T03:32:36.407 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:36.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.695+0000 7f9054600640 1 Processor -- start 2024-05-10T03:32:36.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.695+0000 7f9054600640 1 -- start start 2024-05-10T03:32:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9054600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 0x7f904c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9054600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f904c141d00 0x7f904c0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9054600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904c160510 con 0x7f904c144c70 2024-05-10T03:32:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9054600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904c0ce590 con 0x7f904c141d00 2024-05-10T03:32:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9054600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c15fc70 con 0x7f904c06c7c0 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9051c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f904c144c70 0x7f904c06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43532/0 (socket says 172.21.15.55:43532) 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.698+0000 7f9051c00640 1 -- 172.21.15.55:0/236703390 learned_addr learned my addr 172.21.15.55:0/236703390 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.699+0000 7f9051200640 1 --2- 172.21.15.55:0/236703390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 0x7f904c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.699+0000 7f9052600640 1 --2- 172.21.15.55:0/236703390 >> v2:172.21.15.162:3300/0 conn(0x7f904c141d00 0x7f904c0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.699+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211099822 0 0) 0x7f904c160510 con 0x7f904c144c70 2024-05-10T03:32:36.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.699+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9034003520 con 0x7f904c144c70 2024-05-10T03:32:36.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.699+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998871592 0 0) 0x7f9034003520 con 0x7f904c144c70 2024-05-10T03:32:36.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.700+0000 7f9051200640 1 -- 172.21.15.55:0/236703390 >> v2:172.21.15.162:3300/0 conn(0x7f904c141d00 msgr2=0x7f904c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:36.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.700+0000 7f9051200640 1 --2- 172.21.15.55:0/236703390 >> v2:172.21.15.162:3300/0 conn(0x7f904c141d00 0x7f904c0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.700+0000 7f9051200640 1 -- 172.21.15.55:0/236703390 >> v1:172.21.15.55:6789/0 conn(0x7f904c144c70 legacy=0x7f904c06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.700+0000 7f9051200640 1 -- 172.21.15.55:0/236703390 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c15ff50 con 0x7f904c06c7c0 2024-05-10T03:32:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.701+0000 7f9051200640 1 --2- 172.21.15.55:0/236703390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 0x7f904c15f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90380097a0 tx=0x7f903802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=874a69802f815551 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.702+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400019f0 con 0x7f904c06c7c0 2024-05-10T03:32:36.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.702+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903802ab10 con 0x7f904c06c7c0 2024-05-10T03:32:36.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.702+0000 7f904be00640 1 -- 172.21.15.55:0/236703390 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903802aea0 con 0x7f904c06c7c0 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 -- 172.21.15.55:0/236703390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 msgr2=0x7f904c15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 --2- 172.21.15.55:0/236703390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 0x7f904c15f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90380097a0 tx=0x7f903802a080 comp rx=0 tx=0).stop 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 -- 172.21.15.55:0/236703390 shutdown_connections 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 --2- 172.21.15.55:0/236703390 >> v2:172.21.15.162:3300/0 conn(0x7f904c141d00 0x7f904c0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 --2- 172.21.15.55:0/236703390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c06c7c0 0x7f904c15f5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 -- 172.21.15.55:0/236703390 >> 172.21.15.55:0/236703390 conn(0x7f904c0d0250 msgr2=0x7f904c0d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 -- 172.21.15.55:0/236703390 shutdown_connections 2024-05-10T03:32:36.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.703+0000 7f9054600640 1 -- 172.21.15.55:0/236703390 wait complete. 2024-05-10T03:32:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.706+0000 7f9054600640 1 Processor -- start 2024-05-10T03:32:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.706+0000 7f9054600640 1 -- start start 2024-05-10T03:32:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 0x7f904c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 0x7f904c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c1f2a70 con 0x7f904c141d00 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c13b2f0 con 0x7f904c06c7c0 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.707+0000 7f9054600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c13cb50 con 0x7f904c144c70 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43772/0 (socket says 172.21.15.55:43772) 2024-05-10T03:32:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 -- 172.21.15.55:0/3146404370 learned_addr learned my addr 172.21.15.55:0/3146404370 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:36.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051c00640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 0x7f904c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9052600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 0x7f904c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 -- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 msgr2=0x7f904c15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 0x7f904c15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 -- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 msgr2=0x7f904c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 0x7f904c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051200640 1 -- 172.21.15.55:0/3146404370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038009400 con 0x7f904c141d00 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9051c00640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 0x7f904c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.708+0000 7f9052600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 0x7f904c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9051200640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9038035040 tx=0x7f90380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903803d830 con 0x7f904c141d00 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903803d990 con 0x7f904c141d00 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903803dd20 con 0x7f904c141d00 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c13b570 con 0x7f904c141d00 2024-05-10T03:32:36.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.709+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c13bb30 con 0x7f904c141d00 2024-05-10T03:32:36.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.710+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c067930 con 0x7f904c141d00 2024-05-10T03:32:36.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.712+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9038039040 con 0x7f904c141d00 2024-05-10T03:32:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.713+0000 7f9049600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 0x7f902c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.713+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f90380a9f60 con 0x7f904c141d00 2024-05-10T03:32:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.713+0000 7f9051c00640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 0x7f902c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.714+0000 7f9051c00640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 0x7f902c06d110 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9040005c00 tx=0x7f9040009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:36.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.718+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9038078920 con 0x7f904c141d00 2024-05-10T03:32:36.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.925+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f904c0cdb10 con 0x7f904c141d00 2024-05-10T03:32:36.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.928+0000 7f9049600640 1 -- 172.21.15.55:0/3146404370 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16345 (secure 0 0 0) 0x7f90380a6ff0 con 0x7f904c141d00 2024-05-10T03:32:36.933 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:36.933 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:24.417028+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 msgr2=0x7f902c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 0x7f902c06d110 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9040005c00 tx=0x7f9040009040 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 msgr2=0x7f904c13c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9038035040 tx=0x7f90380046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 shutdown_connections 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f904c144c70 0x7f904c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f904c141d00 0x7f904c13c610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f902c06ac00 0x7f902c06d110 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 --2- 172.21.15.55:0/3146404370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904c06c7c0 0x7f904c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.931+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 >> 172.21.15.55:0/3146404370 conn(0x7f904c0d0250 msgr2=0x7f904c142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:36.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.932+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 shutdown_connections 2024-05-10T03:32:36.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:36.932+0000 7f9054600640 1 -- 172.21.15.55:0/3146404370 wait complete. 2024-05-10T03:32:36.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:11.083041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:11.883212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:36.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:37.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.245+0000 7f76a7200640 1 Processor -- start 2024-05-10T03:32:37.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.245+0000 7f76a7200640 1 -- start start 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.247+0000 7f76a7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a80719f0 0x7f76a806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a806b840 0x7f76a80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a7200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f76a8071e00 0x7f76a8070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a815fdf0 con 0x7f76a80719f0 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a7200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a815f1e0 con 0x7f76a8071e00 2024-05-10T03:32:37.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a815f4c0 con 0x7f76a806b840 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a80719f0 0x7f76a806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a80719f0 0x7f76a806b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43774/0 (socket says 172.21.15.55:43774) 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a5e00640 1 -- 172.21.15.55:0/2077059577 learned_addr learned my addr 172.21.15.55:0/2077059577 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a5400640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a806b840 0x7f76a80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.248+0000 7f76a6800640 1 --2- 172.21.15.55:0/2077059577 >> v2:172.21.15.162:3300/0 conn(0x7f76a8071e00 0x7f76a8070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.249+0000 7f76a5e00640 1 -- 172.21.15.55:0/2077059577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a806b840 msgr2=0x7f76a80700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.249+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a806b840 0x7f76a80700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.249+0000 7f76a5e00640 1 -- 172.21.15.55:0/2077059577 >> v2:172.21.15.162:3300/0 conn(0x7f76a8071e00 msgr2=0x7f76a8070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.249+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2077059577 >> v2:172.21.15.162:3300/0 conn(0x7f76a8071e00 0x7f76a8070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.249+0000 7f76a5e00640 1 -- 172.21.15.55:0/2077059577 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a815f7a0 con 0x7f76a80719f0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.250+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a80719f0 0x7f76a806b300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76a8071940 tx=0x7f769802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fafa8ae6c35185 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.250+0000 7f76a4a00640 1 -- 172.21.15.55:0/2077059577 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769802ae90 con 0x7f76a80719f0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.250+0000 7f76a4a00640 1 -- 172.21.15.55:0/2077059577 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7698032440 con 0x7f76a80719f0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.250+0000 7f76a4a00640 1 -- 172.21.15.55:0/2077059577 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980327d0 con 0x7f76a80719f0 2024-05-10T03:32:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 -- 172.21.15.55:0/2077059577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a80719f0 msgr2=0x7f76a806b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a80719f0 0x7f76a806b300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76a8071940 tx=0x7f769802a140 comp rx=0 tx=0).stop 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 -- 172.21.15.55:0/2077059577 shutdown_connections 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 --2- 172.21.15.55:0/2077059577 >> v2:172.21.15.162:3300/0 conn(0x7f76a8071e00 0x7f76a8070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a806b840 0x7f76a80700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 --2- 172.21.15.55:0/2077059577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a80719f0 0x7f76a806b300 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 -- 172.21.15.55:0/2077059577 >> 172.21.15.55:0/2077059577 conn(0x7f76a806c750 msgr2=0x7f76a80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 -- 172.21.15.55:0/2077059577 shutdown_connections 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.251+0000 7f76a7200640 1 -- 172.21.15.55:0/2077059577 wait complete. 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.253+0000 7f76a7200640 1 Processor -- start 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.254+0000 7f76a7200640 1 -- start start 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 0x7f76a814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 0x7f76a80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814e640 con 0x7f76a806b840 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814ea70 con 0x7f76a80719f0 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c7800 con 0x7f76a8071e00 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.255+0000 7f76a5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43786/0 (socket says 172.21.15.55:43786) 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 -- 172.21.15.55:0/2880443892 learned_addr learned my addr 172.21.15.55:0/2880443892 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a6800640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 0x7f76a80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5400640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 0x7f76a814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 -- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 msgr2=0x7f76a80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 0x7f76a80c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 -- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 msgr2=0x7f76a814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 0x7f76a814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 -- 172.21.15.55:0/2880443892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980094d0 con 0x7f76a806b840 2024-05-10T03:32:37.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a6800640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 0x7f76a80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5400640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 0x7f76a814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.256+0000 7f76a5e00640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f769802a180 tx=0x7f76980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.257+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698030850 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.257+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76980309b0 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.257+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698030d60 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.257+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80ac750 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.257+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80acd10 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.258+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a804e290 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.259+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7698039040 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.260+0000 7f769e000640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 0x7f768406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.260+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f76980aa1b0 con 0x7f76a806b840 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.261+0000 7f76a5400640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 0x7f768406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.262+0000 7f76a5400640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 0x7f768406d110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7690006fd0 tx=0x7f7690008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.265+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f76980750e0 con 0x7f76a806b840 2024-05-10T03:32:37.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.463+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76a8143170 con 0x7f76a806b840 2024-05-10T03:32:37.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.464+0000 7f769e000640 1 -- 172.21.15.55:0/2880443892 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7698078700 con 0x7f76a806b840 2024-05-10T03:32:37.468 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:37.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 msgr2=0x7f768406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 0x7f768406d110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7690006fd0 tx=0x7f7690008040 comp rx=0 tx=0).stop 2024-05-10T03:32:37.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 msgr2=0x7f76a813f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f769802a180 tx=0x7f76980046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 shutdown_connections 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a8071e00 0x7f76a80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f768406ac00 0x7f768406d110 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a80719f0 0x7f76a814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 --2- 172.21.15.55:0/2880443892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a806b840 0x7f76a813f7c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 >> 172.21.15.55:0/2880443892 conn(0x7f76a806c750 msgr2=0x7f76a81425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 shutdown_connections 2024-05-10T03:32:37.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.467+0000 7f76a7200640 1 -- 172.21.15.55:0/2880443892 wait complete. 2024-05-10T03:32:37.495 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:32:37.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.764+0000 7fe4ff200640 1 Processor -- start 2024-05-10T03:32:37.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.764+0000 7fe4ff200640 1 -- start start 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4ff200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4f815e920 0x7fe4f80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4ff200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 0x7fe4f80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4ff200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f80b2a30 con 0x7fe4f806b120 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4ff200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f815a900 con 0x7fe4f815e920 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4ff200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f815eec0 con 0x7fe4f80cdb70 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4fde00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4f815e920 0x7fe4f80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4fde00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4f815e920 0x7fe4f80cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43800/0 (socket says 172.21.15.55:43800) 2024-05-10T03:32:37.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4fde00640 1 -- 172.21.15.55:0/2923955995 learned_addr learned my addr 172.21.15.55:0/2923955995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.767+0000 7fe4fe800640 1 --2- 172.21.15.55:0/2923955995 >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 0x7fe4f80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.768+0000 7fe4fde00640 1 -- 172.21.15.55:0/2923955995 >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 msgr2=0x7fe4f80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.768+0000 7fe4fde00640 1 --2- 172.21.15.55:0/2923955995 >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 0x7fe4f80cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.768+0000 7fe4fde00640 1 -- 172.21.15.55:0/2923955995 >> v1:172.21.15.162:6789/0 conn(0x7fe4f806b120 legacy=0x7fe4f806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.768+0000 7fe4fde00640 1 -- 172.21.15.55:0/2923955995 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f815b8b0 con 0x7fe4f815e920 2024-05-10T03:32:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.768+0000 7fe4fe800640 1 --2- 172.21.15.55:0/2923955995 >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 0x7fe4f80cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.769+0000 7fe4fde00640 1 --2- 172.21.15.55:0/2923955995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f815e920 0x7fe4f80cc240 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80cc780 tx=0x7fe4f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd9a6f46d5ac4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.769+0000 7fe4fca00640 1 -- 172.21.15.55:0/2923955995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0031600 con 0x7fe4f815e920 2024-05-10T03:32:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.769+0000 7fe4fca00640 1 -- 172.21.15.55:0/2923955995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f002d050 con 0x7fe4f815e920 2024-05-10T03:32:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4fca00640 1 -- 172.21.15.55:0/2923955995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0031980 con 0x7fe4f815e920 2024-05-10T03:32:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 -- 172.21.15.55:0/2923955995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f815e920 msgr2=0x7fe4f80cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2923955995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f815e920 0x7fe4f80cc240 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80cc780 tx=0x7fe4f002a730 comp rx=0 tx=0).stop 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 -- 172.21.15.55:0/2923955995 shutdown_connections 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2923955995 >> v2:172.21.15.162:3300/0 conn(0x7fe4f80cdb70 0x7fe4f80cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2923955995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f815e920 0x7fe4f80cc240 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.770+0000 7fe4ff200640 1 -- 172.21.15.55:0/2923955995 >> 172.21.15.55:0/2923955995 conn(0x7fe4f806c750 msgr2=0x7fe4f80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.771+0000 7fe4ff200640 1 -- 172.21.15.55:0/2923955995 shutdown_connections 2024-05-10T03:32:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.771+0000 7fe4ff200640 1 -- 172.21.15.55:0/2923955995 wait complete. 2024-05-10T03:32:37.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.773+0000 7fe4ff200640 1 Processor -- start 2024-05-10T03:32:37.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.773+0000 7fe4ff200640 1 -- start start 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4f80cdb70 0x7fe4f8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 0x7fe4f80b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8146ed0 con 0x7fe4f806b120 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8148c20 con 0x7fe4f815e920 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4ff200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f81497b0 con 0x7fe4f80cdb70 2024-05-10T03:32:37.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4fde00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4fde00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43808/0 (socket says 172.21.15.55:43808) 2024-05-10T03:32:37.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4fde00640 1 -- 172.21.15.55:0/2999683992 learned_addr learned my addr 172.21.15.55:0/2999683992 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.775+0000 7fe4fe800640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 0x7fe4f80b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 -- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4f80cdb70 msgr2=0x7fe4f8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fd400640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4f80cdb70 0x7fe4f8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4f80cdb70 0x7fe4f8148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 -- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 msgr2=0x7fe4f80b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 0x7fe4f80b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 -- 172.21.15.55:0/2999683992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f0009b30 con 0x7fe4f806b120 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fe800640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 0x7fe4f80b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4fde00640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009f40 tx=0x7fe4f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f003b810 con 0x7fe4f806b120 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f003b970 con 0x7fe4f806b120 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f003bd20 con 0x7fe4f806b120 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.776+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f813cb90 con 0x7fe4f806b120 2024-05-10T03:32:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.777+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f8149ce0 con 0x7fe4f806b120 2024-05-10T03:32:37.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.778+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f8149f00 con 0x7fe4f806b120 2024-05-10T03:32:37.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.779+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe4f00073f0 con 0x7fe4f806b120 2024-05-10T03:32:37.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.780+0000 7fe4ee000640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 0x7fe4dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:37.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.780+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fe4f00aa140 con 0x7fe4f806b120 2024-05-10T03:32:37.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.780+0000 7fe4fd400640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 0x7fe4dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:37.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.782+0000 7fe4fd400640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 0x7fe4dc06d140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4f815e040 tx=0x7fe4e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.785+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe4f0075110 con 0x7fe4f806b120 2024-05-10T03:32:37.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.978+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4f804e290 con 0x7fe4f806b120 2024-05-10T03:32:37.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.979+0000 7fe4ee000640 1 -- 172.21.15.55:0/2999683992 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4f0078730 con 0x7fe4f806b120 2024-05-10T03:32:37.983 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.981+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 msgr2=0x7fe4dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 0x7fe4dc06d140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4f815e040 tx=0x7fe4e8009040 comp rx=0 tx=0).stop 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 msgr2=0x7fe4f813c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009f40 tx=0x7fe4f0004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 shutdown_connections 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe4dc06ac30 0x7fe4dc06d140 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4f815e920 0x7fe4f80b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4f80cdb70 0x7fe4f8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 --2- 172.21.15.55:0/2999683992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4f806b120 0x7fe4f813c650 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 >> 172.21.15.55:0/2999683992 conn(0x7fe4f806c750 msgr2=0x7fe4f80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 shutdown_connections 2024-05-10T03:32:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:37.982+0000 7fe4ff200640 1 -- 172.21.15.55:0/2999683992 wait complete. 2024-05-10T03:32:38.013 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:32:38.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.280+0000 7fe9d2800640 1 Processor -- start 2024-05-10T03:32:38.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.280+0000 7fe9d2800640 1 -- start start 2024-05-10T03:32:38.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9cc15b820 0x7fe9cc0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc142010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9cc142550 0x7fe9cc143c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc144e40 con 0x7fe9cc142550 2024-05-10T03:32:38.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc0b4ae0 con 0x7fe9cc15b820 2024-05-10T03:32:38.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.282+0000 7fe9d2800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc142960 con 0x7fe9cc0b46d0 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9cc142550 0x7fe9cc143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe9cc142550 0x7fe9cc143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43812/0 (socket says 172.21.15.55:43812) 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3637494740 learned_addr learned my addr 172.21.15.55:0/3637494740 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d0a00640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1400640 1 --2- 172.21.15.55:0/3637494740 >> v2:172.21.15.162:3300/0 conn(0x7fe9cc15b820 0x7fe9cc0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3637494740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 msgr2=0x7fe9cc142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.283+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.284+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3637494740 >> v2:172.21.15.162:3300/0 conn(0x7fe9cc15b820 msgr2=0x7fe9cc0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.284+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3637494740 >> v2:172.21.15.162:3300/0 conn(0x7fe9cc15b820 0x7fe9cc0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.284+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3637494740 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc142bf0 con 0x7fe9cc142550 2024-05-10T03:32:38.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.285+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc142550 0x7fe9cc143c90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0b4ef0 tx=0x7fe9c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f78db79f4a6e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.285+0000 7fe9cbe00640 1 -- 172.21.15.55:0/3637494740 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4031600 con 0x7fe9cc142550 2024-05-10T03:32:38.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.285+0000 7fe9cbe00640 1 -- 172.21.15.55:0/3637494740 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c402d050 con 0x7fe9cc142550 2024-05-10T03:32:38.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.285+0000 7fe9cbe00640 1 -- 172.21.15.55:0/3637494740 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4031980 con 0x7fe9cc142550 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 -- 172.21.15.55:0/3637494740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc142550 msgr2=0x7fe9cc143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc142550 0x7fe9cc143c90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0b4ef0 tx=0x7fe9c402a6a0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 -- 172.21.15.55:0/3637494740 shutdown_connections 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc142550 0x7fe9cc143c90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3637494740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3637494740 >> v2:172.21.15.162:3300/0 conn(0x7fe9cc15b820 0x7fe9cc0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 -- 172.21.15.55:0/3637494740 >> 172.21.15.55:0/3637494740 conn(0x7fe9cc06b2d0 msgr2=0x7fe9cc06c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:38.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 -- 172.21.15.55:0/3637494740 shutdown_connections 2024-05-10T03:32:38.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.286+0000 7fe9d2800640 1 -- 172.21.15.55:0/3637494740 wait complete. 2024-05-10T03:32:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.289+0000 7fe9d2800640 1 Processor -- start 2024-05-10T03:32:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.289+0000 7fe9d2800640 1 -- start start 2024-05-10T03:32:38.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9cc142550 0x7fe9cc15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1e4670 con 0x7fe9cc15b820 2024-05-10T03:32:38.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0b2010 con 0x7fe9cc142550 2024-05-10T03:32:38.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.290+0000 7fe9d2800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1f7f00 con 0x7fe9cc0b46d0 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43828/0 (socket says 172.21.15.55:43828) 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3722926297 learned_addr learned my addr 172.21.15.55:0/3722926297 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1400640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d0a00640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9cc142550 0x7fe9cc15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 msgr2=0x7fe9cc1f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc1f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9cc142550 msgr2=0x7fe9cc15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9cc142550 0x7fe9cc15b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.291+0000 7fe9d1e00640 1 -- 172.21.15.55:0/3722926297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c4009b30 con 0x7fe9cc15b820 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9d1e00640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009f40 tx=0x7fe9c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c403d860 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c403d9e0 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc0b37a0 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c403dd90 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.292+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1f3d50 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.293+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc0b2180 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.294+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9c403b400 con 0x7fe9cc15b820 2024-05-10T03:32:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.295+0000 7fe9c9600640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 0x7fe9ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.295+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fe9c40aa530 con 0x7fe9cc15b820 2024-05-10T03:32:38.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.296+0000 7fe9d1400640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 0x7fe9ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.297+0000 7fe9d1400640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 0x7fe9ac06d110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.300+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe9cc0b2180 con 0x7fe9cc15b820 2024-05-10T03:32:38.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.502+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9cc0b2180 con 0x7fe9cc15b820 2024-05-10T03:32:38.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.503+0000 7fe9c9600640 1 -- 172.21.15.55:0/3722926297 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9cc0b2180 con 0x7fe9cc15b820 2024-05-10T03:32:38.507 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 msgr2=0x7fe9ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 0x7fe9ac06d110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 msgr2=0x7fe9cc140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009f40 tx=0x7fe9c4004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 shutdown_connections 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9cc15b820 0x7fe9cc140380 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe9ac06ac00 0x7fe9ac06d110 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9cc142550 0x7fe9cc15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 --2- 172.21.15.55:0/3722926297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9cc0b46d0 0x7fe9cc1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 >> 172.21.15.55:0/3722926297 conn(0x7fe9cc06b2d0 msgr2=0x7fe9cc071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 shutdown_connections 2024-05-10T03:32:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.506+0000 7fe9d2800640 1 -- 172.21.15.55:0/3722926297 wait complete. 2024-05-10T03:32:38.539 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:38.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.817+0000 7f92c9e00640 1 Processor -- start 2024-05-10T03:32:38.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.818+0000 7f92c9e00640 1 -- start start 2024-05-10T03:32:38.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.819+0000 7f92c9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40ce5f0 0x7f92c4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.819+0000 7f92c9e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f92c41613c0 0x7f92c40b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.819+0000 7f92c9e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c415edc0 con 0x7f92c415ddd0 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.819+0000 7f92c9e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c41623d0 con 0x7f92c41613c0 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.819+0000 7f92c9e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415e1e0 con 0x7f92c40ce5f0 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.820+0000 7f92c8a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f92c415ddd0 0x7f92c40cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43538/0 (socket says 172.21.15.55:43538) 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.820+0000 7f92c8a00640 1 -- 172.21.15.55:0/1709785819 learned_addr learned my addr 172.21.15.55:0/1709785819 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.820+0000 7f92c9400640 1 --2- 172.21.15.55:0/1709785819 >> v2:172.21.15.162:3300/0 conn(0x7f92c41613c0 0x7f92c40b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.820+0000 7f92c3e00640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40ce5f0 0x7f92c4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c3400640 1 -- 172.21.15.55:0/1709785819 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776083319 0 0) 0x7f92c415edc0 con 0x7f92c415ddd0 2024-05-10T03:32:38.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c3400640 1 -- 172.21.15.55:0/1709785819 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a8003520 con 0x7f92c415ddd0 2024-05-10T03:32:38.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c9400640 1 -- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40ce5f0 msgr2=0x7f92c4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:38.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c9400640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40ce5f0 0x7f92c4157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c9400640 1 -- 172.21.15.55:0/1709785819 >> v1:172.21.15.55:6789/0 conn(0x7f92c415ddd0 legacy=0x7f92c40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.821+0000 7f92c9400640 1 -- 172.21.15.55:0/1709785819 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40cecd0 con 0x7f92c41613c0 2024-05-10T03:32:38.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.823+0000 7f92c9400640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c41613c0 0x7f92c40b1b20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009d90 tx=0x7f92bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb993d72e4e8089 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.826+0000 7f92c3400640 1 -- 172.21.15.55:0/1709785819 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc007740 con 0x7f92c41613c0 2024-05-10T03:32:38.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.826+0000 7f92c3400640 1 -- 172.21.15.55:0/1709785819 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc02d050 con 0x7f92c41613c0 2024-05-10T03:32:38.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.826+0000 7f92c3400640 1 -- 172.21.15.55:0/1709785819 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc007ac0 con 0x7f92c41613c0 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.826+0000 7f92c9e00640 1 -- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c41613c0 msgr2=0x7f92c40b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.826+0000 7f92c9e00640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c41613c0 0x7f92c40b1b20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009d90 tx=0x7f92bc02a660 comp rx=0 tx=0).stop 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 -- 172.21.15.55:0/1709785819 shutdown_connections 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c41613c0 0x7f92c40b1b20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 --2- 172.21.15.55:0/1709785819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40ce5f0 0x7f92c4157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 -- 172.21.15.55:0/1709785819 >> 172.21.15.55:0/1709785819 conn(0x7f92c40d0370 msgr2=0x7f92c40d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 -- 172.21.15.55:0/1709785819 shutdown_connections 2024-05-10T03:32:38.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.827+0000 7f92c9e00640 1 -- 172.21.15.55:0/1709785819 wait complete. 2024-05-10T03:32:38.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.829+0000 7f92c9e00640 1 Processor -- start 2024-05-10T03:32:38.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.829+0000 7f92c9e00640 1 -- start start 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c41613c0 0x7f92c41f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c40c8d50 0x7f92c40c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415a3f0 con 0x7f92c415ddd0 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41f4520 con 0x7f92c40c8d50 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c9e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415a1a0 con 0x7f92c41613c0 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43842/0 (socket says 172.21.15.55:43842) 2024-05-10T03:32:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.831+0000 7f92c8a00640 1 -- 172.21.15.55:0/2928361605 learned_addr learned my addr 172.21.15.55:0/2928361605 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 -- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c41613c0 msgr2=0x7f92c41f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c41613c0 0x7f92c41f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 -- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c40c8d50 msgr2=0x7f92c40c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c9400640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c40c8d50 0x7f92c40c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c40c8d50 0x7f92c40c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 -- 172.21.15.55:0/2928361605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc009a70 con 0x7f92c415ddd0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c8a00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c3e0 tx=0x7f92b400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4005ce0 con 0x7f92c415ddd0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4005e40 con 0x7f92c415ddd0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40072d0 con 0x7f92c415ddd0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.832+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c40c88c0 con 0x7f92c415ddd0 2024-05-10T03:32:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.833+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41563a0 con 0x7f92c415ddd0 2024-05-10T03:32:38.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.834+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c404e290 con 0x7f92c415ddd0 2024-05-10T03:32:38.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.835+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f92b40074f0 con 0x7f92c415ddd0 2024-05-10T03:32:38.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.836+0000 7f92c0c00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 0x7f92a406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:38.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.836+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f92b40889a0 con 0x7f92c415ddd0 2024-05-10T03:32:38.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.836+0000 7f92c3e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 0x7f92a406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:38.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.837+0000 7f92c3e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 0x7f92a406d140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92b0006a90 tx=0x7f92b0006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:38.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:38.840+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f92b4053ab0 con 0x7f92c415ddd0 2024-05-10T03:32:39.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.080+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92c40d15f0 con 0x7f92c415ddd0 2024-05-10T03:32:39.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.081+0000 7f92c0c00640 1 -- 172.21.15.55:0/2928361605 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f92b4017090 con 0x7f92c415ddd0 2024-05-10T03:32:39.085 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:39.085 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:39.086 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:39.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 msgr2=0x7f92a406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 0x7f92a406d140 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92b0006a90 tx=0x7f92b0006a20 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 msgr2=0x7f92c4155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c3e0 tx=0x7f92b400c8a0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 shutdown_connections 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f92a406ac30 0x7f92a406d140 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c40c8d50 0x7f92c40c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c41613c0 0x7f92c41f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 --2- 172.21.15.55:0/2928361605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f92c415ddd0 0x7f92c4155e60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 >> 172.21.15.55:0/2928361605 conn(0x7f92c40d0370 msgr2=0x7f92c415ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 shutdown_connections 2024-05-10T03:32:39.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.084+0000 7f92c9e00640 1 -- 172.21.15.55:0/2928361605 wait complete. 2024-05-10T03:32:39.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:39.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.391+0000 7fdea4e00640 1 Processor -- start 2024-05-10T03:32:39.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.391+0000 7fdea4e00640 1 -- start start 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdea0071db0 0x7fdea0158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea006a4b0 0x7fdea00ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 0x7fdea00b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea00b2490 con 0x7fdea0071db0 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea0070050 con 0x7fdea00b3860 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fdea4e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00708b0 con 0x7fdea006a4b0 2024-05-10T03:32:39.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fde9de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdea0071db0 0x7fdea0158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fde9de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdea0071db0 0x7fdea0158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43852/0 (socket says 172.21.15.55:43852) 2024-05-10T03:32:39.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.394+0000 7fde9de00640 1 -- 172.21.15.55:0/2275838728 learned_addr learned my addr 172.21.15.55:0/2275838728 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:39.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9d400640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea006a4b0 0x7fdea00ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9e800640 1 --2- 172.21.15.55:0/2275838728 >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 0x7fdea00b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9de00640 1 -- 172.21.15.55:0/2275838728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea006a4b0 msgr2=0x7fdea00ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9de00640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea006a4b0 0x7fdea00ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9de00640 1 -- 172.21.15.55:0/2275838728 >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 msgr2=0x7fdea00b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9de00640 1 --2- 172.21.15.55:0/2275838728 >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 0x7fdea00b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9de00640 1 -- 172.21.15.55:0/2275838728 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea015b940 con 0x7fdea0071db0 2024-05-10T03:32:39.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.395+0000 7fde9e800640 1 --2- 172.21.15.55:0/2275838728 >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 0x7fdea00b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:39.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.396+0000 7fde9de00640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0158150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdea006fe20 tx=0x7fde9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85328df95f79d7d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.396+0000 7fde9ca00640 1 -- 172.21.15.55:0/2275838728 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9002ae90 con 0x7fdea0071db0 2024-05-10T03:32:39.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.396+0000 7fde9ca00640 1 -- 172.21.15.55:0/2275838728 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90032440 con 0x7fdea0071db0 2024-05-10T03:32:39.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.396+0000 7fde9ca00640 1 -- 172.21.15.55:0/2275838728 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900327d0 con 0x7fdea0071db0 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 -- 172.21.15.55:0/2275838728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 msgr2=0x7fdea0158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0158150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdea006fe20 tx=0x7fde9002a140 comp rx=0 tx=0).stop 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 -- 172.21.15.55:0/2275838728 shutdown_connections 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 --2- 172.21.15.55:0/2275838728 >> v2:172.21.15.162:3300/0 conn(0x7fdea00b3860 0x7fdea00b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea006a4b0 0x7fdea00ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 --2- 172.21.15.55:0/2275838728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0158150 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 -- 172.21.15.55:0/2275838728 >> 172.21.15.55:0/2275838728 conn(0x7fdea00cbb90 msgr2=0x7fdea00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.397+0000 7fdea4e00640 1 -- 172.21.15.55:0/2275838728 shutdown_connections 2024-05-10T03:32:39.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.398+0000 7fdea4e00640 1 -- 172.21.15.55:0/2275838728 wait complete. 2024-05-10T03:32:39.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.400+0000 7fdea4e00640 1 Processor -- start 2024-05-10T03:32:39.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.400+0000 7fdea4e00640 1 -- start start 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea006a4b0 0x7fdea0156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea00b3860 0x7fdea01597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01389a0 con 0x7fdea0071db0 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0158ee0 con 0x7fdea006a4b0 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fdea4e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea015a0f0 con 0x7fdea00b3860 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43866/0 (socket says 172.21.15.55:43866) 2024-05-10T03:32:39.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 -- 172.21.15.55:0/1719581762 learned_addr learned my addr 172.21.15.55:0/1719581762 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9de00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea006a4b0 0x7fdea0156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 -- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea00b3860 msgr2=0x7fdea01597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea00b3860 0x7fdea01597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 -- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea006a4b0 msgr2=0x7fdea0156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea006a4b0 0x7fdea0156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.402+0000 7fde9d400640 1 -- 172.21.15.55:0/1719581762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900094d0 con 0x7fdea0071db0 2024-05-10T03:32:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fde9d400640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c7f0 tx=0x7fde8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c004220 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c004380 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea015a620 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007840 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.403+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea015a340 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.405+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0138ae0 con 0x7fdea0071db0 2024-05-10T03:32:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.405+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde8c007bb0 con 0x7fdea0071db0 2024-05-10T03:32:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.407+0000 7fde96000640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 0x7fde8006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.407+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fde8c089500 con 0x7fdea0071db0 2024-05-10T03:32:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.408+0000 7fde9de00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 0x7fde8006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.409+0000 7fde9de00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 0x7fde8006d110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde90034000 tx=0x7fde9002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.411+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdea0138ae0 con 0x7fdea0071db0 2024-05-10T03:32:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.613+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea0138ae0 con 0x7fdea0071db0 2024-05-10T03:32:39.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.616+0000 7fde96000640 1 -- 172.21.15.55:0/1719581762 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16345 (secure 0 0 0) 0x7fdea0138ae0 con 0x7fdea0071db0 2024-05-10T03:32:39.621 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:39.621 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:24.417028+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 msgr2=0x7fde8006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 0x7fde8006d110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde90034000 tx=0x7fde9002e000 comp rx=0 tx=0).stop 2024-05-10T03:32:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 msgr2=0x7fdea0138dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c7f0 tx=0x7fde8c00ccb0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 shutdown_connections 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdea00b3860 0x7fdea01597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdea0071db0 0x7fdea0138dc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde8006ac00 0x7fde8006d110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.619+0000 7fdea4e00640 1 --2- 172.21.15.55:0/1719581762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdea006a4b0 0x7fdea0156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.620+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 >> 172.21.15.55:0/1719581762 conn(0x7fdea00cbb90 msgr2=0x7fdea013bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.620+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 shutdown_connections 2024-05-10T03:32:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.620+0000 7fdea4e00640 1 -- 172.21.15.55:0/1719581762 wait complete. 2024-05-10T03:32:39.650 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:32:39.651 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:39.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.924+0000 7f4b7fe00640 1 Processor -- start 2024-05-10T03:32:39.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.924+0000 7f4b7fe00640 1 -- start start 2024-05-10T03:32:39.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b80144710 0x7f4b80144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8006d350 con 0x7f4b800b3190 2024-05-10T03:32:39.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8006c740 con 0x7f4b8006d7e0 2024-05-10T03:32:39.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8006ca20 con 0x7f4b80144710 2024-05-10T03:32:39.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b80144710 0x7f4b80144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4b800b3190 0x7f4b800cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43540/0 (socket says 172.21.15.55:43540) 2024-05-10T03:32:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.927+0000 7f4b7f400640 1 -- 172.21.15.55:0/651199686 learned_addr learned my addr 172.21.15.55:0/651199686 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.928+0000 7f4b7ea00640 1 -- 172.21.15.55:0/651199686 >> v1:172.21.15.162:6789/0 conn(0x7f4b8006d7e0 legacy=0x7f4b800b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.928+0000 7f4b7ea00640 1 -- 172.21.15.55:0/651199686 >> v1:172.21.15.55:6789/0 conn(0x7f4b800b3190 legacy=0x7f4b800cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.928+0000 7f4b7ea00640 1 -- 172.21.15.55:0/651199686 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8006cd00 con 0x7f4b80144710 2024-05-10T03:32:39.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.929+0000 7f4b7ea00640 1 --2- 172.21.15.55:0/651199686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b80144710 0x7f4b80144120 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b700097a0 tx=0x7f4b7002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f87d0ef1d2d55ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.929+0000 7f4b7d600640 1 -- 172.21.15.55:0/651199686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7002adc0 con 0x7f4b80144710 2024-05-10T03:32:39.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.929+0000 7f4b7d600640 1 -- 172.21.15.55:0/651199686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70032440 con 0x7f4b80144710 2024-05-10T03:32:39.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.929+0000 7f4b7d600640 1 -- 172.21.15.55:0/651199686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700327d0 con 0x7f4b80144710 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 -- 172.21.15.55:0/651199686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b80144710 msgr2=0x7f4b80144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/651199686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b80144710 0x7f4b80144120 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b700097a0 tx=0x7f4b7002a0b0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 -- 172.21.15.55:0/651199686 shutdown_connections 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/651199686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b80144710 0x7f4b80144120 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 -- 172.21.15.55:0/651199686 >> 172.21.15.55:0/651199686 conn(0x7f4b8006a110 msgr2=0x7f4b8006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 -- 172.21.15.55:0/651199686 shutdown_connections 2024-05-10T03:32:39.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.930+0000 7f4b7fe00640 1 -- 172.21.15.55:0/651199686 wait complete. 2024-05-10T03:32:39.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.932+0000 7f4b7fe00640 1 Processor -- start 2024-05-10T03:32:39.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.933+0000 7f4b7fe00640 1 -- start start 2024-05-10T03:32:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b800b3190 0x7f4b80133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b80144710 0x7f4b80132a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801406f0 con 0x7f4b8006d7e0 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80140b20 con 0x7f4b80144710 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80131740 con 0x7f4b800b3190 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43888/0 (socket says 172.21.15.55:43888) 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.934+0000 7f4b7ea00640 1 -- 172.21.15.55:0/2237943872 learned_addr learned my addr 172.21.15.55:0/2237943872 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7e000640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b800b3190 0x7f4b80133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 -- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b800b3190 msgr2=0x7f4b80133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b800b3190 0x7f4b80133e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7f400640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b80144710 0x7f4b80132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 -- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b80144710 msgr2=0x7f4b80132a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b80144710 0x7f4b80132a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 -- 172.21.15.55:0/2237943872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70009400 con 0x7f4b8006d7e0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.935+0000 7f4b7ea00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4b700098c0 tx=0x7f4b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.936+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7003b810 con 0x7f4b8006d7e0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.936+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7003b970 con 0x7f4b8006d7e0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.936+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801325a0 con 0x7f4b8006d7e0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.936+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7003bd20 con 0x7f4b8006d7e0 2024-05-10T03:32:39.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.936+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b80131990 con 0x7f4b8006d7e0 2024-05-10T03:32:39.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.937+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b80066ce0 con 0x7f4b8006d7e0 2024-05-10T03:32:39.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.938+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b70039040 con 0x7f4b8006d7e0 2024-05-10T03:32:39.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.939+0000 7f4b76a00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 0x7f4b6006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:39.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.939+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f4b700a9b10 con 0x7f4b8006d7e0 2024-05-10T03:32:39.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.940+0000 7f4b7e000640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 0x7f4b6006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:39.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.941+0000 7f4b7e000640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 0x7f4b6006d140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b3df0 tx=0x7f4b6c0072f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:39.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:39.944+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4b70074a40 con 0x7f4b8006d7e0 2024-05-10T03:32:40.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.175+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b80066ce0 con 0x7f4b8006d7e0 2024-05-10T03:32:40.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.176+0000 7f4b76a00640 1 -- 172.21.15.55:0/2237943872 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7f4b70037020 con 0x7f4b8006d7e0 2024-05-10T03:32:40.180 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:40.181 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:40.181 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 msgr2=0x7f4b6006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 0x7f4b6006d140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b3df0 tx=0x7f4b6c0072f0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 msgr2=0x7f4b801401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4b700098c0 tx=0x7f4b700046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 shutdown_connections 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6006ac30 0x7f4b6006d140 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b80144710 0x7f4b80132a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b800b3190 0x7f4b80133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/2237943872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8006d7e0 0x7f4b801401b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 >> 172.21.15.55:0/2237943872 conn(0x7f4b8006a110 msgr2=0x7f4b800cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:40.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 shutdown_connections 2024-05-10T03:32:40.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.179+0000 7f4b7fe00640 1 -- 172.21.15.55:0/2237943872 wait complete. 2024-05-10T03:32:40.206 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:40.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.475+0000 7fe203000640 1 Processor -- start 2024-05-10T03:32:40.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.475+0000 7fe203000640 1 -- start start 2024-05-10T03:32:40.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.477+0000 7fe203000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1fc0b6680 0x7fe1fc15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.477+0000 7fe203000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1fc0b2170 0x7fe1fc1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.477+0000 7fe203000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc06bac0 con 0x7fe1fc1616e0 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.477+0000 7fe203000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0d2b20 con 0x7fe1fc0b6680 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.477+0000 7fe203000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc15db20 con 0x7fe1fc0b2170 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe202600640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1fc1616e0 0x7fe1fc0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43556/0 (socket says 172.21.15.55:43556) 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe202600640 1 -- 172.21.15.55:0/1390074438 learned_addr learned my addr 172.21.15.55:0/1390074438 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe201c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1fc0b6680 0x7fe1fc15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe201200640 1 --2- 172.21.15.55:0/1390074438 >> v2:172.21.15.162:3300/0 conn(0x7fe1fc0b2170 0x7fe1fc1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe1fbe00640 1 -- 172.21.15.55:0/1390074438 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099117761 0 0) 0x7fe1fc06bac0 con 0x7fe1fc1616e0 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe1fbe00640 1 -- 172.21.15.55:0/1390074438 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e0003520 con 0x7fe1fc1616e0 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe201c00640 1 -- 172.21.15.55:0/1390074438 >> v2:172.21.15.162:3300/0 conn(0x7fe1fc0b2170 msgr2=0x7fe1fc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.478+0000 7fe201c00640 1 --2- 172.21.15.55:0/1390074438 >> v2:172.21.15.162:3300/0 conn(0x7fe1fc0b2170 0x7fe1fc1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.479+0000 7fe201c00640 1 -- 172.21.15.55:0/1390074438 >> v1:172.21.15.55:6789/0 conn(0x7fe1fc1616e0 legacy=0x7fe1fc0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.479+0000 7fe201c00640 1 -- 172.21.15.55:0/1390074438 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0d25a0 con 0x7fe1fc0b6680 2024-05-10T03:32:40.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.480+0000 7fe201c00640 1 --2- 172.21.15.55:0/1390074438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b6680 0x7fe1fc15d5e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0cfaa0 tx=0x7fe1ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bed55a1c5878fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:40.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.480+0000 7fe1fbe00640 1 -- 172.21.15.55:0/1390074438 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02aea0 con 0x7fe1fc0b6680 2024-05-10T03:32:40.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.480+0000 7fe1fbe00640 1 -- 172.21.15.55:0/1390074438 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec02d050 con 0x7fe1fc0b6680 2024-05-10T03:32:40.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.480+0000 7fe1fbe00640 1 -- 172.21.15.55:0/1390074438 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec007940 con 0x7fe1fc0b6680 2024-05-10T03:32:40.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 -- 172.21.15.55:0/1390074438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b6680 msgr2=0x7fe1fc15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 --2- 172.21.15.55:0/1390074438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b6680 0x7fe1fc15d5e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0cfaa0 tx=0x7fe1ec02a680 comp rx=0 tx=0).stop 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 -- 172.21.15.55:0/1390074438 shutdown_connections 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 --2- 172.21.15.55:0/1390074438 >> v2:172.21.15.162:3300/0 conn(0x7fe1fc0b2170 0x7fe1fc1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 --2- 172.21.15.55:0/1390074438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b6680 0x7fe1fc15d5e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 -- 172.21.15.55:0/1390074438 >> 172.21.15.55:0/1390074438 conn(0x7fe1fc0b5730 msgr2=0x7fe1fc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 -- 172.21.15.55:0/1390074438 shutdown_connections 2024-05-10T03:32:40.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.481+0000 7fe203000640 1 -- 172.21.15.55:0/1390074438 wait complete. 2024-05-10T03:32:40.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.484+0000 7fe203000640 1 Processor -- start 2024-05-10T03:32:40.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.484+0000 7fe203000640 1 -- start start 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1fc0b6680 0x7fe1fc14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 0x7fe1fc14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15f320 con 0x7fe1fc0b2170 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc160610 con 0x7fe1fc1616e0 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe203000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15f570 con 0x7fe1fc0b6680 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43900/0 (socket says 172.21.15.55:43900) 2024-05-10T03:32:40.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 -- 172.21.15.55:0/2338307112 learned_addr learned my addr 172.21.15.55:0/2338307112 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201200640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1fc0b6680 0x7fe1fc14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 -- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1fc0b6680 msgr2=0x7fe1fc14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1fc0b6680 0x7fe1fc14d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe201c00640 1 -- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 msgr2=0x7fe1fc14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.486+0000 7fe202600640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 0x7fe1fc14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe201c00640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 0x7fe1fc14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe201c00640 1 -- 172.21.15.55:0/2338307112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec009a70 con 0x7fe1fc0b2170 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe202600640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 0x7fe1fc14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe201c00640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009e80 tx=0x7fe1ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec03b820 con 0x7fe1fc0b2170 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec03b980 con 0x7fe1fc0b2170 2024-05-10T03:32:40.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec03bd10 con 0x7fe1fc0b2170 2024-05-10T03:32:40.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc15f850 con 0x7fe1fc0b2170 2024-05-10T03:32:40.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.487+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc15fe10 con 0x7fe1fc0b2170 2024-05-10T03:32:40.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.489+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc04e290 con 0x7fe1fc0b2170 2024-05-10T03:32:40.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.490+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe1ec031400 con 0x7fe1fc0b2170 2024-05-10T03:32:40.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.491+0000 7fe1f9600640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 0x7fe1dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:40.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.491+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fe1ec0aa6e0 con 0x7fe1fc0b2170 2024-05-10T03:32:40.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.492+0000 7fe201200640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 0x7fe1dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:40.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.493+0000 7fe201200640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 0x7fe1dc06d140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:40.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.496+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe1ec0790a0 con 0x7fe1fc0b2170 2024-05-10T03:32:40.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.725+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1fc0d1510 con 0x7fe1fc0b2170 2024-05-10T03:32:40.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.726+0000 7fe1f9600640 1 -- 172.21.15.55:0/2338307112 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fe1ec009450 con 0x7fe1fc0b2170 2024-05-10T03:32:40.730 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:40.730 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:12.869975+0000","modified":"2024-05-10T03:32:16.337855+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3706115975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3706115975},{"type":"v1","addr":"172.21.15.55:6834","nonce":3706115975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-05-10T03:32:40.730 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 24 2024-05-10T03:32:40.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 msgr2=0x7fe1dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 0x7fe1dc06d140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8008040 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 msgr2=0x7fe1fc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009e80 tx=0x7fe1ec004a40 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 shutdown_connections 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1dc06ac30 0x7fe1dc06d140 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1fc1616e0 0x7fe1fc14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1fc0b6680 0x7fe1fc14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 --2- 172.21.15.55:0/2338307112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1fc0b2170 0x7fe1fc15ecb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 >> 172.21.15.55:0/2338307112 conn(0x7fe1fc0b5730 msgr2=0x7fe1fc15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 shutdown_connections 2024-05-10T03:32:40.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:40.729+0000 7fe203000640 1 -- 172.21.15.55:0/2338307112 wait complete. 2024-05-10T03:32:40.758 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:41.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.021+0000 7f464da00640 1 Processor -- start 2024-05-10T03:32:41.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.021+0000 7f464da00640 1 -- start start 2024-05-10T03:32:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b2170 0x7f464815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46481618b0 0x7f464806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464da00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480d25a0 con 0x7f46480b6680 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464da00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46480d2b20 con 0x7f46481618b0 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464815db20 con 0x7f46480b2170 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.023+0000 7f464d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46481618b0 0x7f464806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46481618b0 0x7f464806aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43906/0 (socket says 172.21.15.55:43906) 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 -- 172.21.15.55:0/320607092 learned_addr learned my addr 172.21.15.55:0/320607092 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f4647e00640 1 --1- 172.21.15.55:0/320607092 >> v1:172.21.15.55:6789/0 conn(0x7f46480b6680 0x7f46481611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43572/0 (socket says 172.21.15.55:43572) 2024-05-10T03:32:41.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f4647400640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b2170 0x7f464815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 -- 172.21.15.55:0/320607092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b2170 msgr2=0x7f464815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b2170 0x7f464815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 -- 172.21.15.55:0/320607092 >> v1:172.21.15.55:6789/0 conn(0x7f46480b6680 legacy=0x7f46481611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f464d000640 1 -- 172.21.15.55:0/320607092 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480b28d0 con 0x7f46481618b0 2024-05-10T03:32:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.024+0000 7f4646a00640 1 -- 172.21.15.55:0/320607092 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021478222 0 0) 0x7f46480d25a0 con 0x7f46480b6680 2024-05-10T03:32:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.025+0000 7f464d000640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46481618b0 0x7f464806aab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4640009870 tx=0x7f464002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c051bac76bb1aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f4646a00640 1 -- 172.21.15.55:0/320607092 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640032440 con 0x7f46481618b0 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f4646a00640 1 -- 172.21.15.55:0/320607092 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400325a0 con 0x7f46481618b0 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f4646a00640 1 -- 172.21.15.55:0/320607092 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640032930 con 0x7f46481618b0 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 -- 172.21.15.55:0/320607092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46481618b0 msgr2=0x7f464806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46481618b0 0x7f464806aab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4640009870 tx=0x7f464002a270 comp rx=0 tx=0).stop 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 -- 172.21.15.55:0/320607092 shutdown_connections 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b2170 0x7f464815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 --2- 172.21.15.55:0/320607092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46481618b0 0x7f464806aab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 -- 172.21.15.55:0/320607092 >> 172.21.15.55:0/320607092 conn(0x7f46480b5730 msgr2=0x7f46480d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.026+0000 7f464da00640 1 -- 172.21.15.55:0/320607092 shutdown_connections 2024-05-10T03:32:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.027+0000 7f464da00640 1 -- 172.21.15.55:0/320607092 wait complete. 2024-05-10T03:32:41.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.029+0000 7f464da00640 1 Processor -- start 2024-05-10T03:32:41.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.029+0000 7f464da00640 1 -- start start 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b6680 0x7f464815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46481618b0 0x7f46480b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4648155430 con 0x7f46480b2170 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464815df10 con 0x7f46481618b0 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.030+0000 7f464da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464815dc60 con 0x7f46480b6680 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43918/0 (socket says 172.21.15.55:43918) 2024-05-10T03:32:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 -- 172.21.15.55:0/3205375845 learned_addr learned my addr 172.21.15.55:0/3205375845 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 -- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b6680 msgr2=0x7f464815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647400640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b6680 0x7f464815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b6680 0x7f464815f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 -- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46481618b0 msgr2=0x7f46480b7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46481618b0 0x7f46480b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 -- 172.21.15.55:0/3205375845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400094d0 con 0x7f46480b2170 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.031+0000 7f4647e00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f463800c7c0 tx=0x7f463800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.032+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800a640 con 0x7f46480b2170 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.032+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463800a7c0 con 0x7f46480b2170 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.032+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800ab70 con 0x7f46480b2170 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.032+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464815e160 con 0x7f46480b2170 2024-05-10T03:32:41.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.032+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464815e720 con 0x7f46480b2170 2024-05-10T03:32:41.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.033+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4648067740 con 0x7f46480b2170 2024-05-10T03:32:41.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.034+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f463800acd0 con 0x7f46480b2170 2024-05-10T03:32:41.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.035+0000 7f463fe00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 0x7f462406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.035+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f46380889c0 con 0x7f46480b2170 2024-05-10T03:32:41.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.036+0000 7f4647400640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 0x7f462406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.037+0000 7f4647400640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 0x7f462406d140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4630006fd0 tx=0x7f4630008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.040+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4638053a10 con 0x7f46480b2170 2024-05-10T03:32:41.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.234+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f464804e290 con 0x7f46480b2170 2024-05-10T03:32:41.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.236+0000 7f463fe00640 1 -- 172.21.15.55:0/3205375845 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16345 (secure 0 0 0) 0x7f4638018020 con 0x7f46480b2170 2024-05-10T03:32:41.240 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:41.241 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":54,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:24.417028+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:11.083041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:11.883212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.38","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 msgr2=0x7f462406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 0x7f462406d140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4630006fd0 tx=0x7f4630008040 comp rx=0 tx=0).stop 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 msgr2=0x7f4648152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f463800c7c0 tx=0x7f463800cc80 comp rx=0 tx=0).stop 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 shutdown_connections 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f462406ac30 0x7f462406d140 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46481618b0 0x7f46480b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46480b6680 0x7f464815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 --2- 172.21.15.55:0/3205375845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46480b2170 0x7f4648152610 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 >> 172.21.15.55:0/3205375845 conn(0x7f46480b5730 msgr2=0x7f464815b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 shutdown_connections 2024-05-10T03:32:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.239+0000 7f464da00640 1 -- 172.21.15.55:0/3205375845 wait complete. 2024-05-10T03:32:41.270 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:32:41.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.541+0000 7f7d2ae00640 1 Processor -- start 2024-05-10T03:32:41.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.541+0000 7f7d2ae00640 1 -- start start 2024-05-10T03:32:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.543+0000 7f7d2ae00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2406b830 0x7f7d24143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.543+0000 7f7d2ae00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240cdbe0 con 0x7f7d240b2040 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.543+0000 7f7d2ae00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2415e8b0 con 0x7f7d241435d0 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.543+0000 7f7d2ae00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240ce140 con 0x7f7d2406b830 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.544+0000 7f7d29a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7d240b2040 0x7f7d240ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43582/0 (socket says 172.21.15.55:43582) 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.544+0000 7f7d29a00640 1 -- 172.21.15.55:0/1816303028 learned_addr learned my addr 172.21.15.55:0/1816303028 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.544+0000 7f7d2a400640 1 --2- 172.21.15.55:0/1816303028 >> v2:172.21.15.55:3300/0 conn(0x7f7d2406b830 0x7f7d24143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.544+0000 7f7d23e00640 1 -- 172.21.15.55:0/1816303028 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245820390 0 0) 0x7f7d240cdbe0 con 0x7f7d240b2040 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.544+0000 7f7d23e00640 1 -- 172.21.15.55:0/1816303028 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c005730 con 0x7f7d240b2040 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.545+0000 7f7d2a400640 1 -- 172.21.15.55:0/1816303028 >> v1:172.21.15.162:6789/0 conn(0x7f7d241435d0 legacy=0x7f7d2406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.545+0000 7f7d2a400640 1 -- 172.21.15.55:0/1816303028 >> v1:172.21.15.55:6789/0 conn(0x7f7d240b2040 legacy=0x7f7d240ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.545+0000 7f7d2a400640 1 -- 172.21.15.55:0/1816303028 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2415ede0 con 0x7f7d2406b830 2024-05-10T03:32:41.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.546+0000 7f7d2a400640 1 --2- 172.21.15.55:0/1816303028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2406b830 0x7f7d24143020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0097d0 tx=0x7f7d1c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8b074002e9f268 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.546+0000 7f7d23e00640 1 -- 172.21.15.55:0/1816303028 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c032440 con 0x7f7d2406b830 2024-05-10T03:32:41.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.546+0000 7f7d23e00640 1 -- 172.21.15.55:0/1816303028 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0325a0 con 0x7f7d2406b830 2024-05-10T03:32:41.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.546+0000 7f7d23e00640 1 -- 172.21.15.55:0/1816303028 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c032910 con 0x7f7d2406b830 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 -- 172.21.15.55:0/1816303028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2406b830 msgr2=0x7f7d24143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/1816303028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2406b830 0x7f7d24143020 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0097d0 tx=0x7f7d1c02a1c0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 -- 172.21.15.55:0/1816303028 shutdown_connections 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/1816303028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2406b830 0x7f7d24143020 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 -- 172.21.15.55:0/1816303028 >> 172.21.15.55:0/1816303028 conn(0x7f7d2406c730 msgr2=0x7f7d240b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:41.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 -- 172.21.15.55:0/1816303028 shutdown_connections 2024-05-10T03:32:41.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.547+0000 7f7d2ae00640 1 -- 172.21.15.55:0/1816303028 wait complete. 2024-05-10T03:32:41.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.549+0000 7f7d2ae00640 1 Processor -- start 2024-05-10T03:32:41.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.549+0000 7f7d2ae00640 1 -- start start 2024-05-10T03:32:41.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d240b2040 0x7f7d24137650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2415d3c0 0x7f7d240b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d241f41c0 con 0x7f7d241435d0 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415c0d0 con 0x7f7d240b2040 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2ae00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2413ee60 con 0x7f7d2415d3c0 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43930/0 (socket says 172.21.15.55:43930) 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 -- 172.21.15.55:0/999403180 learned_addr learned my addr 172.21.15.55:0/999403180 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29a00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d240b2040 0x7f7d24137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d2a400640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2415d3c0 0x7f7d240b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 -- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2415d3c0 msgr2=0x7f7d240b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2415d3c0 0x7f7d240b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.551+0000 7f7d29000640 1 -- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d240b2040 msgr2=0x7f7d24137650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d29000640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d240b2040 0x7f7d24137650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d29000640 1 -- 172.21.15.55:0/999403180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009400 con 0x7f7d241435d0 2024-05-10T03:32:41.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d29000640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c3c0 tx=0x7f7d1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100052d0 con 0x7f7d241435d0 2024-05-10T03:32:41.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10005470 con 0x7f7d241435d0 2024-05-10T03:32:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10006a60 con 0x7f7d241435d0 2024-05-10T03:32:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d24137b90 con 0x7f7d241435d0 2024-05-10T03:32:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.552+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2415c600 con 0x7f7d241435d0 2024-05-10T03:32:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.553+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d240b1680 con 0x7f7d241435d0 2024-05-10T03:32:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.555+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d10006bc0 con 0x7f7d241435d0 2024-05-10T03:32:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.556+0000 7f7d21600640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 0x7f7d0406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.556+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7d100887f0 con 0x7f7d241435d0 2024-05-10T03:32:41.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.556+0000 7f7d29a00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 0x7f7d0406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:41.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.558+0000 7f7d29a00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 0x7f7d0406d110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7d18001560 tx=0x7f7d18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:41.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.561+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d10053850 con 0x7f7d241435d0 2024-05-10T03:32:41.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.790+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7d24140400 con 0x7f7d241435d0 2024-05-10T03:32:41.797 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:32:41.793+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:32:41.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.813+0000 7f7d21600640 1 -- 172.21.15.55:0/999403180 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7d10056e70 con 0x7f7d241435d0 2024-05-10T03:32:41.817 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 msgr2=0x7f7d0406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 0x7f7d0406d110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7d18001560 tx=0x7f7d18009040 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 msgr2=0x7f7d240afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c3c0 tx=0x7f7d1000c880 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 shutdown_connections 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2415d3c0 0x7f7d240b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d241435d0 0x7f7d240afd00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0406ac00 0x7f7d0406d110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 --2- 172.21.15.55:0/999403180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d240b2040 0x7f7d24137650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.815+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 >> 172.21.15.55:0/999403180 conn(0x7f7d2406c730 msgr2=0x7f7d2415e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:41.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.816+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 shutdown_connections 2024-05-10T03:32:41.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:41.816+0000 7f7d2ae00640 1 -- 172.21.15.55:0/999403180 wait complete. 2024-05-10T03:32:41.835 INFO:tasks.ceph.mds.c.smithi055.stderr: -2240> 2024-05-10T03:32:17.244+0000 7f09f7600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:32:41.844 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:32:41.975 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:32:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.101+0000 7feaf3200640 1 Processor -- start 2024-05-10T03:32:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.101+0000 7feaf3200640 1 -- start start 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.103+0000 7feaf3200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feaf415e0b0 0x7feaf40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.103+0000 7feaf3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feaf415a9e0 0x7feaf4164a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.103+0000 7feaf3200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf415e590 con 0x7feaf4159ec0 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.103+0000 7feaf3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40b2960 con 0x7feaf415a9e0 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.103+0000 7feaf3200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40b2c40 con 0x7feaf415e0b0 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.104+0000 7feaf2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feaf415a9e0 0x7feaf4164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.104+0000 7feaf1400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feaf4159ec0 0x7feaf415a2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43604/0 (socket says 172.21.15.55:43604) 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.104+0000 7feaf1400640 1 -- 172.21.15.55:0/1694836406 learned_addr learned my addr 172.21.15.55:0/1694836406 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.104+0000 7feaf1e00640 1 --2- 172.21.15.55:0/1694836406 >> v2:172.21.15.162:3300/0 conn(0x7feaf415e0b0 0x7feaf40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:42.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.104+0000 7feaf2800640 1 -- 172.21.15.55:0/1694836406 >> v2:172.21.15.162:3300/0 conn(0x7feaf415e0b0 msgr2=0x7feaf40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:42.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.105+0000 7feaf2800640 1 --2- 172.21.15.55:0/1694836406 >> v2:172.21.15.162:3300/0 conn(0x7feaf415e0b0 0x7feaf40713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.105+0000 7feaf2800640 1 -- 172.21.15.55:0/1694836406 >> v1:172.21.15.55:6789/0 conn(0x7feaf4159ec0 legacy=0x7feaf415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.105+0000 7feaf2800640 1 -- 172.21.15.55:0/1694836406 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40b2f20 con 0x7feaf415a9e0 2024-05-10T03:32:42.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.105+0000 7feaf0a00640 1 -- 172.21.15.55:0/1694836406 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538206979 0 0) 0x7feaf415e590 con 0x7feaf4159ec0 2024-05-10T03:32:42.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.106+0000 7feaf2800640 1 --2- 172.21.15.55:0/1694836406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf415a9e0 0x7feaf4164a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f3b0 tx=0x7feaec02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2731b387c5f523 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:42.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.106+0000 7feaf0a00640 1 -- 172.21.15.55:0/1694836406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec032790 con 0x7feaf415a9e0 2024-05-10T03:32:42.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.106+0000 7feaf0a00640 1 -- 172.21.15.55:0/1694836406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec0328f0 con 0x7feaf415a9e0 2024-05-10T03:32:42.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.106+0000 7feaf0a00640 1 -- 172.21.15.55:0/1694836406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec032c80 con 0x7feaf415a9e0 2024-05-10T03:32:42.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 -- 172.21.15.55:0/1694836406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf415a9e0 msgr2=0x7feaf4164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 --2- 172.21.15.55:0/1694836406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf415a9e0 0x7feaf4164a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7feaf406f3b0 tx=0x7feaec02c890 comp rx=0 tx=0).stop 2024-05-10T03:32:42.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 -- 172.21.15.55:0/1694836406 shutdown_connections 2024-05-10T03:32:42.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 --2- 172.21.15.55:0/1694836406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf415a9e0 0x7feaf4164a30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 --2- 172.21.15.55:0/1694836406 >> v2:172.21.15.162:3300/0 conn(0x7feaf415e0b0 0x7feaf40713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 -- 172.21.15.55:0/1694836406 >> 172.21.15.55:0/1694836406 conn(0x7feaf40b1290 msgr2=0x7feaf406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:42.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 -- 172.21.15.55:0/1694836406 shutdown_connections 2024-05-10T03:32:42.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.107+0000 7feaf3200640 1 -- 172.21.15.55:0/1694836406 wait complete. 2024-05-10T03:32:42.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.110+0000 7feaf3200640 1 Processor -- start 2024-05-10T03:32:42.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.110+0000 7feaf3200640 1 -- start start 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.111+0000 7feaf3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaf415a9e0 0x7feaf40c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaf415e0b0 0x7feaf40c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf41595c0 con 0x7feaf4159ec0 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40c96c0 con 0x7feaf415e0b0 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40c4af0 con 0x7feaf415a9e0 2024-05-10T03:32:42.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43972/0 (socket says 172.21.15.55:43972) 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 -- 172.21.15.55:0/2492839737 learned_addr learned my addr 172.21.15.55:0/2492839737 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 -- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaf415a9e0 msgr2=0x7feaf40c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaf415a9e0 0x7feaf40c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 -- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaf415e0b0 msgr2=0x7feaf40c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaf415e0b0 0x7feaf40c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.112+0000 7feaf1e00640 1 -- 172.21.15.55:0/2492839737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009a70 con 0x7feaf4159ec0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaf1e00640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e870 tx=0x7feadc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0094f0 con 0x7feaf4159ec0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00c940 con 0x7feaf4159ec0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00ccd0 con 0x7feaf4159ec0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf40c5950 con 0x7feaf4159ec0 2024-05-10T03:32:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.113+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf40c5020 con 0x7feaf4159ec0 2024-05-10T03:32:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.114+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf404e290 con 0x7feaf4159ec0 2024-05-10T03:32:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.116+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feadc013470 con 0x7feaf4159ec0 2024-05-10T03:32:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.116+0000 7feaea000640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 0x7fead006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.117+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6173+0+0 (secure 0 0 0) 0x7feadc0896b0 con 0x7feaf4159ec0 2024-05-10T03:32:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.117+0000 7feaf1400640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 0x7fead006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.118+0000 7feaf1400640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 0x7fead006d110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feae4007fd0 tx=0x7feae4007960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.121+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feadc054590 con 0x7feaf4159ec0 2024-05-10T03:32:42.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.367+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feaf40aebd0 con 0x7feaf4159ec0 2024-05-10T03:32:42.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.869+0000 7feaea000640 1 -- 172.21.15.55:0/2492839737 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7feadc057bb0 con 0x7feaf4159ec0 2024-05-10T03:32:42.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 msgr2=0x7fead006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 0x7fead006d110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feae4007fd0 tx=0x7feae4007960 comp rx=0 tx=0).stop 2024-05-10T03:32:42.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 msgr2=0x7feaf40c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:42.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e870 tx=0x7feadc00ed30 comp rx=0 tx=0).stop 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 shutdown_connections 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fead006ac00 0x7fead006d110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feaf415e0b0 0x7feaf40c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feaf415a9e0 0x7feaf40c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 --2- 172.21.15.55:0/2492839737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf4159ec0 0x7feaf40c5de0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 >> 172.21.15.55:0/2492839737 conn(0x7feaf40b1290 msgr2=0x7feaf40afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 shutdown_connections 2024-05-10T03:32:42.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:42.874+0000 7feaf3200640 1 -- 172.21.15.55:0/2492839737 wait complete. 2024-05-10T03:32:42.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:32:43.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.189+0000 7fd842000640 1 Processor -- start 2024-05-10T03:32:43.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.189+0000 7fd842000640 1 -- start start 2024-05-10T03:32:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.191+0000 7fd842000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c0ce520 0x7fd83c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.191+0000 7fd842000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c0b2240 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.191+0000 7fd842000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c0b2490 con 0x7fd83c0d1330 2024-05-10T03:32:43.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.191+0000 7fd842000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0d2480 con 0x7fd83c0ce520 2024-05-10T03:32:43.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.192+0000 7fd840c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd83c0b1cf0 0x7fd83c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45574/0 (socket says 172.21.15.55:45574) 2024-05-10T03:32:43.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.192+0000 7fd840c00640 1 -- 172.21.15.55:0/1971530430 learned_addr learned my addr 172.21.15.55:0/1971530430 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.192+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932944860 0 0) 0x7fd83c0b2240 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.192+0000 7fd83be00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c0ce520 0x7fd83c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.192+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd824003520 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 590832542 0 0) 0x7fd824003520 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c0ce520 msgr2=0x7fd83c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c0ce520 0x7fd83c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 >> v1:172.21.15.162:6789/0 conn(0x7fd83c0d1330 legacy=0x7fd83c06a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0d2760 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fd830001670 con 0x7fd83c0b1cf0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 >> v1:172.21.15.55:6789/0 conn(0x7fd83c0b1cf0 legacy=0x7fd83c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8240039a0 0x7fd824003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd824004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd824005110 0x7fd82400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0d2480 con 0x7fd8240047c0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400c020 con 0x7fd824005110 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400c1e0 con 0x7fd8240039a0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83be00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd824004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd840c00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8240039a0 0x7fd824003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd841600640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd824005110 0x7fd82400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83be00640 1 -- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8240039a0 msgr2=0x7fd824003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:43.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83be00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8240039a0 0x7fd824003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83be00640 1 -- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd824005110 msgr2=0x7fd82400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:43.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.193+0000 7fd83be00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd824005110 0x7fd82400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.194+0000 7fd83be00640 1 -- 172.21.15.55:0/1971530430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824003520 con 0x7fd8240047c0 2024-05-10T03:32:43.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.194+0000 7fd840c00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8240039a0 0x7fd824003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.195+0000 7fd83be00640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd824004bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ba10 tx=0x7fd8240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b54064e8590382 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.195+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82802ade0 con 0x7fd8240047c0 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.195+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82802e070 con 0x7fd8240047c0 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.195+0000 7fd83b400640 1 -- 172.21.15.55:0/1971530430 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828034620 con 0x7fd8240047c0 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 -- 172.21.15.55:0/1971530430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 msgr2=0x7fd824004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 --2- 172.21.15.55:0/1971530430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd824004bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ba10 tx=0x7fd8240057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd841600640 1 -- 172.21.15.55:0/1971530430 reap_dead start 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 -- 172.21.15.55:0/1971530430 shutdown_connections 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 -- 172.21.15.55:0/1971530430 >> 172.21.15.55:0/1971530430 conn(0x7fd83c0d0850 msgr2=0x7fd83c0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 -- 172.21.15.55:0/1971530430 shutdown_connections 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.196+0000 7fd842000640 1 -- 172.21.15.55:0/1971530430 wait complete. 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.199+0000 7fd842000640 1 Processor -- start 2024-05-10T03:32:43.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.199+0000 7fd842000640 1 -- start start 2024-05-10T03:32:43.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c160550 0x7fd83c1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 0x7fd83c15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c15e560 con 0x7fd8240047c0 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c160190 con 0x7fd83c160960 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.200+0000 7fd842000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c143e90 con 0x7fd83c160550 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46280/0 (socket says 172.21.15.55:46280) 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 -- 172.21.15.55:0/2622956252 learned_addr learned my addr 172.21.15.55:0/2622956252 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd83be00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c160550 0x7fd83c1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd841600640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 0x7fd83c15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 -- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c160550 msgr2=0x7fd83c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c160550 0x7fd83c1433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 -- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 msgr2=0x7fd83c15f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 0x7fd83c15f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd840c00640 1 -- 172.21.15.55:0/2622956252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c15dec0 con 0x7fd8240047c0 2024-05-10T03:32:43.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.201+0000 7fd841600640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 0x7fd83c15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:43.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd840c00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd83000d8c0 tx=0x7fd83000dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:43.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300074e0 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd830007640 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300049e0 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c1440e0 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.202+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c15fed0 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.203+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c067220 con 0x7fd8240047c0 2024-05-10T03:32:43.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.205+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd830004b40 con 0x7fd8240047c0 2024-05-10T03:32:43.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.205+0000 7fd838c00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 0x7fd81c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:43.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.206+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6173+0+0 (secure 0 0 0) 0x7fd830088f60 con 0x7fd8240047c0 2024-05-10T03:32:43.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.206+0000 7fd83be00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 0x7fd81c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:43.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.207+0000 7fd83be00640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 0x7fd81c06d140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd83c15dd10 tx=0x7fd828030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:43.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.211+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd830053ef0 con 0x7fd8240047c0 2024-05-10T03:32:43.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.416+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd83c15aa10 con 0x7fd8240047c0 2024-05-10T03:32:43.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.876+0000 7fd838c00640 1 -- 172.21.15.55:0/2622956252 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd830019070 con 0x7fd8240047c0 2024-05-10T03:32:43.881 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:32:43.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.882+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 msgr2=0x7fd81c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.882+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 0x7fd81c06d140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd83c15dd10 tx=0x7fd828030000 comp rx=0 tx=0).stop 2024-05-10T03:32:43.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.882+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 msgr2=0x7fd83c15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:43.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.882+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd83000d8c0 tx=0x7fd83000dd80 comp rx=0 tx=0).stop 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 shutdown_connections 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd81c06ac30 0x7fd81c06d140 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c160960 0x7fd83c15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83c160550 0x7fd83c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 --2- 172.21.15.55:0/2622956252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8240047c0 0x7fd83c15e980 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.883+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 >> 172.21.15.55:0/2622956252 conn(0x7fd83c0d0850 msgr2=0x7fd83c0b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.884+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 shutdown_connections 2024-05-10T03:32:43.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:43.884+0000 7fd842000640 1 -- 172.21.15.55:0/2622956252 wait complete. 2024-05-10T03:32:43.922 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:32:44.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.208+0000 7f54b4a00640 1 Processor -- start 2024-05-10T03:32:44.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.208+0000 7f54b4a00640 1 -- start start 2024-05-10T03:32:44.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.210+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00ad310 0x7f54b00ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.210+0000 7f54b4a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54b0161e20 0x7f54b015df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.210+0000 7f54b4a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b015ee60 con 0x7f54b00abf60 2024-05-10T03:32:44.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.210+0000 7f54b4a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b0158350 con 0x7f54b0161e20 2024-05-10T03:32:44.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.210+0000 7f54b4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b015e8a0 con 0x7f54b00ad310 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ad400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54b0161e20 0x7f54b015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ae800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f54b00abf60 0x7f54b00ac370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45582/0 (socket says 172.21.15.55:45582) 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ae800640 1 -- 172.21.15.55:0/1756389947 learned_addr learned my addr 172.21.15.55:0/1756389947 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ade00640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00ad310 0x7f54b00ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ad400640 1 -- 172.21.15.55:0/1756389947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00ad310 msgr2=0x7f54b00ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ad400640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00ad310 0x7f54b00ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ad400640 1 -- 172.21.15.55:0/1756389947 >> v1:172.21.15.55:6789/0 conn(0x7f54b00abf60 legacy=0x7f54b00ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54aca00640 1 -- 172.21.15.55:0/1756389947 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527761347 0 0) 0x7f54b015ee60 con 0x7f54b00abf60 2024-05-10T03:32:44.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.211+0000 7f54ad400640 1 -- 172.21.15.55:0/1756389947 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b015eb80 con 0x7f54b0161e20 2024-05-10T03:32:44.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.212+0000 7f54ad400640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b0161e20 0x7f54b015df00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f549c0097d0 tx=0x7f549c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bc27e378cdb0c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:44.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.213+0000 7f54aca00640 1 -- 172.21.15.55:0/1756389947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c02ae40 con 0x7f54b0161e20 2024-05-10T03:32:44.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.213+0000 7f54aca00640 1 -- 172.21.15.55:0/1756389947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c032440 con 0x7f54b0161e20 2024-05-10T03:32:44.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.213+0000 7f54aca00640 1 -- 172.21.15.55:0/1756389947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0327d0 con 0x7f54b0161e20 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 -- 172.21.15.55:0/1756389947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b0161e20 msgr2=0x7f54b015df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b0161e20 0x7f54b015df00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f549c0097d0 tx=0x7f549c02a130 comp rx=0 tx=0).stop 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 -- 172.21.15.55:0/1756389947 shutdown_connections 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b0161e20 0x7f54b015df00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 --2- 172.21.15.55:0/1756389947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00ad310 0x7f54b00ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 -- 172.21.15.55:0/1756389947 >> 172.21.15.55:0/1756389947 conn(0x7f54b00b0cf0 msgr2=0x7f54b00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:44.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 -- 172.21.15.55:0/1756389947 shutdown_connections 2024-05-10T03:32:44.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.214+0000 7f54b4a00640 1 -- 172.21.15.55:0/1756389947 wait complete. 2024-05-10T03:32:44.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.216+0000 7f54b4a00640 1 Processor -- start 2024-05-10T03:32:44.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.216+0000 7f54b4a00640 1 -- start start 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00abf60 0x7f54b015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0161e20 0x7f54b015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01f4550 con 0x7f54b00ad310 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b015bfa0 con 0x7f54b0161e20 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54b4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b015c3d0 con 0x7f54b00abf60 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54ad400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54ad400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46296/0 (socket says 172.21.15.55:46296) 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.218+0000 7f54ad400640 1 -- 172.21.15.55:0/2153076629 learned_addr learned my addr 172.21.15.55:0/2153076629 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ade00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00abf60 0x7f54b015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ae800640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0161e20 0x7f54b015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 -- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00abf60 msgr2=0x7f54b015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00abf60 0x7f54b015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 -- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0161e20 msgr2=0x7f54b015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0161e20 0x7f54b015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 -- 172.21.15.55:0/2153076629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c009400 con 0x7f54b00ad310 2024-05-10T03:32:44.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.219+0000 7f54ad400640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f549c0098f0 tx=0x7f549c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:44.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.220+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c004030 con 0x7f54b00ad310 2024-05-10T03:32:44.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.220+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c004190 con 0x7f54b00ad310 2024-05-10T03:32:44.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.220+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01f3a60 con 0x7f54b00ad310 2024-05-10T03:32:44.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.220+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c030900 con 0x7f54b00ad310 2024-05-10T03:32:44.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.220+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b01f3cb0 con 0x7f54b00ad310 2024-05-10T03:32:44.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.221+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0067290 con 0x7f54b00ad310 2024-05-10T03:32:44.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.222+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f549c030a60 con 0x7f54b00ad310 2024-05-10T03:32:44.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.223+0000 7f54a6000640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 0x7f549006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:44.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.223+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5735+0+0 (secure 0 0 0) 0x7f549c0aa230 con 0x7f54b00ad310 2024-05-10T03:32:44.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.224+0000 7f54ade00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 0x7f549006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:44.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.225+0000 7f54ade00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 0x7f549006d110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54a0006fd0 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:44.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.228+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f549c075360 con 0x7f54b00ad310 2024-05-10T03:32:44.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.427+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54b0157ba0 con 0x7f54b00ad310 2024-05-10T03:32:44.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.895+0000 7f54a6000640 1 -- 172.21.15.55:0/2153076629 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f549c04b0d0 con 0x7f54b00ad310 2024-05-10T03:32:44.900 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:32:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 msgr2=0x7f549006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 0x7f549006d110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54a0006fd0 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 msgr2=0x7f54b01559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f549c0098f0 tx=0x7f549c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 shutdown_connections 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f549006ac00 0x7f549006d110 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f54a0006fd0 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0161e20 0x7f54b015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54b00ad310 0x7f54b01559f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 --2- 172.21.15.55:0/2153076629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54b00abf60 0x7f54b015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.900+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 >> 172.21.15.55:0/2153076629 conn(0x7f54b00b0cf0 msgr2=0x7f54b00af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.901+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 shutdown_connections 2024-05-10T03:32:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:44.901+0000 7f54b4a00640 1 -- 172.21.15.55:0/2153076629 wait complete. 2024-05-10T03:32:44.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:32:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.207+0000 7f07b3200640 1 Processor -- start 2024-05-10T03:32:45.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.207+0000 7f07b3200640 1 -- start start 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.209+0000 7f07b3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161310 0x7f07ac15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.209+0000 7f07b3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07ac161c90 0x7f07ac0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b3200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0d2750 con 0x7f07ac0d2340 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac15e370 con 0x7f07ac161c90 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac15e650 con 0x7f07ac161310 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b2800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f07ac0d2340 0x7f07ac0cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45598/0 (socket says 172.21.15.55:45598) 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b2800640 1 -- 172.21.15.55:0/1769696547 learned_addr learned my addr 172.21.15.55:0/1769696547 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b1400640 1 --2- 172.21.15.55:0/1769696547 >> v2:172.21.15.55:3300/0 conn(0x7f07ac161c90 0x7f07ac0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.210+0000 7f07b1e00640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161310 0x7f07ac15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b0a00640 1 -- 172.21.15.55:0/1769696547 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89429418 0 0) 0x7f07ac0d2750 con 0x7f07ac0d2340 2024-05-10T03:32:45.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b0a00640 1 -- 172.21.15.55:0/1769696547 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0794003520 con 0x7f07ac0d2340 2024-05-10T03:32:45.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b1400640 1 -- 172.21.15.55:0/1769696547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161310 msgr2=0x7f07ac15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b1400640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161310 0x7f07ac15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b1400640 1 -- 172.21.15.55:0/1769696547 >> v1:172.21.15.55:6789/0 conn(0x7f07ac0d2340 legacy=0x7f07ac0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.211+0000 7f07b1400640 1 -- 172.21.15.55:0/1769696547 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac15e930 con 0x7f07ac161c90 2024-05-10T03:32:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.212+0000 7f07b1400640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161c90 0x7f07ac0d1e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f079c0097a0 tx=0x7f079c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43427a2c75faabf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.212+0000 7f07b0a00640 1 -- 172.21.15.55:0/1769696547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c032440 con 0x7f07ac161c90 2024-05-10T03:32:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.212+0000 7f07b0a00640 1 -- 172.21.15.55:0/1769696547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c0325a0 con 0x7f07ac161c90 2024-05-10T03:32:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.212+0000 7f07b0a00640 1 -- 172.21.15.55:0/1769696547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c032930 con 0x7f07ac161c90 2024-05-10T03:32:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 -- 172.21.15.55:0/1769696547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161c90 msgr2=0x7f07ac0d1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161c90 0x7f07ac0d1e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f079c0097a0 tx=0x7f079c02a180 comp rx=0 tx=0).stop 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 -- 172.21.15.55:0/1769696547 shutdown_connections 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161c90 0x7f07ac0d1e00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 --2- 172.21.15.55:0/1769696547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161310 0x7f07ac15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 -- 172.21.15.55:0/1769696547 >> 172.21.15.55:0/1769696547 conn(0x7f07ac0d08a0 msgr2=0x7f07ac0d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 -- 172.21.15.55:0/1769696547 shutdown_connections 2024-05-10T03:32:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.213+0000 7f07b3200640 1 -- 172.21.15.55:0/1769696547 wait complete. 2024-05-10T03:32:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.216+0000 7f07b3200640 1 Processor -- start 2024-05-10T03:32:45.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.216+0000 7f07b3200640 1 -- start start 2024-05-10T03:32:45.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 0x7f07ac151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161c90 0x7f07ac1eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1f4b50 con 0x7f07ac161310 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1e8860 con 0x7f07ac0d2340 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.217+0000 7f07b3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1524b0 con 0x7f07ac161c90 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46302/0 (socket says 172.21.15.55:46302) 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 -- 172.21.15.55:0/2254301271 learned_addr learned my addr 172.21.15.55:0/2254301271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1e00640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 0x7f07ac151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 -- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161c90 msgr2=0x7f07ac1eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b2800640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161c90 0x7f07ac1eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161c90 0x7f07ac1eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 -- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 msgr2=0x7f07ac151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 0x7f07ac151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1400640 1 -- 172.21.15.55:0/2254301271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c009400 con 0x7f07ac161310 2024-05-10T03:32:45.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.218+0000 7f07b1e00640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 0x7f07ac151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:45.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07b1400640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098c0 tx=0x7f079c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c03d830 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c03d990 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c03dd40 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac1e8ab0 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.219+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac1e9070 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.221+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac15db00 con 0x7f07ac161310 2024-05-10T03:32:45.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.222+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f079c039040 con 0x7f07ac161310 2024-05-10T03:32:45.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.222+0000 7f07a6000640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 0x7f079006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.223+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f079c0a9a60 con 0x7f07ac161310 2024-05-10T03:32:45.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.223+0000 7f07b1e00640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 0x7f079006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.225+0000 7f07b1e00640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 0x7f079006d110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009a20 tx=0x7f07a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.227+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f079c074d60 con 0x7f07ac161310 2024-05-10T03:32:45.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.426+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f07ac0c6280 con 0x7f07ac161310 2024-05-10T03:32:45.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.437+0000 7f07a6000640 1 -- 172.21.15.55:0/2254301271 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f079c078380 con 0x7f07ac161310 2024-05-10T03:32:45.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 msgr2=0x7f079006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 0x7f079006d110 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009a20 tx=0x7f07a0008040 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 msgr2=0x7f07ac1ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098c0 tx=0x7f079c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 shutdown_connections 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07ac161c90 0x7f07ac1eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07ac161310 0x7f07ac1ea4c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f079006ac00 0x7f079006d110 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 --2- 172.21.15.55:0/2254301271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07ac0d2340 0x7f07ac151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.440+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 >> 172.21.15.55:0/2254301271 conn(0x7f07ac0d08a0 msgr2=0x7f07ac0b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.441+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 shutdown_connections 2024-05-10T03:32:45.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.441+0000 7f07b3200640 1 -- 172.21.15.55:0/2254301271 wait complete. 2024-05-10T03:32:45.472 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-05-10T03:32:45.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.751+0000 7efef7800640 1 Processor -- start 2024-05-10T03:32:45.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.751+0000 7efef7800640 1 -- start start 2024-05-10T03:32:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efef7800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efef015ab30 0x7efef00715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efef7800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efef015af40 0x7efef0071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efef7800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00cd710 con 0x7efef00cdba0 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efef7800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00cdfb0 con 0x7efef015ab30 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efef7800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef006b4d0 con 0x7efef015af40 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.754+0000 7efeefe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efef015ab30 0x7efef00715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efef015ab30 0x7efef00715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46304/0 (socket says 172.21.15.55:46304) 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 -- 172.21.15.55:0/4233131470 learned_addr learned my addr 172.21.15.55:0/4233131470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efef5800640 1 --2- 172.21.15.55:0/4233131470 >> v2:172.21.15.162:3300/0 conn(0x7efef015af40 0x7efef0071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 -- 172.21.15.55:0/4233131470 >> v2:172.21.15.162:3300/0 conn(0x7efef015af40 msgr2=0x7efef0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:45.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 --2- 172.21.15.55:0/4233131470 >> v2:172.21.15.162:3300/0 conn(0x7efef015af40 0x7efef0071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 -- 172.21.15.55:0/4233131470 >> v1:172.21.15.162:6789/0 conn(0x7efef00cdba0 legacy=0x7efef006a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.755+0000 7efeefe00640 1 -- 172.21.15.55:0/4233131470 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef006b1f0 con 0x7efef015ab30 2024-05-10T03:32:45.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.757+0000 7efeefe00640 1 --2- 172.21.15.55:0/4233131470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef00715e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efef00b4810 tx=0x7efedc02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992ad235aff4ce44 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.757+0000 7efeef400640 1 -- 172.21.15.55:0/4233131470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032710 con 0x7efef015ab30 2024-05-10T03:32:45.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.757+0000 7efeef400640 1 -- 172.21.15.55:0/4233131470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc030050 con 0x7efef015ab30 2024-05-10T03:32:45.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.757+0000 7efeef400640 1 -- 172.21.15.55:0/4233131470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032ab0 con 0x7efef015ab30 2024-05-10T03:32:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 -- 172.21.15.55:0/4233131470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 msgr2=0x7efef00715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 --2- 172.21.15.55:0/4233131470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef00715e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efef00b4810 tx=0x7efedc02c810 comp rx=0 tx=0).stop 2024-05-10T03:32:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 -- 172.21.15.55:0/4233131470 shutdown_connections 2024-05-10T03:32:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 --2- 172.21.15.55:0/4233131470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef00715e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 --2- 172.21.15.55:0/4233131470 >> v2:172.21.15.162:3300/0 conn(0x7efef015af40 0x7efef0071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 -- 172.21.15.55:0/4233131470 >> 172.21.15.55:0/4233131470 conn(0x7efef00cbb90 msgr2=0x7efef00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 -- 172.21.15.55:0/4233131470 shutdown_connections 2024-05-10T03:32:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.758+0000 7efef7800640 1 -- 172.21.15.55:0/4233131470 wait complete. 2024-05-10T03:32:45.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.761+0000 7efef7800640 1 Processor -- start 2024-05-10T03:32:45.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.761+0000 7efef7800640 1 -- start start 2024-05-10T03:32:45.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.762+0000 7efef7800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 0x7efef01426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.762+0000 7efef7800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef7800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efef015af40 0x7efef00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef7800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01f80a0 con 0x7efef015ab30 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef7800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef014e9d0 con 0x7efef00cdba0 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef7800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef01f71d0 con 0x7efef015af40 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46320/0 (socket says 172.21.15.55:46320) 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 -- 172.21.15.55:0/560199776 learned_addr learned my addr 172.21.15.55:0/560199776 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef4e00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 0x7efef01426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef5800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efef015af40 0x7efef00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 -- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efef015af40 msgr2=0x7efef00b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efef015af40 0x7efef00b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 -- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 msgr2=0x7efef01426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 0x7efef01426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:45.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efeefe00640 1 -- 172.21.15.55:0/560199776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc00b770 con 0x7efef015ab30 2024-05-10T03:32:45.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.763+0000 7efef4e00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 0x7efef01426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:45.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efeefe00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efedc02c850 tx=0x7efedc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032710 con 0x7efef015ab30 2024-05-10T03:32:45.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc032870 con 0x7efef015ab30 2024-05-10T03:32:45.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01f7420 con 0x7efef015ab30 2024-05-10T03:32:45.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc032c20 con 0x7efef015ab30 2024-05-10T03:32:45.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.764+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01f79e0 con 0x7efef015ab30 2024-05-10T03:32:45.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.766+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0066f60 con 0x7efef015ab30 2024-05-10T03:32:45.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.767+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efedc048460 con 0x7efef015ab30 2024-05-10T03:32:45.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.768+0000 7efeecc00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 0x7efed006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:45.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.768+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7efedc0aa500 con 0x7efef015ab30 2024-05-10T03:32:45.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.768+0000 7efef4e00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 0x7efed006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:45.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.770+0000 7efef4e00640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 0x7efed006d110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efee0000f60 tx=0x7efee0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:45.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:45.773+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efedc075a60 con 0x7efef015ab30 2024-05-10T03:32:46.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.043+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) v1 -- 0x7efef0067220 con 0x7efef015ab30 2024-05-10T03:32:46.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.909+0000 7efeecc00640 1 -- 172.21.15.55:0/560199776 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v199) v1 ==== 153+0+0 (secure 0 0 0) 0x7efedc0044e0 con 0x7efef015ab30 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 msgr2=0x7efed006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 0x7efed006d110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efee0000f60 tx=0x7efee0009040 comp rx=0 tx=0).stop 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 msgr2=0x7efef0153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efedc02c850 tx=0x7efedc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 shutdown_connections 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efef015af40 0x7efef00b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef015ab30 0x7efef0153ae0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efed006ac00 0x7efed006d110 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:46.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.913+0000 7efef7800640 1 --2- 172.21.15.55:0/560199776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efef00cdba0 0x7efef01426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:46.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.914+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 >> 172.21.15.55:0/560199776 conn(0x7efef00cbb90 msgr2=0x7efef006a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:46.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.914+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 shutdown_connections 2024-05-10T03:32:46.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:46.914+0000 7efef7800640 1 -- 172.21.15.55:0/560199776 wait complete. 2024-05-10T03:32:46.952 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... skipped 'Requires FUSE client to use mds_cluster.is_addr_blocklisted()' 2024-05-10T03:32:46.952 INFO:tasks.cephfs_test_runner:Starting test: test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:32:46.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:47.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.236+0000 7f93fb200640 1 Processor -- start 2024-05-10T03:32:47.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.236+0000 7f93fb200640 1 -- start start 2024-05-10T03:32:47.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.238+0000 7f93fb200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93fc15d4b0 0x7f93fc06c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.238+0000 7f93fb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93fc0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.238+0000 7f93fb200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0b3b20 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.238+0000 7f93fb200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc06cb80 con 0x7f93fc15d4b0 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.238+0000 7f93fb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06deb0 con 0x7f93fc06c770 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.239+0000 7f93f9e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f93fc0ac8e0 0x7f93fc161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45612/0 (socket says 172.21.15.55:45612) 2024-05-10T03:32:47.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.239+0000 7f93f9e00640 1 -- 172.21.15.55:0/4135484698 learned_addr learned my addr 172.21.15.55:0/4135484698 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:47.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.239+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730487492 0 0) 0x7f93fc0b3b20 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.239+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e0003520 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f9400640 1 --2- 172.21.15.55:0/4135484698 >> v2:172.21.15.162:3300/0 conn(0x7f93fc15d4b0 0x7f93fc06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93fa800640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93fc0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846215522 0 0) 0x7f93e0003520 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 msgr2=0x7f93fc0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93fc0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 >> v2:172.21.15.162:3300/0 conn(0x7f93fc15d4b0 msgr2=0x7f93fc06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 --2- 172.21.15.55:0/4135484698 >> v2:172.21.15.162:3300/0 conn(0x7f93fc15d4b0 0x7f93fc06c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc06e190 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f93f40019f0 con 0x7f93fc0ac8e0 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 >> v1:172.21.15.55:6789/0 conn(0x7f93fc0ac8e0 legacy=0x7f93fc161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e00039e0 0x7f93e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93e00047c0 0x7f93e0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06cb80 con 0x7f93e00047c0 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06deb0 con 0x7f93e00039e0 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.240+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e000bb30 con 0x7f93fc06c770 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93e00047c0 0x7f93e0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9e00640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e00039e0 0x7f93e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93fa800640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 -- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 msgr2=0x7f93e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93fc06c770 0x7f93e00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 -- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e00039e0 msgr2=0x7f93e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93e00039e0 0x7f93e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.241+0000 7f93f9400640 1 -- 172.21.15.55:0/4135484698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0003520 con 0x7f93e00047c0 2024-05-10T03:32:47.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.242+0000 7f93f9400640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93e00047c0 0x7f93e0004bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ba30 tx=0x7f93e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18c79a254dc65d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.242+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e402ae10 con 0x7f93e00047c0 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.242+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e402e070 con 0x7f93e00047c0 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.242+0000 7f93f8a00640 1 -- 172.21.15.55:0/4135484698 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4034620 con 0x7f93e00047c0 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 -- 172.21.15.55:0/4135484698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93e00047c0 msgr2=0x7f93e0004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 --2- 172.21.15.55:0/4135484698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93e00047c0 0x7f93e0004bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ba30 tx=0x7f93e00057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fa800640 1 -- 172.21.15.55:0/4135484698 reap_dead start 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 -- 172.21.15.55:0/4135484698 shutdown_connections 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 -- 172.21.15.55:0/4135484698 >> 172.21.15.55:0/4135484698 conn(0x7f93fc070eb0 msgr2=0x7f93fc0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 -- 172.21.15.55:0/4135484698 shutdown_connections 2024-05-10T03:32:47.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.243+0000 7f93fb200640 1 -- 172.21.15.55:0/4135484698 wait complete. 2024-05-10T03:32:47.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.246+0000 7f93fb200640 1 Processor -- start 2024-05-10T03:32:47.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.246+0000 7f93fb200640 1 -- start start 2024-05-10T03:32:47.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 0x7f93fc146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 0x7f93fc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0ce690 con 0x7f93fc1468a0 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1598c0 con 0x7f93fc14f850 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.247+0000 7f93fb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc160500 con 0x7f93e00047c0 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46334/0 (socket says 172.21.15.55:46334) 2024-05-10T03:32:47.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 -- 172.21.15.55:0/1379754804 learned_addr learned my addr 172.21.15.55:0/1379754804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:47.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9e00640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 0x7f93fc146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93fa800640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 0x7f93fc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 -- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 msgr2=0x7f93fc146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 0x7f93fc146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 -- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 msgr2=0x7f93fc1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 0x7f93fc1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9400640 1 -- 172.21.15.55:0/1379754804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc15eec0 con 0x7f93fc1468a0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93f9e00640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 0x7f93fc146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.248+0000 7f93fa800640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 0x7f93fc1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93f9400640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f93e402a900 tx=0x7f93e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4035020 con 0x7f93fc1468a0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e40351a0 con 0x7f93fc1468a0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e403f8c0 con 0x7f93fc1468a0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc146650 con 0x7f93fc1468a0 2024-05-10T03:32:47.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.249+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc0cdcf0 con 0x7f93fc1468a0 2024-05-10T03:32:47.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.250+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc0ce900 con 0x7f93fc1468a0 2024-05-10T03:32:47.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.251+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f93e403fa20 con 0x7f93fc1468a0 2024-05-10T03:32:47.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.252+0000 7f93f2000640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 0x7f93d806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.253+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f93e40aa2d0 con 0x7f93fc1468a0 2024-05-10T03:32:47.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.253+0000 7f93f9e00640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 0x7f93d806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.255+0000 7f93f9e00640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 0x7f93d806d140 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93f4002230 tx=0x7f93f4001bb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.257+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f93e40755d0 con 0x7f93fc1468a0 2024-05-10T03:32:47.315 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:32:47.326 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:32:47.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.462+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93fc15eec0 con 0x7f93fc1468a0 2024-05-10T03:32:47.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.463+0000 7f93f2000640 1 -- 172.21.15.55:0/1379754804 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+12723 (secure 0 0 0) 0x7f93fc15eec0 con 0x7f93fc1468a0 2024-05-10T03:32:47.468 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:47.468 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":57,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:44.885300+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6834/3706115975":"2024-05-11T03:32:41.793275+0000","172.21.15.55:6832/3706115975":"2024-05-11T03:32:41.793275+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.466+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 msgr2=0x7f93d806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.466+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 0x7f93d806d140 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93f4002230 tx=0x7f93f4001bb0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.466+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 msgr2=0x7f93fc151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.466+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f93e402a900 tx=0x7f93e4004730 comp rx=0 tx=0).stop 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 shutdown_connections 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac30 0x7f93d806d140 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93fc14f850 0x7f93fc1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93fc1468a0 0x7f93fc151d90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 --2- 172.21.15.55:0/1379754804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93e00047c0 0x7f93fc146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 >> 172.21.15.55:0/1379754804 conn(0x7f93fc070eb0 msgr2=0x7f93fc0afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:47.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 shutdown_connections 2024-05-10T03:32:47.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.467+0000 7f93fb200640 1 -- 172.21.15.55:0/1379754804 wait complete. 2024-05-10T03:32:47.502 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:47.502 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:47.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.834+0000 7fcd0b600640 1 Processor -- start 2024-05-10T03:32:47.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.835+0000 7fcd0b600640 1 -- start start 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.837+0000 7fcd0b600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcd040cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.837+0000 7fcd0b600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd0406b120 0x7fcd0406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd0b600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd040cee70 con 0x7fcd04070050 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd0b600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd040b3590 con 0x7fcd0406b120 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd0b600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04166830 con 0x7fcd041658d0 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd08c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcd04070050 0x7fcd04161b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45628/0 (socket says 172.21.15.55:45628) 2024-05-10T03:32:47.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd08c00640 1 -- 172.21.15.55:0/610018191 learned_addr learned my addr 172.21.15.55:0/610018191 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756044497 0 0) 0x7fcd040cee70 con 0x7fcd04070050 2024-05-10T03:32:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.838+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce8003520 con 0x7fcd04070050 2024-05-10T03:32:47.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03e00640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcd040cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297144825 0 0) 0x7fcce8003520 con 0x7fcd04070050 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> v2:172.21.15.55:3300/0 conn(0x7fcd0406b120 0x7fcd0406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 msgr2=0x7fcd040cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcd040cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 >> v2:172.21.15.55:3300/0 conn(0x7fcd0406b120 msgr2=0x7fcd0406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 --2- 172.21.15.55:0/610018191 >> v2:172.21.15.55:3300/0 conn(0x7fcd0406b120 0x7fcd0406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0406ba70 con 0x7fcd04070050 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fccf4001670 con 0x7fcd04070050 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> v2:172.21.15.55:3300/0 conn(0x7fcd0406b120 0x7fcd0406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03e00640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcd040cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 >> v1:172.21.15.55:6789/0 conn(0x7fcd04070050 legacy=0x7fcd04161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8003970 0x7fcce8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcce80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce8004d70 0x7fcce8005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040b3590 con 0x7fcce8004d70 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04166830 con 0x7fcce8003970 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.839+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce800bc60 con 0x7fcd041658d0 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce8004d70 0x7fcce8005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd03e00640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcce80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd08c00640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8003970 0x7fcce8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 -- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 msgr2=0x7fcce80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd041658d0 0x7fcce80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 -- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8003970 msgr2=0x7fcce8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8003970 0x7fcce8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.840+0000 7fcd09600640 1 -- 172.21.15.55:0/610018191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce8003520 con 0x7fcce8004d70 2024-05-10T03:32:47.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.841+0000 7fcd09600640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce8004d70 0x7fcce8005240 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcd040aed10 tx=0x7fcce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1c662b348086d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.841+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc02ba60 con 0x7fcce8004d70 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc02bbc0 con 0x7fcce8004d70 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd03400640 1 -- 172.21.15.55:0/610018191 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc033440 con 0x7fcce8004d70 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 -- 172.21.15.55:0/610018191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce8004d70 msgr2=0x7fcce8005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 --2- 172.21.15.55:0/610018191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce8004d70 0x7fcce8005240 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcd040aed10 tx=0x7fcce80057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd09600640 1 -- 172.21.15.55:0/610018191 reap_dead start 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 -- 172.21.15.55:0/610018191 shutdown_connections 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 -- 172.21.15.55:0/610018191 >> 172.21.15.55:0/610018191 conn(0x7fcd0406c120 msgr2=0x7fcd0406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 -- 172.21.15.55:0/610018191 shutdown_connections 2024-05-10T03:32:47.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.842+0000 7fcd0b600640 1 -- 172.21.15.55:0/610018191 wait complete. 2024-05-10T03:32:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.845+0000 7fcd0b600640 1 Processor -- start 2024-05-10T03:32:47.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.845+0000 7fcd0b600640 1 -- start start 2024-05-10T03:32:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8004d70 0x7fcd0415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415b290 0x7fcd040a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0415f180 con 0x7fcd040ad750 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040adce0 con 0x7fcce8004d70 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd0b600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040a8b20 con 0x7fcd0415b290 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46362/0 (socket says 172.21.15.55:46362) 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 -- 172.21.15.55:0/476213926 learned_addr learned my addr 172.21.15.55:0/476213926 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd08c00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8004d70 0x7fcd0415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 -- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415b290 msgr2=0x7fcd040a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415b290 0x7fcd040a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 -- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8004d70 msgr2=0x7fcd0415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8004d70 0x7fcd0415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.847+0000 7fcd03e00640 1 -- 172.21.15.55:0/476213926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040adf30 con 0x7fcd040ad750 2024-05-10T03:32:47.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd03e00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e870 tx=0x7fccf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000ce50 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf000a920 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000acb0 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd040ae210 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.848+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040a87b0 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.849+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04066f60 con 0x7fcd040ad750 2024-05-10T03:32:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.851+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fccf0015480 con 0x7fcd040ad750 2024-05-10T03:32:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.852+0000 7fcd00c00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 0x7fcce406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:47.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.852+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fccf0089cc0 con 0x7fcd040ad750 2024-05-10T03:32:47.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.852+0000 7fcd08c00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 0x7fcce406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.854+0000 7fcd08c00640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 0x7fcce406d110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005a70 tx=0x7fccf4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:47.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:47.857+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fccf0090080 con 0x7fcd040ad750 2024-05-10T03:32:48.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.057+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd04067220 con 0x7fcd040ad750 2024-05-10T03:32:48.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.058+0000 7fcd00c00640 1 -- 172.21.15.55:0/476213926 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccf0054fc0 con 0x7fcd040ad750 2024-05-10T03:32:48.062 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:48.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 msgr2=0x7fcce406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 0x7fcce406d110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005a70 tx=0x7fccf4009040 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 msgr2=0x7fcd0415aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e870 tx=0x7fccf000ed30 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 shutdown_connections 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415b290 0x7fcd040a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd040ad750 0x7fcd0415aa20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce406ac00 0x7fcce406d110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 --2- 172.21.15.55:0/476213926 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcce8004d70 0x7fcd0415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 >> 172.21.15.55:0/476213926 conn(0x7fcd0406c120 msgr2=0x7fcd040b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 shutdown_connections 2024-05-10T03:32:48.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.061+0000 7fcd0b600640 1 -- 172.21.15.55:0/476213926 wait complete. 2024-05-10T03:32:48.091 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-05-10T03:32:48.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.371+0000 7f544b200640 1 Processor -- start 2024-05-10T03:32:48.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.372+0000 7f544b200640 1 -- start start 2024-05-10T03:32:48.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.374+0000 7f544b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c1549e0 0x7f544c0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.374+0000 7f544b200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f544c161a00 0x7f544c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.374+0000 7f544b200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544c0b37a0 con 0x7f544c0b3390 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.374+0000 7f544b200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544c0b39f0 con 0x7f544c161a00 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.374+0000 7f544b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0b3cd0 con 0x7f544c1549e0 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.375+0000 7f544a800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f544c161a00 0x7f544c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.375+0000 7f544a800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f544c161a00 0x7f544c071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:39172/0 (socket says 172.21.15.55:39172) 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.375+0000 7f5443e00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f544c0b3390 0x7f544c1612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:51748/0 (socket says 172.21.15.55:51748) 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.375+0000 7f544a800640 1 -- 172.21.15.55:0/3469513623 learned_addr learned my addr 172.21.15.55:0/3469513623 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:48.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.375+0000 7f5449e00640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c1549e0 0x7f544c0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.376+0000 7f544a800640 1 -- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c1549e0 msgr2=0x7f544c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:48.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.376+0000 7f544a800640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c1549e0 0x7f544c0b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.376+0000 7f544a800640 1 -- 172.21.15.55:0/3469513623 >> v1:172.21.15.162:6789/0 conn(0x7f544c0b3390 legacy=0x7f544c1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.376+0000 7f544a800640 1 -- 172.21.15.55:0/3469513623 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c0b3fb0 con 0x7f544c161a00 2024-05-10T03:32:48.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.377+0000 7f544a800640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c161a00 0x7f544c071100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5444009e50 tx=0x7f544402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d815cd499e5e26e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:48.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.378+0000 7f5449400640 1 -- 172.21.15.55:0/3469513623 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544402aea0 con 0x7f544c161a00 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.378+0000 7f5449400640 1 -- 172.21.15.55:0/3469513623 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544402d050 con 0x7f544c161a00 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.378+0000 7f5449400640 1 -- 172.21.15.55:0/3469513623 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444031820 con 0x7f544c161a00 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 -- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c161a00 msgr2=0x7f544c071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c161a00 0x7f544c071100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5444009e50 tx=0x7f544402a680 comp rx=0 tx=0).stop 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 -- 172.21.15.55:0/3469513623 shutdown_connections 2024-05-10T03:32:48.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c161a00 0x7f544c071100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 --2- 172.21.15.55:0/3469513623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c1549e0 0x7f544c0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 -- 172.21.15.55:0/3469513623 >> 172.21.15.55:0/3469513623 conn(0x7f544c06a110 msgr2=0x7f544c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 -- 172.21.15.55:0/3469513623 shutdown_connections 2024-05-10T03:32:48.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.379+0000 7f544b200640 1 -- 172.21.15.55:0/3469513623 wait complete. 2024-05-10T03:32:48.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.382+0000 7f544b200640 1 Processor -- start 2024-05-10T03:32:48.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.383+0000 7f544b200640 1 -- start start 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c1549e0 0x7f544c1f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 0x7f544c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0cdd10 con 0x7f544c0b3390 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c1f7b40 con 0x7f544c1549e0 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0c4f50 con 0x7f544c148960 2024-05-10T03:32:48.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f5449e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f5449e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46366/0 (socket says 172.21.15.55:46366) 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f5449e00640 1 -- 172.21.15.55:0/834292102 learned_addr learned my addr 172.21.15.55:0/834292102 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f5443e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c1549e0 0x7f544c1f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.385+0000 7f544a800640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 0x7f544c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 -- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 msgr2=0x7f544c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 0x7f544c0c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 -- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c1549e0 msgr2=0x7f544c1f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c1549e0 0x7f544c1f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 -- 172.21.15.55:0/834292102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444009b30 con 0x7f544c0b3390 2024-05-10T03:32:48.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f544a800640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 0x7f544c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:48.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.386+0000 7f5449e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e870 tx=0x7f543c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.387+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c0094f0 con 0x7f544c0b3390 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.387+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00c940 con 0x7f544c0b3390 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.387+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00ccd0 con 0x7f544c0b3390 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.387+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544c0cce40 con 0x7f544c0b3390 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.387+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544c0cd370 con 0x7f544c0b3390 2024-05-10T03:32:48.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.388+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544c04e290 con 0x7f544c0b3390 2024-05-10T03:32:48.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.389+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f543c013470 con 0x7f544c0b3390 2024-05-10T03:32:48.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.391+0000 7f5442000640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 0x7f542806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:48.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.391+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f543c089230 con 0x7f544c0b3390 2024-05-10T03:32:48.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.391+0000 7f5443e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 0x7f542806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:48.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.392+0000 7f5443e00640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 0x7f542806d110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f544c06d0f0 tx=0x7f5434000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:48.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.396+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f543c0545f0 con 0x7f544c0b3390 2024-05-10T03:32:48.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.665+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) v1 -- 0x7f544c160010 con 0x7f544c0b3390 2024-05-10T03:32:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.925+0000 7f5442000640 1 -- 172.21.15.55:0/834292102 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v201) v1 ==== 145+0+0 (secure 0 0 0) 0x7f543c057c10 con 0x7f544c0b3390 2024-05-10T03:32:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 msgr2=0x7f542806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 0x7f542806d110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f544c06d0f0 tx=0x7f5434000ed0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 msgr2=0x7f544c0ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:48.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e870 tx=0x7f543c00ed30 comp rx=0 tx=0).stop 2024-05-10T03:32:48.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 shutdown_connections 2024-05-10T03:32:48.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f544c148960 0x7f544c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f542806ac00 0x7f542806d110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f544c1549e0 0x7f544c1f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 --2- 172.21.15.55:0/834292102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f544c0b3390 0x7f544c0ce130 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.929+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 >> 172.21.15.55:0/834292102 conn(0x7f544c06a110 msgr2=0x7f544c153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.930+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 shutdown_connections 2024-05-10T03:32:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:48.930+0000 7f544b200640 1 -- 172.21.15.55:0/834292102 wait complete. 2024-05-10T03:32:48.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:32:49.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.245+0000 7f666d000640 1 Processor -- start 2024-05-10T03:32:49.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.245+0000 7f666d000640 1 -- start start 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f666815e920 0x7f66680cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f666806b120 0x7f666806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66680710f0 0x7f6668071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666815a900 con 0x7f666806b120 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680b2a30 con 0x7f666815e920 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f666d000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666815eec0 con 0x7f66680710f0 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f6666a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f666806b120 0x7f666806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f6666a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f666806b120 0x7f666806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46376/0 (socket says 172.21.15.55:46376) 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f6666a00640 1 -- 172.21.15.55:0/299394397 learned_addr learned my addr 172.21.15.55:0/299394397 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.248+0000 7f6667e00640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66680710f0 0x7f6668071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6667400640 1 --2- 172.21.15.55:0/299394397 >> v2:172.21.15.162:3300/0 conn(0x7f666815e920 0x7f66680cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6666a00640 1 -- 172.21.15.55:0/299394397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66680710f0 msgr2=0x7f6668071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6666a00640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66680710f0 0x7f6668071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6666a00640 1 -- 172.21.15.55:0/299394397 >> v2:172.21.15.162:3300/0 conn(0x7f666815e920 msgr2=0x7f66680cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6666a00640 1 --2- 172.21.15.55:0/299394397 >> v2:172.21.15.162:3300/0 conn(0x7f666815e920 0x7f66680cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.249+0000 7f6666a00640 1 -- 172.21.15.55:0/299394397 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666806ba70 con 0x7f666806b120 2024-05-10T03:32:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.250+0000 7f6666a00640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f666806b530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6668072190 tx=0x7f665402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b1357b019acd1c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.250+0000 7f6666000640 1 -- 172.21.15.55:0/299394397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654032440 con 0x7f666806b120 2024-05-10T03:32:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.250+0000 7f6666000640 1 -- 172.21.15.55:0/299394397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66540325a0 con 0x7f666806b120 2024-05-10T03:32:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.250+0000 7f6666000640 1 -- 172.21.15.55:0/299394397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654032930 con 0x7f666806b120 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 -- 172.21.15.55:0/299394397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 msgr2=0x7f666806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f666806b530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6668072190 tx=0x7f665402a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 -- 172.21.15.55:0/299394397 shutdown_connections 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66680710f0 0x7f6668071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 --2- 172.21.15.55:0/299394397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f666806b530 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 --2- 172.21.15.55:0/299394397 >> v2:172.21.15.162:3300/0 conn(0x7f666815e920 0x7f66680cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 -- 172.21.15.55:0/299394397 >> 172.21.15.55:0/299394397 conn(0x7f666806c750 msgr2=0x7f66680b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 -- 172.21.15.55:0/299394397 shutdown_connections 2024-05-10T03:32:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.251+0000 7f666d000640 1 -- 172.21.15.55:0/299394397 wait complete. 2024-05-10T03:32:49.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.254+0000 7f666d000640 1 Processor -- start 2024-05-10T03:32:49.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.254+0000 7f666d000640 1 -- start start 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.255+0000 7f666d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.255+0000 7f666d000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 0x7f66680c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.255+0000 7f666d000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f666815e920 0x7f66680a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f666d000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680cb420 con 0x7f666806b120 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f666d000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668132250 con 0x7f66680710f0 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f666d000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66680c8bc0 con 0x7f666815e920 2024-05-10T03:32:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46384/0 (socket says 172.21.15.55:46384) 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 -- 172.21.15.55:0/2689765089 learned_addr learned my addr 172.21.15.55:0/2689765089 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667e00640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f666815e920 0x7f66680a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6666a00640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 0x7f66680c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 -- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f666815e920 msgr2=0x7f66680a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f666815e920 0x7f66680a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 -- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 msgr2=0x7f66680c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 0x7f66680c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6667400640 1 -- 172.21.15.55:0/2689765089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540094d0 con 0x7f666806b120 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.256+0000 7f6666a00640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 0x7f66680c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f6667400640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f665800d620 tx=0x7f665800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665800b620 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665800b780 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665800bb30 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66680c8e10 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.257+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680c93d0 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.258+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666804e290 con 0x7f666806b120 2024-05-10T03:32:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.259+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f665800bc90 con 0x7f666806b120 2024-05-10T03:32:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.260+0000 7f665f400640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 0x7f664806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.260+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6658089760 con 0x7f666806b120 2024-05-10T03:32:49.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.261+0000 7f6666a00640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 0x7f664806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:49.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.262+0000 7f6666a00640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 0x7f664806d110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6654034000 tx=0x7f665402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:49.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.265+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f66580549d0 con 0x7f666806b120 2024-05-10T03:32:49.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.462+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f66680b2300 con 0x7f666806b120 2024-05-10T03:32:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.472+0000 7f665f400640 1 -- 172.21.15.55:0/2689765089 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6658057ff0 con 0x7f666806b120 2024-05-10T03:32:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.474+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 msgr2=0x7f664806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.474+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 0x7f664806d110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6654034000 tx=0x7f665402e000 comp rx=0 tx=0).stop 2024-05-10T03:32:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.474+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 msgr2=0x7f6668133900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.474+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f665800d620 tx=0x7f665800dae0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 shutdown_connections 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f666815e920 0x7f66680a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f664806ac00 0x7f664806d110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66680710f0 0x7f66680c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 --2- 172.21.15.55:0/2689765089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f666806b120 0x7f6668133900 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 >> 172.21.15.55:0/2689765089 conn(0x7f666806c750 msgr2=0x7f666806fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 shutdown_connections 2024-05-10T03:32:49.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:49.475+0000 7f666d000640 1 -- 172.21.15.55:0/2689765089 wait complete. 2024-05-10T03:32:49.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:32:49.506 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:49.612 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:32:49.727 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:32:49.727 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:49.727 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:49.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:32:49.788 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:49.788 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:49.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:32:49.848 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:32:49.848 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:49.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:32:49.908 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:32:49.909 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:32:49.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:32:49.964 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:32:49.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:32:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.247+0000 7f8255200640 1 Processor -- start 2024-05-10T03:32:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.247+0000 7f8255200640 1 -- start start 2024-05-10T03:32:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f8255200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8250144710 0x7f8250144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f8255200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82500b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f8255200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825006d350 con 0x7f82500b2fc0 2024-05-10T03:32:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f8255200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825006c740 con 0x7f8250144710 2024-05-10T03:32:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f8255200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825006ca20 con 0x7f825006d7e0 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f824fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f82500b2fc0 0x7f825015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45642/0 (socket says 172.21.15.55:45642) 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f824fe00640 1 -- 172.21.15.55:0/230845072 learned_addr learned my addr 172.21.15.55:0/230845072 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.249+0000 7f824f400640 1 --2- 172.21.15.55:0/230845072 >> v2:172.21.15.162:3300/0 conn(0x7f8250144710 0x7f8250144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824ea00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82500b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396827591 0 0) 0x7f825006d350 con 0x7f82500b2fc0 2024-05-10T03:32:50.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8234003520 con 0x7f82500b2fc0 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1100740934 0 0) 0x7f8234003520 con 0x7f82500b2fc0 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 msgr2=0x7f82500b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82500b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 >> v2:172.21.15.162:3300/0 conn(0x7f8250144710 msgr2=0x7f8250144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 --2- 172.21.15.55:0/230845072 >> v2:172.21.15.162:3300/0 conn(0x7f8250144710 0x7f8250144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825006cd00 con 0x7f82500b2fc0 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f82480019f0 con 0x7f82500b2fc0 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 >> v1:172.21.15.55:6789/0 conn(0x7f82500b2fc0 legacy=0x7f825015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f82340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8234004d00 0x7f8234005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825006c740 con 0x7f8234003f60 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825006ca20 con 0x7f8234004d00 2024-05-10T03:32:50.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.250+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823400fef0 con 0x7f825006d7e0 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f82340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824f400640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 -- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 msgr2=0x7f82340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f825006d7e0 0x7f82340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824fe00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8234004d00 0x7f8234005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 -- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8234004d00 msgr2=0x7f8234005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8234004d00 0x7f8234005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824ea00640 1 -- 172.21.15.55:0/230845072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234003520 con 0x7f8234003f60 2024-05-10T03:32:50.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.251+0000 7f824fe00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8234004d00 0x7f8234005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:50.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.252+0000 7f824ea00640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f82340047c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ac0 tx=0x7f82340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c3165fb4d9a9d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.252+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031400 con 0x7f8234003f60 2024-05-10T03:32:50.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823802d050 con 0x7f8234003f60 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f824e000640 1 -- 172.21.15.55:0/230845072 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031780 con 0x7f8234003f60 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 -- 172.21.15.55:0/230845072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 msgr2=0x7f82340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 --2- 172.21.15.55:0/230845072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f82340047c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ac0 tx=0x7f82340057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f824fe00640 1 -- 172.21.15.55:0/230845072 reap_dead start 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 -- 172.21.15.55:0/230845072 shutdown_connections 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 -- 172.21.15.55:0/230845072 >> 172.21.15.55:0/230845072 conn(0x7f825006a110 msgr2=0x7f825006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 -- 172.21.15.55:0/230845072 shutdown_connections 2024-05-10T03:32:50.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.253+0000 7f8255200640 1 -- 172.21.15.55:0/230845072 wait complete. 2024-05-10T03:32:50.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.256+0000 7f8255200640 1 Processor -- start 2024-05-10T03:32:50.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.256+0000 7f8255200640 1 -- start start 2024-05-10T03:32:50.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 0x7f82501573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82501578f0 0x7f8250157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501586f0 con 0x7f8234003f60 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250158240 con 0x7f825015ba70 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.257+0000 7f8255200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825015e9a0 con 0x7f82501578f0 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46400/0 (socket says 172.21.15.55:46400) 2024-05-10T03:32:50.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 -- 172.21.15.55:0/3064934051 learned_addr learned my addr 172.21.15.55:0/3064934051 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 -- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82501578f0 msgr2=0x7f8250157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824ea00640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 0x7f82501573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824fe00640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82501578f0 0x7f8250157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82501578f0 0x7f8250157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 -- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 msgr2=0x7f82501573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 0x7f82501573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824f400640 1 -- 172.21.15.55:0/3064934051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825015e3e0 con 0x7f8234003f60 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.258+0000 7f824ea00640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 0x7f82501573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f824f400640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f82400115f0 tx=0x7f8240011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824000f8f0 con 0x7f8234003f60 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824000fa50 con 0x7f8234003f60 2024-05-10T03:32:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824000fde0 con 0x7f8234003f60 2024-05-10T03:32:50.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825015e6c0 con 0x7f8234003f60 2024-05-10T03:32:50.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.259+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825015c870 con 0x7f8234003f60 2024-05-10T03:32:50.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.261+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8250066ce0 con 0x7f8234003f60 2024-05-10T03:32:50.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.262+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8240015460 con 0x7f8234003f60 2024-05-10T03:32:50.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.262+0000 7f8247400640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 0x7f822c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.263+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8240089a70 con 0x7f8234003f60 2024-05-10T03:32:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.263+0000 7f824ea00640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 0x7f822c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.265+0000 7f824ea00640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 0x7f822c06d110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8238033000 tx=0x7f8238008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.267+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8240054e10 con 0x7f8234003f60 2024-05-10T03:32:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.505+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f825004e290 con 0x7f8234003f60 2024-05-10T03:32:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.506+0000 7f8247400640 1 -- 172.21.15.55:0/3064934051 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f8240022de0 con 0x7f8234003f60 2024-05-10T03:32:50.511 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:50.511 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3727952388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3727952388},{"type":"v1","addr":"172.21.15.162:6837","nonce":3727952388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1545920248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1545920248},{"type":"v1","addr":"172.21.15.162:6835","nonce":1545920248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1785125902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1785125902},{"type":"v1","addr":"172.21.15.55:6835","nonce":1785125902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6054,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1183289404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1183289404},{"type":"v1","addr":"172.21.15.55:6834","nonce":1183289404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-05-10T03:32:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2024-05-10T03:32:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 msgr2=0x7f822c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 0x7f822c06d110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8238033000 tx=0x7f8238008030 comp rx=0 tx=0).stop 2024-05-10T03:32:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 msgr2=0x7f825013b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f82400115f0 tx=0x7f8240011ab0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 shutdown_connections 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82501578f0 0x7f8250157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f822c06ac00 0x7f822c06d110 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f825015ba70 0x7f82501573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 --2- 172.21.15.55:0/3064934051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8234003f60 0x7f825013b570 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.509+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 >> 172.21.15.55:0/3064934051 conn(0x7f825006a110 msgr2=0x7f82500b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.510+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 shutdown_connections 2024-05-10T03:32:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.510+0000 7f8255200640 1 -- 172.21.15.55:0/3064934051 wait complete. 2024-05-10T03:32:50.543 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:32:50.543 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:32:50.544 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:32:50.544 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:50.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:32:50.544 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:32:50.544 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:32:50.544 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:50.544 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:32:50.544 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:32:50.545 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:32:50.545 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:50.545 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:32:50.545 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:32:50.545 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:32:50.545 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:32:50.579 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:32:50.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:32:50.580 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:32:50.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:32:50.586 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:32:50.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:32:50.593 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:32:50.594 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:50.594 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:32:50.594 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:32:50.626 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:32:50.626 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:50.632 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:32:50.632 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:50.639 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:32:50.639 DEBUG:teuthology.parallel:result is None 2024-05-10T03:32:50.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:32:50.824 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:32:50.850 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:32:50.853 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:32:50.870 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:32:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.890+0000 7fece3e00640 1 Processor -- start 2024-05-10T03:32:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.890+0000 7fece3e00640 1 -- start start 2024-05-10T03:32:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fece40d1ba0 0x7fece40b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fece415d700 0x7fece415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece40cdf70 0x7fece40ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece415e050 con 0x7fece415d700 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece40b7b50 con 0x7fece40d1ba0 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece40ce8c0 con 0x7fece40cdf70 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fece415d700 0x7fece415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece2a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fece40d1ba0 0x7fece40b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fece415d700 0x7fece415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46430/0 (socket says 172.21.15.55:46430) 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece2000640 1 -- 172.21.15.55:0/3564142775 learned_addr learned my addr 172.21.15.55:0/3564142775 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.892+0000 7fece3400640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece40cdf70 0x7fece40ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.893+0000 7fece2000640 1 -- 172.21.15.55:0/3564142775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece40cdf70 msgr2=0x7fece40ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.893+0000 7fece2000640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece40cdf70 0x7fece40ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.893+0000 7fece2000640 1 -- 172.21.15.55:0/3564142775 >> v2:172.21.15.162:3300/0 conn(0x7fece40d1ba0 msgr2=0x7fece40b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.893+0000 7fece2000640 1 --2- 172.21.15.55:0/3564142775 >> v2:172.21.15.162:3300/0 conn(0x7fece40d1ba0 0x7fece40b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.893+0000 7fece2000640 1 -- 172.21.15.55:0/3564142775 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40b6f40 con 0x7fece415d700 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.894+0000 7fece2000640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece415d700 0x7fece415db10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fece40d1470 tx=0x7fecd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e138c6e92847a20 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.894+0000 7fece1600640 1 -- 172.21.15.55:0/3564142775 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032440 con 0x7fece415d700 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.894+0000 7fece1600640 1 -- 172.21.15.55:0/3564142775 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd00325a0 con 0x7fece415d700 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.894+0000 7fece1600640 1 -- 172.21.15.55:0/3564142775 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032930 con 0x7fece415d700 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 -- 172.21.15.55:0/3564142775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece415d700 msgr2=0x7fece415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece415d700 0x7fece415db10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fece40d1470 tx=0x7fecd002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 -- 172.21.15.55:0/3564142775 shutdown_connections 2024-05-10T03:32:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece40cdf70 0x7fece40ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 --2- 172.21.15.55:0/3564142775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece415d700 0x7fece415db10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 --2- 172.21.15.55:0/3564142775 >> v2:172.21.15.162:3300/0 conn(0x7fece40d1ba0 0x7fece40b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 -- 172.21.15.55:0/3564142775 >> 172.21.15.55:0/3564142775 conn(0x7fece40d0620 msgr2=0x7fece40d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 -- 172.21.15.55:0/3564142775 shutdown_connections 2024-05-10T03:32:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.895+0000 7fece3e00640 1 -- 172.21.15.55:0/3564142775 wait complete. 2024-05-10T03:32:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.897+0000 7fece3e00640 1 Processor -- start 2024-05-10T03:32:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.898+0000 7fece3e00640 1 -- start start 2024-05-10T03:32:50.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40d1ba0 0x7fece4160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece415d700 0x7fece40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece41f49e0 con 0x7fece40cdf70 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece415f660 con 0x7fece40d1ba0 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece415fb90 con 0x7fece415d700 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46436/0 (socket says 172.21.15.55:46436) 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.899+0000 7fece2a00640 1 -- 172.21.15.55:0/1735450777 learned_addr learned my addr 172.21.15.55:0/1735450777 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2000640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40d1ba0 0x7fece4160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 -- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece415d700 msgr2=0x7fece40c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece415d700 0x7fece40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 -- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40d1ba0 msgr2=0x7fece4160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40d1ba0 0x7fece4160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 -- 172.21.15.55:0/1735450777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00094d0 con 0x7fece40cdf70 2024-05-10T03:32:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece2a00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e870 tx=0x7fecd400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40094f0 con 0x7fece40cdf70 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400c8c0 con 0x7fece40cdf70 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400cc70 con 0x7fece40cdf70 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.900+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece40c7e80 con 0x7fece40cdf70 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.901+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece40c7830 con 0x7fece40cdf70 2024-05-10T03:32:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.902+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece404e290 con 0x7fece40cdf70 2024-05-10T03:32:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.903+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fecd4013470 con 0x7fece40cdf70 2024-05-10T03:32:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.904+0000 7fecdb400640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 0x7fecc406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.904+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fecd4089230 con 0x7fece40cdf70 2024-05-10T03:32:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.905+0000 7fece2000640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 0x7fecc406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.905+0000 7fece2000640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 0x7fecc406d140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:50.908+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fecd4054550 con 0x7fece40cdf70 2024-05-10T03:32:51.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.117+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fece40d14a0 con 0x7fece40cdf70 2024-05-10T03:32:51.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.960+0000 7fecdb400640 1 -- 172.21.15.55:0/1735450777 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fecd4057b70 con 0x7fece40cdf70 2024-05-10T03:32:51.964 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 msgr2=0x7fecc406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 0x7fecc406d140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 msgr2=0x7fece414e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e870 tx=0x7fecd400ed30 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 shutdown_connections 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece415d700 0x7fece40c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fecc406ac30 0x7fecc406d140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fece40d1ba0 0x7fece4160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 --2- 172.21.15.55:0/1735450777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fece40cdf70 0x7fece414e480 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 >> 172.21.15.55:0/1735450777 conn(0x7fece40d0620 msgr2=0x7fece415c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:51.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 shutdown_connections 2024-05-10T03:32:51.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:51.964+0000 7fece3e00640 1 -- 172.21.15.55:0/1735450777 wait complete. 2024-05-10T03:32:51.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:32:52.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.268+0000 7f1073e00640 1 Processor -- start 2024-05-10T03:32:52.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.268+0000 7f1073e00640 1 -- start start 2024-05-10T03:32:52.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.270+0000 7f1073e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1074144710 0x7f1074144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 0x7f10740b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107406d350 con 0x7f10740b2fc0 2024-05-10T03:32:52.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107406c740 con 0x7f1074144710 2024-05-10T03:32:52.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107406ca20 con 0x7f107406d7e0 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1072a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1074144710 0x7f1074144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f10740b2fc0 0x7f107415eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45664/0 (socket says 172.21.15.55:45664) 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.271+0000 7f1073400640 1 -- 172.21.15.55:0/1805208836 learned_addr learned my addr 172.21.15.55:0/1805208836 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072000640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 0x7f10740b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072a00640 1 -- 172.21.15.55:0/1805208836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 msgr2=0x7f10740b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:52.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072a00640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 0x7f10740b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072a00640 1 -- 172.21.15.55:0/1805208836 >> v1:172.21.15.55:6789/0 conn(0x7f10740b2fc0 legacy=0x7f107415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072a00640 1 -- 172.21.15.55:0/1805208836 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107406cd00 con 0x7f1074144710 2024-05-10T03:32:52.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1071600640 1 -- 172.21.15.55:0/1805208836 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166256563 0 0) 0x7f107406d350 con 0x7f10740b2fc0 2024-05-10T03:32:52.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.272+0000 7f1072000640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 0x7f10740b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:52.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.273+0000 7f1072a00640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1074144710 0x7f1074144120 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f10640097a0 tx=0x7f106402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9e9e4ac6dc9e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.273+0000 7f1071600640 1 -- 172.21.15.55:0/1805208836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106402ae50 con 0x7f1074144710 2024-05-10T03:32:52.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.273+0000 7f1071600640 1 -- 172.21.15.55:0/1805208836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064032440 con 0x7f1074144710 2024-05-10T03:32:52.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.273+0000 7f1071600640 1 -- 172.21.15.55:0/1805208836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10640327d0 con 0x7f1074144710 2024-05-10T03:32:52.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 -- 172.21.15.55:0/1805208836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1074144710 msgr2=0x7f1074144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1074144710 0x7f1074144120 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f10640097a0 tx=0x7f106402a100 comp rx=0 tx=0).stop 2024-05-10T03:32:52.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 -- 172.21.15.55:0/1805208836 shutdown_connections 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f107406d7e0 0x7f10740b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 --2- 172.21.15.55:0/1805208836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1074144710 0x7f1074144120 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 -- 172.21.15.55:0/1805208836 >> 172.21.15.55:0/1805208836 conn(0x7f107406a110 msgr2=0x7f107406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 -- 172.21.15.55:0/1805208836 shutdown_connections 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.274+0000 7f1073e00640 1 -- 172.21.15.55:0/1805208836 wait complete. 2024-05-10T03:32:52.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.277+0000 7f1073e00640 1 Processor -- start 2024-05-10T03:32:52.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.277+0000 7f1073e00640 1 -- start start 2024-05-10T03:32:52.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.278+0000 7f1073e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.278+0000 7f1073e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10740b2fc0 0x7f1074140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.278+0000 7f1073e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1074144710 0x7f107412e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1073e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107415b4f0 con 0x7f107406d7e0 2024-05-10T03:32:52.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1073e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107412fb90 con 0x7f10740b2fc0 2024-05-10T03:32:52.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1073e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10741418d0 con 0x7f1074144710 2024-05-10T03:32:52.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46466/0 (socket says 172.21.15.55:46466) 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 -- 172.21.15.55:0/1382743802 learned_addr learned my addr 172.21.15.55:0/1382743802 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 -- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1074144710 msgr2=0x7f107412e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1074144710 0x7f107412e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 -- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10740b2fc0 msgr2=0x7f1074140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10740b2fc0 0x7f1074140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.279+0000 7f1072a00640 1 -- 172.21.15.55:0/1382743802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009400 con 0x7f107406d7e0 2024-05-10T03:32:52.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f1072a00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10640098c0 tx=0x7f10640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106403d910 con 0x7f107406d7e0 2024-05-10T03:32:52.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106403da70 con 0x7f107406d7e0 2024-05-10T03:32:52.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106403de20 con 0x7f107406d7e0 2024-05-10T03:32:52.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1074142730 con 0x7f107406d7e0 2024-05-10T03:32:52.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.280+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1074141e00 con 0x7f107406d7e0 2024-05-10T03:32:52.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.281+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10741428c0 con 0x7f107406d7e0 2024-05-10T03:32:52.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.282+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1064039040 con 0x7f107406d7e0 2024-05-10T03:32:52.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.283+0000 7f106aa00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 0x7f105406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.283+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1064032e40 con 0x7f107406d7e0 2024-05-10T03:32:52.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.284+0000 7f1072000640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 0x7f105406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.285+0000 7f1072000640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 0x7f105406d140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.288+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1064074e90 con 0x7f107406d7e0 2024-05-10T03:32:52.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.556+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1074066ce0 con 0x7f107406d7e0 2024-05-10T03:32:52.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.557+0000 7f106aa00640 1 -- 172.21.15.55:0/1382743802 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1064041090 con 0x7f107406d7e0 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.561 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.562 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:52.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.564 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:32:52.566 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.567 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.568 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:32:52.569 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:52.570 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:32:52.571 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.572 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:32:52.573 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:32:52.574 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.574 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:52.574 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:32:52.574 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 msgr2=0x7f105406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 0x7f105406d140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060008040 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 msgr2=0x7f10740adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10640098c0 tx=0x7f10640046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 shutdown_connections 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1074144710 0x7f107412e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f105406ac30 0x7f105406d140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10740b2fc0 0x7f1074140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 --2- 172.21.15.55:0/1382743802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107406d7e0 0x7f10740adbb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 >> 172.21.15.55:0/1382743802 conn(0x7f107406a110 msgr2=0x7f10740aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:52.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 shutdown_connections 2024-05-10T03:32:52.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.560+0000 7f1073e00640 1 -- 172.21.15.55:0/1382743802 wait complete. 2024-05-10T03:32:52.589 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.857+0000 7fb402600640 1 Processor -- start 2024-05-10T03:32:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.857+0000 7fb402600640 1 -- start start 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.859+0000 7fb402600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3fc142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb402600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc06ad20 con 0x7fb3fc0b2550 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb402600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc06a110 con 0x7fb3fc15fe50 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb402600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc06a3f0 con 0x7fb3fc0b2140 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb3fbe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb3fc0b2550 0x7fb3fc06b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54038/0 (socket says 172.21.15.55:54038) 2024-05-10T03:32:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb3fbe00640 1 -- 172.21.15.55:0/3447516660 learned_addr learned my addr 172.21.15.55:0/3447516660 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb3faa00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3fc142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.860+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865492058 0 0) 0x7fb3fc06ad20 con 0x7fb3fc0b2550 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e0003520 con 0x7fb3fc0b2550 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270978158 0 0) 0x7fb3e0003520 con 0x7fb3fc0b2550 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 msgr2=0x7fb3fc142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3fc142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 >> v1:172.21.15.162:6789/0 conn(0x7fb3fc15fe50 legacy=0x7fb3fc141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc06a6d0 con 0x7fb3fc0b2550 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb3f40019f0 con 0x7fb3fc0b2550 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 >> v1:172.21.15.55:6789/0 conn(0x7fb3fc0b2550 legacy=0x7fb3fc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3e00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0003f20 0x7fb3e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0004d00 0x7fb3e0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc06a3f0 con 0x7fb3e0004d00 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000bba0 con 0x7fb3e0003f20 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e000bd60 con 0x7fb3fc0b2140 2024-05-10T03:32:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.861+0000 7fb3fbe00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0004d00 0x7fb3e0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fb400640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3e00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3faa00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0003f20 0x7fb3e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fbe00640 1 -- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 msgr2=0x7fb3e00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fbe00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3fc0b2140 0x7fb3e00039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fbe00640 1 -- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0003f20 msgr2=0x7fb3e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fbe00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e0003f20 0x7fb3e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.862+0000 7fb3fbe00640 1 -- 172.21.15.55:0/3447516660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0003520 con 0x7fb3e0004d00 2024-05-10T03:32:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.863+0000 7fb3fbe00640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0004d00 0x7fb3e0005110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400a820 tx=0x7fb3e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9eee0843bf423d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.863+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f402ba30 con 0x7fb3e0004d00 2024-05-10T03:32:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.863+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f402bbb0 con 0x7fb3e0004d00 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.863+0000 7fb3fa000640 1 -- 172.21.15.55:0/3447516660 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4034470 con 0x7fb3e0004d00 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 -- 172.21.15.55:0/3447516660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0004d00 msgr2=0x7fb3e0005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 --2- 172.21.15.55:0/3447516660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3e0004d00 0x7fb3e0005110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400a820 tx=0x7fb3e00057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb3fbe00640 1 -- 172.21.15.55:0/3447516660 reap_dead start 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 -- 172.21.15.55:0/3447516660 shutdown_connections 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 -- 172.21.15.55:0/3447516660 >> 172.21.15.55:0/3447516660 conn(0x7fb3fc0cbb90 msgr2=0x7fb3fc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 -- 172.21.15.55:0/3447516660 shutdown_connections 2024-05-10T03:32:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.864+0000 7fb402600640 1 -- 172.21.15.55:0/3447516660 wait complete. 2024-05-10T03:32:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.866+0000 7fb402600640 1 Processor -- start 2024-05-10T03:32:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.867+0000 7fb402600640 1 -- start start 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3e0004d00 0x7fb3fc1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3fc142fe0 0x7fb3fc13c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0702f0 con 0x7fb3fc142bd0 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1f71e0 con 0x7fb3fc142fe0 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb402600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1f6fc0 con 0x7fb3e0004d00 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb3faa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb3faa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44236/0 (socket says 172.21.15.55:44236) 2024-05-10T03:32:52.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.868+0000 7fb3faa00640 1 -- 172.21.15.55:0/2878457614 learned_addr learned my addr 172.21.15.55:0/2878457614 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3fb400640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3e0004d00 0x7fb3fc1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 -- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3e0004d00 msgr2=0x7fb3fc1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3e0004d00 0x7fb3fc1f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 -- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3fc142fe0 msgr2=0x7fb3fc13c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3fc142fe0 0x7fb3fc13c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 -- 172.21.15.55:0/2878457614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc1f7430 con 0x7fb3fc142bd0 2024-05-10T03:32:52.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3faa00640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8011600 tx=0x7fb3e8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.869+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8015870 con 0x7fb3fc142bd0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.870+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80159d0 con 0x7fb3fc142bd0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.870+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8015d40 con 0x7fb3fc142bd0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.870+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc1f7710 con 0x7fb3fc142bd0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.870+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc1f7cd0 con 0x7fb3fc142bd0 2024-05-10T03:32:52.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.871+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc066f60 con 0x7fb3fc142bd0 2024-05-10T03:32:52.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.872+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb3e800f490 con 0x7fb3fc142bd0 2024-05-10T03:32:52.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.873+0000 7fb3f3400640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 0x7fb3dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:52.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.873+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb3e8089730 con 0x7fb3fc142bd0 2024-05-10T03:32:52.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.873+0000 7fb3fb400640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 0x7fb3dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:52.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.875+0000 7fb3fb400640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 0x7fb3dc06d140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec001590 tx=0x7fb3ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:52.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:52.878+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb3e80549f0 con 0x7fb3fc142bd0 2024-05-10T03:32:53.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.084+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3fc067220 con 0x7fb3fc142bd0 2024-05-10T03:32:53.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.086+0000 7fb3f3400640 1 -- 172.21.15.55:0/2878457614 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+12594 (secure 0 0 0) 0x7fb3e8022090 con 0x7fb3fc142bd0 2024-05-10T03:32:53.089 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:53.090 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":62,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:51.940094+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:53.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 msgr2=0x7fb3dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 0x7fb3dc06d140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec001590 tx=0x7fb3ec009040 comp rx=0 tx=0).stop 2024-05-10T03:32:53.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 msgr2=0x7fb3fc13bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8011600 tx=0x7fb3e8011ac0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 shutdown_connections 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb3dc06ac30 0x7fb3dc06d140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3fc142fe0 0x7fb3fc13c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3fc142bd0 0x7fb3fc13bef0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 --2- 172.21.15.55:0/2878457614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3e0004d00 0x7fb3fc1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.089+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 >> 172.21.15.55:0/2878457614 conn(0x7fb3fc0cbb90 msgr2=0x7fb3fc0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.090+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 shutdown_connections 2024-05-10T03:32:53.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.090+0000 7fb402600640 1 -- 172.21.15.55:0/2878457614 wait complete. 2024-05-10T03:32:53.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:32:53.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:32:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.401+0000 7fcd0b000640 1 Processor -- start 2024-05-10T03:32:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.401+0000 7fcd0b000640 1 -- start start 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fcd0b000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd04141a30 0x7fcd040cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fcd0b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415bce0 0x7fcd0415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fcd0b000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd04071ce0 con 0x7fcd04070050 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fcd0b000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd04070f40 con 0x7fcd04141a30 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fcd0b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0415f280 con 0x7fcd0415bce0 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fccfbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd04141a30 0x7fcd040cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fccfbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd04141a30 0x7fcd040cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44244/0 (socket says 172.21.15.55:44244) 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.404+0000 7fccfbe00640 1 -- 172.21.15.55:0/2984485958 learned_addr learned my addr 172.21.15.55:0/2984485958 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.405+0000 7fccfbe00640 1 -- 172.21.15.55:0/2984485958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415bce0 msgr2=0x7fcd0415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.405+0000 7fccfbe00640 1 --2- 172.21.15.55:0/2984485958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415bce0 0x7fcd0415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.405+0000 7fccfbe00640 1 -- 172.21.15.55:0/2984485958 >> v1:172.21.15.162:6789/0 conn(0x7fcd04070050 legacy=0x7fcd0415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.405+0000 7fccfbe00640 1 -- 172.21.15.55:0/2984485958 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04070460 con 0x7fcd04141a30 2024-05-10T03:32:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.406+0000 7fccfbe00640 1 --2- 172.21.15.55:0/2984485958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040cd750 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5f00025c7b0ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.406+0000 7fcd03400640 1 -- 172.21.15.55:0/2984485958 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec032440 con 0x7fcd04141a30 2024-05-10T03:32:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.406+0000 7fcd03400640 1 -- 172.21.15.55:0/2984485958 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec0325a0 con 0x7fcd04141a30 2024-05-10T03:32:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.406+0000 7fcd03400640 1 -- 172.21.15.55:0/2984485958 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec032930 con 0x7fcd04141a30 2024-05-10T03:32:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.407+0000 7fcd0b000640 1 -- 172.21.15.55:0/2984485958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 msgr2=0x7fcd040cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.407+0000 7fcd0b000640 1 --2- 172.21.15.55:0/2984485958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040cd750 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec02a210 comp rx=0 tx=0).stop 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 -- 172.21.15.55:0/2984485958 shutdown_connections 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 --2- 172.21.15.55:0/2984485958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd0415bce0 0x7fcd0415c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 --2- 172.21.15.55:0/2984485958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040cd750 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 -- 172.21.15.55:0/2984485958 >> 172.21.15.55:0/2984485958 conn(0x7fcd0406c120 msgr2=0x7fcd0406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 -- 172.21.15.55:0/2984485958 shutdown_connections 2024-05-10T03:32:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.408+0000 7fcd0b000640 1 -- 172.21.15.55:0/2984485958 wait complete. 2024-05-10T03:32:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.411+0000 7fcd0b000640 1 Processor -- start 2024-05-10T03:32:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.411+0000 7fcd0b000640 1 -- start start 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd04070050 0x7fcd04158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 0x7fcd04140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0413d210 con 0x7fcd04141a30 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04144da0 con 0x7fcd0415bce0 2024-05-10T03:32:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.413+0000 7fcd0b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041421f0 con 0x7fcd04070050 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44248/0 (socket says 172.21.15.55:44248) 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 -- 172.21.15.55:0/1271106633 learned_addr learned my addr 172.21.15.55:0/1271106633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fcd03e00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd04070050 0x7fcd04158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fcd09000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 0x7fcd04140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 -- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd04070050 msgr2=0x7fcd04158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd04070050 0x7fcd04158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 -- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 msgr2=0x7fcd04140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 0x7fcd04140100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fccfbe00640 1 -- 172.21.15.55:0/1271106633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0094d0 con 0x7fcd04141a30 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.414+0000 7fcd09000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 0x7fcd04140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fccfbe00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec032440 con 0x7fcd04141a30 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec030ba0 con 0x7fcd04141a30 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec03b400 con 0x7fcd04141a30 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041424d0 con 0x7fcd04141a30 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.415+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd04142a90 con 0x7fcd04141a30 2024-05-10T03:32:53.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.416+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04066f60 con 0x7fcd04141a30 2024-05-10T03:32:53.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.417+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fccec039040 con 0x7fcd04141a30 2024-05-10T03:32:53.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.418+0000 7fcd00c00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 0x7fcce006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.418+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fccec0a9760 con 0x7fcd04141a30 2024-05-10T03:32:53.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.419+0000 7fcd03e00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 0x7fcce006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.420+0000 7fcd03e00640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 0x7fcce006d110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006fd0 tx=0x7fccf4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.423+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fccec074ab0 con 0x7fcd04141a30 2024-05-10T03:32:53.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.614+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd04070460 con 0x7fcd04141a30 2024-05-10T03:32:53.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.615+0000 7fcd00c00640 1 -- 172.21.15.55:0/1271106633 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd04070460 con 0x7fcd04141a30 2024-05-10T03:32:53.618 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 msgr2=0x7fcce006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 0x7fcce006d110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fccf4006fd0 tx=0x7fccf4008040 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 msgr2=0x7fcd040a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fccec009870 tx=0x7fccec0046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 shutdown_connections 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcce006ac00 0x7fcce006d110 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd0415bce0 0x7fcd04140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd04141a30 0x7fcd040a9240 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 --2- 172.21.15.55:0/1271106633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd04070050 0x7fcd04158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 >> 172.21.15.55:0/1271106633 conn(0x7fcd0406c120 msgr2=0x7fcd040cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:53.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 shutdown_connections 2024-05-10T03:32:53.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.618+0000 7fcd0b000640 1 -- 172.21.15.55:0/1271106633 wait complete. 2024-05-10T03:32:53.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:32:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.927+0000 7fe2dac00640 1 Processor -- start 2024-05-10T03:32:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.927+0000 7fe2dac00640 1 -- start start 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.929+0000 7fe2dac00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2d415fe50 0x7fe2d4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2dac00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 0x7fe2d4142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2dac00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d406ad20 con 0x7fe2d40b2550 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2dac00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d4141f50 con 0x7fe2d415fe50 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2dac00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d406a110 con 0x7fe2d40b2140 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2d8c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe2d40b2550 0x7fe2d406b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54054/0 (socket says 172.21.15.55:54054) 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2d8c00640 1 -- 172.21.15.55:0/2187527172 learned_addr learned my addr 172.21.15.55:0/2187527172 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/2187527172 >> v2:172.21.15.55:3300/0 conn(0x7fe2d415fe50 0x7fe2d4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2d3e00640 1 -- 172.21.15.55:0/2187527172 >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 msgr2=0x7fe2d4142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.930+0000 7fe2cbe00640 1 --2- 172.21.15.55:0/2187527172 >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 0x7fe2d4142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/2187527172 >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 0x7fe2d4142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2d3400640 1 -- 172.21.15.55:0/2187527172 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666659004 0 0) 0x7fe2d406ad20 con 0x7fe2d40b2550 2024-05-10T03:32:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2d3e00640 1 -- 172.21.15.55:0/2187527172 >> v1:172.21.15.55:6789/0 conn(0x7fe2d40b2550 legacy=0x7fe2d406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2d3e00640 1 -- 172.21.15.55:0/2187527172 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d406a3f0 con 0x7fe2d415fe50 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.931+0000 7fe2cbe00640 1 --2- 172.21.15.55:0/2187527172 >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 0x7fe2d4142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.932+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/2187527172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d415fe50 0x7fe2d4141a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2d41454b0 tx=0x7fe2c402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbaafba6be48fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.932+0000 7fe2d3400640 1 -- 172.21.15.55:0/2187527172 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4032440 con 0x7fe2d415fe50 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.932+0000 7fe2d3400640 1 -- 172.21.15.55:0/2187527172 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c40325a0 con 0x7fe2d415fe50 2024-05-10T03:32:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.932+0000 7fe2d3400640 1 -- 172.21.15.55:0/2187527172 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4032930 con 0x7fe2d415fe50 2024-05-10T03:32:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 -- 172.21.15.55:0/2187527172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d415fe50 msgr2=0x7fe2d4141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 --2- 172.21.15.55:0/2187527172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d415fe50 0x7fe2d4141a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2d41454b0 tx=0x7fe2c402a250 comp rx=0 tx=0).stop 2024-05-10T03:32:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 -- 172.21.15.55:0/2187527172 shutdown_connections 2024-05-10T03:32:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 --2- 172.21.15.55:0/2187527172 >> v2:172.21.15.162:3300/0 conn(0x7fe2d40b2140 0x7fe2d4142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 --2- 172.21.15.55:0/2187527172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d415fe50 0x7fe2d4141a10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 -- 172.21.15.55:0/2187527172 >> 172.21.15.55:0/2187527172 conn(0x7fe2d40cbb90 msgr2=0x7fe2d40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 -- 172.21.15.55:0/2187527172 shutdown_connections 2024-05-10T03:32:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.933+0000 7fe2dac00640 1 -- 172.21.15.55:0/2187527172 wait complete. 2024-05-10T03:32:53.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.936+0000 7fe2dac00640 1 Processor -- start 2024-05-10T03:32:53.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.936+0000 7fe2dac00640 1 -- start start 2024-05-10T03:32:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2d40b2550 0x7fe2d415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2d415fe50 0x7fe2d41340e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4134c80 con 0x7fe2d40b2140 2024-05-10T03:32:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d415bee0 con 0x7fe2d40b2550 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.937+0000 7fe2dac00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d415c130 con 0x7fe2d415fe50 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44266/0 (socket says 172.21.15.55:44266) 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 -- 172.21.15.55:0/687651355 learned_addr learned my addr 172.21.15.55:0/687651355 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 -- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2d415fe50 msgr2=0x7fe2d41340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2d415fe50 0x7fe2d41340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 -- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2d40b2550 msgr2=0x7fe2d415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2d40b2550 0x7fe2d415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.938+0000 7fe2d3e00640 1 -- 172.21.15.55:0/687651355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4009400 con 0x7fe2d40b2140 2024-05-10T03:32:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2d3e00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4030b80 con 0x7fe2d40b2140 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c4030d00 con 0x7fe2d40b2140 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c403d680 con 0x7fe2d40b2140 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d415c410 con 0x7fe2d40b2140 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.939+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d415c9d0 con 0x7fe2d40b2140 2024-05-10T03:32:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.940+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d4066f60 con 0x7fe2d40b2140 2024-05-10T03:32:53.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.941+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe2c4039040 con 0x7fe2d40b2140 2024-05-10T03:32:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.943+0000 7fe2d0c00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 0x7fe2b006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.943+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe2c40a9dd0 con 0x7fe2d40b2140 2024-05-10T03:32:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.944+0000 7fe2cbe00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 0x7fe2b006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:53.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.945+0000 7fe2cbe00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 0x7fe2b006d140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006fd0 tx=0x7fe2bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:53.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:53.948+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe2c4075140 con 0x7fe2d40b2140 2024-05-10T03:32:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.149+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe2d4067220 con 0x7fe2d40b2140 2024-05-10T03:32:54.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.150+0000 7fe2d0c00640 1 -- 172.21.15.55:0/687651355 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe2c4041060 con 0x7fe2d40b2140 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:32:54.153 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.152+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 msgr2=0x7fe2b006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.152+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 0x7fe2b006d140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006fd0 tx=0x7fe2bc008040 comp rx=0 tx=0).stop 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.152+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 msgr2=0x7fe2d4136580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.152+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c40046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 shutdown_connections 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2d415fe50 0x7fe2d41340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2b006ac30 0x7fe2b006d140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2d40b2550 0x7fe2d415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 --2- 172.21.15.55:0/687651355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40b2140 0x7fe2d4136580 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 >> 172.21.15.55:0/687651355 conn(0x7fe2d40cbb90 msgr2=0x7fe2d415efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 shutdown_connections 2024-05-10T03:32:54.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.153+0000 7fe2dac00640 1 -- 172.21.15.55:0/687651355 wait complete. 2024-05-10T03:32:54.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:32:54.181 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:32:54.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.461+0000 7fa6e1c00640 1 Processor -- start 2024-05-10T03:32:54.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.462+0000 7fa6e1c00640 1 -- start start 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.464+0000 7fa6e1c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6dc06b7c0 0x7fa6dc15b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6e1c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc0ad730 con 0x7fa6dc0cce40 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6e1c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc06bbd0 con 0x7fa6dc070dc0 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6e1c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc0ad170 con 0x7fa6dc06b7c0 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6da000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6dc0cce40 0x7fa6dc06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54056/0 (socket says 172.21.15.55:54056) 2024-05-10T03:32:54.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6da000640 1 -- 172.21.15.55:0/2407451678 learned_addr learned my addr 172.21.15.55:0/2407451678 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:54.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2407451678 >> v2:172.21.15.162:3300/0 conn(0x7fa6dc06b7c0 0x7fa6dc15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.465+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181575553 0 0) 0x7fa6c8001110 con 0x7fa6dc0cce40 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c0003520 con 0x7fa6dc0cce40 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 211295102 0 0) 0x7fa6c0003520 con 0x7fa6dc0cce40 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 >> v2:172.21.15.162:3300/0 conn(0x7fa6dc06b7c0 msgr2=0x7fa6dc15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 --2- 172.21.15.55:0/2407451678 >> v2:172.21.15.162:3300/0 conn(0x7fa6dc06b7c0 0x7fa6dc15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 >> v1:172.21.15.162:6789/0 conn(0x7fa6dc070dc0 legacy=0x7fa6dc0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0ad450 con 0x7fa6dc0cce40 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa6c8001800 con 0x7fa6dc0cce40 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 >> v1:172.21.15.55:6789/0 conn(0x7fa6dc0cce40 legacy=0x7fa6dc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c0003970 0x7fa6c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6c0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c0005180 0x7fa6c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0ad170 con 0x7fa6c00047c0 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c000c120 con 0x7fa6c0003970 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.466+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c000c2e0 con 0x7fa6c0005180 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6c0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c0003970 0x7fa6c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6db400640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c0005180 0x7fa6c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 -- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c0005180 msgr2=0x7fa6c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c0005180 0x7fa6c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 -- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c0003970 msgr2=0x7fa6c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c0003970 0x7fa6c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6da000640 1 -- 172.21.15.55:0/2407451678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0003520 con 0x7fa6c00047c0 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c0003970 0x7fa6c0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6db400640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c0005180 0x7fa6c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:54.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.467+0000 7fa6db400640 1 -- 172.21.15.55:0/2407451678 reap_dead start 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.468+0000 7fa6da000640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6c0004bd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800c240 tx=0x7fa6c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11f485b5514d1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.468+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c802e070 con 0x7fa6c00047c0 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c80058d0 con 0x7fa6c00047c0 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6d9600640 1 -- 172.21.15.55:0/2407451678 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8005ce0 con 0x7fa6c00047c0 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 msgr2=0x7fa6c0004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6c0004bd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800c240 tx=0x7fa6c00057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2407451678 shutdown_connections 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2407451678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6c0004bd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.469+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2407451678 >> 172.21.15.55:0/2407451678 conn(0x7fa6dc0b0cf0 msgr2=0x7fa6dc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.470+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2407451678 shutdown_connections 2024-05-10T03:32:54.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.470+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2407451678 wait complete. 2024-05-10T03:32:54.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.472+0000 7fa6e1c00640 1 Processor -- start 2024-05-10T03:32:54.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.472+0000 7fa6e1c00640 1 -- start start 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6dc06c880 0x7fa6dc06cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6dc0c8370 0x7fa6dc0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15c580 con 0x7fa6c00047c0 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c8e50 con 0x7fa6dc0c8370 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6e1c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0c90a0 con 0x7fa6dc06c880 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6daa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6daa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44286/0 (socket says 172.21.15.55:44286) 2024-05-10T03:32:54.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.474+0000 7fa6daa00640 1 -- 172.21.15.55:0/2675096496 learned_addr learned my addr 172.21.15.55:0/2675096496 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6da000640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6dc06c880 0x7fa6dc06cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6db400640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6dc0c8370 0x7fa6dc0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 -- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6dc06c880 msgr2=0x7fa6dc06cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6dc06c880 0x7fa6dc06cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 -- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6dc0c8370 msgr2=0x7fa6dc0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6dc0c8370 0x7fa6dc0c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 -- 172.21.15.55:0/2675096496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0c9380 con 0x7fa6c00047c0 2024-05-10T03:32:54.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6daa00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00f3d0 tx=0x7fa6cc00f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01a070 con 0x7fa6c00047c0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc003990 con 0x7fa6c00047c0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc003d00 con 0x7fa6c00047c0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.475+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc09b470 con 0x7fa6c00047c0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.476+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc09ba30 con 0x7fa6c00047c0 2024-05-10T03:32:54.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.477+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc09bbf0 con 0x7fa6c00047c0 2024-05-10T03:32:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.478+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa6cc004020 con 0x7fa6c00047c0 2024-05-10T03:32:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.479+0000 7fa6d2a00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 0x7fa6bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.479+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa6cc089e60 con 0x7fa6c00047c0 2024-05-10T03:32:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.479+0000 7fa6da000640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 0x7fa6bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.481+0000 7fa6da000640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 0x7fa6bc06d140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8033000 tx=0x7fa6c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:54.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.483+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa6cc0551b0 con 0x7fa6c00047c0 2024-05-10T03:32:54.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:54.682+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6dc0b1a40 con 0x7fa6c00047c0 2024-05-10T03:32:55.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.005+0000 7fa6d2a00640 1 -- 172.21.15.55:0/2675096496 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6cc003e60 con 0x7fa6c00047c0 2024-05-10T03:32:55.008 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.009+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 msgr2=0x7fa6bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.009+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 0x7fa6bc06d140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8033000 tx=0x7fa6c802e000 comp rx=0 tx=0).stop 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.009+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 msgr2=0x7fa6dc15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.009+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00f3d0 tx=0x7fa6cc00f890 comp rx=0 tx=0).stop 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 shutdown_connections 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa6bc06ac30 0x7fa6bc06d140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6dc0c8370 0x7fa6dc0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6dc06c880 0x7fa6dc06cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 --2- 172.21.15.55:0/2675096496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c00047c0 0x7fa6dc15bdc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 >> 172.21.15.55:0/2675096496 conn(0x7fa6dc0b0cf0 msgr2=0x7fa6dc071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 shutdown_connections 2024-05-10T03:32:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.010+0000 7fa6e1c00640 1 -- 172.21.15.55:0/2675096496 wait complete. 2024-05-10T03:32:55.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:32:55.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.329+0000 7f1d9b400640 1 Processor -- start 2024-05-10T03:32:55.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.329+0000 7f1d9b400640 1 -- start start 2024-05-10T03:32:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.332+0000 7f1d9b400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d94161cc0 0x7f1d9406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.332+0000 7f1d9b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9406b660 0x7f1d940708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.332+0000 7f1d9b400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d94071a60 con 0x7f1d940ccf40 2024-05-10T03:32:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.332+0000 7f1d9b400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940cd350 con 0x7f1d94161cc0 2024-05-10T03:32:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.332+0000 7f1d9b400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9406ba70 con 0x7f1d9406b660 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.333+0000 7f1d93e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d94161cc0 0x7f1d9406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.333+0000 7f1d93e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d94161cc0 0x7f1d9406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:45880/0 (socket says 172.21.15.55:45880) 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.333+0000 7f1d99400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9406b660 0x7f1d940708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.333+0000 7f1d93e00640 1 -- 172.21.15.55:0/3374564452 learned_addr learned my addr 172.21.15.55:0/3374564452 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.334+0000 7f1d93e00640 1 -- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9406b660 msgr2=0x7f1d940708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.334+0000 7f1d93e00640 1 --2- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9406b660 0x7f1d940708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.334+0000 7f1d93e00640 1 -- 172.21.15.55:0/3374564452 >> v1:172.21.15.162:6789/0 conn(0x7f1d940ccf40 legacy=0x7f1d940b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.334+0000 7f1d93e00640 1 -- 172.21.15.55:0/3374564452 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d941620d0 con 0x7f1d94161cc0 2024-05-10T03:32:55.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.335+0000 7f1d93e00640 1 --2- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d94161cc0 0x7f1d9406b120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009870 tx=0x7f1d8002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa7685edecd2574 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:55.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.336+0000 7f1d93400640 1 -- 172.21.15.55:0/3374564452 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80032440 con 0x7f1d94161cc0 2024-05-10T03:32:55.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.336+0000 7f1d93400640 1 -- 172.21.15.55:0/3374564452 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800325a0 con 0x7f1d94161cc0 2024-05-10T03:32:55.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.336+0000 7f1d93400640 1 -- 172.21.15.55:0/3374564452 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80032930 con 0x7f1d94161cc0 2024-05-10T03:32:55.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 -- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d94161cc0 msgr2=0x7f1d9406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 --2- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d94161cc0 0x7f1d9406b120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009870 tx=0x7f1d8002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 -- 172.21.15.55:0/3374564452 shutdown_connections 2024-05-10T03:32:55.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 --2- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9406b660 0x7f1d940708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 --2- 172.21.15.55:0/3374564452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d94161cc0 0x7f1d9406b120 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 -- 172.21.15.55:0/3374564452 >> 172.21.15.55:0/3374564452 conn(0x7f1d9406c120 msgr2=0x7f1d9406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:55.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.337+0000 7f1d9b400640 1 -- 172.21.15.55:0/3374564452 shutdown_connections 2024-05-10T03:32:55.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.338+0000 7f1d9b400640 1 -- 172.21.15.55:0/3374564452 wait complete. 2024-05-10T03:32:55.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.341+0000 7f1d9b400640 1 Processor -- start 2024-05-10T03:32:55.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.341+0000 7f1d9b400640 1 -- start start 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9406b660 0x7f1d9415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d940ccf40 0x7f1d94164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94163560 con 0x7f1d94161cc0 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415e330 con 0x7f1d9406b660 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.343+0000 7f1d9b400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415ca70 con 0x7f1d940ccf40 2024-05-10T03:32:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.344+0000 7f1d99400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.344+0000 7f1d99400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44302/0 (socket says 172.21.15.55:44302) 2024-05-10T03:32:55.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.344+0000 7f1d99400640 1 -- 172.21.15.55:0/4292700535 learned_addr learned my addr 172.21.15.55:0/4292700535 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:55.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.344+0000 7f1d98a00640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9406b660 0x7f1d9415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.344+0000 7f1d93e00640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d940ccf40 0x7f1d94164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 -- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d940ccf40 msgr2=0x7f1d94164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d940ccf40 0x7f1d94164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 -- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9406b660 msgr2=0x7f1d9415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:55.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9406b660 0x7f1d9415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:55.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 -- 172.21.15.55:0/4292700535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800094d0 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d99400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ef90 tx=0x7f1d8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c009b20 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00f040 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9415be60 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.345+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0146c0 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.346+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9415c420 con 0x7f1d94161cc0 2024-05-10T03:32:55.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.347+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d94153ad0 con 0x7f1d94161cc0 2024-05-10T03:32:55.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.348+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1d8c014820 con 0x7f1d94161cc0 2024-05-10T03:32:55.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.349+0000 7f1d90c00640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 0x7f1d7406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:55.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.349+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f1d8c08a5c0 con 0x7f1d94161cc0 2024-05-10T03:32:55.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.350+0000 7f1d98a00640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 0x7f1d7406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:55.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.351+0000 7f1d98a00640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 0x7f1d7406d110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d940ceb80 tx=0x7f1d88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:55.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.353+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1d8c055820 con 0x7f1d94161cc0 2024-05-10T03:32:55.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:55.553+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1d94066f60 con 0x7f1d94161cc0 2024-05-10T03:32:56.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.030+0000 7f1d90c00640 1 -- 172.21.15.55:0/4292700535 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1d8c009e60 con 0x7f1d94161cc0 2024-05-10T03:32:56.034 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:32:56.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.038+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 msgr2=0x7f1d7406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.038+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 0x7f1d7406d110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d940ceb80 tx=0x7f1d88009040 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.038+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 msgr2=0x7f1d94164b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.038+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ef90 tx=0x7f1d8c00c490 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 shutdown_connections 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d94161cc0 0x7f1d94164b80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d940ccf40 0x7f1d94164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1d7406ac00 0x7f1d7406d110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 --2- 172.21.15.55:0/4292700535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9406b660 0x7f1d9415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 >> 172.21.15.55:0/4292700535 conn(0x7f1d9406c120 msgr2=0x7f1d9406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.040+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 shutdown_connections 2024-05-10T03:32:56.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.041+0000 7f1d9b400640 1 -- 172.21.15.55:0/4292700535 wait complete. 2024-05-10T03:32:56.090 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:32:56.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.344+0000 7f0917200640 1 Processor -- start 2024-05-10T03:32:56.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.344+0000 7f0917200640 1 -- start start 2024-05-10T03:32:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f0917200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0910070120 0x7f09100b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f0917200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100cf7a0 con 0x7f0910144a30 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f0917200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100b2570 con 0x7f09100cfc30 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f0917200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100cef40 con 0x7f0910070120 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f0907e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0910070120 0x7f09100b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f090fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0910144a30 0x7f091006fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54062/0 (socket says 172.21.15.55:54062) 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.347+0000 7f090fe00640 1 -- 172.21.15.55:0/126998907 learned_addr learned my addr 172.21.15.55:0/126998907 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.348+0000 7f0907e00640 1 -- 172.21.15.55:0/126998907 >> v1:172.21.15.162:6789/0 conn(0x7f09100cfc30 legacy=0x7f09100d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.348+0000 7f0907e00640 1 -- 172.21.15.55:0/126998907 >> v1:172.21.15.55:6789/0 conn(0x7f0910144a30 legacy=0x7f091006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.348+0000 7f0907e00640 1 -- 172.21.15.55:0/126998907 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100cf150 con 0x7f0910070120 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.349+0000 7f0907e00640 1 --2- 172.21.15.55:0/126998907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910070120 0x7f09100b6120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f08f80097a0 tx=0x7f08f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7ccd2c0f1299a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.350+0000 7f090f400640 1 -- 172.21.15.55:0/126998907 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8032440 con 0x7f0910070120 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.350+0000 7f090f400640 1 -- 172.21.15.55:0/126998907 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f80325a0 con 0x7f0910070120 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.350+0000 7f090f400640 1 -- 172.21.15.55:0/126998907 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8032930 con 0x7f0910070120 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.350+0000 7f0917200640 1 -- 172.21.15.55:0/126998907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910070120 msgr2=0x7f09100b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.350+0000 7f0917200640 1 --2- 172.21.15.55:0/126998907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910070120 0x7f09100b6120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f08f80097a0 tx=0x7f08f802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.351+0000 7f0917200640 1 -- 172.21.15.55:0/126998907 shutdown_connections 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.351+0000 7f0917200640 1 --2- 172.21.15.55:0/126998907 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910070120 0x7f09100b6120 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.351+0000 7f0917200640 1 -- 172.21.15.55:0/126998907 >> 172.21.15.55:0/126998907 conn(0x7f09100b36e0 msgr2=0x7f09100d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:56.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.351+0000 7f0917200640 1 -- 172.21.15.55:0/126998907 shutdown_connections 2024-05-10T03:32:56.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.351+0000 7f0917200640 1 -- 172.21.15.55:0/126998907 wait complete. 2024-05-10T03:32:56.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.353+0000 7f0917200640 1 Processor -- start 2024-05-10T03:32:56.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.353+0000 7f0917200640 1 -- start start 2024-05-10T03:32:56.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09100cfc30 0x7f09101f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 0x7f091006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c9fe0 con 0x7f0910144a30 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c8550 con 0x7f09100cfc30 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0917200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09100cbd50 con 0x7f091006d7d0 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0907e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0907e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44332/0 (socket says 172.21.15.55:44332) 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.355+0000 7f0907e00640 1 -- 172.21.15.55:0/470906395 learned_addr learned my addr 172.21.15.55:0/470906395 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0915200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 0x7f091006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 -- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 msgr2=0x7f091006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 0x7f091006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 -- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09100cfc30 msgr2=0x7f09101f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09100cfc30 0x7f09101f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 -- 172.21.15.55:0/470906395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009400 con 0x7f0910144a30 2024-05-10T03:32:56.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0915200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 0x7f091006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:56.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f0907e00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8030b80 con 0x7f0910144a30 2024-05-10T03:32:56.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8030ce0 con 0x7f0910144a30 2024-05-10T03:32:56.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.356+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f803b640 con 0x7f0910144a30 2024-05-10T03:32:56.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.357+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09101f8140 con 0x7f0910144a30 2024-05-10T03:32:56.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.357+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09100cbaa0 con 0x7f0910144a30 2024-05-10T03:32:56.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.358+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0910067220 con 0x7f0910144a30 2024-05-10T03:32:56.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.359+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08f8039040 con 0x7f0910144a30 2024-05-10T03:32:56.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.361+0000 7f090cc00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 0x7f08ec06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.361+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f08f80a9b10 con 0x7f0910144a30 2024-05-10T03:32:56.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.362+0000 7f090fe00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 0x7f08ec06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.363+0000 7f090fe00640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 0x7f08ec06d110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0900001550 tx=0x7f0900007a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.366+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f08f8074c10 con 0x7f0910144a30 2024-05-10T03:32:56.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.600+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f09100cbfa0 con 0x7f0910144a30 2024-05-10T03:32:56.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.615+0000 7f090cc00640 1 -- 172.21.15.55:0/470906395 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f08f804c020 con 0x7f0910144a30 2024-05-10T03:32:56.619 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 9 and data pool 10 2024-05-10T03:32:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.620+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 msgr2=0x7f08ec06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.620+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 0x7f08ec06d110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0900001550 tx=0x7f0900007a40 comp rx=0 tx=0).stop 2024-05-10T03:32:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.620+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 msgr2=0x7f091013d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.620+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f08f8000c00 tx=0x7f08f80046c0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 shutdown_connections 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091006d7d0 0x7f091006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0910144a30 0x7f091013d930 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08ec06ac00 0x7f08ec06d110 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 --2- 172.21.15.55:0/470906395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09100cfc30 0x7f09101f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.626+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 >> 172.21.15.55:0/470906395 conn(0x7f09100b36e0 msgr2=0x7f0910142190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:56.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.627+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 shutdown_connections 2024-05-10T03:32:56.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.628+0000 7f0917200640 1 -- 172.21.15.55:0/470906395 wait complete. 2024-05-10T03:32:56.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:56.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.908+0000 7fb6bd200640 1 Processor -- start 2024-05-10T03:32:56.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.908+0000 7fb6bd200640 1 -- start start 2024-05-10T03:32:56.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.910+0000 7fb6bd200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6b81549e0 0x7fb6b80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.910+0000 7fb6bd200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b80b37a0 con 0x7fb6b80b3390 2024-05-10T03:32:56.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.910+0000 7fb6bd200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b80b39f0 con 0x7fb6b8161a00 2024-05-10T03:32:56.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.910+0000 7fb6bd200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b80b3cd0 con 0x7fb6b81549e0 2024-05-10T03:32:56.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.911+0000 7fb6b6a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb6b80b3390 0x7fb6b81612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54078/0 (socket says 172.21.15.55:54078) 2024-05-10T03:32:56.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.911+0000 7fb6b6a00640 1 -- 172.21.15.55:0/1061774290 learned_addr learned my addr 172.21.15.55:0/1061774290 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.911+0000 7fb6b7400640 1 --2- 172.21.15.55:0/1061774290 >> v2:172.21.15.162:3300/0 conn(0x7fb6b81549e0 0x7fb6b80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.911+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545374311 0 0) 0x7fb6b80b37a0 con 0x7fb6b80b3390 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.911+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb69c003520 con 0x7fb6b80b3390 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4294536018 0 0) 0x7fb69c003520 con 0x7fb6b80b3390 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 >> v2:172.21.15.162:3300/0 conn(0x7fb6b81549e0 msgr2=0x7fb6b80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 --2- 172.21.15.55:0/1061774290 >> v2:172.21.15.162:3300/0 conn(0x7fb6b81549e0 0x7fb6b80b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 >> v1:172.21.15.162:6789/0 conn(0x7fb6b8161a00 legacy=0x7fb6b8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80b3fb0 con 0x7fb6b80b3390 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb6a8001670 con 0x7fb6b80b3390 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 >> v1:172.21.15.55:6789/0 conn(0x7fb6b80b3390 legacy=0x7fb6b81612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 0x7fb69c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0047c0 0x7fb69c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb69c005310 0x7fb69c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b3cd0 con 0x7fb69c0039e0 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c00c210 con 0x7fb69c0047c0 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c00c3d0 con 0x7fb69c005310 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.912+0000 7fb6b7400640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 0x7fb69c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b6a00640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0047c0 0x7fb69c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7e00640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb69c005310 0x7fb69c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7400640 1 -- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb69c005310 msgr2=0x7fb69c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7400640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb69c005310 0x7fb69c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7400640 1 -- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0047c0 msgr2=0x7fb69c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7400640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0047c0 0x7fb69c004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7400640 1 -- 172.21.15.55:0/1061774290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c003520 con 0x7fb69c0039e0 2024-05-10T03:32:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7e00640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb69c005310 0x7fb69c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.913+0000 7fb6b7e00640 1 -- 172.21.15.55:0/1061774290 reap_dead start 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.914+0000 7fb6b7400640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 0x7fb69c003e50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009a90 tx=0x7fb69c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef110ad2d37ae15a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.914+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0031400 con 0x7fb69c0039e0 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.914+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a002d050 con 0x7fb69c0039e0 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.914+0000 7fb6b6000640 1 -- 172.21.15.55:0/1061774290 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00317a0 con 0x7fb69c0039e0 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 -- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 msgr2=0x7fb69c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 0x7fb69c003e50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009a90 tx=0x7fb69c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 -- 172.21.15.55:0/1061774290 shutdown_connections 2024-05-10T03:32:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 --2- 172.21.15.55:0/1061774290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb69c0039e0 0x7fb69c003e50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 -- 172.21.15.55:0/1061774290 >> 172.21.15.55:0/1061774290 conn(0x7fb6b806a110 msgr2=0x7fb6b806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 -- 172.21.15.55:0/1061774290 shutdown_connections 2024-05-10T03:32:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.915+0000 7fb6bd200640 1 -- 172.21.15.55:0/1061774290 wait complete. 2024-05-10T03:32:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.917+0000 7fb6bd200640 1 Processor -- start 2024-05-10T03:32:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.917+0000 7fb6bd200640 1 -- start start 2024-05-10T03:32:56.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0039e0 0x7fb6b81418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 0x7fb6b813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80ce9a0 con 0x7fb6b8071510 2024-05-10T03:32:56.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b813ede0 con 0x7fb69c0039e0 2024-05-10T03:32:56.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6bd200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80cb940 con 0x7fb6b80cdf90 2024-05-10T03:32:56.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6b7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44358/0 (socket says 172.21.15.55:44358) 2024-05-10T03:32:56.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.919+0000 7fb6b7e00640 1 -- 172.21.15.55:0/487557563 learned_addr learned my addr 172.21.15.55:0/487557563 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:56.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7400640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0039e0 0x7fb6b81418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b6a00640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 0x7fb6b813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 -- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 msgr2=0x7fb6b813e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 0x7fb6b813e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 -- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0039e0 msgr2=0x7fb6b81418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:56.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0039e0 0x7fb6b81418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:56.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 -- 172.21.15.55:0/487557563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80ce470 con 0x7fb6b8071510 2024-05-10T03:32:56.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b6a00640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 0x7fb6b813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6b7e00640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80ad190 tx=0x7fb6b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0003b70 con 0x7fb6b8071510 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b0003cd0 con 0x7fb6b8071510 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b8141de0 con 0x7fb6b8071510 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00052b0 con 0x7fb6b8071510 2024-05-10T03:32:56.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.920+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b813eae0 con 0x7fb6b8071510 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.921+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80ce720 con 0x7fb6b8071510 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.923+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb6b0005410 con 0x7fb6b8071510 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.924+0000 7fb6af400640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 0x7fb69406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.924+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb6b0088e10 con 0x7fb6b8071510 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.924+0000 7fb6b7400640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 0x7fb69406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:56.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.926+0000 7fb6b7400640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 0x7fb69406d110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0033000 tx=0x7fb6a0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:56.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:56.928+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb6b008c030 con 0x7fb6b8071510 2024-05-10T03:32:57.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.120+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6b804e290 con 0x7fb6b8071510 2024-05-10T03:32:57.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.123+0000 7fb6af400640 1 -- 172.21.15.55:0/487557563 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16261 (secure 0 0 0) 0x7fb6b0018070 con 0x7fb6b8071510 2024-05-10T03:32:57.126 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:57.127 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":65,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:56.602042+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:57.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 msgr2=0x7fb69406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 0x7fb69406d110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0033000 tx=0x7fb6a0008030 comp rx=0 tx=0).stop 2024-05-10T03:32:57.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 msgr2=0x7fb6b8071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80ad190 tx=0x7fb6b000dc00 comp rx=0 tx=0).stop 2024-05-10T03:32:57.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 shutdown_connections 2024-05-10T03:32:57.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6b8071510 0x7fb6b8071920 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6b80cdf90 0x7fb6b813e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb69406ac00 0x7fb69406d110 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 --2- 172.21.15.55:0/487557563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb69c0039e0 0x7fb6b81418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 >> 172.21.15.55:0/487557563 conn(0x7fb6b806a110 msgr2=0x7fb6b81605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:57.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 shutdown_connections 2024-05-10T03:32:57.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.126+0000 7fb6bd200640 1 -- 172.21.15.55:0/487557563 wait complete. 2024-05-10T03:32:57.154 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:32:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.431+0000 7fd518000640 1 Processor -- start 2024-05-10T03:32:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.431+0000 7fd518000640 1 -- start start 2024-05-10T03:32:57.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.433+0000 7fd518000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd510141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.433+0000 7fd518000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd5100b2140 0x7fd510142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.433+0000 7fd518000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51006ad20 con 0x7fd5100b2550 2024-05-10T03:32:57.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.433+0000 7fd518000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd510141f50 con 0x7fd5100b2140 2024-05-10T03:32:57.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.433+0000 7fd518000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006a110 con 0x7fd51015fe50 2024-05-10T03:32:57.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd516000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd5100b2550 0x7fd51006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54088/0 (socket says 172.21.15.55:54088) 2024-05-10T03:32:57.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd516000640 1 -- 172.21.15.55:0/1754566421 learned_addr learned my addr 172.21.15.55:0/1754566421 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784155996 0 0) 0x7fd51006ad20 con 0x7fd5100b2550 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd514c00640 1 --2- 172.21.15.55:0/1754566421 >> v2:172.21.15.162:3300/0 conn(0x7fd5100b2140 0x7fd510142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f8003520 con 0x7fd5100b2550 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.434+0000 7fd515600640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd510141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218441782 0 0) 0x7fd4f8003520 con 0x7fd5100b2550 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 msgr2=0x7fd510141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd510141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 >> v2:172.21.15.162:3300/0 conn(0x7fd5100b2140 msgr2=0x7fd510142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 --2- 172.21.15.55:0/1754566421 >> v2:172.21.15.162:3300/0 conn(0x7fd5100b2140 0x7fd510142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51006a3f0 con 0x7fd5100b2550 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fd5080019f0 con 0x7fd5100b2550 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 >> v1:172.21.15.55:6789/0 conn(0x7fd5100b2550 legacy=0x7fd51006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f8003970 0x7fd4f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd4f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4f8004d00 0x7fd4f8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510141f50 con 0x7fd4f8004d00 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006a110 con 0x7fd4f8003970 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800ff80 con 0x7fd51015fe50 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd516000640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4f8004d00 0x7fd4f8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.435+0000 7fd515600640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f8003970 0x7fd4f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd514c00640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd4f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd516000640 1 -- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 msgr2=0x7fd4f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd516000640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd51015fe50 0x7fd4f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd516000640 1 -- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f8003970 msgr2=0x7fd4f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd516000640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f8003970 0x7fd4f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd516000640 1 -- 172.21.15.55:0/1754566421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8003520 con 0x7fd4f8004d00 2024-05-10T03:32:57.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.436+0000 7fd515600640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f8003970 0x7fd4f8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.437+0000 7fd516000640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4f8004d00 0x7fd4f8005110 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd50800a830 tx=0x7fd4f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35feeba90e7ca4e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.437+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50802ba50 con 0x7fd4f8004d00 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.437+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50802bbb0 con 0x7fd4f8004d00 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.437+0000 7fd50fe00640 1 -- 172.21.15.55:0/1754566421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508034470 con 0x7fd4f8004d00 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 -- 172.21.15.55:0/1754566421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4f8004d00 msgr2=0x7fd4f8005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 --2- 172.21.15.55:0/1754566421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4f8004d00 0x7fd4f8005110 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd50800a830 tx=0x7fd4f80057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd516000640 1 -- 172.21.15.55:0/1754566421 reap_dead start 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 -- 172.21.15.55:0/1754566421 shutdown_connections 2024-05-10T03:32:57.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 -- 172.21.15.55:0/1754566421 >> 172.21.15.55:0/1754566421 conn(0x7fd5100cbb90 msgr2=0x7fd5100cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:57.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 -- 172.21.15.55:0/1754566421 shutdown_connections 2024-05-10T03:32:57.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.438+0000 7fd518000640 1 -- 172.21.15.55:0/1754566421 wait complete. 2024-05-10T03:32:57.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.441+0000 7fd518000640 1 Processor -- start 2024-05-10T03:32:57.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.441+0000 7fd518000640 1 -- start start 2024-05-10T03:32:57.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 0x7fd51014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5100a85f0 0x7fd51015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51014ce70 con 0x7fd51014c2d0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015d8f0 con 0x7fd5100a85f0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.442+0000 7fd518000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51014ec80 con 0x7fd4f8004d00 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44376/0 (socket says 172.21.15.55:44376) 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 -- 172.21.15.55:0/3531869036 learned_addr learned my addr 172.21.15.55:0/3531869036 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd514c00640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5100a85f0 0x7fd51015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd515600640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 0x7fd51014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 -- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 msgr2=0x7fd51014e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 0x7fd51014e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 -- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5100a85f0 msgr2=0x7fd51015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5100a85f0 0x7fd51015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd516000640 1 -- 172.21.15.55:0/3531869036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51013fe70 con 0x7fd51014c2d0 2024-05-10T03:32:57.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.443+0000 7fd515600640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 0x7fd51014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:32:57.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd516000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd50802b430 tx=0x7fd51015bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:57.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50803b040 con 0x7fd51014c2d0 2024-05-10T03:32:57.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508034670 con 0x7fd51014c2d0 2024-05-10T03:32:57.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd510140150 con 0x7fd51014c2d0 2024-05-10T03:32:57.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080349e0 con 0x7fd51014c2d0 2024-05-10T03:32:57.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.444+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51014cbb0 con 0x7fd51014c2d0 2024-05-10T03:32:57.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.446+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51015daa0 con 0x7fd51014c2d0 2024-05-10T03:32:57.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.447+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd508031070 con 0x7fd51014c2d0 2024-05-10T03:32:57.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.448+0000 7fd50d600640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 0x7fd4f006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.448+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd5080a9cd0 con 0x7fd51014c2d0 2024-05-10T03:32:57.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.448+0000 7fd515600640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 0x7fd4f006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.450+0000 7fd515600640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 0x7fd4f006d140 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd504006fd0 tx=0x7fd504008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:57.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.452+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd51015daa0 con 0x7fd51014c2d0 2024-05-10T03:32:57.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.655+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd51015daa0 con 0x7fd51014c2d0 2024-05-10T03:32:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.658+0000 7fd50d600640 1 -- 172.21.15.55:0/3531869036 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fd51015daa0 con 0x7fd51014c2d0 2024-05-10T03:32:57.661 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:32:57.661 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:32:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 msgr2=0x7fd4f006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 0x7fd4f006d140 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd504006fd0 tx=0x7fd504008040 comp rx=0 tx=0).stop 2024-05-10T03:32:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 msgr2=0x7fd51015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd50802b430 tx=0x7fd51015bd20 comp rx=0 tx=0).stop 2024-05-10T03:32:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 shutdown_connections 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd51014c2d0 0x7fd51015cb20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd4f006ac30 0x7fd4f006d140 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5100a85f0 0x7fd51015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 --2- 172.21.15.55:0/3531869036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f8004d00 0x7fd51014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 >> 172.21.15.55:0/3531869036 conn(0x7fd5100cbb90 msgr2=0x7fd5100cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 shutdown_connections 2024-05-10T03:32:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.661+0000 7fd518000640 1 -- 172.21.15.55:0/3531869036 wait complete. 2024-05-10T03:32:57.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:32:57.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.977+0000 7f4b8da00640 1 Processor -- start 2024-05-10T03:32:57.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.977+0000 7f4b8da00640 1 -- start start 2024-05-10T03:32:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.979+0000 7f4b8da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b880719f0 0x7f4b8806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.979+0000 7f4b8da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b88071e00 0x7f4b88141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.979+0000 7f4b8da00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8815fdf0 con 0x7f4b8806b840 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.979+0000 7f4b8da00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8815f1e0 con 0x7f4b88071e00 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.979+0000 7f4b8da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8815f4c0 con 0x7f4b880719f0 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b88071e00 0x7f4b88141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4b88071e00 0x7f4b88141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44388/0 (socket says 172.21.15.55:44388) 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 -- 172.21.15.55:0/2706794733 learned_addr learned my addr 172.21.15.55:0/2706794733 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b87e00640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b880719f0 0x7f4b8806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 -- 172.21.15.55:0/2706794733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b880719f0 msgr2=0x7f4b8806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b880719f0 0x7f4b8806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 -- 172.21.15.55:0/2706794733 >> v1:172.21.15.162:6789/0 conn(0x7f4b8806b840 legacy=0x7f4b880700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:57.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.980+0000 7f4b8d000640 1 -- 172.21.15.55:0/2706794733 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8815f7a0 con 0x7f4b88071e00 2024-05-10T03:32:57.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.981+0000 7f4b8d000640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b88071e00 0x7f4b88141600 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009e50 tx=0x7f4b8002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9b55de0dd9c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:57.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.982+0000 7f4b86a00640 1 -- 172.21.15.55:0/2706794733 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80031600 con 0x7f4b88071e00 2024-05-10T03:32:57.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.982+0000 7f4b86a00640 1 -- 172.21.15.55:0/2706794733 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8002d050 con 0x7f4b88071e00 2024-05-10T03:32:57.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.982+0000 7f4b86a00640 1 -- 172.21.15.55:0/2706794733 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80031980 con 0x7f4b88071e00 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 -- 172.21.15.55:0/2706794733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b88071e00 msgr2=0x7f4b88141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b88071e00 0x7f4b88141600 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009e50 tx=0x7f4b8002a630 comp rx=0 tx=0).stop 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 -- 172.21.15.55:0/2706794733 shutdown_connections 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b88071e00 0x7f4b88141600 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 --2- 172.21.15.55:0/2706794733 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b880719f0 0x7f4b8806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 -- 172.21.15.55:0/2706794733 >> 172.21.15.55:0/2706794733 conn(0x7f4b8806c750 msgr2=0x7f4b880b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 -- 172.21.15.55:0/2706794733 shutdown_connections 2024-05-10T03:32:57.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.983+0000 7f4b8da00640 1 -- 172.21.15.55:0/2706794733 wait complete. 2024-05-10T03:32:57.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.985+0000 7f4b8da00640 1 Processor -- start 2024-05-10T03:32:57.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.985+0000 7f4b8da00640 1 -- start start 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b880719f0 0x7f4b881f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b88071e00 0x7f4b88143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881f4100 con 0x7f4b8806b840 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881e7190 con 0x7f4b880719f0 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b8da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881e73e0 con 0x7f4b88071e00 2024-05-10T03:32:57.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44390/0 (socket says 172.21.15.55:44390) 2024-05-10T03:32:57.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.987+0000 7f4b87e00640 1 -- 172.21.15.55:0/643045434 learned_addr learned my addr 172.21.15.55:0/643045434 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 -- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b88071e00 msgr2=0x7f4b88143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b8d000640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b88071e00 0x7f4b88143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b88071e00 0x7f4b88143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 -- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b880719f0 msgr2=0x7f4b881f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87400640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b880719f0 0x7f4b881f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b880719f0 0x7f4b881f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 -- 172.21.15.55:0/643045434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009b30 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b87e00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d5f0 tx=0x7f4b7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.988+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780125f0 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.989+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78012750 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.989+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78012b00 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.989+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b881e8c70 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.989+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881e8340 con 0x7f4b8806b840 2024-05-10T03:32:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.990+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8804e290 con 0x7f4b8806b840 2024-05-10T03:32:57.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.991+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4b78012c60 con 0x7f4b8806b840 2024-05-10T03:32:57.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.993+0000 7f4b7fe00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 0x7f4b6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.993+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4b780898a0 con 0x7f4b8806b840 2024-05-10T03:32:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.993+0000 7f4b87400640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 0x7f4b6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.995+0000 7f4b87400640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 0x7f4b6806d140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b74006fd0 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:58.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:57.998+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4b780549d0 con 0x7f4b8806b840 2024-05-10T03:32:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:58.245+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b881e75e0 con 0x7f4b8806b840 2024-05-10T03:32:59.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.051+0000 7f4b7fe00640 1 -- 172.21.15.55:0/643045434 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b78057ff0 con 0x7f4b8806b840 2024-05-10T03:32:59.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 msgr2=0x7f4b6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 0x7f4b6806d140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b74006fd0 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-05-10T03:32:59.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 msgr2=0x7f4b881e9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d5f0 tx=0x7f4b7800dab0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 shutdown_connections 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b88071e00 0x7f4b88143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4b6806ac30 0x7f4b6806d140 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b880719f0 0x7f4b881f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 --2- 172.21.15.55:0/643045434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b8806b840 0x7f4b881e9100 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 >> 172.21.15.55:0/643045434 conn(0x7f4b8806c750 msgr2=0x7f4b8806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 shutdown_connections 2024-05-10T03:32:59.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.055+0000 7f4b8da00640 1 -- 172.21.15.55:0/643045434 wait complete. 2024-05-10T03:32:59.087 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:32:59.087 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:32:59.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.360+0000 7f9566600640 1 Processor -- start 2024-05-10T03:32:59.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.361+0000 7f9566600640 1 -- start start 2024-05-10T03:32:59.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.363+0000 7f9566600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95600710f0 0x7f95601405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.363+0000 7f9566600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 0x7f9560140b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.363+0000 7f9566600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956015fdf0 con 0x7f95600719f0 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.363+0000 7f9566600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956015f1e0 con 0x7f95600710f0 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.363+0000 7f9566600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956015f4c0 con 0x7f9560071e00 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f9565200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f95600719f0 0x7f956006b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54090/0 (socket says 172.21.15.55:54090) 2024-05-10T03:32:59.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f9565200640 1 -- 172.21.15.55:0/1090200906 learned_addr learned my addr 172.21.15.55:0/1090200906 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f9565c00640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 0x7f9560140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f955fe00640 1 --2- 172.21.15.55:0/1090200906 >> v2:172.21.15.162:3300/0 conn(0x7f95600710f0 0x7f95601405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349993043 0 0) 0x7f956015fdf0 con 0x7f95600719f0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.364+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9548003520 con 0x7f95600719f0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755048613 0 0) 0x7f9548003520 con 0x7f95600719f0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 msgr2=0x7f9560140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 0x7f9560140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 >> v2:172.21.15.162:3300/0 conn(0x7f95600710f0 msgr2=0x7f95601405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 --2- 172.21.15.55:0/1090200906 >> v2:172.21.15.162:3300/0 conn(0x7f95600710f0 0x7f95601405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956015f7a0 con 0x7f95600719f0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f9554001670 con 0x7f95600719f0 2024-05-10T03:32:59.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 >> v1:172.21.15.55:6789/0 conn(0x7f95600719f0 legacy=0x7f956006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95480039e0 0x7f9548003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95480047c0 0x7f9548004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 0x7f9548005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956015f1e0 con 0x7f95480047c0 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956015f4c0 con 0x7f95480039e0 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954800ba80 con 0x7f9560071e00 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955fe00640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95480047c0 0x7f9548004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955fe00640 1 -- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 msgr2=0x7f9548005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955fe00640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9560071e00 0x7f9548005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955fe00640 1 -- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95480039e0 msgr2=0x7f9548003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.365+0000 7f955fe00640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95480039e0 0x7f9548003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.366+0000 7f955fe00640 1 -- 172.21.15.55:0/1090200906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548003520 con 0x7f95480047c0 2024-05-10T03:32:59.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f955fe00640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95480047c0 0x7f9548004bd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a90 tx=0x7f95480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f18fa420563094 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c031400 con 0x7f95480047c0 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c02d050 con 0x7f95480047c0 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f955f400640 1 -- 172.21.15.55:0/1090200906 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c031780 con 0x7f95480047c0 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f9566600640 1 -- 172.21.15.55:0/1090200906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95480047c0 msgr2=0x7f9548004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.367+0000 7f9566600640 1 --2- 172.21.15.55:0/1090200906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95480047c0 0x7f9548004bd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a90 tx=0x7f95480057f0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.368+0000 7f9565c00640 1 -- 172.21.15.55:0/1090200906 reap_dead start 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.368+0000 7f9566600640 1 -- 172.21.15.55:0/1090200906 shutdown_connections 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.368+0000 7f9566600640 1 -- 172.21.15.55:0/1090200906 >> 172.21.15.55:0/1090200906 conn(0x7f956006c750 msgr2=0x7f95600b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.368+0000 7f9566600640 1 -- 172.21.15.55:0/1090200906 shutdown_connections 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.368+0000 7f9566600640 1 -- 172.21.15.55:0/1090200906 wait complete. 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.370+0000 7f9566600640 1 Processor -- start 2024-05-10T03:32:59.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.370+0000 7f9566600640 1 -- start start 2024-05-10T03:32:59.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9560070050 0x7f9560070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95600caa40 0x7f95600cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600c8740 con 0x7f95601f1f70 2024-05-10T03:32:59.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600709a0 con 0x7f9560070050 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9566600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600aa9e0 con 0x7f95600caa40 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9565200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9565200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44416/0 (socket says 172.21.15.55:44416) 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f9565200640 1 -- 172.21.15.55:0/1540305324 learned_addr learned my addr 172.21.15.55:0/1540305324 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.372+0000 7f955fe00640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9560070050 0x7f9560070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565c00640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95600caa40 0x7f95600cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 -- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95600caa40 msgr2=0x7f95600cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95600caa40 0x7f95600cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 -- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9560070050 msgr2=0x7f9560070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9560070050 0x7f9560070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 -- 172.21.15.55:0/1540305324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600c8990 con 0x7f95601f1f70 2024-05-10T03:32:59.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f9565200640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f955400caa0 tx=0x7f955400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554004cb0 con 0x7f95601f1f70 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554004e10 con 0x7f95601f1f70 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.373+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95540072d0 con 0x7f95601f1f70 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.374+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600c8c70 con 0x7f95601f1f70 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.374+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600c9230 con 0x7f95601f1f70 2024-05-10T03:32:59.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.375+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95601f0d90 con 0x7f95601f1f70 2024-05-10T03:32:59.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.376+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9554007430 con 0x7f95601f1f70 2024-05-10T03:32:59.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.377+0000 7f955cc00640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 0x7f954006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:32:59.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.377+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9554088a40 con 0x7f95601f1f70 2024-05-10T03:32:59.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.378+0000 7f955fe00640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 0x7f954006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:32:59.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.379+0000 7f955fe00640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 0x7f954006d110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f954c033000 tx=0x7f954c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:32:59.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.382+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9554053ad0 con 0x7f95601f1f70 2024-05-10T03:32:59.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:32:59.619+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f95600cda00 con 0x7f95601f1f70 2024-05-10T03:33:00.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.052+0000 7f955cc00640 1 -- 172.21.15.55:0/1540305324 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9554018070 con 0x7f95601f1f70 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 msgr2=0x7f954006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 0x7f954006d110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f954c033000 tx=0x7f954c008030 comp rx=0 tx=0).stop 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 msgr2=0x7f95600aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f955400caa0 tx=0x7f955400cf60 comp rx=0 tx=0).stop 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 shutdown_connections 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95600caa40 0x7f95600cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f954006ac00 0x7f954006d110 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9560070050 0x7f9560070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 --2- 172.21.15.55:0/1540305324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95601f1f70 0x7f95600aa4a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 >> 172.21.15.55:0/1540305324 conn(0x7f956006c750 msgr2=0x7f956015e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:00.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.057+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 shutdown_connections 2024-05-10T03:33:00.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.058+0000 7f9566600640 1 -- 172.21.15.55:0/1540305324 wait complete. 2024-05-10T03:33:00.092 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:33:00.092 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.373+0000 7fa633200640 1 Processor -- start 2024-05-10T03:33:00.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.374+0000 7fa633200640 1 -- start start 2024-05-10T03:33:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.376+0000 7fa633200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15a5d0 0x7fa62c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.376+0000 7fa633200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa62c0b4880 0x7fa62c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.376+0000 7fa633200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c15ad30 con 0x7fa62c0cb730 2024-05-10T03:33:00.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.376+0000 7fa633200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c15b010 con 0x7fa62c0b4880 2024-05-10T03:33:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.376+0000 7fa633200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c15b2f0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa62c0b4880 0x7fa62c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa631e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15a5d0 0x7fa62c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa62c0b4880 0x7fa62c0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44426/0 (socket says 172.21.15.55:44426) 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 -- 172.21.15.55:0/2444273406 learned_addr learned my addr 172.21.15.55:0/2444273406 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 -- 172.21.15.55:0/2444273406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15a5d0 msgr2=0x7fa62c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 --2- 172.21.15.55:0/2444273406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15a5d0 0x7fa62c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.377+0000 7fa632800640 1 -- 172.21.15.55:0/2444273406 >> v1:172.21.15.162:6789/0 conn(0x7fa62c0cb730 legacy=0x7fa62c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.378+0000 7fa632800640 1 -- 172.21.15.55:0/2444273406 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c15b5d0 con 0x7fa62c0b4880 2024-05-10T03:33:00.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.379+0000 7fa632800640 1 --2- 172.21.15.55:0/2444273406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c0b4880 0x7fa62c0ccc80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373cc0da0e4008f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.379+0000 7fa630a00640 1 -- 172.21.15.55:0/2444273406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628009600 con 0x7fa62c0b4880 2024-05-10T03:33:00.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.379+0000 7fa630a00640 1 -- 172.21.15.55:0/2444273406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa628009760 con 0x7fa62c0b4880 2024-05-10T03:33:00.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.379+0000 7fa630a00640 1 -- 172.21.15.55:0/2444273406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628009af0 con 0x7fa62c0b4880 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 -- 172.21.15.55:0/2444273406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c0b4880 msgr2=0x7fa62c0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 --2- 172.21.15.55:0/2444273406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c0b4880 0x7fa62c0ccc80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d150 tx=0x7fa62802c930 comp rx=0 tx=0).stop 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 -- 172.21.15.55:0/2444273406 shutdown_connections 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 --2- 172.21.15.55:0/2444273406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c0b4880 0x7fa62c0ccc80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 --2- 172.21.15.55:0/2444273406 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15a5d0 0x7fa62c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 -- 172.21.15.55:0/2444273406 >> 172.21.15.55:0/2444273406 conn(0x7fa62c06a110 msgr2=0x7fa62c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 -- 172.21.15.55:0/2444273406 shutdown_connections 2024-05-10T03:33:00.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.380+0000 7fa633200640 1 -- 172.21.15.55:0/2444273406 wait complete. 2024-05-10T03:33:00.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.382+0000 7fa633200640 1 Processor -- start 2024-05-10T03:33:00.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.383+0000 7fa633200640 1 -- start start 2024-05-10T03:33:00.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa62c0cb730 0x7fa62c13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15e450 0x7fa62c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c140040 con 0x7fa62c15a5d0 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c140cf0 con 0x7fa62c0cb730 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa633200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c1407f0 con 0x7fa62c15e450 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa631400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa631400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44428/0 (socket says 172.21.15.55:44428) 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.384+0000 7fa631400640 1 -- 172.21.15.55:0/1921797628 learned_addr learned my addr 172.21.15.55:0/1921797628 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631e00640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa62c0cb730 0x7fa62c13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa632800640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15e450 0x7fa62c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 -- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15e450 msgr2=0x7fa62c156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15e450 0x7fa62c156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 -- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa62c0cb730 msgr2=0x7fa62c13fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa62c0cb730 0x7fa62c13fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 -- 172.21.15.55:0/1921797628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62800bd40 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.385+0000 7fa631400640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c3f0 tx=0x7fa61c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.386+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0052d0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.386+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c005430 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.386+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c15d3b0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.386+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c003850 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.386+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c15d970 con 0x7fa62c15a5d0 2024-05-10T03:33:00.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.387+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c1575b0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.388+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa61c0039b0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.389+0000 7fa626000640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 0x7fa61006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.389+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa61c088a00 con 0x7fa62c15a5d0 2024-05-10T03:33:00.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.390+0000 7fa631e00640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 0x7fa61006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.391+0000 7fa631e00640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 0x7fa61006d110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa62c072370 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.394+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa61c053ac0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.632+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa62c066ce0 con 0x7fa62c15a5d0 2024-05-10T03:33:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.633+0000 7fa626000640 1 -- 172.21.15.55:0/1921797628 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fa61c017090 con 0x7fa62c15a5d0 2024-05-10T03:33:00.636 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:00.637 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 msgr2=0x7fa61006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 0x7fa61006d110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa62c072370 tx=0x7fa620008040 comp rx=0 tx=0).stop 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 msgr2=0x7fa62c1402b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c3f0 tx=0x7fa61c00c7f0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 shutdown_connections 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa62c15e450 0x7fa62c156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa62c15a5d0 0x7fa62c1402b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa61006ac00 0x7fa61006d110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 --2- 172.21.15.55:0/1921797628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa62c0cb730 0x7fa62c13fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 >> 172.21.15.55:0/1921797628 conn(0x7fa62c06a110 msgr2=0x7fa62c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 shutdown_connections 2024-05-10T03:33:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.636+0000 7fa633200640 1 -- 172.21.15.55:0/1921797628 wait complete. 2024-05-10T03:33:00.666 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:00.946 INFO:tasks.ceph.mds.c.smithi055.stderr:2024-05-10T03:33:00.943+0000 7f1934a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:33:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.945+0000 7f81a4a00640 1 Processor -- start 2024-05-10T03:33:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.945+0000 7f81a4a00640 1 -- start start 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f81a4a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81a0144710 0x7f81a0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f81a4a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a006d350 con 0x7f81a006d7e0 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f81a4a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a006c740 con 0x7f81a0144710 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f81a4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a006ca20 con 0x7f81a00b3190 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f819f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81a0144710 0x7f81a0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f819ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f81a006d7e0 0x7f81a00b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54096/0 (socket says 172.21.15.55:54096) 2024-05-10T03:33:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.948+0000 7f819ea00640 1 -- 172.21.15.55:0/245477867 learned_addr learned my addr 172.21.15.55:0/245477867 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819fe00640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819f400640 1 -- 172.21.15.55:0/245477867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a00b3190 msgr2=0x7f81a00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819f400640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819f400640 1 -- 172.21.15.55:0/245477867 >> v1:172.21.15.55:6789/0 conn(0x7f81a006d7e0 legacy=0x7f81a00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819f400640 1 -- 172.21.15.55:0/245477867 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a006cd00 con 0x7f81a0144710 2024-05-10T03:33:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.949+0000 7f819e000640 1 -- 172.21.15.55:0/245477867 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512228266 0 0) 0x7f81a006d350 con 0x7f81a006d7e0 2024-05-10T03:33:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.950+0000 7f819f400640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a0144710 0x7f81a0144120 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f81900097a0 tx=0x7f819002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d6a5378898e695 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.950+0000 7f819e000640 1 -- 172.21.15.55:0/245477867 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819002adb0 con 0x7f81a0144710 2024-05-10T03:33:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.950+0000 7f819e000640 1 -- 172.21.15.55:0/245477867 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190032440 con 0x7f81a0144710 2024-05-10T03:33:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.950+0000 7f819e000640 1 -- 172.21.15.55:0/245477867 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900327d0 con 0x7f81a0144710 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 -- 172.21.15.55:0/245477867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a0144710 msgr2=0x7f81a0144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a0144710 0x7f81a0144120 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f81900097a0 tx=0x7f819002a060 comp rx=0 tx=0).stop 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 -- 172.21.15.55:0/245477867 shutdown_connections 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 --2- 172.21.15.55:0/245477867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a0144710 0x7f81a0144120 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 -- 172.21.15.55:0/245477867 >> 172.21.15.55:0/245477867 conn(0x7f81a006a110 msgr2=0x7f81a006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 -- 172.21.15.55:0/245477867 shutdown_connections 2024-05-10T03:33:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.951+0000 7f81a4a00640 1 -- 172.21.15.55:0/245477867 wait complete. 2024-05-10T03:33:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.954+0000 7f81a4a00640 1 Processor -- start 2024-05-10T03:33:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.954+0000 7f81a4a00640 1 -- start start 2024-05-10T03:33:00.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.955+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a006d7e0 0x7f81a015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.955+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f81a4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 0x7f81a00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f81a4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a015c000 con 0x7f81a00b3190 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f81a4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a013fe60 con 0x7f81a0144710 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f81a4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0138470 con 0x7f81a006d7e0 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44450/0 (socket says 172.21.15.55:44450) 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 -- 172.21.15.55:0/17733727 learned_addr learned my addr 172.21.15.55:0/17733727 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819f400640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a006d7e0 0x7f81a015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819fe00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 0x7f81a00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 -- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a006d7e0 msgr2=0x7f81a015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a006d7e0 0x7f81a015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 -- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 msgr2=0x7f81a00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 0x7f81a00cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819ea00640 1 -- 172.21.15.55:0/17733727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190009400 con 0x7f81a00b3190 2024-05-10T03:33:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.956+0000 7f819fe00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 0x7f81a00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f819ea00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f818800c430 tx=0x7f818800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81880074e0 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188007660 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188004a20 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a0138750 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.957+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a0138d10 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.959+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0066ce0 con 0x7f81a00b3190 2024-05-10T03:33:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.960+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8188004b80 con 0x7f81a00b3190 2024-05-10T03:33:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.960+0000 7f8197400640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 0x7f817c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.961+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8188089350 con 0x7f81a00b3190 2024-05-10T03:33:00.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.961+0000 7f819f400640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 0x7f817c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:00.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.962+0000 7f819f400640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 0x7f817c06d110 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8190034000 tx=0x7f819002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:00.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:00.965+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8188054360 con 0x7f81a00b3190 2024-05-10T03:33:01.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.158+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81a004e290 con 0x7f81a00b3190 2024-05-10T03:33:01.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.160+0000 7f8197400640 1 -- 172.21.15.55:0/17733727 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f8188057980 con 0x7f81a00b3190 2024-05-10T03:33:01.164 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:01.164 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 msgr2=0x7f817c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 0x7f817c06d110 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8190034000 tx=0x7f819002e000 comp rx=0 tx=0).stop 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 msgr2=0x7f81a00cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f818800c430 tx=0x7f818800c8f0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 shutdown_connections 2024-05-10T03:33:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f817c06ac00 0x7f817c06d110 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a0144710 0x7f81a00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81a00b3190 0x7f81a00cd9c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 --2- 172.21.15.55:0/17733727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81a006d7e0 0x7f81a015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 >> 172.21.15.55:0/17733727 conn(0x7f81a006a110 msgr2=0x7f81a00b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 shutdown_connections 2024-05-10T03:33:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.163+0000 7f81a4a00640 1 -- 172.21.15.55:0/17733727 wait complete. 2024-05-10T03:33:01.193 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:33:01.193 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:33:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.470+0000 7f64e8800640 1 Processor -- start 2024-05-10T03:33:01.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.471+0000 7f64e8800640 1 -- start start 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.472+0000 7f64e8800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00b6bf0 0x7f64e015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.472+0000 7f64e8800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f64e00ceb50 0x7f64e00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.472+0000 7f64e8800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e006bcb0 con 0x7f64e00cdef0 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.472+0000 7f64e8800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e006c270 con 0x7f64e00ceb50 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.472+0000 7f64e8800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e006bf90 con 0x7f64e00b6bf0 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f64e00ceb50 0x7f64e00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f64e00cdef0 0x7f64e00ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54098/0 (socket says 172.21.15.55:54098) 2024-05-10T03:33:01.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e5400640 1 -- 172.21.15.55:0/1394780338 learned_addr learned my addr 172.21.15.55:0/1394780338 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e5e00640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00b6bf0 0x7f64e015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e4a00640 1 -- 172.21.15.55:0/1394780338 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306033359 0 0) 0x7f64e006bcb0 con 0x7f64e00cdef0 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.473+0000 7f64e4a00640 1 -- 172.21.15.55:0/1394780338 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c8003520 con 0x7f64e00cdef0 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.474+0000 7f64e6800640 1 -- 172.21.15.55:0/1394780338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00b6bf0 msgr2=0x7f64e015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.474+0000 7f64e6800640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00b6bf0 0x7f64e015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.474+0000 7f64e6800640 1 -- 172.21.15.55:0/1394780338 >> v1:172.21.15.55:6789/0 conn(0x7f64e00cdef0 legacy=0x7f64e00ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.474+0000 7f64e6800640 1 -- 172.21.15.55:0/1394780338 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00d1360 con 0x7f64e00ceb50 2024-05-10T03:33:01.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.475+0000 7f64e6800640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00ceb50 0x7f64e00d1b60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009d90 tx=0x7f64dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e02873bd7f6499 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:01.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e4a00640 1 -- 172.21.15.55:0/1394780338 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc007740 con 0x7f64e00ceb50 2024-05-10T03:33:01.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e4a00640 1 -- 172.21.15.55:0/1394780338 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc02d050 con 0x7f64e00ceb50 2024-05-10T03:33:01.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e4a00640 1 -- 172.21.15.55:0/1394780338 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc007aa0 con 0x7f64e00ceb50 2024-05-10T03:33:01.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 -- 172.21.15.55:0/1394780338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00ceb50 msgr2=0x7f64e00d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00ceb50 0x7f64e00d1b60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009d90 tx=0x7f64dc02a6f0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 -- 172.21.15.55:0/1394780338 shutdown_connections 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00ceb50 0x7f64e00d1b60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 --2- 172.21.15.55:0/1394780338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00b6bf0 0x7f64e015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.476+0000 7f64e8800640 1 -- 172.21.15.55:0/1394780338 >> 172.21.15.55:0/1394780338 conn(0x7f64e00d32c0 msgr2=0x7f64e00d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.477+0000 7f64e8800640 1 -- 172.21.15.55:0/1394780338 shutdown_connections 2024-05-10T03:33:01.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.477+0000 7f64e8800640 1 -- 172.21.15.55:0/1394780338 wait complete. 2024-05-10T03:33:01.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.479+0000 7f64e8800640 1 Processor -- start 2024-05-10T03:33:01.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.479+0000 7f64e8800640 1 -- start start 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 0x7f64e00b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e0150800 0x7f64e015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0150fe0 con 0x7f64e00b6bf0 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b5550 con 0x7f64e0150800 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e8800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e015f370 con 0x7f64e00cdef0 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44464/0 (socket says 172.21.15.55:44464) 2024-05-10T03:33:01.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.481+0000 7f64e5e00640 1 -- 172.21.15.55:0/326751268 learned_addr learned my addr 172.21.15.55:0/326751268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5400640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 0x7f64e00b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 -- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 msgr2=0x7f64e00b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e6800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e0150800 0x7f64e015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 0x7f64e00b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 -- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e0150800 msgr2=0x7f64e015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e0150800 0x7f64e015f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 -- 172.21.15.55:0/326751268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc009a70 con 0x7f64e00b6bf0 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5400640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 0x7f64e00b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:01.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.482+0000 7f64e5e00640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d5f0 tx=0x7f64d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:01.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.483+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40125f0 con 0x7f64e00b6bf0 2024-05-10T03:33:01.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.483+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4012750 con 0x7f64e00b6bf0 2024-05-10T03:33:01.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.483+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e015e760 con 0x7f64e00b6bf0 2024-05-10T03:33:01.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.483+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4012b00 con 0x7f64e00b6bf0 2024-05-10T03:33:01.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.483+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e015ed20 con 0x7f64e00b6bf0 2024-05-10T03:33:01.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.484+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e0067220 con 0x7f64e00b6bf0 2024-05-10T03:33:01.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.485+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f64d4012c60 con 0x7f64e00b6bf0 2024-05-10T03:33:01.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.487+0000 7f64da000640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 0x7f64c406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:01.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.487+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f64d4089a70 con 0x7f64e00b6bf0 2024-05-10T03:33:01.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.487+0000 7f64e5400640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 0x7f64c406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:01.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.489+0000 7f64e5400640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 0x7f64c406d110 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64d0005880 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:01.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.491+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f64d4054a70 con 0x7f64e00b6bf0 2024-05-10T03:33:01.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.709+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64e00cf0c0 con 0x7f64c406ac00 2024-05-10T03:33:01.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.733+0000 7f64da000640 1 -- 172.21.15.55:0/326751268 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7f64d0001c40 con 0x7f64c406ac00 2024-05-10T03:33:01.740 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:01.742 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.738+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 msgr2=0x7f64c406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.738+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 0x7f64c406d110 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64d0005880 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 msgr2=0x7f64e014f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f64d400d5f0 tx=0x7f64d400dab0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 shutdown_connections 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f64c406ac00 0x7f64c406d110 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e0150800 0x7f64e015f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e00cdef0 0x7f64e00b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 --2- 172.21.15.55:0/326751268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64e00b6bf0 0x7f64e014f540 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 >> 172.21.15.55:0/326751268 conn(0x7f64e00d32c0 msgr2=0x7f64e00d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:01.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 shutdown_connections 2024-05-10T03:33:01.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:01.739+0000 7f64e8800640 1 -- 172.21.15.55:0/326751268 wait complete. 2024-05-10T03:33:01.770 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-05-10T03:33:01.025700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248068,"kb_used_data":7276,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726652,"statfs":{"total":773094113280,"available":772840091648,"internally_reserved":0,"allocated":7450624,"data_stored":3953214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002264"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.649746+0000","last_change":"2024-05-10T03:32:56.646709+0000","last_active":"2024-05-10T03:32:56.649746+0000","last_peered":"2024-05-10T03:32:56.649746+0000","last_clean":"2024-05-10T03:32:56.649746+0000","last_became_active":"2024-05-10T03:32:56.646355+0000","last_became_peered":"2024-05-10T03:32:56.646355+0000","last_unstale":"2024-05-10T03:32:56.649746+0000","last_undegraded":"2024-05-10T03:32:56.649746+0000","last_fullsized":"2024-05-10T03:32:56.649746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636382+0000","last_change":"2024-05-10T03:32:56.631374+0000","last_active":"2024-05-10T03:32:56.636382+0000","last_peered":"2024-05-10T03:32:56.636382+0000","last_clean":"2024-05-10T03:32:56.636382+0000","last_became_active":"2024-05-10T03:32:56.631050+0000","last_became_peered":"2024-05-10T03:32:56.631050+0000","last_unstale":"2024-05-10T03:32:56.636382+0000","last_undegraded":"2024-05-10T03:32:56.636382+0000","last_fullsized":"2024-05-10T03:32:56.636382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665260+0000","last_change":"2024-05-10T03:32:56.661084+0000","last_active":"2024-05-10T03:32:56.665260+0000","last_peered":"2024-05-10T03:32:56.665260+0000","last_clean":"2024-05-10T03:32:56.665260+0000","last_became_active":"2024-05-10T03:32:56.660004+0000","last_became_peered":"2024-05-10T03:32:56.660004+0000","last_unstale":"2024-05-10T03:32:56.665260+0000","last_undegraded":"2024-05-10T03:32:56.665260+0000","last_fullsized":"2024-05-10T03:32:56.665260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:22:48.257013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653660+0000","last_change":"2024-05-10T03:32:56.640314+0000","last_active":"2024-05-10T03:32:56.653660+0000","last_peered":"2024-05-10T03:32:56.653660+0000","last_clean":"2024-05-10T03:32:56.653660+0000","last_became_active":"2024-05-10T03:32:56.639782+0000","last_became_peered":"2024-05-10T03:32:56.639782+0000","last_unstale":"2024-05-10T03:32:56.653660+0000","last_undegraded":"2024-05-10T03:32:56.653660+0000","last_fullsized":"2024-05-10T03:32:56.653660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:35.393359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664945+0000","last_change":"2024-05-10T03:32:56.662090+0000","last_active":"2024-05-10T03:32:56.664945+0000","last_peered":"2024-05-10T03:32:56.664945+0000","last_clean":"2024-05-10T03:32:56.664945+0000","last_became_active":"2024-05-10T03:32:56.659903+0000","last_became_peered":"2024-05-10T03:32:56.659903+0000","last_unstale":"2024-05-10T03:32:56.664945+0000","last_undegraded":"2024-05-10T03:32:56.664945+0000","last_fullsized":"2024-05-10T03:32:56.664945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:49.229236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655345+0000","last_change":"2024-05-10T03:32:56.654302+0000","last_active":"2024-05-10T03:32:56.655345+0000","last_peered":"2024-05-10T03:32:56.655345+0000","last_clean":"2024-05-10T03:32:56.655345+0000","last_became_active":"2024-05-10T03:32:56.654025+0000","last_became_peered":"2024-05-10T03:32:56.654025+0000","last_unstale":"2024-05-10T03:32:56.655345+0000","last_undegraded":"2024-05-10T03:32:56.655345+0000","last_fullsized":"2024-05-10T03:32:56.655345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:49.068473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663477+0000","last_change":"2024-05-10T03:32:56.662031+0000","last_active":"2024-05-10T03:32:56.663477+0000","last_peered":"2024-05-10T03:32:56.663477+0000","last_clean":"2024-05-10T03:32:56.663477+0000","last_became_active":"2024-05-10T03:32:56.661718+0000","last_became_peered":"2024-05-10T03:32:56.661718+0000","last_unstale":"2024-05-10T03:32:56.663477+0000","last_undegraded":"2024-05-10T03:32:56.663477+0000","last_fullsized":"2024-05-10T03:32:56.663477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:25:26.923232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.662802+0000","last_change":"2024-05-10T03:32:56.655348+0000","last_active":"2024-05-10T03:32:56.662802+0000","last_peered":"2024-05-10T03:32:56.662802+0000","last_clean":"2024-05-10T03:32:56.662802+0000","last_became_active":"2024-05-10T03:32:56.654956+0000","last_became_peered":"2024-05-10T03:32:56.654956+0000","last_unstale":"2024-05-10T03:32:56.662802+0000","last_undegraded":"2024-05-10T03:32:56.662802+0000","last_fullsized":"2024-05-10T03:32:56.662802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:11:33.192350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671920+0000","last_change":"2024-05-10T03:32:56.669938+0000","last_active":"2024-05-10T03:32:56.671920+0000","last_peered":"2024-05-10T03:32:56.671920+0000","last_clean":"2024-05-10T03:32:56.671920+0000","last_became_active":"2024-05-10T03:32:56.669578+0000","last_became_peered":"2024-05-10T03:32:56.669578+0000","last_unstale":"2024-05-10T03:32:56.671920+0000","last_undegraded":"2024-05-10T03:32:56.671920+0000","last_fullsized":"2024-05-10T03:32:56.671920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.666677+0000","last_change":"2024-05-10T03:32:56.663080+0000","last_active":"2024-05-10T03:32:56.666677+0000","last_peered":"2024-05-10T03:32:56.666677+0000","last_clean":"2024-05-10T03:32:56.666677+0000","last_became_active":"2024-05-10T03:32:56.661792+0000","last_became_peered":"2024-05-10T03:32:56.661792+0000","last_unstale":"2024-05-10T03:32:56.666677+0000","last_undegraded":"2024-05-10T03:32:56.666677+0000","last_fullsized":"2024-05-10T03:32:56.666677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:02:43.422052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650916+0000","last_change":"2024-05-10T03:32:56.643923+0000","last_active":"2024-05-10T03:32:56.650916+0000","last_peered":"2024-05-10T03:32:56.650916+0000","last_clean":"2024-05-10T03:32:56.650916+0000","last_became_active":"2024-05-10T03:32:56.643395+0000","last_became_peered":"2024-05-10T03:32:56.643395+0000","last_unstale":"2024-05-10T03:32:56.650916+0000","last_undegraded":"2024-05-10T03:32:56.650916+0000","last_fullsized":"2024-05-10T03:32:56.650916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663790+0000","last_change":"2024-05-10T03:32:56.659437+0000","last_active":"2024-05-10T03:32:56.663790+0000","last_peered":"2024-05-10T03:32:56.663790+0000","last_clean":"2024-05-10T03:32:56.663790+0000","last_became_active":"2024-05-10T03:32:56.658059+0000","last_became_peered":"2024-05-10T03:32:56.658059+0000","last_unstale":"2024-05-10T03:32:56.663790+0000","last_undegraded":"2024-05-10T03:32:56.663790+0000","last_fullsized":"2024-05-10T03:32:56.663790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:20.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651635+0000","last_change":"2024-05-10T03:32:56.648678+0000","last_active":"2024-05-10T03:32:56.651635+0000","last_peered":"2024-05-10T03:32:56.651635+0000","last_clean":"2024-05-10T03:32:56.651635+0000","last_became_active":"2024-05-10T03:32:56.648161+0000","last_became_peered":"2024-05-10T03:32:56.648161+0000","last_unstale":"2024-05-10T03:32:56.651635+0000","last_undegraded":"2024-05-10T03:32:56.651635+0000","last_fullsized":"2024-05-10T03:32:56.651635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:02:43.422052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.666974+0000","last_change":"2024-05-10T03:32:56.661740+0000","last_active":"2024-05-10T03:32:56.666974+0000","last_peered":"2024-05-10T03:32:56.666974+0000","last_clean":"2024-05-10T03:32:56.666974+0000","last_became_active":"2024-05-10T03:32:56.661436+0000","last_became_peered":"2024-05-10T03:32:56.661436+0000","last_unstale":"2024-05-10T03:32:56.666974+0000","last_undegraded":"2024-05-10T03:32:56.666974+0000","last_fullsized":"2024-05-10T03:32:56.666974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664786+0000","last_change":"2024-05-10T03:32:56.656818+0000","last_active":"2024-05-10T03:32:56.664786+0000","last_peered":"2024-05-10T03:32:56.664786+0000","last_clean":"2024-05-10T03:32:56.664786+0000","last_became_active":"2024-05-10T03:32:56.656524+0000","last_became_peered":"2024-05-10T03:32:56.656524+0000","last_unstale":"2024-05-10T03:32:56.664786+0000","last_undegraded":"2024-05-10T03:32:56.664786+0000","last_fullsized":"2024-05-10T03:32:56.664786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:19:53.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.644742+0000","last_change":"2024-05-10T03:32:56.641041+0000","last_active":"2024-05-10T03:32:56.644742+0000","last_peered":"2024-05-10T03:32:56.644742+0000","last_clean":"2024-05-10T03:32:56.644742+0000","last_became_active":"2024-05-10T03:32:56.640721+0000","last_became_peered":"2024-05-10T03:32:56.640721+0000","last_unstale":"2024-05-10T03:32:56.644742+0000","last_undegraded":"2024-05-10T03:32:56.644742+0000","last_fullsized":"2024-05-10T03:32:56.644742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:47.060463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656076+0000","last_change":"2024-05-10T03:32:56.655062+0000","last_active":"2024-05-10T03:32:56.656076+0000","last_peered":"2024-05-10T03:32:56.656076+0000","last_clean":"2024-05-10T03:32:56.656076+0000","last_became_active":"2024-05-10T03:32:56.654770+0000","last_became_peered":"2024-05-10T03:32:56.654770+0000","last_unstale":"2024-05-10T03:32:56.656076+0000","last_undegraded":"2024-05-10T03:32:56.656076+0000","last_fullsized":"2024-05-10T03:32:56.656076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:49.229236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653168+0000","last_change":"2024-05-10T03:32:56.644887+0000","last_active":"2024-05-10T03:32:56.653168+0000","last_peered":"2024-05-10T03:32:56.653168+0000","last_clean":"2024-05-10T03:32:56.653168+0000","last_became_active":"2024-05-10T03:32:56.644581+0000","last_became_peered":"2024-05-10T03:32:56.644581+0000","last_unstale":"2024-05-10T03:32:56.653168+0000","last_undegraded":"2024-05-10T03:32:56.653168+0000","last_fullsized":"2024-05-10T03:32:56.653168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656262+0000","last_change":"2024-05-10T03:32:56.646920+0000","last_active":"2024-05-10T03:32:56.656262+0000","last_peered":"2024-05-10T03:32:56.656262+0000","last_clean":"2024-05-10T03:32:56.656262+0000","last_became_active":"2024-05-10T03:32:56.646518+0000","last_became_peered":"2024-05-10T03:32:56.646518+0000","last_unstale":"2024-05-10T03:32:56.656262+0000","last_undegraded":"2024-05-10T03:32:56.656262+0000","last_fullsized":"2024-05-10T03:32:56.656262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:22.441460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.648870+0000","last_change":"2024-05-10T03:32:56.646411+0000","last_active":"2024-05-10T03:32:56.648870+0000","last_peered":"2024-05-10T03:32:56.648870+0000","last_clean":"2024-05-10T03:32:56.648870+0000","last_became_active":"2024-05-10T03:32:56.646060+0000","last_became_peered":"2024-05-10T03:32:56.646060+0000","last_unstale":"2024-05-10T03:32:56.648870+0000","last_undegraded":"2024-05-10T03:32:56.648870+0000","last_fullsized":"2024-05-10T03:32:56.648870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:34:53.800064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647872+0000","last_change":"2024-05-10T03:32:56.639358+0000","last_active":"2024-05-10T03:32:56.647872+0000","last_peered":"2024-05-10T03:32:56.647872+0000","last_clean":"2024-05-10T03:32:56.647872+0000","last_became_active":"2024-05-10T03:32:56.638583+0000","last_became_peered":"2024-05-10T03:32:56.638583+0000","last_unstale":"2024-05-10T03:32:56.647872+0000","last_undegraded":"2024-05-10T03:32:56.647872+0000","last_fullsized":"2024-05-10T03:32:56.647872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:00.061901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663836+0000","last_change":"2024-05-10T03:32:56.662898+0000","last_active":"2024-05-10T03:32:56.663836+0000","last_peered":"2024-05-10T03:32:56.663836+0000","last_clean":"2024-05-10T03:32:56.663836+0000","last_became_active":"2024-05-10T03:32:56.662625+0000","last_became_peered":"2024-05-10T03:32:56.662625+0000","last_unstale":"2024-05-10T03:32:56.663836+0000","last_undegraded":"2024-05-10T03:32:56.663836+0000","last_fullsized":"2024-05-10T03:32:56.663836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:14:48.700520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663780+0000","last_change":"2024-05-10T03:32:56.662076+0000","last_active":"2024-05-10T03:32:56.663780+0000","last_peered":"2024-05-10T03:32:56.663780+0000","last_clean":"2024-05-10T03:32:56.663780+0000","last_became_active":"2024-05-10T03:32:56.661738+0000","last_became_peered":"2024-05-10T03:32:56.661738+0000","last_unstale":"2024-05-10T03:32:56.663780+0000","last_undegraded":"2024-05-10T03:32:56.663780+0000","last_fullsized":"2024-05-10T03:32:56.663780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:48.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664829+0000","last_change":"2024-05-10T03:32:56.655233+0000","last_active":"2024-05-10T03:32:56.664829+0000","last_peered":"2024-05-10T03:32:56.664829+0000","last_clean":"2024-05-10T03:32:56.664829+0000","last_became_active":"2024-05-10T03:32:56.654914+0000","last_became_peered":"2024-05-10T03:32:56.654914+0000","last_unstale":"2024-05-10T03:32:56.664829+0000","last_undegraded":"2024-05-10T03:32:56.664829+0000","last_fullsized":"2024-05-10T03:32:56.664829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:03.475802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.649415+0000","last_change":"2024-05-10T03:32:56.638635+0000","last_active":"2024-05-10T03:32:56.649415+0000","last_peered":"2024-05-10T03:32:56.649415+0000","last_clean":"2024-05-10T03:32:56.649415+0000","last_became_active":"2024-05-10T03:32:56.638203+0000","last_became_peered":"2024-05-10T03:32:56.638203+0000","last_unstale":"2024-05-10T03:32:56.649415+0000","last_undegraded":"2024-05-10T03:32:56.649415+0000","last_fullsized":"2024-05-10T03:32:56.649415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:25.353955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.648511+0000","last_change":"2024-05-10T03:32:56.641752+0000","last_active":"2024-05-10T03:32:56.648511+0000","last_peered":"2024-05-10T03:32:56.648511+0000","last_clean":"2024-05-10T03:32:56.648511+0000","last_became_active":"2024-05-10T03:32:56.641055+0000","last_became_peered":"2024-05-10T03:32:56.641055+0000","last_unstale":"2024-05-10T03:32:56.648511+0000","last_undegraded":"2024-05-10T03:32:56.648511+0000","last_fullsized":"2024-05-10T03:32:56.648511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:01:33.945466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650889+0000","last_change":"2024-05-10T03:32:56.643720+0000","last_active":"2024-05-10T03:32:56.650889+0000","last_peered":"2024-05-10T03:32:56.650889+0000","last_clean":"2024-05-10T03:32:56.650889+0000","last_became_active":"2024-05-10T03:32:56.642824+0000","last_became_peered":"2024-05-10T03:32:56.642824+0000","last_unstale":"2024-05-10T03:32:56.650889+0000","last_undegraded":"2024-05-10T03:32:56.650889+0000","last_fullsized":"2024-05-10T03:32:56.650889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:35.712429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658076+0000","last_change":"2024-05-10T03:32:56.656891+0000","last_active":"2024-05-10T03:32:56.658076+0000","last_peered":"2024-05-10T03:32:56.658076+0000","last_clean":"2024-05-10T03:32:56.658076+0000","last_became_active":"2024-05-10T03:32:56.656575+0000","last_became_peered":"2024-05-10T03:32:56.656575+0000","last_unstale":"2024-05-10T03:32:56.658076+0000","last_undegraded":"2024-05-10T03:32:56.658076+0000","last_fullsized":"2024-05-10T03:32:56.658076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671434+0000","last_change":"2024-05-10T03:32:56.666376+0000","last_active":"2024-05-10T03:32:56.671434+0000","last_peered":"2024-05-10T03:32:56.671434+0000","last_clean":"2024-05-10T03:32:56.671434+0000","last_became_active":"2024-05-10T03:32:56.665995+0000","last_became_peered":"2024-05-10T03:32:56.665995+0000","last_unstale":"2024-05-10T03:32:56.671434+0000","last_undegraded":"2024-05-10T03:32:56.671434+0000","last_fullsized":"2024-05-10T03:32:56.671434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651590+0000","last_change":"2024-05-10T03:32:56.648373+0000","last_active":"2024-05-10T03:32:56.651590+0000","last_peered":"2024-05-10T03:32:56.651590+0000","last_clean":"2024-05-10T03:32:56.651590+0000","last_became_active":"2024-05-10T03:32:56.647975+0000","last_became_peered":"2024-05-10T03:32:56.647975+0000","last_unstale":"2024-05-10T03:32:56.651590+0000","last_undegraded":"2024-05-10T03:32:56.651590+0000","last_fullsized":"2024-05-10T03:32:56.651590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:15:38.802682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650435+0000","last_change":"2024-05-10T03:32:56.638669+0000","last_active":"2024-05-10T03:32:56.650435+0000","last_peered":"2024-05-10T03:32:56.650435+0000","last_clean":"2024-05-10T03:32:56.650435+0000","last_became_active":"2024-05-10T03:32:56.638288+0000","last_became_peered":"2024-05-10T03:32:56.638288+0000","last_unstale":"2024-05-10T03:32:56.650435+0000","last_undegraded":"2024-05-10T03:32:56.650435+0000","last_fullsized":"2024-05-10T03:32:56.650435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:48:28.869006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.635374+0000","last_change":"2024-05-10T03:32:56.632738+0000","last_active":"2024-05-10T03:32:56.635374+0000","last_peered":"2024-05-10T03:32:56.635374+0000","last_clean":"2024-05-10T03:32:56.635374+0000","last_became_active":"2024-05-10T03:32:56.632386+0000","last_became_peered":"2024-05-10T03:32:56.632386+0000","last_unstale":"2024-05-10T03:32:56.635374+0000","last_undegraded":"2024-05-10T03:32:56.635374+0000","last_fullsized":"2024-05-10T03:32:56.635374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:20.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655994+0000","last_change":"2024-05-10T03:32:56.647475+0000","last_active":"2024-05-10T03:32:56.655994+0000","last_peered":"2024-05-10T03:32:56.655994+0000","last_clean":"2024-05-10T03:32:56.655994+0000","last_became_active":"2024-05-10T03:32:56.647060+0000","last_became_peered":"2024-05-10T03:32:56.647060+0000","last_unstale":"2024-05-10T03:32:56.655994+0000","last_undegraded":"2024-05-10T03:32:56.655994+0000","last_fullsized":"2024-05-10T03:32:56.655994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.638555+0000","last_change":"2024-05-10T03:32:56.064165+0000","last_active":"2024-05-10T03:32:56.638555+0000","last_peered":"2024-05-10T03:32:56.638555+0000","last_clean":"2024-05-10T03:32:56.638555+0000","last_became_active":"2024-05-10T03:32:56.063684+0000","last_became_peered":"2024-05-10T03:32:56.063684+0000","last_unstale":"2024-05-10T03:32:56.638555+0000","last_undegraded":"2024-05-10T03:32:56.638555+0000","last_fullsized":"2024-05-10T03:32:56.638555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:40.673453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658296+0000","last_change":"2024-05-10T03:32:56.651496+0000","last_active":"2024-05-10T03:32:56.658296+0000","last_peered":"2024-05-10T03:32:56.658296+0000","last_clean":"2024-05-10T03:32:56.658296+0000","last_became_active":"2024-05-10T03:32:56.651166+0000","last_became_peered":"2024-05-10T03:32:56.651166+0000","last_unstale":"2024-05-10T03:32:56.658296+0000","last_undegraded":"2024-05-10T03:32:56.658296+0000","last_fullsized":"2024-05-10T03:32:56.658296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:55.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636284+0000","last_change":"2024-05-10T03:32:56.058681+0000","last_active":"2024-05-10T03:32:56.636284+0000","last_peered":"2024-05-10T03:32:56.636284+0000","last_clean":"2024-05-10T03:32:56.636284+0000","last_became_active":"2024-05-10T03:32:56.058341+0000","last_became_peered":"2024-05-10T03:32:56.058341+0000","last_unstale":"2024-05-10T03:32:56.636284+0000","last_undegraded":"2024-05-10T03:32:56.636284+0000","last_fullsized":"2024-05-10T03:32:56.636284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:22:47.236001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.639812+0000","last_change":"2024-05-10T03:32:56.637636+0000","last_active":"2024-05-10T03:32:56.639812+0000","last_peered":"2024-05-10T03:32:56.639812+0000","last_clean":"2024-05-10T03:32:56.639812+0000","last_became_active":"2024-05-10T03:32:56.636267+0000","last_became_peered":"2024-05-10T03:32:56.636267+0000","last_unstale":"2024-05-10T03:32:56.639812+0000","last_undegraded":"2024-05-10T03:32:56.639812+0000","last_fullsized":"2024-05-10T03:32:56.639812+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:07.669782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.640484+0000","last_change":"2024-05-10T03:32:56.056232+0000","last_active":"2024-05-10T03:32:56.640484+0000","last_peered":"2024-05-10T03:32:56.640484+0000","last_clean":"2024-05-10T03:32:56.640484+0000","last_became_active":"2024-05-10T03:32:56.055911+0000","last_became_peered":"2024-05-10T03:32:56.055911+0000","last_unstale":"2024-05-10T03:32:56.640484+0000","last_undegraded":"2024-05-10T03:32:56.640484+0000","last_fullsized":"2024-05-10T03:32:56.640484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:02.454790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.668963+0000","last_change":"2024-05-10T03:32:56.665949+0000","last_active":"2024-05-10T03:32:56.668963+0000","last_peered":"2024-05-10T03:32:56.668963+0000","last_clean":"2024-05-10T03:32:56.668963+0000","last_became_active":"2024-05-10T03:32:56.665576+0000","last_became_peered":"2024-05-10T03:32:56.665576+0000","last_unstale":"2024-05-10T03:32:56.668963+0000","last_undegraded":"2024-05-10T03:32:56.668963+0000","last_fullsized":"2024-05-10T03:32:56.668963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:25.353955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634502+0000","last_change":"2024-05-10T03:32:56.065654+0000","last_active":"2024-05-10T03:32:56.634502+0000","last_peered":"2024-05-10T03:32:56.634502+0000","last_clean":"2024-05-10T03:32:56.634502+0000","last_became_active":"2024-05-10T03:32:56.064205+0000","last_became_peered":"2024-05-10T03:32:56.064205+0000","last_unstale":"2024-05-10T03:32:56.634502+0000","last_undegraded":"2024-05-10T03:32:56.634502+0000","last_fullsized":"2024-05-10T03:32:56.634502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:38.601647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.637728+0000","last_change":"2024-05-10T03:32:56.635413+0000","last_active":"2024-05-10T03:32:56.637728+0000","last_peered":"2024-05-10T03:32:56.637728+0000","last_clean":"2024-05-10T03:32:56.637728+0000","last_became_active":"2024-05-10T03:32:56.634954+0000","last_became_peered":"2024-05-10T03:32:56.634954+0000","last_unstale":"2024-05-10T03:32:56.637728+0000","last_undegraded":"2024-05-10T03:32:56.637728+0000","last_fullsized":"2024-05-10T03:32:56.637728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:04.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.630214+0000","last_change":"2024-05-10T03:32:56.054732+0000","last_active":"2024-05-10T03:32:56.630214+0000","last_peered":"2024-05-10T03:32:56.630214+0000","last_clean":"2024-05-10T03:32:56.630214+0000","last_became_active":"2024-05-10T03:32:56.054391+0000","last_became_peered":"2024-05-10T03:32:56.054391+0000","last_unstale":"2024-05-10T03:32:56.630214+0000","last_undegraded":"2024-05-10T03:32:56.630214+0000","last_fullsized":"2024-05-10T03:32:56.630214+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:35:39.520715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.646213+0000","last_change":"2024-05-10T03:32:56.644484+0000","last_active":"2024-05-10T03:32:56.646213+0000","last_peered":"2024-05-10T03:32:56.646213+0000","last_clean":"2024-05-10T03:32:56.646213+0000","last_became_active":"2024-05-10T03:32:56.643771+0000","last_became_peered":"2024-05-10T03:32:56.643771+0000","last_unstale":"2024-05-10T03:32:56.646213+0000","last_undegraded":"2024-05-10T03:32:56.646213+0000","last_fullsized":"2024-05-10T03:32:56.646213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634064+0000","last_change":"2024-05-10T03:32:56.054043+0000","last_active":"2024-05-10T03:32:56.634064+0000","last_peered":"2024-05-10T03:32:56.634064+0000","last_clean":"2024-05-10T03:32:56.634064+0000","last_became_active":"2024-05-10T03:32:56.051572+0000","last_became_peered":"2024-05-10T03:32:56.051572+0000","last_unstale":"2024-05-10T03:32:56.634064+0000","last_undegraded":"2024-05-10T03:32:56.634064+0000","last_fullsized":"2024-05-10T03:32:56.634064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:17.868723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656843+0000","last_change":"2024-05-10T03:32:56.645639+0000","last_active":"2024-05-10T03:32:56.656843+0000","last_peered":"2024-05-10T03:32:56.656843+0000","last_clean":"2024-05-10T03:32:56.656843+0000","last_became_active":"2024-05-10T03:32:56.645308+0000","last_became_peered":"2024-05-10T03:32:56.645308+0000","last_unstale":"2024-05-10T03:32:56.656843+0000","last_undegraded":"2024-05-10T03:32:56.656843+0000","last_fullsized":"2024-05-10T03:32:56.656843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:35.393359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626644+0000","last_change":"2024-05-10T03:32:56.059410+0000","last_active":"2024-05-10T03:32:56.626644+0000","last_peered":"2024-05-10T03:32:56.626644+0000","last_clean":"2024-05-10T03:32:56.626644+0000","last_became_active":"2024-05-10T03:32:56.059088+0000","last_became_peered":"2024-05-10T03:32:56.059088+0000","last_unstale":"2024-05-10T03:32:56.626644+0000","last_undegraded":"2024-05-10T03:32:56.626644+0000","last_fullsized":"2024-05-10T03:32:56.626644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:06.648770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.645222+0000","last_change":"2024-05-10T03:32:56.643809+0000","last_active":"2024-05-10T03:32:56.645222+0000","last_peered":"2024-05-10T03:32:56.645222+0000","last_clean":"2024-05-10T03:32:56.645222+0000","last_became_active":"2024-05-10T03:32:56.643444+0000","last_became_peered":"2024-05-10T03:32:56.643444+0000","last_unstale":"2024-05-10T03:32:56.645222+0000","last_undegraded":"2024-05-10T03:32:56.645222+0000","last_fullsized":"2024-05-10T03:32:56.645222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:05.641768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.633459+0000","last_change":"2024-05-10T03:32:56.054544+0000","last_active":"2024-05-10T03:32:56.633459+0000","last_peered":"2024-05-10T03:32:56.633459+0000","last_clean":"2024-05-10T03:32:56.633459+0000","last_became_active":"2024-05-10T03:32:56.052424+0000","last_became_peered":"2024-05-10T03:32:56.052424+0000","last_unstale":"2024-05-10T03:32:56.633459+0000","last_undegraded":"2024-05-10T03:32:56.633459+0000","last_fullsized":"2024-05-10T03:32:56.633459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.854201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636613+0000","last_change":"2024-05-10T03:32:56.632190+0000","last_active":"2024-05-10T03:32:56.636613+0000","last_peered":"2024-05-10T03:32:56.636613+0000","last_clean":"2024-05-10T03:32:56.636613+0000","last_became_active":"2024-05-10T03:32:56.631782+0000","last_became_peered":"2024-05-10T03:32:56.631782+0000","last_unstale":"2024-05-10T03:32:56.636613+0000","last_undegraded":"2024-05-10T03:32:56.636613+0000","last_fullsized":"2024-05-10T03:32:56.636613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.660580+0000","last_change":"2024-05-10T03:32:56.065803+0000","last_active":"2024-05-10T03:32:56.660580+0000","last_peered":"2024-05-10T03:32:56.660580+0000","last_clean":"2024-05-10T03:32:56.660580+0000","last_became_active":"2024-05-10T03:32:56.065252+0000","last_became_peered":"2024-05-10T03:32:56.065252+0000","last_unstale":"2024-05-10T03:32:56.660580+0000","last_undegraded":"2024-05-10T03:32:56.660580+0000","last_fullsized":"2024-05-10T03:32:56.660580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:47:45.628490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.639860+0000","last_change":"2024-05-10T03:32:56.638389+0000","last_active":"2024-05-10T03:32:56.639860+0000","last_peered":"2024-05-10T03:32:56.639860+0000","last_clean":"2024-05-10T03:32:56.639860+0000","last_became_active":"2024-05-10T03:32:56.638083+0000","last_became_peered":"2024-05-10T03:32:56.638083+0000","last_unstale":"2024-05-10T03:32:56.639860+0000","last_undegraded":"2024-05-10T03:32:56.639860+0000","last_fullsized":"2024-05-10T03:32:56.639860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:56.340378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650319+0000","last_change":"2024-05-10T03:32:56.061445+0000","last_active":"2024-05-10T03:32:56.650319+0000","last_peered":"2024-05-10T03:32:56.650319+0000","last_clean":"2024-05-10T03:32:56.650319+0000","last_became_active":"2024-05-10T03:32:56.060999+0000","last_became_peered":"2024-05-10T03:32:56.060999+0000","last_unstale":"2024-05-10T03:32:56.650319+0000","last_undegraded":"2024-05-10T03:32:56.650319+0000","last_fullsized":"2024-05-10T03:32:56.650319+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:51:24.605065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.657693+0000","last_change":"2024-05-10T03:32:56.650427+0000","last_active":"2024-05-10T03:32:56.657693+0000","last_peered":"2024-05-10T03:32:56.657693+0000","last_clean":"2024-05-10T03:32:56.657693+0000","last_became_active":"2024-05-10T03:32:56.649323+0000","last_became_peered":"2024-05-10T03:32:56.649323+0000","last_unstale":"2024-05-10T03:32:56.657693+0000","last_undegraded":"2024-05-10T03:32:56.657693+0000","last_fullsized":"2024-05-10T03:32:56.657693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665205+0000","last_change":"2024-05-10T03:32:56.065871+0000","last_active":"2024-05-10T03:32:56.665205+0000","last_peered":"2024-05-10T03:32:56.665205+0000","last_clean":"2024-05-10T03:32:56.665205+0000","last_became_active":"2024-05-10T03:32:56.065242+0000","last_became_peered":"2024-05-10T03:32:56.065242+0000","last_unstale":"2024-05-10T03:32:56.665205+0000","last_undegraded":"2024-05-10T03:32:56.665205+0000","last_fullsized":"2024-05-10T03:32:56.665205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:40.673453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653066+0000","last_change":"2024-05-10T03:32:56.644734+0000","last_active":"2024-05-10T03:32:56.653066+0000","last_peered":"2024-05-10T03:32:56.653066+0000","last_clean":"2024-05-10T03:32:56.653066+0000","last_became_active":"2024-05-10T03:32:56.644346+0000","last_became_peered":"2024-05-10T03:32:56.644346+0000","last_unstale":"2024-05-10T03:32:56.653066+0000","last_undegraded":"2024-05-10T03:32:56.653066+0000","last_fullsized":"2024-05-10T03:32:56.653066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:44.777127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667525+0000","last_change":"2024-05-10T03:32:56.055660+0000","last_active":"2024-05-10T03:32:56.667525+0000","last_peered":"2024-05-10T03:32:56.667525+0000","last_clean":"2024-05-10T03:32:56.667525+0000","last_became_active":"2024-05-10T03:32:56.055271+0000","last_became_peered":"2024-05-10T03:32:56.055271+0000","last_unstale":"2024-05-10T03:32:56.667525+0000","last_undegraded":"2024-05-10T03:32:56.667525+0000","last_fullsized":"2024-05-10T03:32:56.667525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:47:45.628490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664233+0000","last_change":"2024-05-10T03:32:56.658899+0000","last_active":"2024-05-10T03:32:56.664233+0000","last_peered":"2024-05-10T03:32:56.664233+0000","last_clean":"2024-05-10T03:32:56.664233+0000","last_became_active":"2024-05-10T03:32:56.656381+0000","last_became_peered":"2024-05-10T03:32:56.656381+0000","last_unstale":"2024-05-10T03:32:56.664233+0000","last_undegraded":"2024-05-10T03:32:56.664233+0000","last_fullsized":"2024-05-10T03:32:56.664233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:16.750300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664834+0000","last_change":"2024-05-10T03:32:56.058583+0000","last_active":"2024-05-10T03:32:56.664834+0000","last_peered":"2024-05-10T03:32:56.664834+0000","last_clean":"2024-05-10T03:32:56.664834+0000","last_became_active":"2024-05-10T03:32:56.058260+0000","last_became_peered":"2024-05-10T03:32:56.058260+0000","last_unstale":"2024-05-10T03:32:56.664834+0000","last_undegraded":"2024-05-10T03:32:56.664834+0000","last_fullsized":"2024-05-10T03:32:56.664834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:43.756115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647412+0000","last_change":"2024-05-10T03:32:56.639319+0000","last_active":"2024-05-10T03:32:56.647412+0000","last_peered":"2024-05-10T03:32:56.647412+0000","last_clean":"2024-05-10T03:32:56.647412+0000","last_became_active":"2024-05-10T03:32:56.638858+0000","last_became_peered":"2024-05-10T03:32:56.638858+0000","last_unstale":"2024-05-10T03:32:56.647412+0000","last_undegraded":"2024-05-10T03:32:56.647412+0000","last_fullsized":"2024-05-10T03:32:56.647412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:29.455124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.617703+0000","last_change":"2024-05-10T03:32:56.062821+0000","last_active":"2024-05-10T03:32:56.617703+0000","last_peered":"2024-05-10T03:32:56.617703+0000","last_clean":"2024-05-10T03:32:56.617703+0000","last_became_active":"2024-05-10T03:32:56.062345+0000","last_became_peered":"2024-05-10T03:32:56.062345+0000","last_unstale":"2024-05-10T03:32:56.617703+0000","last_undegraded":"2024-05-10T03:32:56.617703+0000","last_fullsized":"2024-05-10T03:32:56.617703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:24.419859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656821+0000","last_change":"2024-05-10T03:32:56.644966+0000","last_active":"2024-05-10T03:32:56.656821+0000","last_peered":"2024-05-10T03:32:56.656821+0000","last_clean":"2024-05-10T03:32:56.656821+0000","last_became_active":"2024-05-10T03:32:56.643950+0000","last_became_peered":"2024-05-10T03:32:56.643950+0000","last_unstale":"2024-05-10T03:32:56.656821+0000","last_undegraded":"2024-05-10T03:32:56.656821+0000","last_fullsized":"2024-05-10T03:32:56.656821+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:55.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650664+0000","last_change":"2024-05-10T03:32:56.063611+0000","last_active":"2024-05-10T03:32:56.650664+0000","last_peered":"2024-05-10T03:32:56.650664+0000","last_clean":"2024-05-10T03:32:56.650664+0000","last_became_active":"2024-05-10T03:32:56.063210+0000","last_became_peered":"2024-05-10T03:32:56.063210+0000","last_unstale":"2024-05-10T03:32:56.650664+0000","last_undegraded":"2024-05-10T03:32:56.650664+0000","last_fullsized":"2024-05-10T03:32:56.650664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:14:15.148949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663448+0000","last_change":"2024-05-10T03:32:56.660894+0000","last_active":"2024-05-10T03:32:56.663448+0000","last_peered":"2024-05-10T03:32:56.663448+0000","last_clean":"2024-05-10T03:32:56.663448+0000","last_became_active":"2024-05-10T03:32:56.660580+0000","last_became_peered":"2024-05-10T03:32:56.660580+0000","last_unstale":"2024-05-10T03:32:56.663448+0000","last_undegraded":"2024-05-10T03:32:56.663448+0000","last_fullsized":"2024-05-10T03:32:56.663448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667381+0000","last_change":"2024-05-10T03:32:56.052809+0000","last_active":"2024-05-10T03:32:56.667381+0000","last_peered":"2024-05-10T03:32:56.667381+0000","last_clean":"2024-05-10T03:32:56.667381+0000","last_became_active":"2024-05-10T03:32:56.052454+0000","last_became_peered":"2024-05-10T03:32:56.052454+0000","last_unstale":"2024-05-10T03:32:56.667381+0000","last_undegraded":"2024-05-10T03:32:56.667381+0000","last_fullsized":"2024-05-10T03:32:56.667381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:50.141087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671453+0000","last_change":"2024-05-10T03:32:56.666617+0000","last_active":"2024-05-10T03:32:56.671453+0000","last_peered":"2024-05-10T03:32:56.671453+0000","last_clean":"2024-05-10T03:32:56.671453+0000","last_became_active":"2024-05-10T03:32:56.665890+0000","last_became_peered":"2024-05-10T03:32:56.665890+0000","last_unstale":"2024-05-10T03:32:56.671453+0000","last_undegraded":"2024-05-10T03:32:56.671453+0000","last_fullsized":"2024-05-10T03:32:56.671453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:01:24.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653999+0000","last_change":"2024-05-10T03:32:56.063651+0000","last_active":"2024-05-10T03:32:56.653999+0000","last_peered":"2024-05-10T03:32:56.653999+0000","last_clean":"2024-05-10T03:32:56.653999+0000","last_became_active":"2024-05-10T03:32:56.063292+0000","last_became_peered":"2024-05-10T03:32:56.063292+0000","last_unstale":"2024-05-10T03:32:56.653999+0000","last_undegraded":"2024-05-10T03:32:56.653999+0000","last_fullsized":"2024-05-10T03:32:56.653999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:34.372347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664962+0000","last_change":"2024-05-10T03:32:56.660043+0000","last_active":"2024-05-10T03:32:56.664962+0000","last_peered":"2024-05-10T03:32:56.664962+0000","last_clean":"2024-05-10T03:32:56.664962+0000","last_became_active":"2024-05-10T03:32:56.659720+0000","last_became_peered":"2024-05-10T03:32:56.659720+0000","last_unstale":"2024-05-10T03:32:56.664962+0000","last_undegraded":"2024-05-10T03:32:56.664962+0000","last_fullsized":"2024-05-10T03:32:56.664962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:44.288664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667593+0000","last_change":"2024-05-10T03:32:56.060479+0000","last_active":"2024-05-10T03:32:56.667593+0000","last_peered":"2024-05-10T03:32:56.667593+0000","last_clean":"2024-05-10T03:32:56.667593+0000","last_became_active":"2024-05-10T03:32:56.058343+0000","last_became_peered":"2024-05-10T03:32:56.058343+0000","last_unstale":"2024-05-10T03:32:56.667593+0000","last_undegraded":"2024-05-10T03:32:56.667593+0000","last_fullsized":"2024-05-10T03:32:56.667593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:08.424775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655945+0000","last_change":"2024-05-10T03:32:56.651151+0000","last_active":"2024-05-10T03:32:56.655945+0000","last_peered":"2024-05-10T03:32:56.655945+0000","last_clean":"2024-05-10T03:32:56.655945+0000","last_became_active":"2024-05-10T03:32:56.650633+0000","last_became_peered":"2024-05-10T03:32:56.650633+0000","last_unstale":"2024-05-10T03:32:56.655945+0000","last_undegraded":"2024-05-10T03:32:56.655945+0000","last_fullsized":"2024-05-10T03:32:56.655945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:44.288664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665314+0000","last_change":"2024-05-10T03:32:56.065790+0000","last_active":"2024-05-10T03:32:56.665314+0000","last_peered":"2024-05-10T03:32:56.665314+0000","last_clean":"2024-05-10T03:32:56.665314+0000","last_became_active":"2024-05-10T03:32:56.065192+0000","last_became_peered":"2024-05-10T03:32:56.065192+0000","last_unstale":"2024-05-10T03:32:56.665314+0000","last_undegraded":"2024-05-10T03:32:56.665314+0000","last_fullsized":"2024-05-10T03:32:56.665314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656730+0000","last_change":"2024-05-10T03:32:56.646506+0000","last_active":"2024-05-10T03:32:56.656730+0000","last_peered":"2024-05-10T03:32:56.656730+0000","last_clean":"2024-05-10T03:32:56.656730+0000","last_became_active":"2024-05-10T03:32:56.646171+0000","last_became_peered":"2024-05-10T03:32:56.646171+0000","last_unstale":"2024-05-10T03:32:56.656730+0000","last_undegraded":"2024-05-10T03:32:56.656730+0000","last_fullsized":"2024-05-10T03:32:56.656730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626874+0000","last_change":"2024-05-10T03:32:56.049848+0000","last_active":"2024-05-10T03:32:56.626874+0000","last_peered":"2024-05-10T03:32:56.626874+0000","last_clean":"2024-05-10T03:32:56.626874+0000","last_became_active":"2024-05-10T03:32:56.049439+0000","last_became_peered":"2024-05-10T03:32:56.049439+0000","last_unstale":"2024-05-10T03:32:56.626874+0000","last_undegraded":"2024-05-10T03:32:56.626874+0000","last_fullsized":"2024-05-10T03:32:56.626874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:05:26.132081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634745+0000","last_change":"2024-05-10T03:32:56.631763+0000","last_active":"2024-05-10T03:32:56.634745+0000","last_peered":"2024-05-10T03:32:56.634745+0000","last_clean":"2024-05-10T03:32:56.634745+0000","last_became_active":"2024-05-10T03:32:56.631455+0000","last_became_peered":"2024-05-10T03:32:56.631455+0000","last_unstale":"2024-05-10T03:32:56.634745+0000","last_undegraded":"2024-05-10T03:32:56.634745+0000","last_fullsized":"2024-05-10T03:32:56.634745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.889735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.638287+0000","last_change":"2024-05-10T03:32:56.058835+0000","last_active":"2024-05-10T03:32:56.638287+0000","last_peered":"2024-05-10T03:32:56.638287+0000","last_clean":"2024-05-10T03:32:56.638287+0000","last_became_active":"2024-05-10T03:32:56.058238+0000","last_became_peered":"2024-05-10T03:32:56.058238+0000","last_unstale":"2024-05-10T03:32:56.638287+0000","last_undegraded":"2024-05-10T03:32:56.638287+0000","last_fullsized":"2024-05-10T03:32:56.638287+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:43.134365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647856+0000","last_change":"2024-05-10T03:32:56.640892+0000","last_active":"2024-05-10T03:32:56.647856+0000","last_peered":"2024-05-10T03:32:56.647856+0000","last_clean":"2024-05-10T03:32:56.647856+0000","last_became_active":"2024-05-10T03:32:56.640546+0000","last_became_peered":"2024-05-10T03:32:56.640546+0000","last_unstale":"2024-05-10T03:32:56.647856+0000","last_undegraded":"2024-05-10T03:32:56.647856+0000","last_fullsized":"2024-05-10T03:32:56.647856+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:19:53.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.617998+0000","last_change":"2024-05-10T03:32:56.061608+0000","last_active":"2024-05-10T03:32:56.617998+0000","last_peered":"2024-05-10T03:32:56.617998+0000","last_clean":"2024-05-10T03:32:56.617998+0000","last_became_active":"2024-05-10T03:32:56.061201+0000","last_became_peered":"2024-05-10T03:32:56.061201+0000","last_unstale":"2024-05-10T03:32:56.617998+0000","last_undegraded":"2024-05-10T03:32:56.617998+0000","last_fullsized":"2024-05-10T03:32:56.617998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:09.188366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653385+0000","last_change":"2024-05-10T03:32:56.642021+0000","last_active":"2024-05-10T03:32:56.653385+0000","last_peered":"2024-05-10T03:32:56.653385+0000","last_clean":"2024-05-10T03:32:56.653385+0000","last_became_active":"2024-05-10T03:32:56.641650+0000","last_became_peered":"2024-05-10T03:32:56.641650+0000","last_unstale":"2024-05-10T03:32:56.653385+0000","last_undegraded":"2024-05-10T03:32:56.653385+0000","last_fullsized":"2024-05-10T03:32:56.653385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:16.750300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.613287+0000","last_change":"2024-05-10T03:32:56.059212+0000","last_active":"2024-05-10T03:32:56.613287+0000","last_peered":"2024-05-10T03:32:56.613287+0000","last_clean":"2024-05-10T03:32:56.613287+0000","last_became_active":"2024-05-10T03:32:56.058916+0000","last_became_peered":"2024-05-10T03:32:56.058916+0000","last_unstale":"2024-05-10T03:32:56.613287+0000","last_undegraded":"2024-05-10T03:32:56.613287+0000","last_fullsized":"2024-05-10T03:32:56.613287+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:02.479608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655367+0000","last_change":"2024-05-10T03:32:56.654543+0000","last_active":"2024-05-10T03:32:56.655367+0000","last_peered":"2024-05-10T03:32:56.655367+0000","last_clean":"2024-05-10T03:32:56.655367+0000","last_became_active":"2024-05-10T03:32:56.654286+0000","last_became_peered":"2024-05-10T03:32:56.654286+0000","last_unstale":"2024-05-10T03:32:56.655367+0000","last_undegraded":"2024-05-10T03:32:56.655367+0000","last_fullsized":"2024-05-10T03:32:56.655367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:29.455124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.662274+0000","last_change":"2024-05-10T03:32:56.056422+0000","last_active":"2024-05-10T03:32:56.662274+0000","last_peered":"2024-05-10T03:32:56.662274+0000","last_clean":"2024-05-10T03:32:56.662274+0000","last_became_active":"2024-05-10T03:32:56.056073+0000","last_became_peered":"2024-05-10T03:32:56.056073+0000","last_unstale":"2024-05-10T03:32:56.662274+0000","last_undegraded":"2024-05-10T03:32:56.662274+0000","last_fullsized":"2024-05-10T03:32:56.662274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:10.624076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.633654+0000","last_change":"2024-05-10T03:32:56.628240+0000","last_active":"2024-05-10T03:32:56.633654+0000","last_peered":"2024-05-10T03:32:56.633654+0000","last_clean":"2024-05-10T03:32:56.633654+0000","last_became_active":"2024-05-10T03:32:56.627944+0000","last_became_peered":"2024-05-10T03:32:56.627944+0000","last_unstale":"2024-05-10T03:32:56.633654+0000","last_undegraded":"2024-05-10T03:32:56.633654+0000","last_fullsized":"2024-05-10T03:32:56.633654+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:54.022090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.621724+0000","last_change":"2024-05-10T03:32:56.051891+0000","last_active":"2024-05-10T03:32:56.621724+0000","last_peered":"2024-05-10T03:32:56.621724+0000","last_clean":"2024-05-10T03:32:56.621724+0000","last_became_active":"2024-05-10T03:32:56.051570+0000","last_became_peered":"2024-05-10T03:32:56.051570+0000","last_unstale":"2024-05-10T03:32:56.621724+0000","last_undegraded":"2024-05-10T03:32:56.621724+0000","last_fullsized":"2024-05-10T03:32:56.621724+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:15.729288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651477+0000","last_change":"2024-05-10T03:32:56.641971+0000","last_active":"2024-05-10T03:32:56.651477+0000","last_peered":"2024-05-10T03:32:56.651477+0000","last_clean":"2024-05-10T03:32:56.651477+0000","last_became_active":"2024-05-10T03:32:56.641422+0000","last_became_peered":"2024-05-10T03:32:56.641422+0000","last_unstale":"2024-05-10T03:32:56.651477+0000","last_undegraded":"2024-05-10T03:32:56.651477+0000","last_fullsized":"2024-05-10T03:32:56.651477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.889735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.613586+0000","last_change":"2024-05-10T03:32:56.054270+0000","last_active":"2024-05-10T03:32:56.613586+0000","last_peered":"2024-05-10T03:32:56.613586+0000","last_clean":"2024-05-10T03:32:56.613586+0000","last_became_active":"2024-05-10T03:32:56.053940+0000","last_became_peered":"2024-05-10T03:32:56.053940+0000","last_unstale":"2024-05-10T03:32:56.613586+0000","last_undegraded":"2024-05-10T03:32:56.613586+0000","last_fullsized":"2024-05-10T03:32:56.613586+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:01:23.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656707+0000","last_change":"2024-05-10T03:32:56.646070+0000","last_active":"2024-05-10T03:32:56.656707+0000","last_peered":"2024-05-10T03:32:56.656707+0000","last_clean":"2024-05-10T03:32:56.656707+0000","last_became_active":"2024-05-10T03:32:56.645738+0000","last_became_peered":"2024-05-10T03:32:56.645738+0000","last_unstale":"2024-05-10T03:32:56.656707+0000","last_undegraded":"2024-05-10T03:32:56.656707+0000","last_fullsized":"2024-05-10T03:32:56.656707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.624607+0000","last_change":"2024-05-10T03:32:56.053264+0000","last_active":"2024-05-10T03:32:56.624607+0000","last_peered":"2024-05-10T03:32:56.624607+0000","last_clean":"2024-05-10T03:32:56.624607+0000","last_became_active":"2024-05-10T03:32:56.052057+0000","last_became_peered":"2024-05-10T03:32:56.052057+0000","last_unstale":"2024-05-10T03:32:56.624607+0000","last_undegraded":"2024-05-10T03:32:56.624607+0000","last_fullsized":"2024-05-10T03:32:56.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655949+0000","last_change":"2024-05-10T03:32:56.645370+0000","last_active":"2024-05-10T03:32:56.655949+0000","last_peered":"2024-05-10T03:32:56.655949+0000","last_clean":"2024-05-10T03:32:56.655949+0000","last_became_active":"2024-05-10T03:32:56.645019+0000","last_became_peered":"2024-05-10T03:32:56.645019+0000","last_unstale":"2024-05-10T03:32:56.655949+0000","last_undegraded":"2024-05-10T03:32:56.655949+0000","last_fullsized":"2024-05-10T03:32:56.655949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.652618+0000","last_change":"2024-05-10T03:32:56.063908+0000","last_active":"2024-05-10T03:32:56.652618+0000","last_peered":"2024-05-10T03:32:56.652618+0000","last_clean":"2024-05-10T03:32:56.652618+0000","last_became_active":"2024-05-10T03:32:56.063545+0000","last_became_peered":"2024-05-10T03:32:56.063545+0000","last_unstale":"2024-05-10T03:32:56.652618+0000","last_undegraded":"2024-05-10T03:32:56.652618+0000","last_fullsized":"2024-05-10T03:32:56.652618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:14.589377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.661968+0000","last_change":"2024-05-10T03:32:56.656067+0000","last_active":"2024-05-10T03:32:56.661968+0000","last_peered":"2024-05-10T03:32:56.661968+0000","last_clean":"2024-05-10T03:32:56.661968+0000","last_became_active":"2024-05-10T03:32:56.655703+0000","last_became_peered":"2024-05-10T03:32:56.655703+0000","last_unstale":"2024-05-10T03:32:56.661968+0000","last_undegraded":"2024-05-10T03:32:56.661968+0000","last_fullsized":"2024-05-10T03:32:56.661968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626349+0000","last_change":"2024-05-10T03:32:56.064429+0000","last_active":"2024-05-10T03:32:56.626349+0000","last_peered":"2024-05-10T03:32:56.626349+0000","last_clean":"2024-05-10T03:32:56.626349+0000","last_became_active":"2024-05-10T03:32:56.063414+0000","last_became_peered":"2024-05-10T03:32:56.063414+0000","last_unstale":"2024-05-10T03:32:56.626349+0000","last_undegraded":"2024-05-10T03:32:56.626349+0000","last_fullsized":"2024-05-10T03:32:56.626349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:19.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.645601+0000","last_change":"2024-05-10T03:32:56.643800+0000","last_active":"2024-05-10T03:32:56.645601+0000","last_peered":"2024-05-10T03:32:56.645601+0000","last_clean":"2024-05-10T03:32:56.645601+0000","last_became_active":"2024-05-10T03:32:56.643478+0000","last_became_peered":"2024-05-10T03:32:56.643478+0000","last_unstale":"2024-05-10T03:32:56.645601+0000","last_undegraded":"2024-05-10T03:32:56.645601+0000","last_fullsized":"2024-05-10T03:32:56.645601+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:22.441460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626758+0000","last_change":"2024-05-10T03:32:56.059457+0000","last_active":"2024-05-10T03:32:56.626758+0000","last_peered":"2024-05-10T03:32:56.626758+0000","last_clean":"2024-05-10T03:32:56.626758+0000","last_became_active":"2024-05-10T03:32:56.059128+0000","last_became_peered":"2024-05-10T03:32:56.059128+0000","last_unstale":"2024-05-10T03:32:56.626758+0000","last_undegraded":"2024-05-10T03:32:56.626758+0000","last_fullsized":"2024-05-10T03:32:56.626758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:48.047461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.644648+0000","last_change":"2024-05-10T03:32:56.641168+0000","last_active":"2024-05-10T03:32:56.644648+0000","last_peered":"2024-05-10T03:32:56.644648+0000","last_clean":"2024-05-10T03:32:56.644648+0000","last_became_active":"2024-05-10T03:32:56.640761+0000","last_became_peered":"2024-05-10T03:32:56.640761+0000","last_unstale":"2024-05-10T03:32:56.644648+0000","last_undegraded":"2024-05-10T03:32:56.644648+0000","last_fullsized":"2024-05-10T03:32:56.644648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:29.456781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.621204+0000","last_change":"2024-05-10T03:32:56.061249+0000","last_active":"2024-05-10T03:32:56.621204+0000","last_peered":"2024-05-10T03:32:56.621204+0000","last_clean":"2024-05-10T03:32:56.621204+0000","last_became_active":"2024-05-10T03:32:56.060931+0000","last_became_peered":"2024-05-10T03:32:56.060931+0000","last_unstale":"2024-05-10T03:32:56.621204+0000","last_undegraded":"2024-05-10T03:32:56.621204+0000","last_fullsized":"2024-05-10T03:32:56.621204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:14:15.148949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658308+0000","last_change":"2024-05-10T03:32:56.644951+0000","last_active":"2024-05-10T03:32:56.658308+0000","last_peered":"2024-05-10T03:32:56.658308+0000","last_clean":"2024-05-10T03:32:56.658308+0000","last_became_active":"2024-05-10T03:32:56.644583+0000","last_became_peered":"2024-05-10T03:32:56.644583+0000","last_unstale":"2024-05-10T03:32:56.658308+0000","last_undegraded":"2024-05-10T03:32:56.658308+0000","last_fullsized":"2024-05-10T03:32:56.658308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.623608+0000","last_change":"2024-05-10T03:32:56.055989+0000","last_active":"2024-05-10T03:32:56.623608+0000","last_peered":"2024-05-10T03:32:56.623608+0000","last_clean":"2024-05-10T03:32:56.623608+0000","last_became_active":"2024-05-10T03:32:56.054447+0000","last_became_peered":"2024-05-10T03:32:56.054447+0000","last_unstale":"2024-05-10T03:32:56.623608+0000","last_undegraded":"2024-05-10T03:32:56.623608+0000","last_fullsized":"2024-05-10T03:32:56.623608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:53:27.762809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.045385+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:56.045385+0000","last_peered":"2024-05-10T03:32:56.045385+0000","last_clean":"2024-05-10T03:32:56.045385+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:56.045385+0000","last_undegraded":"2024-05-10T03:32:56.045385+0000","last_fullsized":"2024-05-10T03:32:56.045385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1396736,"data_stored":941150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7490000000000001,"5min":1.877,"15min":1.877},"last":1.264}]},{"osd":1,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1439999999999999,"15min":1.1419999999999999},"min":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.853},"last":1.159},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2450000000000001,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9039999999999999,"5min":1.925,"15min":1.925},"last":0.746}]},{"osd":2,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.363,"15min":1.383},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.77,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3899999999999999,"15min":1.4319999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8460000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.2909999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3720000000000001,"15min":1.3859999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":2.222,"15min":2.222},"last":1.319},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.353,"15min":1.3340000000000001},"min":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9550000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.421}]},{"osd":4,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.446,"15min":1.4690000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9790000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5189999999999999,"15min":1.5069999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2,"5min":2.323,"15min":2.323},"last":1.204}]},{"osd":5,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.528,"15min":1.51},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.619,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6879999999999999,"15min":1.696},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1709999999999998,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.7130000000000001,"15min":1.7250000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6519999999999999,"15min":1.6399999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.097,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.393}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":350040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.087,"15min":1.075},"min":{"1min":0.74099999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3089999999999999,"15min":1.345},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.835,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.91800000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8160000000000001,"5min":3.113,"15min":3.113},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.423,"15min":1.48},"min":{"1min":0.76900000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.7589999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.282,"15min":1.244},"min":{"1min":0.98199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4299999999999999,"15min":1.504},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.859,"5min":3.569,"15min":3.569},"last":0.98299999999999998}]},{"osd":3,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.5449999999999999,"15min":1.613},"min":{"1min":1.012,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.863,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3200000000000001,"15min":1.3080000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.9099999999999999,"5min":1.931,"15min":1.931},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.998,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5449999999999999,"15min":1.579},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.9650000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.89100000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.4039999999999999,"15min":1.3919999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.6120000000000001,"15min":1.6200000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.117,"5min":2.472,"15min":2.472},"last":1.1759999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.657,"15min":1.669},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4830000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.627,"15min":1.647},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.847,"5min":2.359,"15min":2.359},"last":1.05}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5900000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.655,"5min":1.851,"15min":1.851},"last":1.4179999999999999}]},{"osd":1,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.78600000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.653,"5min":1.855,"15min":1.855},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.214,"15min":1.222},"min":{"1min":0.96699999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4219999999999999,"5min":1.954,"15min":1.954},"last":1.1639999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.242,"15min":1.226},"min":{"1min":0.81299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7490000000000001,"5min":2.359,"15min":2.359},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3089999999999999,"15min":1.331},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4970000000000001}]},{"osd":3,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.3560000000000001},"min":{"1min":1.034,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6479999999999999,"5min":1.881,"15min":1.881},"last":1.552},{"interface":"front","average":{"1min":1.413,"5min":1.3360000000000001,"15min":1.298},"min":{"1min":0.76800000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9830000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.234}]},{"osd":4,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.90200000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8540000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.069},{"interface":"front","average":{"1min":1.482,"5min":1.593,"15min":1.603},"min":{"1min":0.78100000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9670000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.3}]},{"osd":6,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.516,"15min":1.512},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1320000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5509999999999999,"15min":1.552},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.669,"5min":1.8009999999999999,"15min":1.8129999999999999},"min":{"1min":0.877,"5min":0.753,"15min":0.753},"max":{"1min":2.1739999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.6200000000000001,"15min":1.613},"min":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.98}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.159,"15min":1.173},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6919999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1259999999999999,"15min":1.1359999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.591,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1779999999999999,"15min":1.1599999999999999},"min":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.613,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.323},{"interface":"front","average":{"1min":1.139,"5min":1.248,"15min":1.2729999999999999},"min":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.72,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.79700000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5289999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.369,"15min":1.397},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.726,"5min":2.29,"15min":2.29},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.87,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.379,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.5649999999999999,"15min":1.581},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.782,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9079999999999999,"5min":2.512,"15min":2.512},"last":1.258},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.504,"15min":1.552},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5510000000000002,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.052}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.5569999999999999,"15min":1.583},"min":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4769999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.629},{"interface":"front","average":{"1min":1.51,"5min":1.724,"15min":1.7769999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.093,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.0880000000000001}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.6519999999999999,"15min":1.6899999999999999},"min":{"1min":0.63700000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.6160000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.242},{"interface":"front","average":{"1min":1.623,"5min":1.736,"15min":1.7430000000000001},"min":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6349999999999998,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.6830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":786432,"data_stored":350006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95399999999999996,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.96999999999999997},"min":{"1min":0.54500000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.575,"5min":1.708,"15min":1.708},"last":1.02}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.393,"5min":1.603,"15min":1.603},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.228,"15min":1.25},"min":{"1min":0.623,"5min":0.377,"15min":0.377},"max":{"1min":1.5329999999999999,"5min":1.964,"15min":1.964},"last":1.4399999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7450000000000001,"5min":1.99,"15min":1.99},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.3140000000000001,"15min":1.3600000000000001},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.679,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.633}]},{"osd":4,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.9019999999999999,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3999999999999999,"15min":1.3819999999999999},"min":{"1min":0.90200000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.948,"5min":2.464,"15min":2.464},"last":1.9650000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.613,"15min":1.6739999999999999},"min":{"1min":1.012,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2709999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":2.2400000000000002},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0449999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":2.359}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.5640000000000001,"15min":1.5760000000000001},"min":{"1min":1.123,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.113,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":2.081},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":1.7949999999999999},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4470000000000001,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":2.4329999999999998}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5840000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1139999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.504},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6359999999999999,"15min":1.657},"min":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.0419999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.8049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":786432,"data_stored":350006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.772,"15min":1.772},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.423,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.365},{"interface":"front","average":{"1min":1.145,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.925,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.536}]},{"osd":3,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.093,"15min":1.1200000000000001},"min":{"1min":0.57699999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":2.169,"15min":2.169},"last":1.157},{"interface":"front","average":{"1min":1.143,"5min":1.2250000000000001,"15min":1.2549999999999999},"min":{"1min":0.76800000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8200000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.8520000000000001}]},{"osd":4,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.488,"15min":1.5349999999999999},"min":{"1min":0.97199999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8160000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.46,"15min":1.538},"min":{"1min":0.86299999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7689999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.6399999999999999}]},{"osd":5,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4810000000000001,"15min":1.4890000000000001},"min":{"1min":0.91200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9259999999999999,"5min":2.508,"15min":2.508},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6100000000000001,"15min":1.6240000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.278,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0859999999999999}]},{"osd":6,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.93100000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.927,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.77},{"interface":"front","average":{"1min":1.524,"5min":1.5660000000000001,"15min":1.593},"min":{"1min":0.97199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3799999999999999,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":2.044}]},{"osd":7,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.627,"15min":1.655},"min":{"1min":1.018,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.2000000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.887},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.6739999999999999},"min":{"1min":0.95699999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.468,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":2.1379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.92100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5089999999999999,"5min":1.607,"15min":1.607},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0469999999999999,"15min":1.085},"min":{"1min":0.52100000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.7030000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.98099999999999998,"15min":1.004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.747},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.986,"5min":2.444,"15min":2.444},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.319},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.879,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2729999999999999,"15min":1.3009999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9399999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.92800000000000005}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.98499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3159999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5329999999999999,"15min":1.585},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.101,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.266}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5209999999999999,"15min":1.585},"min":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2450000000000001,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.573,"15min":1.631},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0190000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.409}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.6459999999999999,"15min":1.7190000000000001},"min":{"1min":1.004,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4470000000000001,"5min":3.48,"15min":3.48},"last":1.522},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.6599999999999999,"15min":1.73},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.004,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.427}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.6839999999999999,"15min":1.7330000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.1859999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.72,"15min":1.782},"min":{"1min":1.113,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.5449999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1819999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1368064,"data_stored":933279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.95699999999999996,"15min":1.022},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4370000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.87},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.089,"15min":1.1870000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2589999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.1499999999999999,"15min":1.22},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.593,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.022},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.149,"15min":1.2170000000000001},"min":{"1min":0.56299999999999994,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5680000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.115}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.056,"15min":1.103},"min":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6160000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.052},{"interface":"front","average":{"1min":1.149,"5min":1.278,"15min":1.3240000000000001},"min":{"1min":0.69799999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.78,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.2649999999999999}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5409999999999999,"15min":1.633},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.958,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.611},{"interface":"front","average":{"1min":1.302,"5min":1.401,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.768,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.3680000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.4239999999999999,"15min":1.4730000000000001},"min":{"1min":0.80700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7110000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.47},{"interface":"front","average":{"1min":1.373,"5min":1.5489999999999999,"15min":1.627},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.873,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.5820000000000001}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.593,"15min":1.6419999999999999},"min":{"1min":0.98099999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.9410000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.647,"15min":1.74},"min":{"1min":1.069,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.831,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6879999999999999,"15min":1.792},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2.0840000000000001,"5min":3.149,"15min":3.149},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5369999999999999,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9019999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.54}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:33:01.778 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:33:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.094+0000 7f2994600640 1 Processor -- start 2024-05-10T03:33:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.094+0000 7f2994600640 1 -- start start 2024-05-10T03:33:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.096+0000 7f2994600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f298c0b0c40 0x7f298c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.096+0000 7f2994600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0c7060 con 0x7f298c0b1590 2024-05-10T03:33:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.096+0000 7f2994600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0cd3b0 con 0x7f298c0cc0a0 2024-05-10T03:33:02.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.096+0000 7f2994600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c0c7300 con 0x7f298c0b0c40 2024-05-10T03:33:02.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f298c0b1590 0x7f298c0cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54112/0 (socket says 172.21.15.55:54112) 2024-05-10T03:33:02.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f298c0b0c40 0x7f298c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991200640 1 -- 172.21.15.55:0/176648353 learned_addr learned my addr 172.21.15.55:0/176648353 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:02.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f298be00640 1 -- 172.21.15.55:0/176648353 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110019767 0 0) 0x7f298c0c7060 con 0x7f298c0b1590 2024-05-10T03:33:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f298be00640 1 -- 172.21.15.55:0/176648353 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003520 con 0x7f298c0b1590 2024-05-10T03:33:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991c00640 1 -- 172.21.15.55:0/176648353 >> v1:172.21.15.162:6789/0 conn(0x7f298c0cc0a0 legacy=0x7f298c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991c00640 1 -- 172.21.15.55:0/176648353 >> v1:172.21.15.55:6789/0 conn(0x7f298c0b1590 legacy=0x7f298c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.097+0000 7f2991c00640 1 -- 172.21.15.55:0/176648353 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0c75e0 con 0x7f298c0b0c40 2024-05-10T03:33:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f2991c00640 1 --2- 172.21.15.55:0/176648353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b0c40 0x7f298c0b1050 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f29800097a0 tx=0x7f298002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c401a035816b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f298be00640 1 -- 172.21.15.55:0/176648353 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980032440 con 0x7f298c0b0c40 2024-05-10T03:33:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f298be00640 1 -- 172.21.15.55:0/176648353 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800325a0 con 0x7f298c0b0c40 2024-05-10T03:33:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f298be00640 1 -- 172.21.15.55:0/176648353 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980032910 con 0x7f298c0b0c40 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f2994600640 1 -- 172.21.15.55:0/176648353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b0c40 msgr2=0x7f298c0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.099+0000 7f2994600640 1 --2- 172.21.15.55:0/176648353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b0c40 0x7f298c0b1050 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f29800097a0 tx=0x7f298002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.100+0000 7f2994600640 1 -- 172.21.15.55:0/176648353 shutdown_connections 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.100+0000 7f2994600640 1 --2- 172.21.15.55:0/176648353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b0c40 0x7f298c0b1050 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.100+0000 7f2994600640 1 -- 172.21.15.55:0/176648353 >> 172.21.15.55:0/176648353 conn(0x7f298c0cdbe0 msgr2=0x7f298c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.100+0000 7f2994600640 1 -- 172.21.15.55:0/176648353 shutdown_connections 2024-05-10T03:33:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.100+0000 7f2994600640 1 -- 172.21.15.55:0/176648353 wait complete. 2024-05-10T03:33:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.102+0000 7f2994600640 1 Processor -- start 2024-05-10T03:33:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.102+0000 7f2994600640 1 -- start start 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f298c0b0c40 0x7f298c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f298c0cc0a0 0x7f298c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c15cae0 con 0x7f298c0b1590 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0affd0 con 0x7f298c0b0c40 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2994600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0b0220 con 0x7f298c0cc0a0 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44494/0 (socket says 172.21.15.55:44494) 2024-05-10T03:33:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 -- 172.21.15.55:0/226156773 learned_addr learned my addr 172.21.15.55:0/226156773 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991c00640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f298c0b0c40 0x7f298c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 -- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f298c0cc0a0 msgr2=0x7f298c0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f298c0cc0a0 0x7f298c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 -- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f298c0b0c40 msgr2=0x7f298c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f298c0b0c40 0x7f298c152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.104+0000 7f2991200640 1 -- 172.21.15.55:0/226156773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009400 con 0x7f298c0b1590 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2991200640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f297800c390 tx=0x7f297800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978005ce0 con 0x7f298c0b1590 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978005e40 con 0x7f298c0b1590 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c06b220 con 0x7f298c0b1590 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780072f0 con 0x7f298c0b1590 2024-05-10T03:33:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.105+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c06b7e0 con 0x7f298c0b1590 2024-05-10T03:33:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.106+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c15ba70 con 0x7f298c0b1590 2024-05-10T03:33:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.107+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2978007450 con 0x7f298c0b1590 2024-05-10T03:33:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.108+0000 7f2989600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.108+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2978088bd0 con 0x7f298c0b1590 2024-05-10T03:33:02.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.109+0000 7f2991c00640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.110+0000 7f2991c00640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2980034000 tx=0x7f298002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.114+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2978053b40 con 0x7f298c0b1590 2024-05-10T03:33:02.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.310+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f298c0ccbc0 con 0x7f296c06ac00 2024-05-10T03:33:02.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.333+0000 7f2989600640 1 -- 172.21.15.55:0/226156773 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7f29800040c0 con 0x7f296c06ac00 2024-05-10T03:33:02.341 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:02.342 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:33:02.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.340+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 msgr2=0x7f296c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.340+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2980034000 tx=0x7f298002e000 comp rx=0 tx=0).stop 2024-05-10T03:33:02.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.340+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 msgr2=0x7f298c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.340+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f297800c390 tx=0x7f297800c850 comp rx=0 tx=0).stop 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 shutdown_connections 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f298c0cc0a0 0x7f298c0b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f298c0b1590 0x7f298c0b5970 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f296c06ac00 0x7f296c06d110 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 --2- 172.21.15.55:0/226156773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f298c0b0c40 0x7f298c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 >> 172.21.15.55:0/226156773 conn(0x7f298c0cdbe0 msgr2=0x7f298c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 shutdown_connections 2024-05-10T03:33:02.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.341+0000 7f2994600640 1 -- 172.21.15.55:0/226156773 wait complete. 2024-05-10T03:33:02.372 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-05-10T03:33:01.025700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248068,"kb_used_data":7276,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754726652,"statfs":{"total":773094113280,"available":772840091648,"internally_reserved":0,"allocated":7450624,"data_stored":3953214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":246402616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002264"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.649746+0000","last_change":"2024-05-10T03:32:56.646709+0000","last_active":"2024-05-10T03:32:56.649746+0000","last_peered":"2024-05-10T03:32:56.649746+0000","last_clean":"2024-05-10T03:32:56.649746+0000","last_became_active":"2024-05-10T03:32:56.646355+0000","last_became_peered":"2024-05-10T03:32:56.646355+0000","last_unstale":"2024-05-10T03:32:56.649746+0000","last_undegraded":"2024-05-10T03:32:56.649746+0000","last_fullsized":"2024-05-10T03:32:56.649746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636382+0000","last_change":"2024-05-10T03:32:56.631374+0000","last_active":"2024-05-10T03:32:56.636382+0000","last_peered":"2024-05-10T03:32:56.636382+0000","last_clean":"2024-05-10T03:32:56.636382+0000","last_became_active":"2024-05-10T03:32:56.631050+0000","last_became_peered":"2024-05-10T03:32:56.631050+0000","last_unstale":"2024-05-10T03:32:56.636382+0000","last_undegraded":"2024-05-10T03:32:56.636382+0000","last_fullsized":"2024-05-10T03:32:56.636382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665260+0000","last_change":"2024-05-10T03:32:56.661084+0000","last_active":"2024-05-10T03:32:56.665260+0000","last_peered":"2024-05-10T03:32:56.665260+0000","last_clean":"2024-05-10T03:32:56.665260+0000","last_became_active":"2024-05-10T03:32:56.660004+0000","last_became_peered":"2024-05-10T03:32:56.660004+0000","last_unstale":"2024-05-10T03:32:56.665260+0000","last_undegraded":"2024-05-10T03:32:56.665260+0000","last_fullsized":"2024-05-10T03:32:56.665260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:22:48.257013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653660+0000","last_change":"2024-05-10T03:32:56.640314+0000","last_active":"2024-05-10T03:32:56.653660+0000","last_peered":"2024-05-10T03:32:56.653660+0000","last_clean":"2024-05-10T03:32:56.653660+0000","last_became_active":"2024-05-10T03:32:56.639782+0000","last_became_peered":"2024-05-10T03:32:56.639782+0000","last_unstale":"2024-05-10T03:32:56.653660+0000","last_undegraded":"2024-05-10T03:32:56.653660+0000","last_fullsized":"2024-05-10T03:32:56.653660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:35.393359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664945+0000","last_change":"2024-05-10T03:32:56.662090+0000","last_active":"2024-05-10T03:32:56.664945+0000","last_peered":"2024-05-10T03:32:56.664945+0000","last_clean":"2024-05-10T03:32:56.664945+0000","last_became_active":"2024-05-10T03:32:56.659903+0000","last_became_peered":"2024-05-10T03:32:56.659903+0000","last_unstale":"2024-05-10T03:32:56.664945+0000","last_undegraded":"2024-05-10T03:32:56.664945+0000","last_fullsized":"2024-05-10T03:32:56.664945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:49.229236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655345+0000","last_change":"2024-05-10T03:32:56.654302+0000","last_active":"2024-05-10T03:32:56.655345+0000","last_peered":"2024-05-10T03:32:56.655345+0000","last_clean":"2024-05-10T03:32:56.655345+0000","last_became_active":"2024-05-10T03:32:56.654025+0000","last_became_peered":"2024-05-10T03:32:56.654025+0000","last_unstale":"2024-05-10T03:32:56.655345+0000","last_undegraded":"2024-05-10T03:32:56.655345+0000","last_fullsized":"2024-05-10T03:32:56.655345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:49.068473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663477+0000","last_change":"2024-05-10T03:32:56.662031+0000","last_active":"2024-05-10T03:32:56.663477+0000","last_peered":"2024-05-10T03:32:56.663477+0000","last_clean":"2024-05-10T03:32:56.663477+0000","last_became_active":"2024-05-10T03:32:56.661718+0000","last_became_peered":"2024-05-10T03:32:56.661718+0000","last_unstale":"2024-05-10T03:32:56.663477+0000","last_undegraded":"2024-05-10T03:32:56.663477+0000","last_fullsized":"2024-05-10T03:32:56.663477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:25:26.923232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.662802+0000","last_change":"2024-05-10T03:32:56.655348+0000","last_active":"2024-05-10T03:32:56.662802+0000","last_peered":"2024-05-10T03:32:56.662802+0000","last_clean":"2024-05-10T03:32:56.662802+0000","last_became_active":"2024-05-10T03:32:56.654956+0000","last_became_peered":"2024-05-10T03:32:56.654956+0000","last_unstale":"2024-05-10T03:32:56.662802+0000","last_undegraded":"2024-05-10T03:32:56.662802+0000","last_fullsized":"2024-05-10T03:32:56.662802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:11:33.192350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671920+0000","last_change":"2024-05-10T03:32:56.669938+0000","last_active":"2024-05-10T03:32:56.671920+0000","last_peered":"2024-05-10T03:32:56.671920+0000","last_clean":"2024-05-10T03:32:56.671920+0000","last_became_active":"2024-05-10T03:32:56.669578+0000","last_became_peered":"2024-05-10T03:32:56.669578+0000","last_unstale":"2024-05-10T03:32:56.671920+0000","last_undegraded":"2024-05-10T03:32:56.671920+0000","last_fullsized":"2024-05-10T03:32:56.671920+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.666677+0000","last_change":"2024-05-10T03:32:56.663080+0000","last_active":"2024-05-10T03:32:56.666677+0000","last_peered":"2024-05-10T03:32:56.666677+0000","last_clean":"2024-05-10T03:32:56.666677+0000","last_became_active":"2024-05-10T03:32:56.661792+0000","last_became_peered":"2024-05-10T03:32:56.661792+0000","last_unstale":"2024-05-10T03:32:56.666677+0000","last_undegraded":"2024-05-10T03:32:56.666677+0000","last_fullsized":"2024-05-10T03:32:56.666677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:02:43.422052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650916+0000","last_change":"2024-05-10T03:32:56.643923+0000","last_active":"2024-05-10T03:32:56.650916+0000","last_peered":"2024-05-10T03:32:56.650916+0000","last_clean":"2024-05-10T03:32:56.650916+0000","last_became_active":"2024-05-10T03:32:56.643395+0000","last_became_peered":"2024-05-10T03:32:56.643395+0000","last_unstale":"2024-05-10T03:32:56.650916+0000","last_undegraded":"2024-05-10T03:32:56.650916+0000","last_fullsized":"2024-05-10T03:32:56.650916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663790+0000","last_change":"2024-05-10T03:32:56.659437+0000","last_active":"2024-05-10T03:32:56.663790+0000","last_peered":"2024-05-10T03:32:56.663790+0000","last_clean":"2024-05-10T03:32:56.663790+0000","last_became_active":"2024-05-10T03:32:56.658059+0000","last_became_peered":"2024-05-10T03:32:56.658059+0000","last_unstale":"2024-05-10T03:32:56.663790+0000","last_undegraded":"2024-05-10T03:32:56.663790+0000","last_fullsized":"2024-05-10T03:32:56.663790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:20.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651635+0000","last_change":"2024-05-10T03:32:56.648678+0000","last_active":"2024-05-10T03:32:56.651635+0000","last_peered":"2024-05-10T03:32:56.651635+0000","last_clean":"2024-05-10T03:32:56.651635+0000","last_became_active":"2024-05-10T03:32:56.648161+0000","last_became_peered":"2024-05-10T03:32:56.648161+0000","last_unstale":"2024-05-10T03:32:56.651635+0000","last_undegraded":"2024-05-10T03:32:56.651635+0000","last_fullsized":"2024-05-10T03:32:56.651635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:02:43.422052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.666974+0000","last_change":"2024-05-10T03:32:56.661740+0000","last_active":"2024-05-10T03:32:56.666974+0000","last_peered":"2024-05-10T03:32:56.666974+0000","last_clean":"2024-05-10T03:32:56.666974+0000","last_became_active":"2024-05-10T03:32:56.661436+0000","last_became_peered":"2024-05-10T03:32:56.661436+0000","last_unstale":"2024-05-10T03:32:56.666974+0000","last_undegraded":"2024-05-10T03:32:56.666974+0000","last_fullsized":"2024-05-10T03:32:56.666974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664786+0000","last_change":"2024-05-10T03:32:56.656818+0000","last_active":"2024-05-10T03:32:56.664786+0000","last_peered":"2024-05-10T03:32:56.664786+0000","last_clean":"2024-05-10T03:32:56.664786+0000","last_became_active":"2024-05-10T03:32:56.656524+0000","last_became_peered":"2024-05-10T03:32:56.656524+0000","last_unstale":"2024-05-10T03:32:56.664786+0000","last_undegraded":"2024-05-10T03:32:56.664786+0000","last_fullsized":"2024-05-10T03:32:56.664786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:19:53.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.644742+0000","last_change":"2024-05-10T03:32:56.641041+0000","last_active":"2024-05-10T03:32:56.644742+0000","last_peered":"2024-05-10T03:32:56.644742+0000","last_clean":"2024-05-10T03:32:56.644742+0000","last_became_active":"2024-05-10T03:32:56.640721+0000","last_became_peered":"2024-05-10T03:32:56.640721+0000","last_unstale":"2024-05-10T03:32:56.644742+0000","last_undegraded":"2024-05-10T03:32:56.644742+0000","last_fullsized":"2024-05-10T03:32:56.644742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:47.060463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656076+0000","last_change":"2024-05-10T03:32:56.655062+0000","last_active":"2024-05-10T03:32:56.656076+0000","last_peered":"2024-05-10T03:32:56.656076+0000","last_clean":"2024-05-10T03:32:56.656076+0000","last_became_active":"2024-05-10T03:32:56.654770+0000","last_became_peered":"2024-05-10T03:32:56.654770+0000","last_unstale":"2024-05-10T03:32:56.656076+0000","last_undegraded":"2024-05-10T03:32:56.656076+0000","last_fullsized":"2024-05-10T03:32:56.656076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:49.229236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653168+0000","last_change":"2024-05-10T03:32:56.644887+0000","last_active":"2024-05-10T03:32:56.653168+0000","last_peered":"2024-05-10T03:32:56.653168+0000","last_clean":"2024-05-10T03:32:56.653168+0000","last_became_active":"2024-05-10T03:32:56.644581+0000","last_became_peered":"2024-05-10T03:32:56.644581+0000","last_unstale":"2024-05-10T03:32:56.653168+0000","last_undegraded":"2024-05-10T03:32:56.653168+0000","last_fullsized":"2024-05-10T03:32:56.653168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656262+0000","last_change":"2024-05-10T03:32:56.646920+0000","last_active":"2024-05-10T03:32:56.656262+0000","last_peered":"2024-05-10T03:32:56.656262+0000","last_clean":"2024-05-10T03:32:56.656262+0000","last_became_active":"2024-05-10T03:32:56.646518+0000","last_became_peered":"2024-05-10T03:32:56.646518+0000","last_unstale":"2024-05-10T03:32:56.656262+0000","last_undegraded":"2024-05-10T03:32:56.656262+0000","last_fullsized":"2024-05-10T03:32:56.656262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:22.441460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.648870+0000","last_change":"2024-05-10T03:32:56.646411+0000","last_active":"2024-05-10T03:32:56.648870+0000","last_peered":"2024-05-10T03:32:56.648870+0000","last_clean":"2024-05-10T03:32:56.648870+0000","last_became_active":"2024-05-10T03:32:56.646060+0000","last_became_peered":"2024-05-10T03:32:56.646060+0000","last_unstale":"2024-05-10T03:32:56.648870+0000","last_undegraded":"2024-05-10T03:32:56.648870+0000","last_fullsized":"2024-05-10T03:32:56.648870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:34:53.800064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647872+0000","last_change":"2024-05-10T03:32:56.639358+0000","last_active":"2024-05-10T03:32:56.647872+0000","last_peered":"2024-05-10T03:32:56.647872+0000","last_clean":"2024-05-10T03:32:56.647872+0000","last_became_active":"2024-05-10T03:32:56.638583+0000","last_became_peered":"2024-05-10T03:32:56.638583+0000","last_unstale":"2024-05-10T03:32:56.647872+0000","last_undegraded":"2024-05-10T03:32:56.647872+0000","last_fullsized":"2024-05-10T03:32:56.647872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:07:00.061901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663836+0000","last_change":"2024-05-10T03:32:56.662898+0000","last_active":"2024-05-10T03:32:56.663836+0000","last_peered":"2024-05-10T03:32:56.663836+0000","last_clean":"2024-05-10T03:32:56.663836+0000","last_became_active":"2024-05-10T03:32:56.662625+0000","last_became_peered":"2024-05-10T03:32:56.662625+0000","last_unstale":"2024-05-10T03:32:56.663836+0000","last_undegraded":"2024-05-10T03:32:56.663836+0000","last_fullsized":"2024-05-10T03:32:56.663836+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:14:48.700520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663780+0000","last_change":"2024-05-10T03:32:56.662076+0000","last_active":"2024-05-10T03:32:56.663780+0000","last_peered":"2024-05-10T03:32:56.663780+0000","last_clean":"2024-05-10T03:32:56.663780+0000","last_became_active":"2024-05-10T03:32:56.661738+0000","last_became_peered":"2024-05-10T03:32:56.661738+0000","last_unstale":"2024-05-10T03:32:56.663780+0000","last_undegraded":"2024-05-10T03:32:56.663780+0000","last_fullsized":"2024-05-10T03:32:56.663780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:48.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664829+0000","last_change":"2024-05-10T03:32:56.655233+0000","last_active":"2024-05-10T03:32:56.664829+0000","last_peered":"2024-05-10T03:32:56.664829+0000","last_clean":"2024-05-10T03:32:56.664829+0000","last_became_active":"2024-05-10T03:32:56.654914+0000","last_became_peered":"2024-05-10T03:32:56.654914+0000","last_unstale":"2024-05-10T03:32:56.664829+0000","last_undegraded":"2024-05-10T03:32:56.664829+0000","last_fullsized":"2024-05-10T03:32:56.664829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:03.475802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.649415+0000","last_change":"2024-05-10T03:32:56.638635+0000","last_active":"2024-05-10T03:32:56.649415+0000","last_peered":"2024-05-10T03:32:56.649415+0000","last_clean":"2024-05-10T03:32:56.649415+0000","last_became_active":"2024-05-10T03:32:56.638203+0000","last_became_peered":"2024-05-10T03:32:56.638203+0000","last_unstale":"2024-05-10T03:32:56.649415+0000","last_undegraded":"2024-05-10T03:32:56.649415+0000","last_fullsized":"2024-05-10T03:32:56.649415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:25.353955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.648511+0000","last_change":"2024-05-10T03:32:56.641752+0000","last_active":"2024-05-10T03:32:56.648511+0000","last_peered":"2024-05-10T03:32:56.648511+0000","last_clean":"2024-05-10T03:32:56.648511+0000","last_became_active":"2024-05-10T03:32:56.641055+0000","last_became_peered":"2024-05-10T03:32:56.641055+0000","last_unstale":"2024-05-10T03:32:56.648511+0000","last_undegraded":"2024-05-10T03:32:56.648511+0000","last_fullsized":"2024-05-10T03:32:56.648511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:01:33.945466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650889+0000","last_change":"2024-05-10T03:32:56.643720+0000","last_active":"2024-05-10T03:32:56.650889+0000","last_peered":"2024-05-10T03:32:56.650889+0000","last_clean":"2024-05-10T03:32:56.650889+0000","last_became_active":"2024-05-10T03:32:56.642824+0000","last_became_peered":"2024-05-10T03:32:56.642824+0000","last_unstale":"2024-05-10T03:32:56.650889+0000","last_undegraded":"2024-05-10T03:32:56.650889+0000","last_fullsized":"2024-05-10T03:32:56.650889+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:35.712429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658076+0000","last_change":"2024-05-10T03:32:56.656891+0000","last_active":"2024-05-10T03:32:56.658076+0000","last_peered":"2024-05-10T03:32:56.658076+0000","last_clean":"2024-05-10T03:32:56.658076+0000","last_became_active":"2024-05-10T03:32:56.656575+0000","last_became_peered":"2024-05-10T03:32:56.656575+0000","last_unstale":"2024-05-10T03:32:56.658076+0000","last_undegraded":"2024-05-10T03:32:56.658076+0000","last_fullsized":"2024-05-10T03:32:56.658076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671434+0000","last_change":"2024-05-10T03:32:56.666376+0000","last_active":"2024-05-10T03:32:56.671434+0000","last_peered":"2024-05-10T03:32:56.671434+0000","last_clean":"2024-05-10T03:32:56.671434+0000","last_became_active":"2024-05-10T03:32:56.665995+0000","last_became_peered":"2024-05-10T03:32:56.665995+0000","last_unstale":"2024-05-10T03:32:56.671434+0000","last_undegraded":"2024-05-10T03:32:56.671434+0000","last_fullsized":"2024-05-10T03:32:56.671434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651590+0000","last_change":"2024-05-10T03:32:56.648373+0000","last_active":"2024-05-10T03:32:56.651590+0000","last_peered":"2024-05-10T03:32:56.651590+0000","last_clean":"2024-05-10T03:32:56.651590+0000","last_became_active":"2024-05-10T03:32:56.647975+0000","last_became_peered":"2024-05-10T03:32:56.647975+0000","last_unstale":"2024-05-10T03:32:56.651590+0000","last_undegraded":"2024-05-10T03:32:56.651590+0000","last_fullsized":"2024-05-10T03:32:56.651590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:15:38.802682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650435+0000","last_change":"2024-05-10T03:32:56.638669+0000","last_active":"2024-05-10T03:32:56.650435+0000","last_peered":"2024-05-10T03:32:56.650435+0000","last_clean":"2024-05-10T03:32:56.650435+0000","last_became_active":"2024-05-10T03:32:56.638288+0000","last_became_peered":"2024-05-10T03:32:56.638288+0000","last_unstale":"2024-05-10T03:32:56.650435+0000","last_undegraded":"2024-05-10T03:32:56.650435+0000","last_fullsized":"2024-05-10T03:32:56.650435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:48:28.869006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.635374+0000","last_change":"2024-05-10T03:32:56.632738+0000","last_active":"2024-05-10T03:32:56.635374+0000","last_peered":"2024-05-10T03:32:56.635374+0000","last_clean":"2024-05-10T03:32:56.635374+0000","last_became_active":"2024-05-10T03:32:56.632386+0000","last_became_peered":"2024-05-10T03:32:56.632386+0000","last_unstale":"2024-05-10T03:32:56.635374+0000","last_undegraded":"2024-05-10T03:32:56.635374+0000","last_fullsized":"2024-05-10T03:32:56.635374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:20.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655994+0000","last_change":"2024-05-10T03:32:56.647475+0000","last_active":"2024-05-10T03:32:56.655994+0000","last_peered":"2024-05-10T03:32:56.655994+0000","last_clean":"2024-05-10T03:32:56.655994+0000","last_became_active":"2024-05-10T03:32:56.647060+0000","last_became_peered":"2024-05-10T03:32:56.647060+0000","last_unstale":"2024-05-10T03:32:56.655994+0000","last_undegraded":"2024-05-10T03:32:56.655994+0000","last_fullsized":"2024-05-10T03:32:56.655994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.638555+0000","last_change":"2024-05-10T03:32:56.064165+0000","last_active":"2024-05-10T03:32:56.638555+0000","last_peered":"2024-05-10T03:32:56.638555+0000","last_clean":"2024-05-10T03:32:56.638555+0000","last_became_active":"2024-05-10T03:32:56.063684+0000","last_became_peered":"2024-05-10T03:32:56.063684+0000","last_unstale":"2024-05-10T03:32:56.638555+0000","last_undegraded":"2024-05-10T03:32:56.638555+0000","last_fullsized":"2024-05-10T03:32:56.638555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:40.673453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658296+0000","last_change":"2024-05-10T03:32:56.651496+0000","last_active":"2024-05-10T03:32:56.658296+0000","last_peered":"2024-05-10T03:32:56.658296+0000","last_clean":"2024-05-10T03:32:56.658296+0000","last_became_active":"2024-05-10T03:32:56.651166+0000","last_became_peered":"2024-05-10T03:32:56.651166+0000","last_unstale":"2024-05-10T03:32:56.658296+0000","last_undegraded":"2024-05-10T03:32:56.658296+0000","last_fullsized":"2024-05-10T03:32:56.658296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:55.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636284+0000","last_change":"2024-05-10T03:32:56.058681+0000","last_active":"2024-05-10T03:32:56.636284+0000","last_peered":"2024-05-10T03:32:56.636284+0000","last_clean":"2024-05-10T03:32:56.636284+0000","last_became_active":"2024-05-10T03:32:56.058341+0000","last_became_peered":"2024-05-10T03:32:56.058341+0000","last_unstale":"2024-05-10T03:32:56.636284+0000","last_undegraded":"2024-05-10T03:32:56.636284+0000","last_fullsized":"2024-05-10T03:32:56.636284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:22:47.236001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.639812+0000","last_change":"2024-05-10T03:32:56.637636+0000","last_active":"2024-05-10T03:32:56.639812+0000","last_peered":"2024-05-10T03:32:56.639812+0000","last_clean":"2024-05-10T03:32:56.639812+0000","last_became_active":"2024-05-10T03:32:56.636267+0000","last_became_peered":"2024-05-10T03:32:56.636267+0000","last_unstale":"2024-05-10T03:32:56.639812+0000","last_undegraded":"2024-05-10T03:32:56.639812+0000","last_fullsized":"2024-05-10T03:32:56.639812+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:07.669782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.640484+0000","last_change":"2024-05-10T03:32:56.056232+0000","last_active":"2024-05-10T03:32:56.640484+0000","last_peered":"2024-05-10T03:32:56.640484+0000","last_clean":"2024-05-10T03:32:56.640484+0000","last_became_active":"2024-05-10T03:32:56.055911+0000","last_became_peered":"2024-05-10T03:32:56.055911+0000","last_unstale":"2024-05-10T03:32:56.640484+0000","last_undegraded":"2024-05-10T03:32:56.640484+0000","last_fullsized":"2024-05-10T03:32:56.640484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:02.454790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.668963+0000","last_change":"2024-05-10T03:32:56.665949+0000","last_active":"2024-05-10T03:32:56.668963+0000","last_peered":"2024-05-10T03:32:56.668963+0000","last_clean":"2024-05-10T03:32:56.668963+0000","last_became_active":"2024-05-10T03:32:56.665576+0000","last_became_peered":"2024-05-10T03:32:56.665576+0000","last_unstale":"2024-05-10T03:32:56.668963+0000","last_undegraded":"2024-05-10T03:32:56.668963+0000","last_fullsized":"2024-05-10T03:32:56.668963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:25.353955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634502+0000","last_change":"2024-05-10T03:32:56.065654+0000","last_active":"2024-05-10T03:32:56.634502+0000","last_peered":"2024-05-10T03:32:56.634502+0000","last_clean":"2024-05-10T03:32:56.634502+0000","last_became_active":"2024-05-10T03:32:56.064205+0000","last_became_peered":"2024-05-10T03:32:56.064205+0000","last_unstale":"2024-05-10T03:32:56.634502+0000","last_undegraded":"2024-05-10T03:32:56.634502+0000","last_fullsized":"2024-05-10T03:32:56.634502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:38.601647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.637728+0000","last_change":"2024-05-10T03:32:56.635413+0000","last_active":"2024-05-10T03:32:56.637728+0000","last_peered":"2024-05-10T03:32:56.637728+0000","last_clean":"2024-05-10T03:32:56.637728+0000","last_became_active":"2024-05-10T03:32:56.634954+0000","last_became_peered":"2024-05-10T03:32:56.634954+0000","last_unstale":"2024-05-10T03:32:56.637728+0000","last_undegraded":"2024-05-10T03:32:56.637728+0000","last_fullsized":"2024-05-10T03:32:56.637728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:04.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.630214+0000","last_change":"2024-05-10T03:32:56.054732+0000","last_active":"2024-05-10T03:32:56.630214+0000","last_peered":"2024-05-10T03:32:56.630214+0000","last_clean":"2024-05-10T03:32:56.630214+0000","last_became_active":"2024-05-10T03:32:56.054391+0000","last_became_peered":"2024-05-10T03:32:56.054391+0000","last_unstale":"2024-05-10T03:32:56.630214+0000","last_undegraded":"2024-05-10T03:32:56.630214+0000","last_fullsized":"2024-05-10T03:32:56.630214+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:35:39.520715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.646213+0000","last_change":"2024-05-10T03:32:56.644484+0000","last_active":"2024-05-10T03:32:56.646213+0000","last_peered":"2024-05-10T03:32:56.646213+0000","last_clean":"2024-05-10T03:32:56.646213+0000","last_became_active":"2024-05-10T03:32:56.643771+0000","last_became_peered":"2024-05-10T03:32:56.643771+0000","last_unstale":"2024-05-10T03:32:56.646213+0000","last_undegraded":"2024-05-10T03:32:56.646213+0000","last_fullsized":"2024-05-10T03:32:56.646213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634064+0000","last_change":"2024-05-10T03:32:56.054043+0000","last_active":"2024-05-10T03:32:56.634064+0000","last_peered":"2024-05-10T03:32:56.634064+0000","last_clean":"2024-05-10T03:32:56.634064+0000","last_became_active":"2024-05-10T03:32:56.051572+0000","last_became_peered":"2024-05-10T03:32:56.051572+0000","last_unstale":"2024-05-10T03:32:56.634064+0000","last_undegraded":"2024-05-10T03:32:56.634064+0000","last_fullsized":"2024-05-10T03:32:56.634064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:17.868723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656843+0000","last_change":"2024-05-10T03:32:56.645639+0000","last_active":"2024-05-10T03:32:56.656843+0000","last_peered":"2024-05-10T03:32:56.656843+0000","last_clean":"2024-05-10T03:32:56.656843+0000","last_became_active":"2024-05-10T03:32:56.645308+0000","last_became_peered":"2024-05-10T03:32:56.645308+0000","last_unstale":"2024-05-10T03:32:56.656843+0000","last_undegraded":"2024-05-10T03:32:56.656843+0000","last_fullsized":"2024-05-10T03:32:56.656843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:35.393359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626644+0000","last_change":"2024-05-10T03:32:56.059410+0000","last_active":"2024-05-10T03:32:56.626644+0000","last_peered":"2024-05-10T03:32:56.626644+0000","last_clean":"2024-05-10T03:32:56.626644+0000","last_became_active":"2024-05-10T03:32:56.059088+0000","last_became_peered":"2024-05-10T03:32:56.059088+0000","last_unstale":"2024-05-10T03:32:56.626644+0000","last_undegraded":"2024-05-10T03:32:56.626644+0000","last_fullsized":"2024-05-10T03:32:56.626644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:06.648770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.645222+0000","last_change":"2024-05-10T03:32:56.643809+0000","last_active":"2024-05-10T03:32:56.645222+0000","last_peered":"2024-05-10T03:32:56.645222+0000","last_clean":"2024-05-10T03:32:56.645222+0000","last_became_active":"2024-05-10T03:32:56.643444+0000","last_became_peered":"2024-05-10T03:32:56.643444+0000","last_unstale":"2024-05-10T03:32:56.645222+0000","last_undegraded":"2024-05-10T03:32:56.645222+0000","last_fullsized":"2024-05-10T03:32:56.645222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:05.641768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.633459+0000","last_change":"2024-05-10T03:32:56.054544+0000","last_active":"2024-05-10T03:32:56.633459+0000","last_peered":"2024-05-10T03:32:56.633459+0000","last_clean":"2024-05-10T03:32:56.633459+0000","last_became_active":"2024-05-10T03:32:56.052424+0000","last_became_peered":"2024-05-10T03:32:56.052424+0000","last_unstale":"2024-05-10T03:32:56.633459+0000","last_undegraded":"2024-05-10T03:32:56.633459+0000","last_fullsized":"2024-05-10T03:32:56.633459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:27:16.854201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.636613+0000","last_change":"2024-05-10T03:32:56.632190+0000","last_active":"2024-05-10T03:32:56.636613+0000","last_peered":"2024-05-10T03:32:56.636613+0000","last_clean":"2024-05-10T03:32:56.636613+0000","last_became_active":"2024-05-10T03:32:56.631782+0000","last_became_peered":"2024-05-10T03:32:56.631782+0000","last_unstale":"2024-05-10T03:32:56.636613+0000","last_undegraded":"2024-05-10T03:32:56.636613+0000","last_fullsized":"2024-05-10T03:32:56.636613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.660580+0000","last_change":"2024-05-10T03:32:56.065803+0000","last_active":"2024-05-10T03:32:56.660580+0000","last_peered":"2024-05-10T03:32:56.660580+0000","last_clean":"2024-05-10T03:32:56.660580+0000","last_became_active":"2024-05-10T03:32:56.065252+0000","last_became_peered":"2024-05-10T03:32:56.065252+0000","last_unstale":"2024-05-10T03:32:56.660580+0000","last_undegraded":"2024-05-10T03:32:56.660580+0000","last_fullsized":"2024-05-10T03:32:56.660580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:47:45.628490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.639860+0000","last_change":"2024-05-10T03:32:56.638389+0000","last_active":"2024-05-10T03:32:56.639860+0000","last_peered":"2024-05-10T03:32:56.639860+0000","last_clean":"2024-05-10T03:32:56.639860+0000","last_became_active":"2024-05-10T03:32:56.638083+0000","last_became_peered":"2024-05-10T03:32:56.638083+0000","last_unstale":"2024-05-10T03:32:56.639860+0000","last_undegraded":"2024-05-10T03:32:56.639860+0000","last_fullsized":"2024-05-10T03:32:56.639860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:33:56.340378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650319+0000","last_change":"2024-05-10T03:32:56.061445+0000","last_active":"2024-05-10T03:32:56.650319+0000","last_peered":"2024-05-10T03:32:56.650319+0000","last_clean":"2024-05-10T03:32:56.650319+0000","last_became_active":"2024-05-10T03:32:56.060999+0000","last_became_peered":"2024-05-10T03:32:56.060999+0000","last_unstale":"2024-05-10T03:32:56.650319+0000","last_undegraded":"2024-05-10T03:32:56.650319+0000","last_fullsized":"2024-05-10T03:32:56.650319+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:51:24.605065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.657693+0000","last_change":"2024-05-10T03:32:56.650427+0000","last_active":"2024-05-10T03:32:56.657693+0000","last_peered":"2024-05-10T03:32:56.657693+0000","last_clean":"2024-05-10T03:32:56.657693+0000","last_became_active":"2024-05-10T03:32:56.649323+0000","last_became_peered":"2024-05-10T03:32:56.649323+0000","last_unstale":"2024-05-10T03:32:56.657693+0000","last_undegraded":"2024-05-10T03:32:56.657693+0000","last_fullsized":"2024-05-10T03:32:56.657693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:10.209378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665205+0000","last_change":"2024-05-10T03:32:56.065871+0000","last_active":"2024-05-10T03:32:56.665205+0000","last_peered":"2024-05-10T03:32:56.665205+0000","last_clean":"2024-05-10T03:32:56.665205+0000","last_became_active":"2024-05-10T03:32:56.065242+0000","last_became_peered":"2024-05-10T03:32:56.065242+0000","last_unstale":"2024-05-10T03:32:56.665205+0000","last_undegraded":"2024-05-10T03:32:56.665205+0000","last_fullsized":"2024-05-10T03:32:56.665205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:40.673453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653066+0000","last_change":"2024-05-10T03:32:56.644734+0000","last_active":"2024-05-10T03:32:56.653066+0000","last_peered":"2024-05-10T03:32:56.653066+0000","last_clean":"2024-05-10T03:32:56.653066+0000","last_became_active":"2024-05-10T03:32:56.644346+0000","last_became_peered":"2024-05-10T03:32:56.644346+0000","last_unstale":"2024-05-10T03:32:56.653066+0000","last_undegraded":"2024-05-10T03:32:56.653066+0000","last_fullsized":"2024-05-10T03:32:56.653066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:44.777127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667525+0000","last_change":"2024-05-10T03:32:56.055660+0000","last_active":"2024-05-10T03:32:56.667525+0000","last_peered":"2024-05-10T03:32:56.667525+0000","last_clean":"2024-05-10T03:32:56.667525+0000","last_became_active":"2024-05-10T03:32:56.055271+0000","last_became_peered":"2024-05-10T03:32:56.055271+0000","last_unstale":"2024-05-10T03:32:56.667525+0000","last_undegraded":"2024-05-10T03:32:56.667525+0000","last_fullsized":"2024-05-10T03:32:56.667525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:47:45.628490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664233+0000","last_change":"2024-05-10T03:32:56.658899+0000","last_active":"2024-05-10T03:32:56.664233+0000","last_peered":"2024-05-10T03:32:56.664233+0000","last_clean":"2024-05-10T03:32:56.664233+0000","last_became_active":"2024-05-10T03:32:56.656381+0000","last_became_peered":"2024-05-10T03:32:56.656381+0000","last_unstale":"2024-05-10T03:32:56.664233+0000","last_undegraded":"2024-05-10T03:32:56.664233+0000","last_fullsized":"2024-05-10T03:32:56.664233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:16.750300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664834+0000","last_change":"2024-05-10T03:32:56.058583+0000","last_active":"2024-05-10T03:32:56.664834+0000","last_peered":"2024-05-10T03:32:56.664834+0000","last_clean":"2024-05-10T03:32:56.664834+0000","last_became_active":"2024-05-10T03:32:56.058260+0000","last_became_peered":"2024-05-10T03:32:56.058260+0000","last_unstale":"2024-05-10T03:32:56.664834+0000","last_undegraded":"2024-05-10T03:32:56.664834+0000","last_fullsized":"2024-05-10T03:32:56.664834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:43.756115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647412+0000","last_change":"2024-05-10T03:32:56.639319+0000","last_active":"2024-05-10T03:32:56.647412+0000","last_peered":"2024-05-10T03:32:56.647412+0000","last_clean":"2024-05-10T03:32:56.647412+0000","last_became_active":"2024-05-10T03:32:56.638858+0000","last_became_peered":"2024-05-10T03:32:56.638858+0000","last_unstale":"2024-05-10T03:32:56.647412+0000","last_undegraded":"2024-05-10T03:32:56.647412+0000","last_fullsized":"2024-05-10T03:32:56.647412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:29.455124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.617703+0000","last_change":"2024-05-10T03:32:56.062821+0000","last_active":"2024-05-10T03:32:56.617703+0000","last_peered":"2024-05-10T03:32:56.617703+0000","last_clean":"2024-05-10T03:32:56.617703+0000","last_became_active":"2024-05-10T03:32:56.062345+0000","last_became_peered":"2024-05-10T03:32:56.062345+0000","last_unstale":"2024-05-10T03:32:56.617703+0000","last_undegraded":"2024-05-10T03:32:56.617703+0000","last_fullsized":"2024-05-10T03:32:56.617703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:24.419859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656821+0000","last_change":"2024-05-10T03:32:56.644966+0000","last_active":"2024-05-10T03:32:56.656821+0000","last_peered":"2024-05-10T03:32:56.656821+0000","last_clean":"2024-05-10T03:32:56.656821+0000","last_became_active":"2024-05-10T03:32:56.643950+0000","last_became_peered":"2024-05-10T03:32:56.643950+0000","last_unstale":"2024-05-10T03:32:56.656821+0000","last_undegraded":"2024-05-10T03:32:56.656821+0000","last_fullsized":"2024-05-10T03:32:56.656821+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:44:55.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.650664+0000","last_change":"2024-05-10T03:32:56.063611+0000","last_active":"2024-05-10T03:32:56.650664+0000","last_peered":"2024-05-10T03:32:56.650664+0000","last_clean":"2024-05-10T03:32:56.650664+0000","last_became_active":"2024-05-10T03:32:56.063210+0000","last_became_peered":"2024-05-10T03:32:56.063210+0000","last_unstale":"2024-05-10T03:32:56.650664+0000","last_undegraded":"2024-05-10T03:32:56.650664+0000","last_fullsized":"2024-05-10T03:32:56.650664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:14:15.148949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.663448+0000","last_change":"2024-05-10T03:32:56.660894+0000","last_active":"2024-05-10T03:32:56.663448+0000","last_peered":"2024-05-10T03:32:56.663448+0000","last_clean":"2024-05-10T03:32:56.663448+0000","last_became_active":"2024-05-10T03:32:56.660580+0000","last_became_peered":"2024-05-10T03:32:56.660580+0000","last_unstale":"2024-05-10T03:32:56.663448+0000","last_undegraded":"2024-05-10T03:32:56.663448+0000","last_fullsized":"2024-05-10T03:32:56.663448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667381+0000","last_change":"2024-05-10T03:32:56.052809+0000","last_active":"2024-05-10T03:32:56.667381+0000","last_peered":"2024-05-10T03:32:56.667381+0000","last_clean":"2024-05-10T03:32:56.667381+0000","last_became_active":"2024-05-10T03:32:56.052454+0000","last_became_peered":"2024-05-10T03:32:56.052454+0000","last_unstale":"2024-05-10T03:32:56.667381+0000","last_undegraded":"2024-05-10T03:32:56.667381+0000","last_fullsized":"2024-05-10T03:32:56.667381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:12:50.141087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.671453+0000","last_change":"2024-05-10T03:32:56.666617+0000","last_active":"2024-05-10T03:32:56.671453+0000","last_peered":"2024-05-10T03:32:56.671453+0000","last_clean":"2024-05-10T03:32:56.671453+0000","last_became_active":"2024-05-10T03:32:56.665890+0000","last_became_peered":"2024-05-10T03:32:56.665890+0000","last_unstale":"2024-05-10T03:32:56.671453+0000","last_undegraded":"2024-05-10T03:32:56.671453+0000","last_fullsized":"2024-05-10T03:32:56.671453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:01:24.236015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653999+0000","last_change":"2024-05-10T03:32:56.063651+0000","last_active":"2024-05-10T03:32:56.653999+0000","last_peered":"2024-05-10T03:32:56.653999+0000","last_clean":"2024-05-10T03:32:56.653999+0000","last_became_active":"2024-05-10T03:32:56.063292+0000","last_became_peered":"2024-05-10T03:32:56.063292+0000","last_unstale":"2024-05-10T03:32:56.653999+0000","last_undegraded":"2024-05-10T03:32:56.653999+0000","last_fullsized":"2024-05-10T03:32:56.653999+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:34.372347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.664962+0000","last_change":"2024-05-10T03:32:56.660043+0000","last_active":"2024-05-10T03:32:56.664962+0000","last_peered":"2024-05-10T03:32:56.664962+0000","last_clean":"2024-05-10T03:32:56.664962+0000","last_became_active":"2024-05-10T03:32:56.659720+0000","last_became_peered":"2024-05-10T03:32:56.659720+0000","last_unstale":"2024-05-10T03:32:56.664962+0000","last_undegraded":"2024-05-10T03:32:56.664962+0000","last_fullsized":"2024-05-10T03:32:56.664962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:44.288664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.667593+0000","last_change":"2024-05-10T03:32:56.060479+0000","last_active":"2024-05-10T03:32:56.667593+0000","last_peered":"2024-05-10T03:32:56.667593+0000","last_clean":"2024-05-10T03:32:56.667593+0000","last_became_active":"2024-05-10T03:32:56.058343+0000","last_became_peered":"2024-05-10T03:32:56.058343+0000","last_unstale":"2024-05-10T03:32:56.667593+0000","last_undegraded":"2024-05-10T03:32:56.667593+0000","last_fullsized":"2024-05-10T03:32:56.667593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:15:08.424775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655945+0000","last_change":"2024-05-10T03:32:56.651151+0000","last_active":"2024-05-10T03:32:56.655945+0000","last_peered":"2024-05-10T03:32:56.655945+0000","last_clean":"2024-05-10T03:32:56.655945+0000","last_became_active":"2024-05-10T03:32:56.650633+0000","last_became_peered":"2024-05-10T03:32:56.650633+0000","last_unstale":"2024-05-10T03:32:56.655945+0000","last_undegraded":"2024-05-10T03:32:56.655945+0000","last_fullsized":"2024-05-10T03:32:56.655945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:44.288664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.665314+0000","last_change":"2024-05-10T03:32:56.065790+0000","last_active":"2024-05-10T03:32:56.665314+0000","last_peered":"2024-05-10T03:32:56.665314+0000","last_clean":"2024-05-10T03:32:56.665314+0000","last_became_active":"2024-05-10T03:32:56.065192+0000","last_became_peered":"2024-05-10T03:32:56.065192+0000","last_unstale":"2024-05-10T03:32:56.665314+0000","last_undegraded":"2024-05-10T03:32:56.665314+0000","last_fullsized":"2024-05-10T03:32:56.665314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656730+0000","last_change":"2024-05-10T03:32:56.646506+0000","last_active":"2024-05-10T03:32:56.656730+0000","last_peered":"2024-05-10T03:32:56.656730+0000","last_clean":"2024-05-10T03:32:56.656730+0000","last_became_active":"2024-05-10T03:32:56.646171+0000","last_became_peered":"2024-05-10T03:32:56.646171+0000","last_unstale":"2024-05-10T03:32:56.656730+0000","last_undegraded":"2024-05-10T03:32:56.656730+0000","last_fullsized":"2024-05-10T03:32:56.656730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:39.622659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626874+0000","last_change":"2024-05-10T03:32:56.049848+0000","last_active":"2024-05-10T03:32:56.626874+0000","last_peered":"2024-05-10T03:32:56.626874+0000","last_clean":"2024-05-10T03:32:56.626874+0000","last_became_active":"2024-05-10T03:32:56.049439+0000","last_became_peered":"2024-05-10T03:32:56.049439+0000","last_unstale":"2024-05-10T03:32:56.626874+0000","last_undegraded":"2024-05-10T03:32:56.626874+0000","last_fullsized":"2024-05-10T03:32:56.626874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:05:26.132081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.634745+0000","last_change":"2024-05-10T03:32:56.631763+0000","last_active":"2024-05-10T03:32:56.634745+0000","last_peered":"2024-05-10T03:32:56.634745+0000","last_clean":"2024-05-10T03:32:56.634745+0000","last_became_active":"2024-05-10T03:32:56.631455+0000","last_became_peered":"2024-05-10T03:32:56.631455+0000","last_unstale":"2024-05-10T03:32:56.634745+0000","last_undegraded":"2024-05-10T03:32:56.634745+0000","last_fullsized":"2024-05-10T03:32:56.634745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.889735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.638287+0000","last_change":"2024-05-10T03:32:56.058835+0000","last_active":"2024-05-10T03:32:56.638287+0000","last_peered":"2024-05-10T03:32:56.638287+0000","last_clean":"2024-05-10T03:32:56.638287+0000","last_became_active":"2024-05-10T03:32:56.058238+0000","last_became_peered":"2024-05-10T03:32:56.058238+0000","last_unstale":"2024-05-10T03:32:56.638287+0000","last_undegraded":"2024-05-10T03:32:56.638287+0000","last_fullsized":"2024-05-10T03:32:56.638287+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:43.134365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.647856+0000","last_change":"2024-05-10T03:32:56.640892+0000","last_active":"2024-05-10T03:32:56.647856+0000","last_peered":"2024-05-10T03:32:56.647856+0000","last_clean":"2024-05-10T03:32:56.647856+0000","last_became_active":"2024-05-10T03:32:56.640546+0000","last_became_peered":"2024-05-10T03:32:56.640546+0000","last_unstale":"2024-05-10T03:32:56.647856+0000","last_undegraded":"2024-05-10T03:32:56.647856+0000","last_fullsized":"2024-05-10T03:32:56.647856+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:19:53.865032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.617998+0000","last_change":"2024-05-10T03:32:56.061608+0000","last_active":"2024-05-10T03:32:56.617998+0000","last_peered":"2024-05-10T03:32:56.617998+0000","last_clean":"2024-05-10T03:32:56.617998+0000","last_became_active":"2024-05-10T03:32:56.061201+0000","last_became_peered":"2024-05-10T03:32:56.061201+0000","last_unstale":"2024-05-10T03:32:56.617998+0000","last_undegraded":"2024-05-10T03:32:56.617998+0000","last_fullsized":"2024-05-10T03:32:56.617998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:44:09.188366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.653385+0000","last_change":"2024-05-10T03:32:56.642021+0000","last_active":"2024-05-10T03:32:56.653385+0000","last_peered":"2024-05-10T03:32:56.653385+0000","last_clean":"2024-05-10T03:32:56.653385+0000","last_became_active":"2024-05-10T03:32:56.641650+0000","last_became_peered":"2024-05-10T03:32:56.641650+0000","last_unstale":"2024-05-10T03:32:56.653385+0000","last_undegraded":"2024-05-10T03:32:56.653385+0000","last_fullsized":"2024-05-10T03:32:56.653385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:16.750300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.613287+0000","last_change":"2024-05-10T03:32:56.059212+0000","last_active":"2024-05-10T03:32:56.613287+0000","last_peered":"2024-05-10T03:32:56.613287+0000","last_clean":"2024-05-10T03:32:56.613287+0000","last_became_active":"2024-05-10T03:32:56.058916+0000","last_became_peered":"2024-05-10T03:32:56.058916+0000","last_unstale":"2024-05-10T03:32:56.613287+0000","last_undegraded":"2024-05-10T03:32:56.613287+0000","last_fullsized":"2024-05-10T03:32:56.613287+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:02.479608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655367+0000","last_change":"2024-05-10T03:32:56.654543+0000","last_active":"2024-05-10T03:32:56.655367+0000","last_peered":"2024-05-10T03:32:56.655367+0000","last_clean":"2024-05-10T03:32:56.655367+0000","last_became_active":"2024-05-10T03:32:56.654286+0000","last_became_peered":"2024-05-10T03:32:56.654286+0000","last_unstale":"2024-05-10T03:32:56.655367+0000","last_undegraded":"2024-05-10T03:32:56.655367+0000","last_fullsized":"2024-05-10T03:32:56.655367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:30:29.455124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.662274+0000","last_change":"2024-05-10T03:32:56.056422+0000","last_active":"2024-05-10T03:32:56.662274+0000","last_peered":"2024-05-10T03:32:56.662274+0000","last_clean":"2024-05-10T03:32:56.662274+0000","last_became_active":"2024-05-10T03:32:56.056073+0000","last_became_peered":"2024-05-10T03:32:56.056073+0000","last_unstale":"2024-05-10T03:32:56.662274+0000","last_undegraded":"2024-05-10T03:32:56.662274+0000","last_fullsized":"2024-05-10T03:32:56.662274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:10.624076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.633654+0000","last_change":"2024-05-10T03:32:56.628240+0000","last_active":"2024-05-10T03:32:56.633654+0000","last_peered":"2024-05-10T03:32:56.633654+0000","last_clean":"2024-05-10T03:32:56.633654+0000","last_became_active":"2024-05-10T03:32:56.627944+0000","last_became_peered":"2024-05-10T03:32:56.627944+0000","last_unstale":"2024-05-10T03:32:56.633654+0000","last_undegraded":"2024-05-10T03:32:56.633654+0000","last_fullsized":"2024-05-10T03:32:56.633654+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:54.022090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.621724+0000","last_change":"2024-05-10T03:32:56.051891+0000","last_active":"2024-05-10T03:32:56.621724+0000","last_peered":"2024-05-10T03:32:56.621724+0000","last_clean":"2024-05-10T03:32:56.621724+0000","last_became_active":"2024-05-10T03:32:56.051570+0000","last_became_peered":"2024-05-10T03:32:56.051570+0000","last_unstale":"2024-05-10T03:32:56.621724+0000","last_undegraded":"2024-05-10T03:32:56.621724+0000","last_fullsized":"2024-05-10T03:32:56.621724+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:03:15.729288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.651477+0000","last_change":"2024-05-10T03:32:56.641971+0000","last_active":"2024-05-10T03:32:56.651477+0000","last_peered":"2024-05-10T03:32:56.651477+0000","last_clean":"2024-05-10T03:32:56.651477+0000","last_became_active":"2024-05-10T03:32:56.641422+0000","last_became_peered":"2024-05-10T03:32:56.641422+0000","last_unstale":"2024-05-10T03:32:56.651477+0000","last_undegraded":"2024-05-10T03:32:56.651477+0000","last_fullsized":"2024-05-10T03:32:56.651477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:18.889735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.613586+0000","last_change":"2024-05-10T03:32:56.054270+0000","last_active":"2024-05-10T03:32:56.613586+0000","last_peered":"2024-05-10T03:32:56.613586+0000","last_clean":"2024-05-10T03:32:56.613586+0000","last_became_active":"2024-05-10T03:32:56.053940+0000","last_became_peered":"2024-05-10T03:32:56.053940+0000","last_unstale":"2024-05-10T03:32:56.613586+0000","last_undegraded":"2024-05-10T03:32:56.613586+0000","last_fullsized":"2024-05-10T03:32:56.613586+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:01:23.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.656707+0000","last_change":"2024-05-10T03:32:56.646070+0000","last_active":"2024-05-10T03:32:56.656707+0000","last_peered":"2024-05-10T03:32:56.656707+0000","last_clean":"2024-05-10T03:32:56.656707+0000","last_became_active":"2024-05-10T03:32:56.645738+0000","last_became_peered":"2024-05-10T03:32:56.645738+0000","last_unstale":"2024-05-10T03:32:56.656707+0000","last_undegraded":"2024-05-10T03:32:56.656707+0000","last_fullsized":"2024-05-10T03:32:56.656707+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:03.500620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.624607+0000","last_change":"2024-05-10T03:32:56.053264+0000","last_active":"2024-05-10T03:32:56.624607+0000","last_peered":"2024-05-10T03:32:56.624607+0000","last_clean":"2024-05-10T03:32:56.624607+0000","last_became_active":"2024-05-10T03:32:56.052057+0000","last_became_peered":"2024-05-10T03:32:56.052057+0000","last_unstale":"2024-05-10T03:32:56.624607+0000","last_undegraded":"2024-05-10T03:32:56.624607+0000","last_fullsized":"2024-05-10T03:32:56.624607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:38:21.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.655949+0000","last_change":"2024-05-10T03:32:56.645370+0000","last_active":"2024-05-10T03:32:56.655949+0000","last_peered":"2024-05-10T03:32:56.655949+0000","last_clean":"2024-05-10T03:32:56.655949+0000","last_became_active":"2024-05-10T03:32:56.645019+0000","last_became_peered":"2024-05-10T03:32:56.645019+0000","last_unstale":"2024-05-10T03:32:56.655949+0000","last_undegraded":"2024-05-10T03:32:56.655949+0000","last_fullsized":"2024-05-10T03:32:56.655949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:25.440872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.652618+0000","last_change":"2024-05-10T03:32:56.063908+0000","last_active":"2024-05-10T03:32:56.652618+0000","last_peered":"2024-05-10T03:32:56.652618+0000","last_clean":"2024-05-10T03:32:56.652618+0000","last_became_active":"2024-05-10T03:32:56.063545+0000","last_became_peered":"2024-05-10T03:32:56.063545+0000","last_unstale":"2024-05-10T03:32:56.652618+0000","last_undegraded":"2024-05-10T03:32:56.652618+0000","last_fullsized":"2024-05-10T03:32:56.652618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:05:14.589377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.661968+0000","last_change":"2024-05-10T03:32:56.656067+0000","last_active":"2024-05-10T03:32:56.661968+0000","last_peered":"2024-05-10T03:32:56.661968+0000","last_clean":"2024-05-10T03:32:56.661968+0000","last_became_active":"2024-05-10T03:32:56.655703+0000","last_became_peered":"2024-05-10T03:32:56.655703+0000","last_unstale":"2024-05-10T03:32:56.661968+0000","last_undegraded":"2024-05-10T03:32:56.661968+0000","last_fullsized":"2024-05-10T03:32:56.661968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:23:11.645088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626349+0000","last_change":"2024-05-10T03:32:56.064429+0000","last_active":"2024-05-10T03:32:56.626349+0000","last_peered":"2024-05-10T03:32:56.626349+0000","last_clean":"2024-05-10T03:32:56.626349+0000","last_became_active":"2024-05-10T03:32:56.063414+0000","last_became_peered":"2024-05-10T03:32:56.063414+0000","last_unstale":"2024-05-10T03:32:56.626349+0000","last_undegraded":"2024-05-10T03:32:56.626349+0000","last_fullsized":"2024-05-10T03:32:56.626349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:19.213770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.645601+0000","last_change":"2024-05-10T03:32:56.643800+0000","last_active":"2024-05-10T03:32:56.645601+0000","last_peered":"2024-05-10T03:32:56.645601+0000","last_clean":"2024-05-10T03:32:56.645601+0000","last_became_active":"2024-05-10T03:32:56.643478+0000","last_became_peered":"2024-05-10T03:32:56.643478+0000","last_unstale":"2024-05-10T03:32:56.645601+0000","last_undegraded":"2024-05-10T03:32:56.645601+0000","last_fullsized":"2024-05-10T03:32:56.645601+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:33:22.441460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.626758+0000","last_change":"2024-05-10T03:32:56.059457+0000","last_active":"2024-05-10T03:32:56.626758+0000","last_peered":"2024-05-10T03:32:56.626758+0000","last_clean":"2024-05-10T03:32:56.626758+0000","last_became_active":"2024-05-10T03:32:56.059128+0000","last_became_peered":"2024-05-10T03:32:56.059128+0000","last_unstale":"2024-05-10T03:32:56.626758+0000","last_undegraded":"2024-05-10T03:32:56.626758+0000","last_fullsized":"2024-05-10T03:32:56.626758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:48.047461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.644648+0000","last_change":"2024-05-10T03:32:56.641168+0000","last_active":"2024-05-10T03:32:56.644648+0000","last_peered":"2024-05-10T03:32:56.644648+0000","last_clean":"2024-05-10T03:32:56.644648+0000","last_became_active":"2024-05-10T03:32:56.640761+0000","last_became_peered":"2024-05-10T03:32:56.640761+0000","last_unstale":"2024-05-10T03:32:56.644648+0000","last_undegraded":"2024-05-10T03:32:56.644648+0000","last_fullsized":"2024-05-10T03:32:56.644648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:50:29.456781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.621204+0000","last_change":"2024-05-10T03:32:56.061249+0000","last_active":"2024-05-10T03:32:56.621204+0000","last_peered":"2024-05-10T03:32:56.621204+0000","last_clean":"2024-05-10T03:32:56.621204+0000","last_became_active":"2024-05-10T03:32:56.060931+0000","last_became_peered":"2024-05-10T03:32:56.060931+0000","last_unstale":"2024-05-10T03:32:56.621204+0000","last_undegraded":"2024-05-10T03:32:56.621204+0000","last_fullsized":"2024-05-10T03:32:56.621204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:14:15.148949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.658308+0000","last_change":"2024-05-10T03:32:56.644951+0000","last_active":"2024-05-10T03:32:56.658308+0000","last_peered":"2024-05-10T03:32:56.658308+0000","last_clean":"2024-05-10T03:32:56.658308+0000","last_became_active":"2024-05-10T03:32:56.644583+0000","last_became_peered":"2024-05-10T03:32:56.644583+0000","last_unstale":"2024-05-10T03:32:56.658308+0000","last_undegraded":"2024-05-10T03:32:56.658308+0000","last_fullsized":"2024-05-10T03:32:56.658308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:55.997060+0000","last_clean_scrub_stamp":"2024-05-10T03:32:55.997060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:44.155377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.623608+0000","last_change":"2024-05-10T03:32:56.055989+0000","last_active":"2024-05-10T03:32:56.623608+0000","last_peered":"2024-05-10T03:32:56.623608+0000","last_clean":"2024-05-10T03:32:56.623608+0000","last_became_active":"2024-05-10T03:32:56.054447+0000","last_became_peered":"2024-05-10T03:32:56.054447+0000","last_unstale":"2024-05-10T03:32:56.623608+0000","last_undegraded":"2024-05-10T03:32:56.623608+0000","last_fullsized":"2024-05-10T03:32:56.623608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:32:54.976048+0000","last_clean_scrub_stamp":"2024-05-10T03:32:54.976048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:53:27.762809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-10T03:32:56.045385+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:32:56.045385+0000","last_peered":"2024-05-10T03:32:56.045385+0000","last_clean":"2024-05-10T03:32:56.045385+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:32:56.045385+0000","last_undegraded":"2024-05-10T03:32:56.045385+0000","last_fullsized":"2024-05-10T03:32:56.045385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1396736,"data_stored":941150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7490000000000001,"5min":1.877,"15min":1.877},"last":1.264}]},{"osd":1,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1439999999999999,"15min":1.1419999999999999},"min":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.853},"last":1.159},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2450000000000001,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9039999999999999,"5min":1.925,"15min":1.925},"last":0.746}]},{"osd":2,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.363,"15min":1.383},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.77,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3899999999999999,"15min":1.4319999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8460000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.2909999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3720000000000001,"15min":1.3859999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":2.222,"15min":2.222},"last":1.319},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.353,"15min":1.3340000000000001},"min":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9550000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.421}]},{"osd":4,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.446,"15min":1.4690000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9790000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5189999999999999,"15min":1.5069999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2,"5min":2.323,"15min":2.323},"last":1.204}]},{"osd":5,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.528,"15min":1.51},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.619,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6879999999999999,"15min":1.696},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1709999999999998,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.7130000000000001,"15min":1.7250000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6519999999999999,"15min":1.6399999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.097,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.393}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":350040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.087,"15min":1.075},"min":{"1min":0.74099999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.57799999999999996},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3089999999999999,"15min":1.345},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.835,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.91800000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8160000000000001,"5min":3.113,"15min":3.113},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.423,"15min":1.48},"min":{"1min":0.76900000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.7589999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.74099999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.282,"15min":1.244},"min":{"1min":0.98199999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4299999999999999,"15min":1.504},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.859,"5min":3.569,"15min":3.569},"last":0.98299999999999998}]},{"osd":3,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.5449999999999999,"15min":1.613},"min":{"1min":1.012,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.863,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3200000000000001,"15min":1.3080000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.9099999999999999,"5min":1.931,"15min":1.931},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.998,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5449999999999999,"15min":1.579},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.9650000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.89100000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.4039999999999999,"15min":1.3919999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7410000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.6120000000000001,"15min":1.6200000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.117,"5min":2.472,"15min":2.472},"last":1.1759999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:36 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.657,"15min":1.669},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4830000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.627,"15min":1.647},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.847,"5min":2.359,"15min":2.359},"last":1.05}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.071,"15min":1.0640000000000001},"min":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5900000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.655,"5min":1.851,"15min":1.851},"last":1.4179999999999999}]},{"osd":1,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.78600000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.653,"5min":1.855,"15min":1.855},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.214,"15min":1.222},"min":{"1min":0.96699999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4219999999999999,"5min":1.954,"15min":1.954},"last":1.1639999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.242,"15min":1.226},"min":{"1min":0.81299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7490000000000001,"5min":2.359,"15min":2.359},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.3089999999999999,"15min":1.331},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4970000000000001}]},{"osd":3,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.3560000000000001},"min":{"1min":1.034,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.6479999999999999,"5min":1.881,"15min":1.881},"last":1.552},{"interface":"front","average":{"1min":1.413,"5min":1.3360000000000001,"15min":1.298},"min":{"1min":0.76800000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9830000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.234}]},{"osd":4,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.90200000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8540000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.069},{"interface":"front","average":{"1min":1.482,"5min":1.593,"15min":1.603},"min":{"1min":0.78100000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9670000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.3}]},{"osd":6,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.516,"15min":1.512},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1320000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5509999999999999,"15min":1.552},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri May 10 03:32:33 2024","interfaces":[{"interface":"back","average":{"1min":1.669,"5min":1.8009999999999999,"15min":1.8129999999999999},"min":{"1min":0.877,"5min":0.753,"15min":0.753},"max":{"1min":2.1739999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.6200000000000001,"15min":1.613},"min":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.98}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.159,"15min":1.173},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6919999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1259999999999999,"15min":1.1359999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.591,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1779999999999999,"15min":1.1599999999999999},"min":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.613,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.323},{"interface":"front","average":{"1min":1.139,"5min":1.248,"15min":1.2729999999999999},"min":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.72,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.79700000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5289999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.369,"15min":1.397},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.726,"5min":2.29,"15min":2.29},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.87,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.379,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.5649999999999999,"15min":1.581},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.782,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9079999999999999,"5min":2.512,"15min":2.512},"last":1.258},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.504,"15min":1.552},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5510000000000002,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.052}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.5569999999999999,"15min":1.583},"min":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4769999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.629},{"interface":"front","average":{"1min":1.51,"5min":1.724,"15min":1.7769999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.093,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.0880000000000001}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.6519999999999999,"15min":1.6899999999999999},"min":{"1min":0.63700000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.6160000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.242},{"interface":"front","average":{"1min":1.623,"5min":1.736,"15min":1.7430000000000001},"min":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6349999999999998,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.6830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":786432,"data_stored":350006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95399999999999996,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.96999999999999997},"min":{"1min":0.54500000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.575,"5min":1.708,"15min":1.708},"last":1.02}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.393,"5min":1.603,"15min":1.603},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.228,"15min":1.25},"min":{"1min":0.623,"5min":0.377,"15min":0.377},"max":{"1min":1.5329999999999999,"5min":1.964,"15min":1.964},"last":1.4399999999999999}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7450000000000001,"5min":1.99,"15min":1.99},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.3140000000000001,"15min":1.3600000000000001},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.679,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.633}]},{"osd":4,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.9019999999999999,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3999999999999999,"15min":1.3819999999999999},"min":{"1min":0.90200000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.948,"5min":2.464,"15min":2.464},"last":1.9650000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.613,"15min":1.6739999999999999},"min":{"1min":1.012,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2709999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":2.2400000000000002},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0449999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":2.359}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.5640000000000001,"15min":1.5760000000000001},"min":{"1min":1.123,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.113,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":2.081},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":1.7949999999999999},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4470000000000001,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":2.4329999999999998}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5840000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1139999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.504},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6359999999999999,"15min":1.657},"min":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.0419999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.8049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":786432,"data_stored":350006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3220000000000001,"5min":1.772,"15min":1.772},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":0.98199999999999998},"min":{"1min":0.55600000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.423,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.3049999999999999}]},{"osd":1,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.504,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.365},{"interface":"front","average":{"1min":1.145,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.66700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.925,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.536}]},{"osd":3,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.093,"15min":1.1200000000000001},"min":{"1min":0.57699999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":2.169,"15min":2.169},"last":1.157},{"interface":"front","average":{"1min":1.143,"5min":1.2250000000000001,"15min":1.2549999999999999},"min":{"1min":0.76800000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8200000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.8520000000000001}]},{"osd":4,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.488,"15min":1.5349999999999999},"min":{"1min":0.97199999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8160000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.46,"15min":1.538},"min":{"1min":0.86299999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7689999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.6399999999999999}]},{"osd":5,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4810000000000001,"15min":1.4890000000000001},"min":{"1min":0.91200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9259999999999999,"5min":2.508,"15min":2.508},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6100000000000001,"15min":1.6240000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.278,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0859999999999999}]},{"osd":6,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.93100000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.927,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.77},{"interface":"front","average":{"1min":1.524,"5min":1.5660000000000001,"15min":1.593},"min":{"1min":0.97199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3799999999999999,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":2.044}]},{"osd":7,"last update":"Fri May 10 03:32:34 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.627,"15min":1.655},"min":{"1min":1.018,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.2000000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.887},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.6739999999999999},"min":{"1min":0.95699999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.468,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":2.1379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":774144,"data_stored":342911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.92100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5089999999999999,"5min":1.607,"15min":1.607},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0469999999999999,"15min":1.085},"min":{"1min":0.52100000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.7030000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.98099999999999998,"15min":1.004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.747},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.986,"5min":2.444,"15min":2.444},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.319},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.879,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2729999999999999,"15min":1.3009999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9399999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.92800000000000005}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.98499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3159999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5329999999999999,"15min":1.585},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.101,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.266}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5209999999999999,"15min":1.585},"min":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2450000000000001,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.573,"15min":1.631},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0190000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.409}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.6459999999999999,"15min":1.7190000000000001},"min":{"1min":1.004,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4470000000000001,"5min":3.48,"15min":3.48},"last":1.522},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.6599999999999999,"15min":1.73},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.004,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.427}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.6839999999999999,"15min":1.7330000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.1859999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.72,"15min":1.782},"min":{"1min":1.113,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.5449999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1819999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1368064,"data_stored":933279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.95699999999999996,"15min":1.022},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4370000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.87},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.089,"15min":1.1870000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2589999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.1499999999999999,"15min":1.22},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.593,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.022},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.149,"15min":1.2170000000000001},"min":{"1min":0.56299999999999994,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5680000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.115}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.056,"15min":1.103},"min":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6160000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.052},{"interface":"front","average":{"1min":1.149,"5min":1.278,"15min":1.3240000000000001},"min":{"1min":0.69799999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.78,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.2649999999999999}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5409999999999999,"15min":1.633},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.958,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.611},{"interface":"front","average":{"1min":1.302,"5min":1.401,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.768,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.3680000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.4239999999999999,"15min":1.4730000000000001},"min":{"1min":0.80700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7110000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.47},{"interface":"front","average":{"1min":1.373,"5min":1.5489999999999999,"15min":1.627},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.873,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.5820000000000001}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.593,"15min":1.6419999999999999},"min":{"1min":0.98099999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.9410000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.647,"15min":1.74},"min":{"1min":1.069,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.831,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6879999999999999,"15min":1.792},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2.0840000000000001,"5min":3.149,"15min":3.149},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5369999999999999,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9019999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.54}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:33:02.379 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:33:02.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:33:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.709+0000 7f5dfb200640 1 Processor -- start 2024-05-10T03:33:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.709+0000 7f5dfb200640 1 -- start start 2024-05-10T03:33:02.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.711+0000 7f5dfb200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 0x7f5df406a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.711+0000 7f5dfb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 0x7f5df406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.711+0000 7f5dfb200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df40cdab0 con 0x7f5df40b2140 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.711+0000 7f5dfb200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40ce730 con 0x7f5df40b3160 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.711+0000 7f5dfb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40cdd90 con 0x7f5df406a750 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.712+0000 7f5debe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 0x7f5df406a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.712+0000 7f5debe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 0x7f5df406a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46046/0 (socket says 172.21.15.55:46046) 2024-05-10T03:33:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.712+0000 7f5debe00640 1 -- 172.21.15.55:0/2198786714 learned_addr learned my addr 172.21.15.55:0/2198786714 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.712+0000 7f5df9200640 1 --2- 172.21.15.55:0/2198786714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 0x7f5df406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.713+0000 7f5df9200640 1 -- 172.21.15.55:0/2198786714 >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 msgr2=0x7f5df406a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.713+0000 7f5df9200640 1 --2- 172.21.15.55:0/2198786714 >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 0x7f5df406a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.713+0000 7f5df9200640 1 -- 172.21.15.55:0/2198786714 >> v1:172.21.15.162:6789/0 conn(0x7f5df40b2140 legacy=0x7f5df40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.714+0000 7f5df9200640 1 -- 172.21.15.55:0/2198786714 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40ce070 con 0x7f5df406a750 2024-05-10T03:33:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.715+0000 7f5df9200640 1 --2- 172.21.15.55:0/2198786714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 0x7f5df406ab60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009d90 tx=0x7f5dec02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25516936725ff6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.715+0000 7f5df3400640 1 -- 172.21.15.55:0/2198786714 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031600 con 0x7f5df406a750 2024-05-10T03:33:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5df3400640 1 -- 172.21.15.55:0/2198786714 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec02d050 con 0x7f5df406a750 2024-05-10T03:33:02.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5df3400640 1 -- 172.21.15.55:0/2198786714 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031980 con 0x7f5df406a750 2024-05-10T03:33:02.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 -- 172.21.15.55:0/2198786714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 msgr2=0x7f5df406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 --2- 172.21.15.55:0/2198786714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 0x7f5df406ab60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009d90 tx=0x7f5dec02a5d0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 -- 172.21.15.55:0/2198786714 shutdown_connections 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 --2- 172.21.15.55:0/2198786714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df406a750 0x7f5df406ab60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 --2- 172.21.15.55:0/2198786714 >> v2:172.21.15.162:3300/0 conn(0x7f5df40b3160 0x7f5df406a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 -- 172.21.15.55:0/2198786714 >> 172.21.15.55:0/2198786714 conn(0x7f5df40cbb90 msgr2=0x7f5df40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.716+0000 7f5dfb200640 1 -- 172.21.15.55:0/2198786714 shutdown_connections 2024-05-10T03:33:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.717+0000 7f5dfb200640 1 -- 172.21.15.55:0/2198786714 wait complete. 2024-05-10T03:33:02.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.719+0000 7f5dfb200640 1 Processor -- start 2024-05-10T03:33:02.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.719+0000 7f5dfb200640 1 -- start start 2024-05-10T03:33:02.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.720+0000 7f5dfb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df406a750 0x7f5df415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.720+0000 7f5dfb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df40b2140 0x7f5df415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.720+0000 7f5dfb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5dfb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41586a0 con 0x7f5df415e690 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5dfb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df415a540 con 0x7f5df406a750 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5dfb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df4142e40 con 0x7f5df40b2140 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5df9200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5df9200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44510/0 (socket says 172.21.15.55:44510) 2024-05-10T03:33:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5df9200640 1 -- 172.21.15.55:0/4158594456 learned_addr learned my addr 172.21.15.55:0/4158594456 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5debe00640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df40b2140 0x7f5df415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.721+0000 7f5df3e00640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df406a750 0x7f5df415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 -- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df40b2140 msgr2=0x7f5df415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df40b2140 0x7f5df415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 -- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df406a750 msgr2=0x7f5df415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df406a750 0x7f5df415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 -- 172.21.15.55:0/4158594456 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec009a70 con 0x7f5df415e690 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df9200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02abc0 tx=0x7f5dec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec03b820 con 0x7f5df415e690 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec03b980 con 0x7f5df415e690 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec03bcf0 con 0x7f5df415e690 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.722+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df41593f0 con 0x7f5df415e690 2024-05-10T03:33:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.723+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df4158ac0 con 0x7f5df415e690 2024-05-10T03:33:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.724+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df41587e0 con 0x7f5df415e690 2024-05-10T03:33:02.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.725+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5dec03a3a0 con 0x7f5df415e690 2024-05-10T03:33:02.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.726+0000 7f5df0c00640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 0x7f5dd006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:02.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.726+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5dec0aabc0 con 0x7f5df415e690 2024-05-10T03:33:02.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.727+0000 7f5df3e00640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 0x7f5dd006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:02.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.729+0000 7f5df3e00640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 0x7f5dd006d110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5de4000f60 tx=0x7f5de4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:02.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:02.732+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5df41587e0 con 0x7f5df415e690 2024-05-10T03:33:03.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.005+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5df41587e0 con 0x7f5df415e690 2024-05-10T03:33:03.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.013+0000 7f5df0c00640 1 -- 172.21.15.55:0/4158594456 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5df41587e0 con 0x7f5df415e690 2024-05-10T03:33:03.016 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:33:03.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 msgr2=0x7f5dd006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 0x7f5dd006d110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5de4000f60 tx=0x7f5de4009040 comp rx=0 tx=0).stop 2024-05-10T03:33:03.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 msgr2=0x7f5df4141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5dec02abc0 tx=0x7f5dec004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:03.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 shutdown_connections 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5df415e690 0x7f5df4141d30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5df40b2140 0x7f5df415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5dd006ac00 0x7f5dd006d110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 --2- 172.21.15.55:0/4158594456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df406a750 0x7f5df415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 >> 172.21.15.55:0/4158594456 conn(0x7f5df40cbb90 msgr2=0x7f5df40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 shutdown_connections 2024-05-10T03:33:03.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.016+0000 7f5dfb200640 1 -- 172.21.15.55:0/4158594456 wait complete. 2024-05-10T03:33:03.045 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:03.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.327+0000 7f34b2e00640 1 Processor -- start 2024-05-10T03:33:03.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.327+0000 7f34b2e00640 1 -- start start 2024-05-10T03:33:03.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.329+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f34ac15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.329+0000 7f34b2e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0d29c0 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.329+0000 7f34b2e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0d2ca0 con 0x7f34ac0b1f90 2024-05-10T03:33:03.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.329+0000 7f34b2e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0d2f80 con 0x7f34ac161c20 2024-05-10T03:33:03.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.330+0000 7f34b1000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f34ac0ce6a0 0x7f34ac0d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45736/0 (socket says 172.21.15.55:45736) 2024-05-10T03:33:03.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.330+0000 7f34b1000640 1 -- 172.21.15.55:0/3111889133 learned_addr learned my addr 172.21.15.55:0/3111889133 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.330+0000 7f34b2400640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f34ac15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.330+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706841368 0 0) 0x7f34ac0d29c0 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.330+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3490003520 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707474087 0 0) 0x7f3490003520 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 msgr2=0x7f34ac15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f34ac15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 >> v1:172.21.15.162:6789/0 conn(0x7f34ac0b1f90 legacy=0x7f34ac0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0d1290 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f34980019f0 con 0x7f34ac0ce6a0 2024-05-10T03:33:03.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 >> v1:172.21.15.55:6789/0 conn(0x7f34ac0ce6a0 legacy=0x7f34ac0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3490003a10 0x7f3490003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34900047c0 0x7f3490004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f3490005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0d2f80 con 0x7f34900047c0 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349000bb10 con 0x7f3490003a10 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349000bcd0 con 0x7f34ac161c20 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.331+0000 7f34b1000640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34900047c0 0x7f3490004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1a00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3490003a10 0x7f3490003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1000640 1 -- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 msgr2=0x7f3490005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b2400640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f3490005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1000640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f3490005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1000640 1 -- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3490003a10 msgr2=0x7f3490003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1000640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3490003a10 0x7f3490003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1000640 1 -- 172.21.15.55:0/3111889133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490003520 con 0x7f34900047c0 2024-05-10T03:33:03.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b2400640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac161c20 0x7f3490005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:03.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.332+0000 7f34b1a00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3490003a10 0x7f3490003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.333+0000 7f34b1000640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34900047c0 0x7f3490004bd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f349800a800 tx=0x7f34900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561713e6b1dfebd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.333+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349802ba10 con 0x7f34900047c0 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.333+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349802bb70 con 0x7f34900047c0 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.333+0000 7f34abe00640 1 -- 172.21.15.55:0/3111889133 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498034470 con 0x7f34900047c0 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 -- 172.21.15.55:0/3111889133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34900047c0 msgr2=0x7f3490004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3111889133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34900047c0 0x7f3490004bd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f349800a800 tx=0x7f34900057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2400640 1 -- 172.21.15.55:0/3111889133 reap_dead start 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 -- 172.21.15.55:0/3111889133 shutdown_connections 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 -- 172.21.15.55:0/3111889133 >> 172.21.15.55:0/3111889133 conn(0x7f34ac0d0530 msgr2=0x7f34ac0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 -- 172.21.15.55:0/3111889133 shutdown_connections 2024-05-10T03:33:03.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.334+0000 7f34b2e00640 1 -- 172.21.15.55:0/3111889133 wait complete. 2024-05-10T03:33:03.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.337+0000 7f34b2e00640 1 Processor -- start 2024-05-10T03:33:03.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.337+0000 7f34b2e00640 1 -- start start 2024-05-10T03:33:03.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34900047c0 0x7f34ac1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 0x7f34ac0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac14e280 con 0x7f34ac0b3c40 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac14a750 con 0x7f34900047c0 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.338+0000 7f34b2e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0b4050 con 0x7f34ac14fac0 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53814/0 (socket says 172.21.15.55:53814) 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 -- 172.21.15.55:0/3525325864 learned_addr learned my addr 172.21.15.55:0/3525325864 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b1000640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 0x7f34ac0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b1a00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34900047c0 0x7f34ac1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 -- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 msgr2=0x7f34ac0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 0x7f34ac0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 -- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34900047c0 msgr2=0x7f34ac1f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34900047c0 0x7f34ac1f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b2400640 1 -- 172.21.15.55:0/3525325864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac14a9a0 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.339+0000 7f34b1000640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 0x7f34ac0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34b2400640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0b1f40 tx=0x7f34a4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400e870 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400e9f0 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac14ac80 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400eda0 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.340+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac14b240 con 0x7f34ac0b3c40 2024-05-10T03:33:03.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.341+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac04e290 con 0x7f34ac0b3c40 2024-05-10T03:33:03.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.342+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f34a4020410 con 0x7f34ac0b3c40 2024-05-10T03:33:03.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.344+0000 7f34a9600640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 0x7f348c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:03.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.344+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f34a4088c80 con 0x7f34ac0b3c40 2024-05-10T03:33:03.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.345+0000 7f34b1a00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 0x7f348c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:03.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.346+0000 7f34b1a00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 0x7f348c06d140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f349c001750 tx=0x7f349c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.349+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f34a4053db0 con 0x7f34ac0b3c40 2024-05-10T03:33:03.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.589+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34ac0ccd60 con 0x7f34ac0b3c40 2024-05-10T03:33:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.590+0000 7f34a9600640 1 -- 172.21.15.55:0/3525325864 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f34a40573d0 con 0x7f34ac0b3c40 2024-05-10T03:33:03.593 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:03.593 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:03.593 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.592+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 msgr2=0x7f348c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.592+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 0x7f348c06d140 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f349c001750 tx=0x7f349c009040 comp rx=0 tx=0).stop 2024-05-10T03:33:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.592+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 msgr2=0x7f34ac14ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.592+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0b1f40 tx=0x7f34a4010c20 comp rx=0 tx=0).stop 2024-05-10T03:33:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 shutdown_connections 2024-05-10T03:33:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34ac0b3c40 0x7f34ac14ba40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34ac14fac0 0x7f34ac0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f348c06ac30 0x7f348c06d140 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 --2- 172.21.15.55:0/3525325864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34900047c0 0x7f34ac1f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 >> 172.21.15.55:0/3525325864 conn(0x7f34ac0d0530 msgr2=0x7f34ac06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 shutdown_connections 2024-05-10T03:33:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:03.593+0000 7f34b2e00640 1 -- 172.21.15.55:0/3525325864 wait complete. 2024-05-10T03:33:04.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:32:56.602949+0000', 'modified': '2024-05-10T03:33:00.044266+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6147': {'gid': 6147, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/1356084610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 1356084610}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 1356084610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:33:04.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:33:04.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T03:33:04.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.315+0000 7fe7cfe00640 1 Processor -- start 2024-05-10T03:33:04.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.316+0000 7fe7cfe00640 1 -- start start 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cfe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7d01435d0 0x7fe7d0143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cfe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d00cdbe0 con 0x7fe7d00b2040 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cfe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d015e8b0 con 0x7fe7d006b120 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cfe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d00ce140 con 0x7fe7d01435d0 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe7d00b2040 0x7fe7d00ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45750/0 (socket says 172.21.15.55:45750) 2024-05-10T03:33:04.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.318+0000 7fe7cea00640 1 -- 172.21.15.55:0/2807305066 learned_addr learned my addr 172.21.15.55:0/2807305066 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:04.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/2807305066 >> v2:172.21.15.162:3300/0 conn(0x7fe7d01435d0 0x7fe7d0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117520348 0 0) 0x7fe7d00cdbe0 con 0x7fe7d00b2040 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b4003520 con 0x7fe7d00b2040 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623130490 0 0) 0x7fe7b4003520 con 0x7fe7d00b2040 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 >> v2:172.21.15.162:3300/0 conn(0x7fe7d01435d0 msgr2=0x7fe7d0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 --2- 172.21.15.55:0/2807305066 >> v2:172.21.15.162:3300/0 conn(0x7fe7d01435d0 0x7fe7d0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 >> v1:172.21.15.162:6789/0 conn(0x7fe7d006b120 legacy=0x7fe7d006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d015ede0 con 0x7fe7d00b2040 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe7c0001670 con 0x7fe7d00b2040 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 >> v1:172.21.15.55:6789/0 conn(0x7fe7d00b2040 legacy=0x7fe7d00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 0x7fe7b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7b40047c0 0x7fe7b4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4005210 0x7fe7b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00ce140 con 0x7fe7b4003970 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b400c1a0 con 0x7fe7b4005210 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.319+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b400c360 con 0x7fe7b40047c0 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 0x7fe7b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7b40047c0 0x7fe7b4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 -- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7b40047c0 msgr2=0x7fe7b4004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7b40047c0 0x7fe7b4004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 -- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4005210 msgr2=0x7fe7b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4005210 0x7fe7b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cf400640 1 -- 172.21.15.55:0/2807305066 reap_dead start 2024-05-10T03:33:04.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.320+0000 7fe7cea00640 1 -- 172.21.15.55:0/2807305066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4003520 con 0x7fe7b4003970 2024-05-10T03:33:04.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.321+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 0x7fe7b4003de0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007c60 tx=0x7fe7b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e634952373b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0005ad0 con 0x7fe7b4003970 2024-05-10T03:33:04.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c0005c50 con 0x7fe7b4003970 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7ce000640 1 -- 172.21.15.55:0/2807305066 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0007070 con 0x7fe7b4003970 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7cfe00640 1 -- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 msgr2=0x7fe7b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 0x7fe7b4003de0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007c60 tx=0x7fe7b40057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7cfe00640 1 -- 172.21.15.55:0/2807305066 shutdown_connections 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/2807305066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7b4003970 0x7fe7b4003de0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.322+0000 7fe7cfe00640 1 -- 172.21.15.55:0/2807305066 >> 172.21.15.55:0/2807305066 conn(0x7fe7d006c730 msgr2=0x7fe7d00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.323+0000 7fe7cfe00640 1 -- 172.21.15.55:0/2807305066 shutdown_connections 2024-05-10T03:33:04.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.323+0000 7fe7cfe00640 1 -- 172.21.15.55:0/2807305066 wait complete. 2024-05-10T03:33:04.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.325+0000 7fe7cfe00640 1 Processor -- start 2024-05-10T03:33:04.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.325+0000 7fe7cfe00640 1 -- start start 2024-05-10T03:33:04.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 0x7fe7d0159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 0x7fe7d0141780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d015bd60 con 0x7fe7d015b410 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d01f47a0 con 0x7fe7b4003970 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7cfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d015bf90 con 0x7fe7d0141370 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53826/0 (socket says 172.21.15.55:53826) 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.327+0000 7fe7c7e00640 1 -- 172.21.15.55:0/3399510596 learned_addr learned my addr 172.21.15.55:0/3399510596 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7cf400640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 0x7fe7d0141780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7cea00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 0x7fe7d0159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 -- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 msgr2=0x7fe7d0141780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 0x7fe7d0141780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 -- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 msgr2=0x7fe7d0159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 0x7fe7d0159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 -- 172.21.15.55:0/3399510596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d0141cc0 con 0x7fe7d015b410 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7cf400640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 0x7fe7d0141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7cea00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 0x7fe7d0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80116d0 tx=0x7fe7b8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c6a00640 1 -- 172.21.15.55:0/3399510596 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800f9c0 con 0x7fe7d015b410 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c6a00640 1 -- 172.21.15.55:0/3399510596 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b800fb20 con 0x7fe7d015b410 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7c6a00640 1 -- 172.21.15.55:0/3399510596 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800feb0 con 0x7fe7d015b410 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.328+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d015d280 con 0x7fe7d015b410 2024-05-10T03:33:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.329+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d015c4c0 con 0x7fe7d015b410 2024-05-10T03:33:04.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.331+0000 7fe7c6a00640 1 -- 172.21.15.55:0/3399510596 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe7b8015460 con 0x7fe7d015b410 2024-05-10T03:33:04.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.333+0000 7fe7c6a00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 0x7fe7ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.333+0000 7fe7c6a00640 1 -- 172.21.15.55:0/3399510596 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7b8089db0 con 0x7fe7d015b410 2024-05-10T03:33:04.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.333+0000 7fe7cea00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 0x7fe7ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.334+0000 7fe7dc5e3740 1 Processor -- start 2024-05-10T03:33:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.334+0000 7fe7dc5e3740 1 -- start start 2024-05-10T03:33:04.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7cea00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 0x7fe7ac06d110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0035000 tx=0x7fe7c0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb917c270 0x55bfb9545cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb9560a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9484a70 con 0x55bfb915b590 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9482200 con 0x55bfb917c270 2024-05-10T03:33:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.335+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb947daf0 con 0x55bfb91533b0 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53838/0 (socket says 172.21.15.55:53838) 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 -- 172.21.15.55:0/3272934508 learned_addr learned my addr 172.21.15.55:0/3272934508 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cea00640 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb917c270 0x55bfb9545cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb9560a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 -- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 msgr2=0x55bfb9560a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb9560a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 -- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb917c270 msgr2=0x55bfb9545cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb917c270 0x55bfb9545cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.336+0000 7fe7cf400640 1 -- 172.21.15.55:0/3272934508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb9448b90 con 0x55bfb915b590 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.337+0000 7fe7cf400640 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00cd750 tx=0x7fe7c8001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3b2933ad32a871 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.337+0000 7fe7bf000640 1 -- 172.21.15.55:0/3272934508 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800b070 con 0x55bfb915b590 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.337+0000 7fe7bf000640 1 -- 172.21.15.55:0/3272934508 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c80020d0 con 0x55bfb915b590 2024-05-10T03:33:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.337+0000 7fe7bf000640 1 -- 172.21.15.55:0/3272934508 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800d920 con 0x55bfb915b590 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/3272934508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 msgr2=0x55bfb9560f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00cd750 tx=0x7fe7c8001e80 comp rx=0 tx=0).stop 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/3272934508 shutdown_connections 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb915b590 0x55bfb9560f40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb9560a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/3272934508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb917c270 0x55bfb9545cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/3272934508 >> 172.21.15.55:0/3272934508 conn(0x55bfb91703c0 msgr2=0x55bfb951b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/3272934508 shutdown_connections 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.338+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/3272934508 wait complete. 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.339+0000 7fe7dc5e3740 1 -- start start 2024-05-10T03:33:04.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.339+0000 7fe7dc5e3740 1 Processor -- start 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb91533b0 0x55bfb95495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 0x55bfb95364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb941f220 con 0x55bfb917c270 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb941eec0 con 0x55bfb91533b0 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb91e4f20 con 0x55bfb915b590 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7dc5e3740 10 client.0 _ll_register_callbacks cb 0x55bfb9477160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53846/0 (socket says 172.21.15.55:53846) 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.341+0000 7fe7cf400640 1 -- 172.21.15.55:0/849845132 learned_addr learned my addr 172.21.15.55:0/849845132 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 0x55bfb95364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cea00640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb91533b0 0x55bfb95495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 -- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 msgr2=0x55bfb95364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 0x55bfb95364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 -- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb91533b0 msgr2=0x55bfb95495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb91533b0 0x55bfb95495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 -- 172.21.15.55:0/849845132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d015e8b0 con 0x55bfb917c270 2024-05-10T03:33:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 0x55bfb95364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7cf400640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f80 tx=0x7fe7c80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a055b1e432f5bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.342+0000 7fe7b3400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80074e0 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfb94a7c90 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7c8007640 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c8007800 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7dc5e3740 10 client.6279 fetch_fsmap learned FSMap version 31 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55bfb947c170 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bfb94a7c90 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe7c8013470 con 0x55bfb917c270 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.343+0000 7fe7dc5e3740 10 client.6279 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7dc5e3740 10 client.6279 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7dc5e3740 20 client.6279 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:04.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb913ae40 0x55bfb955d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7dc5e3740 4 client.6279 mds_command: new command op to 6147 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:33:04.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 --> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb91713c0 con 0x55bfb913ae40 2024-05-10T03:33:04.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.344+0000 7fe7cea00640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb913ae40 0x55bfb955d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.345+0000 7fe7cea00640 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb913ae40 0x55bfb955d560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c986552c7e6dc391 server_cookie=5c5f17dd5cb4a3e0 in_seq=0 out_seq=0 2024-05-10T03:33:04.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.345+0000 7fe7b3400640 10 client.6279 ms_handle_connect on v2:172.21.15.55:6834/1356084610 2024-05-10T03:33:04.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.345+0000 7fe7b3400640 1 -- 172.21.15.55:0/849845132 <== mds.0 v2:172.21.15.55:6834/1356084610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfb91713c0 con 0x55bfb913ae40 2024-05-10T03:33:04.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.345+0000 7fe7b3400640 10 client.6279 handle_command_reply: tid=0 2024-05-10T03:33:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.345+0000 7fe7dc5e3740 1 client.6279 shutdown 2024-05-10T03:33:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 msgr2=0x55bfb955f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f80 tx=0x7fe7c80014a0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 shutdown_connections 2024-05-10T03:33:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb913ae40 0x55bfb955d560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb955f210 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb915b590 0x55bfb95364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/849845132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb91533b0 0x55bfb95495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 >> 172.21.15.55:0/849845132 conn(0x55bfb91703c0 msgr2=0x55bfb9562540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7b3400640 0 client.6279 ms_handle_reset on v2:172.21.15.55:6834/1356084610 2024-05-10T03:33:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 shutdown_connections 2024-05-10T03:33:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/849845132 wait complete. 2024-05-10T03:33:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.346+0000 7fe7dc5e3740 20 client.6279 trim_cache size 0 max 16384 2024-05-10T03:33:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.364+0000 7fe7dc5e3740 1 Processor -- start 2024-05-10T03:33:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.364+0000 7fe7dc5e3740 1 -- start start 2024-05-10T03:33:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb955ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb917c270 0x55bfb955e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9404820 con 0x55bfb91533b0 2024-05-10T03:33:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb95606c0 con 0x55bfb915b590 2024-05-10T03:33:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.365+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb955dbe0 con 0x55bfb917c270 2024-05-10T03:33:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53858/0 (socket says 172.21.15.55:53858) 2024-05-10T03:33:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 -- 172.21.15.55:0/2201461128 learned_addr learned my addr 172.21.15.55:0/2201461128 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 -- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb917c270 msgr2=0x55bfb955e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb917c270 0x55bfb955e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 -- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 msgr2=0x55bfb955ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb955ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 -- 172.21.15.55:0/2201461128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb94b1130 con 0x55bfb91533b0 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.366+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001cd0 tx=0x7fe7c0009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5d9fe4b1e586a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7bfa00640 1 -- 172.21.15.55:0/2201461128 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00526a0 con 0x55bfb91533b0 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7bfa00640 1 -- 172.21.15.55:0/2201461128 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c0040070 con 0x55bfb91533b0 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7bfa00640 1 -- 172.21.15.55:0/2201461128 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00047e0 con 0x55bfb91533b0 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2201461128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 msgr2=0x55bfb95c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0001cd0 tx=0x7fe7c0009030 comp rx=0 tx=0).stop 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2201461128 shutdown_connections 2024-05-10T03:33:04.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb917c270 0x55bfb955e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb955ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2201461128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb91533b0 0x55bfb95c0f30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.367+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2201461128 >> 172.21.15.55:0/2201461128 conn(0x55bfb91703c0 msgr2=0x55bfb951b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.368+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2201461128 shutdown_connections 2024-05-10T03:33:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.368+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2201461128 wait complete. 2024-05-10T03:33:04.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.369+0000 7fe7dc5e3740 1 -- start start 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.369+0000 7fe7dc5e3740 1 Processor -- start 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb95156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb9540ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb95b3e80 con 0x55bfb917c270 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb957f9a0 con 0x55bfb915b590 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb9515ce0 con 0x55bfb91533b0 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.370+0000 7fe7dc5e3740 10 client.0 _ll_register_callbacks cb 0x7fe7c8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53872/0 (socket says 172.21.15.55:53872) 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 -- 172.21.15.55:0/2982101780 learned_addr learned my addr 172.21.15.55:0/2982101780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb95156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb9540ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 -- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 msgr2=0x55bfb95156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb95156e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 -- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 msgr2=0x55bfb9540ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb9540ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7cf400640 1 -- 172.21.15.55:0/2982101780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b4003520 con 0x55bfb917c270 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.371+0000 7fe7c7e00640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb9540ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7cf400640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55bfb955c810 tx=0x7fe7c8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604f91fe8eb71a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7b3400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80074e0 con 0x55bfb917c270 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7c8013980 con 0x55bfb917c270 2024-05-10T03:33:04.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7c80020d0 con 0x55bfb917c270 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c8007640 con 0x55bfb917c270 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7dc5e3740 10 client.6285 fetch_fsmap learned FSMap version 31 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55bfb941e430 con 0x55bfb917c270 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.372+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c80020d0 con 0x55bfb917c270 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe7c80077a0 con 0x55bfb917c270 2024-05-10T03:33:04.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 10 client.6285 fetch_fsmap finished waiting for FSMap version 31 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 10 client.6285 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 20 client.6285 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb957f560 0x55bfb955dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 4 client.6285 mds_command: new command op to 6147 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 --> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bfb91826c0 con 0x55bfb957f560 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.373+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb957f560 0x55bfb955dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7cea00640 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb957f560 0x55bfb955dd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361f9569ce99ba0 server_cookie=358c982e6c84c4e in_seq=0 out_seq=0 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7b3400640 10 client.6285 ms_handle_connect on v2:172.21.15.55:6834/1356084610 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7b3400640 1 -- 172.21.15.55:0/2982101780 <== mds.0 v2:172.21.15.55:6834/1356084610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bfb91826c0 con 0x55bfb957f560 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7b3400640 10 client.6285 handle_command_reply: tid=0 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 client.6285 shutdown 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 msgr2=0x55bfb95b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55bfb955c810 tx=0x7fe7c8009040 comp rx=0 tx=0).stop 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 shutdown_connections 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:6834/1356084610,v1:172.21.15.55:6835/1356084610] conn(0x55bfb957f560 0x55bfb955dd70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bfb917c270 0x55bfb95b3940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bfb915b590 0x55bfb9540ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 --2- 172.21.15.55:0/2982101780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bfb91533b0 0x55bfb95156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7b3400640 0 client.6285 ms_handle_reset on v2:172.21.15.55:6834/1356084610 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.374+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 >> 172.21.15.55:0/2982101780 conn(0x55bfb91703c0 msgr2=0x55bfb955fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 shutdown_connections 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7dc5e3740 1 -- 172.21.15.55:0/2982101780 wait complete. 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7dc5e3740 20 client.6285 trim_cache size 0 max 16384 2024-05-10T03:33:04.383 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6147, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.7662800819999998, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 31, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 66, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 66, 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.503114764999999 2024-05-10T03:33:04.384 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:33:04.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 msgr2=0x7fe7ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 0x7fe7ac06d110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0035000 tx=0x7fe7c0030000 comp rx=0 tx=0).stop 2024-05-10T03:33:04.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 msgr2=0x7fe7d015b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80116d0 tx=0x7fe7b8011b90 comp rx=0 tx=0).stop 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 shutdown_connections 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d0141370 0x7fe7d0141780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d015b410 0x7fe7d015b820 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe7ac06ac00 0x7fe7ac06d110 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 --2- 172.21.15.55:0/3399510596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7b4003970 0x7fe7d0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.375+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 >> 172.21.15.55:0/3399510596 conn(0x7fe7d006c730 msgr2=0x7fe7d01409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.376+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 shutdown_connections 2024-05-10T03:33:04.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.376+0000 7fe7cfe00640 1 -- 172.21.15.55:0/3399510596 wait complete. 2024-05-10T03:33:04.406 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:33:04.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.686+0000 7ff756600640 1 Processor -- start 2024-05-10T03:33:04.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.686+0000 7ff756600640 1 -- start start 2024-05-10T03:33:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff756600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7501596c0 0x7ff75006c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff756600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff750071730 0x7ff750071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff756600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff750072080 con 0x7ff750071060 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff756600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75006c9c0 con 0x7ff7501596c0 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff756600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff750159ad0 con 0x7ff750071730 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.689+0000 7ff755200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff750071060 0x7ff7500cbc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45752/0 (socket says 172.21.15.55:45752) 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff755200640 1 -- 172.21.15.55:0/2018185332 learned_addr learned my addr 172.21.15.55:0/2018185332 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74fe00640 1 --2- 172.21.15.55:0/2018185332 >> v2:172.21.15.55:3300/0 conn(0x7ff7501596c0 0x7ff75006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff755c00640 1 --2- 172.21.15.55:0/2018185332 >> v2:172.21.15.162:3300/0 conn(0x7ff750071730 0x7ff750071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74f400640 1 -- 172.21.15.55:0/2018185332 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753224637 0 0) 0x7ff750072080 con 0x7ff750071060 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74f400640 1 -- 172.21.15.55:0/2018185332 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff734003520 con 0x7ff750071060 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74fe00640 1 -- 172.21.15.55:0/2018185332 >> v2:172.21.15.162:3300/0 conn(0x7ff750071730 msgr2=0x7ff750071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74fe00640 1 --2- 172.21.15.55:0/2018185332 >> v2:172.21.15.162:3300/0 conn(0x7ff750071730 0x7ff750071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74fe00640 1 -- 172.21.15.55:0/2018185332 >> v1:172.21.15.55:6789/0 conn(0x7ff750071060 legacy=0x7ff7500cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.690+0000 7ff74fe00640 1 -- 172.21.15.55:0/2018185332 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75006cbb0 con 0x7ff7501596c0 2024-05-10T03:33:04.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.691+0000 7ff74fe00640 1 --2- 172.21.15.55:0/2018185332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75006c480 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b4d10 tx=0x7ff73c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569f2c79fdc6a8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.692+0000 7ff74f400640 1 -- 172.21.15.55:0/2018185332 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c02ae60 con 0x7ff7501596c0 2024-05-10T03:33:04.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.692+0000 7ff74f400640 1 -- 172.21.15.55:0/2018185332 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c032440 con 0x7ff7501596c0 2024-05-10T03:33:04.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.692+0000 7ff74f400640 1 -- 172.21.15.55:0/2018185332 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0327b0 con 0x7ff7501596c0 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.692+0000 7ff756600640 1 -- 172.21.15.55:0/2018185332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 msgr2=0x7ff75006c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.692+0000 7ff756600640 1 --2- 172.21.15.55:0/2018185332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75006c480 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b4d10 tx=0x7ff73c02a110 comp rx=0 tx=0).stop 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 -- 172.21.15.55:0/2018185332 shutdown_connections 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 --2- 172.21.15.55:0/2018185332 >> v2:172.21.15.162:3300/0 conn(0x7ff750071730 0x7ff750071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 --2- 172.21.15.55:0/2018185332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75006c480 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 -- 172.21.15.55:0/2018185332 >> 172.21.15.55:0/2018185332 conn(0x7ff75006a110 msgr2=0x7ff75006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 -- 172.21.15.55:0/2018185332 shutdown_connections 2024-05-10T03:33:04.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.693+0000 7ff756600640 1 -- 172.21.15.55:0/2018185332 wait complete. 2024-05-10T03:33:04.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.695+0000 7ff756600640 1 Processor -- start 2024-05-10T03:33:04.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.695+0000 7ff756600640 1 -- start start 2024-05-10T03:33:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff750071060 0x7ff7500ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff750071730 0x7ff75014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7501f01d0 con 0x7ff7501596c0 2024-05-10T03:33:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7501f1d60 con 0x7ff750071730 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff756600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7501ef6e0 con 0x7ff750071060 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff755c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff755c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53896/0 (socket says 172.21.15.55:53896) 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.697+0000 7ff755c00640 1 -- 172.21.15.55:0/3128187019 learned_addr learned my addr 172.21.15.55:0/3128187019 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755200640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff750071060 0x7ff7500ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff74fe00640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff750071730 0x7ff75014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 -- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff750071060 msgr2=0x7ff7500ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff750071060 0x7ff7500ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 -- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff750071730 msgr2=0x7ff75014b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff750071730 0x7ff75014b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 -- 172.21.15.55:0/3128187019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009400 con 0x7ff7501596c0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.698+0000 7ff755c00640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff748007e60 tx=0x7ff74800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.699+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748014420 con 0x7ff7501596c0 2024-05-10T03:33:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.699+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7480145a0 con 0x7ff7501596c0 2024-05-10T03:33:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.699+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748014950 con 0x7ff7501596c0 2024-05-10T03:33:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.699+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7501ef930 con 0x7ff7501596c0 2024-05-10T03:33:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.699+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7501efef0 con 0x7ff7501596c0 2024-05-10T03:33:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.700+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75004e290 con 0x7ff7501596c0 2024-05-10T03:33:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.701+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff748014ab0 con 0x7ff7501596c0 2024-05-10T03:33:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.702+0000 7ff74cc00640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 0x7ff73006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.702+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff748089ed0 con 0x7ff7501596c0 2024-05-10T03:33:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.703+0000 7ff755200640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 0x7ff73006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:04.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.704+0000 7ff755200640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 0x7ff73006d110 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff740005880 tx=0x7ff740009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:04.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.707+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff748054e40 con 0x7ff7501596c0 2024-05-10T03:33:04.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.946+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff75006cf20 con 0x7ff7501596c0 2024-05-10T03:33:04.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.947+0000 7ff74cc00640 1 -- 172.21.15.55:0/3128187019 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff748058460 con 0x7ff7501596c0 2024-05-10T03:33:04.950 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 msgr2=0x7ff73006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 0x7ff73006d110 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff740005880 tx=0x7ff740009040 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 msgr2=0x7ff75014bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff748007e60 tx=0x7ff74800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 shutdown_connections 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7501596c0 0x7ff75014bc70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff73006ac00 0x7ff73006d110 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff750071730 0x7ff75014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 --2- 172.21.15.55:0/3128187019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff750071060 0x7ff7500ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.950+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 >> 172.21.15.55:0/3128187019 conn(0x7ff75006a110 msgr2=0x7ff7500aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.951+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 shutdown_connections 2024-05-10T03:33:04.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:04.951+0000 7ff756600640 1 -- 172.21.15.55:0/3128187019 wait complete. 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:33:04.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:04.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:04.980 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:33:05.032 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:33:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:33:05.032 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:33:05.033 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.034 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:33:05.035 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:05.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.036 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:33:05.119 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:33:05.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.123 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:33:05.176 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:33:05.176 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:33:05.176 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:33:05.176 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:33:05.176 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:33:05.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.177 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:33:05.177 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:33:05.177 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:33:05.177 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:33:05.177 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:33:05.178 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:33:05.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.329 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:33:05.381 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:33:05.381 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:33:05.382 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:05.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:33:05.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:05.469 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:05.526 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:05.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:05.526 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:05.580 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:33:05.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:05.794 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:05.795 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:05.795 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:05.822 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:05.823 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:05.823 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:05.823 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:05.823 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:05.858 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:05.858 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:05.858 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:05.859 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:05.859 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:05.859 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:33:05.859 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:33:05.859 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:33:05.860 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:33:05.860 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:33:05.860 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:33:05.860 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:33:05.861 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:33:05.866 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:33:05.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.866 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:05.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:05.951 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:33:06.021 INFO:teuthology.orchestra.run.smithi055.stdout:client5563 2024-05-10T03:33:06.022 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:06.022 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:06.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:33:06.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.351+0000 7f1097000640 1 Processor -- start 2024-05-10T03:33:06.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.352+0000 7f1097000640 1 -- start start 2024-05-10T03:33:06.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1097000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f10900b2d00 0x7f10901471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1097000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1090162840 0x7f1090147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1097000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090162c50 con 0x7f1090144b60 2024-05-10T03:33:06.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1097000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10900b1f60 con 0x7f1090162840 2024-05-10T03:33:06.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1097000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1090162ed0 con 0x7f10900b2d00 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.354+0000 7f1096600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1090162840 0x7f1090147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1090162840 0x7f1090147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53910/0 (socket says 172.21.15.55:53910) 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 -- 172.21.15.55:0/1536714933 learned_addr learned my addr 172.21.15.55:0/1536714933 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1095c00640 1 --2- 172.21.15.55:0/1536714933 >> v2:172.21.15.162:3300/0 conn(0x7f10900b2d00 0x7f10901471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 -- 172.21.15.55:0/1536714933 >> v2:172.21.15.162:3300/0 conn(0x7f10900b2d00 msgr2=0x7f10901471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 --2- 172.21.15.55:0/1536714933 >> v2:172.21.15.162:3300/0 conn(0x7f10900b2d00 0x7f10901471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 -- 172.21.15.55:0/1536714933 >> v1:172.21.15.162:6789/0 conn(0x7f1090144b60 legacy=0x7f1090162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:06.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.355+0000 7f1096600640 1 -- 172.21.15.55:0/1536714933 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900b2220 con 0x7f1090162840 2024-05-10T03:33:06.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.356+0000 7f1096600640 1 --2- 172.21.15.55:0/1536714933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090162840 0x7f1090147730 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f10900aeb80 tx=0x7f108802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1948399d449a274 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:06.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.357+0000 7f108fe00640 1 -- 172.21.15.55:0/1536714933 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088031600 con 0x7f1090162840 2024-05-10T03:33:06.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.357+0000 7f108fe00640 1 -- 172.21.15.55:0/1536714933 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108802d050 con 0x7f1090162840 2024-05-10T03:33:06.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.357+0000 7f108fe00640 1 -- 172.21.15.55:0/1536714933 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088031980 con 0x7f1090162840 2024-05-10T03:33:06.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.357+0000 7f1097000640 1 -- 172.21.15.55:0/1536714933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090162840 msgr2=0x7f1090147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:06.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.357+0000 7f1097000640 1 --2- 172.21.15.55:0/1536714933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090162840 0x7f1090147730 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f10900aeb80 tx=0x7f108802a630 comp rx=0 tx=0).stop 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 -- 172.21.15.55:0/1536714933 shutdown_connections 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 --2- 172.21.15.55:0/1536714933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090162840 0x7f1090147730 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 --2- 172.21.15.55:0/1536714933 >> v2:172.21.15.162:3300/0 conn(0x7f10900b2d00 0x7f10901471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 -- 172.21.15.55:0/1536714933 >> 172.21.15.55:0/1536714933 conn(0x7f109006a110 msgr2=0x7f109006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 -- 172.21.15.55:0/1536714933 shutdown_connections 2024-05-10T03:33:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.358+0000 7f1097000640 1 -- 172.21.15.55:0/1536714933 wait complete. 2024-05-10T03:33:06.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.360+0000 7f1097000640 1 Processor -- start 2024-05-10T03:33:06.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.360+0000 7f1097000640 1 -- start start 2024-05-10T03:33:06.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 0x7f10901418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090162840 0x7f109013ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109013fbe0 con 0x7f1090144b60 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090156f90 con 0x7f1090162840 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1097000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1090141e20 con 0x7f10900b2d00 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1095200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1095200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53926/0 (socket says 172.21.15.55:53926) 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.362+0000 7f1095200640 1 -- 172.21.15.55:0/1631497778 learned_addr learned my addr 172.21.15.55:0/1631497778 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1096600640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090162840 0x7f109013ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095c00640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 0x7f10901418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 -- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 msgr2=0x7f10901418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 0x7f10901418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 -- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090162840 msgr2=0x7f109013ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090162840 0x7f109013ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 -- 172.21.15.55:0/1631497778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088009b30 con 0x7f1090144b60 2024-05-10T03:33:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095c00640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 0x7f10901418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1095200640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f107c00b3f0 tx=0x7f107c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011830 con 0x7f1090144b60 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011990 con 0x7f1090144b60 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10901571e0 con 0x7f1090144b60 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.363+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011d40 con 0x7f1090144b60 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.364+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10901577a0 con 0x7f1090144b60 2024-05-10T03:33:06.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.365+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090066ce0 con 0x7f1090144b60 2024-05-10T03:33:06.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.366+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f107c01a400 con 0x7f1090144b60 2024-05-10T03:33:06.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.367+0000 7f108d600640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 0x7f107006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:06.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.367+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f107c089460 con 0x7f1090144b60 2024-05-10T03:33:06.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.368+0000 7f1095c00640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 0x7f107006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:06.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.369+0000 7f1095c00640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 0x7f107006d110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1084006fd0 tx=0x7f1084008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:06.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.371+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f107c054470 con 0x7f1090144b60 2024-05-10T03:33:06.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.612+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f109004e290 con 0x7f1090144b60 2024-05-10T03:33:06.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.613+0000 7f108d600640 1 -- 172.21.15.55:0/1631497778 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f107c057a90 con 0x7f1090144b60 2024-05-10T03:33:06.616 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 msgr2=0x7f107006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 0x7f107006d110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1084006fd0 tx=0x7f1084008040 comp rx=0 tx=0).stop 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 msgr2=0x7f10901f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f107c00b3f0 tx=0x7f107c00b8b0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 shutdown_connections 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f107006ac00 0x7f107006d110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1090162840 0x7f109013ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1090144b60 0x7f10901f7870 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 --2- 172.21.15.55:0/1631497778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10900b2d00 0x7f10901418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 >> 172.21.15.55:0/1631497778 conn(0x7f109006a110 msgr2=0x7f1090071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 shutdown_connections 2024-05-10T03:33:06.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:06.616+0000 7f1097000640 1 -- 172.21.15.55:0/1631497778 wait complete. 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:33:06.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:33:06.646 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:06.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:06.646 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:06.704 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:33:06.705 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:33:06.706 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:06.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:06.707 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:33:06.789 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:33:06.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:06.794 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:33:06.845 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:33:06.846 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:33:06.846 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:33:06.846 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:33:06.846 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:33:06.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:06.846 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:33:06.846 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:33:06.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:33:06.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:33:06.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:33:06.847 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:33:06.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:06.993 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:33:07.042 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:33:07.042 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:33:07.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:07.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:07.044 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:33:07.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:07.127 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:07.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:07.181 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:07.230 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:33:07.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:07.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:07.426 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:07.426 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:07.426 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:07.453 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:07.453 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:07.453 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:07.453 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:07.453 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:33:07.486 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:33:07.487 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:33:07.488 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:33:07.488 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:33:07.488 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:33:07.488 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:33:07.488 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:33:07.493 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:33:07.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:07.493 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:07.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:07.560 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:33:07.616 INFO:teuthology.orchestra.run.smithi162.stdout:client5572 2024-05-10T03:33:07.616 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:07.616 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:07.664 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:33:07.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:07.780 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:07.884 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:33:07.884 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:07.884 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:07.946 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:07.946 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:08.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:08.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.362+0000 7fc6c9600640 1 Processor -- start 2024-05-10T03:33:08.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.362+0000 7fc6c9600640 1 -- start start 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 0x7fc6c40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c41659c0 0x7fc6c4165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6c4162380 0x7fc6c4162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40711c0 con 0x7fc6c4162380 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c406baf0 con 0x7fc6c406b6e0 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c9600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40715e0 con 0x7fc6c41659c0 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6c4162380 0x7fc6c4162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6c4162380 0x7fc6c4162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53942/0 (socket says 172.21.15.55:53942) 2024-05-10T03:33:08.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.365+0000 7fc6c3400640 1 -- 172.21.15.55:0/785222917 learned_addr learned my addr 172.21.15.55:0/785222917 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c2000640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c41659c0 0x7fc6c4165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c2a00640 1 --2- 172.21.15.55:0/785222917 >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 0x7fc6c40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c3400640 1 -- 172.21.15.55:0/785222917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c41659c0 msgr2=0x7fc6c4165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c3400640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c41659c0 0x7fc6c4165dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c3400640 1 -- 172.21.15.55:0/785222917 >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 msgr2=0x7fc6c40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:08.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c3400640 1 --2- 172.21.15.55:0/785222917 >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 0x7fc6c40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c3400640 1 -- 172.21.15.55:0/785222917 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4166310 con 0x7fc6c4162380 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.366+0000 7fc6c2a00640 1 --2- 172.21.15.55:0/785222917 >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 0x7fc6c40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.367+0000 7fc6c3400640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c4162380 0x7fc6c4162790 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406bcb0 tx=0x7fc6bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d09b0763a41b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.367+0000 7fc6c1600640 1 -- 172.21.15.55:0/785222917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc02ae90 con 0x7fc6c4162380 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.367+0000 7fc6c1600640 1 -- 172.21.15.55:0/785222917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc032440 con 0x7fc6c4162380 2024-05-10T03:33:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.367+0000 7fc6c1600640 1 -- 172.21.15.55:0/785222917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0327d0 con 0x7fc6c4162380 2024-05-10T03:33:08.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 -- 172.21.15.55:0/785222917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c4162380 msgr2=0x7fc6c4162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c4162380 0x7fc6c4162790 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406bcb0 tx=0x7fc6bc02a140 comp rx=0 tx=0).stop 2024-05-10T03:33:08.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 -- 172.21.15.55:0/785222917 shutdown_connections 2024-05-10T03:33:08.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c4162380 0x7fc6c4162790 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 --2- 172.21.15.55:0/785222917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c41659c0 0x7fc6c4165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 --2- 172.21.15.55:0/785222917 >> v2:172.21.15.162:3300/0 conn(0x7fc6c406b6e0 0x7fc6c40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 -- 172.21.15.55:0/785222917 >> 172.21.15.55:0/785222917 conn(0x7fc6c40cbb90 msgr2=0x7fc6c40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 -- 172.21.15.55:0/785222917 shutdown_connections 2024-05-10T03:33:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.368+0000 7fc6c9600640 1 -- 172.21.15.55:0/785222917 wait complete. 2024-05-10T03:33:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.371+0000 7fc6c9600640 1 Processor -- start 2024-05-10T03:33:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.371+0000 7fc6c9600640 1 -- start start 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c406b6e0 0x7fc6c4156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6c4162380 0x7fc6c40a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4157250 con 0x7fc6c41659c0 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41f8940 con 0x7fc6c4162380 2024-05-10T03:33:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c9600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4160360 con 0x7fc6c406b6e0 2024-05-10T03:33:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53946/0 (socket says 172.21.15.55:53946) 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.373+0000 7fc6c3400640 1 -- 172.21.15.55:0/3615284842 learned_addr learned my addr 172.21.15.55:0/3615284842 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c2a00640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c406b6e0 0x7fc6c4156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c2000640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6c4162380 0x7fc6c40a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 -- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c406b6e0 msgr2=0x7fc6c4156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c406b6e0 0x7fc6c4156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 -- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6c4162380 msgr2=0x7fc6c40a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6c4162380 0x7fc6c40a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 -- 172.21.15.55:0/3615284842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0094d0 con 0x7fc6c41659c0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.374+0000 7fc6c3400640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02a180 tx=0x7fc6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.375+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc030990 con 0x7fc6c41659c0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.375+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc030b10 con 0x7fc6c41659c0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.375+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc03b400 con 0x7fc6c41659c0 2024-05-10T03:33:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.375+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c41611c0 con 0x7fc6c41659c0 2024-05-10T03:33:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.375+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c4160890 con 0x7fc6c41659c0 2024-05-10T03:33:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.376+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c4161350 con 0x7fc6c41659c0 2024-05-10T03:33:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.377+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc6bc039040 con 0x7fc6c41659c0 2024-05-10T03:33:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.378+0000 7fc6baa00640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 0x7fc6a006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.378+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc6bc0a9cd0 con 0x7fc6c41659c0 2024-05-10T03:33:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.379+0000 7fc6c2a00640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 0x7fc6a006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.380+0000 7fc6c2a00640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 0x7fc6a006d160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406d310 tx=0x7fc6ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.383+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc6bc074d50 con 0x7fc6c41659c0 2024-05-10T03:33:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.592+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6c4066f60 con 0x7fc6c41659c0 2024-05-10T03:33:08.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.596+0000 7fc6baa00640 1 -- 172.21.15.55:0/3615284842 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fc6bc037090 con 0x7fc6c41659c0 2024-05-10T03:33:08.599 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:08.599 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.598+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 msgr2=0x7fc6a006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.598+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 0x7fc6a006d160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406d310 tx=0x7fc6ac008040 comp rx=0 tx=0).stop 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.598+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 msgr2=0x7fc6c40affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.598+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc02a180 tx=0x7fc6bc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 shutdown_connections 2024-05-10T03:33:08.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6c41659c0 0x7fc6c40affa0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc6a006ac50 0x7fc6a006d160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6c4162380 0x7fc6c40a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 --2- 172.21.15.55:0/3615284842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6c406b6e0 0x7fc6c4156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 >> 172.21.15.55:0/3615284842 conn(0x7fc6c40cbb90 msgr2=0x7fc6c415ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 shutdown_connections 2024-05-10T03:33:08.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.599+0000 7fc6c9600640 1 -- 172.21.15.55:0/3615284842 wait complete. 2024-05-10T03:33:08.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:54.741096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:55.615018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:08.632 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:08.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.921+0000 7fac97200640 1 Processor -- start 2024-05-10T03:33:08.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.921+0000 7fac97200640 1 -- start start 2024-05-10T03:33:08.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac9806b840 0x7fac980700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac98070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9815fdf0 con 0x7fac9806b840 2024-05-10T03:33:08.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9815f1e0 con 0x7fac980719f0 2024-05-10T03:33:08.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.923+0000 7fac97200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9815f4c0 con 0x7fac98071e00 2024-05-10T03:33:08.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.924+0000 7fac95e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.924+0000 7fac8fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac9806b840 0x7fac980700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.924+0000 7fac95e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:41962/0 (socket says 172.21.15.55:41962) 2024-05-10T03:33:08.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.924+0000 7fac95e00640 1 -- 172.21.15.55:0/1831224135 learned_addr learned my addr 172.21.15.55:0/1831224135 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.924+0000 7fac96800640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac98070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac8fe00640 1 -- 172.21.15.55:0/1831224135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 msgr2=0x7fac98070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac8fe00640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac98070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac8fe00640 1 -- 172.21.15.55:0/1831224135 >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 msgr2=0x7fac9806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac8fe00640 1 --2- 172.21.15.55:0/1831224135 >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac8fe00640 1 -- 172.21.15.55:0/1831224135 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9815f7a0 con 0x7fac9806b840 2024-05-10T03:33:08.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac96800640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac98070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:08.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.925+0000 7fac95e00640 1 --2- 172.21.15.55:0/1831224135 >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:33:08.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.926+0000 7fac8fe00640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9806b840 0x7fac980700e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fac980b41f0 tx=0x7fac8002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b30834cef55e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.926+0000 7fac95400640 1 -- 172.21.15.55:0/1831224135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8002add0 con 0x7fac9806b840 2024-05-10T03:33:08.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.926+0000 7fac95400640 1 -- 172.21.15.55:0/1831224135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac80032440 con 0x7fac9806b840 2024-05-10T03:33:08.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.926+0000 7fac95400640 1 -- 172.21.15.55:0/1831224135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac800327d0 con 0x7fac9806b840 2024-05-10T03:33:08.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 -- 172.21.15.55:0/1831224135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9806b840 msgr2=0x7fac980700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9806b840 0x7fac980700e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fac980b41f0 tx=0x7fac8002a080 comp rx=0 tx=0).stop 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 -- 172.21.15.55:0/1831224135 shutdown_connections 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac98070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 --2- 172.21.15.55:0/1831224135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9806b840 0x7fac980700e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 --2- 172.21.15.55:0/1831224135 >> v2:172.21.15.162:3300/0 conn(0x7fac980719f0 0x7fac9806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 -- 172.21.15.55:0/1831224135 >> 172.21.15.55:0/1831224135 conn(0x7fac9806c750 msgr2=0x7fac980b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:08.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 -- 172.21.15.55:0/1831224135 shutdown_connections 2024-05-10T03:33:08.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.927+0000 7fac97200640 1 -- 172.21.15.55:0/1831224135 wait complete. 2024-05-10T03:33:08.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.930+0000 7fac97200640 1 Processor -- start 2024-05-10T03:33:08.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.930+0000 7fac97200640 1 -- start start 2024-05-10T03:33:08.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac9806b840 0x7fac9815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac980b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9813fa60 con 0x7fac980719f0 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c4e10 con 0x7fac9806b840 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.931+0000 7fac97200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98132c90 con 0x7fac98071e00 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53972/0 (socket says 172.21.15.55:53972) 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 -- 172.21.15.55:0/57521163 learned_addr learned my addr 172.21.15.55:0/57521163 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac95e00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac9806b840 0x7fac9815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac96800640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac980b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 -- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 msgr2=0x7fac980b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac980b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 -- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac9806b840 msgr2=0x7fac9815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac9806b840 0x7fac9815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.932+0000 7fac8fe00640 1 -- 172.21.15.55:0/57521163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac880094d0 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac8fe00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac80009890 tx=0x7fac800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8003d910 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8003da90 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8003de40 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac80009400 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.933+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980c8480 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.934+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98131670 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.935+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fac80039040 con 0x7fac980719f0 2024-05-10T03:33:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.937+0000 7fac8e000640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 0x7fac7406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:08.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.937+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac800a9f40 con 0x7fac980719f0 2024-05-10T03:33:08.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.937+0000 7fac95e00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 0x7fac7406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:08.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.939+0000 7fac95e00640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 0x7fac7406d110 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fac88009bd0 tx=0x7fac88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:08.941+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fac98131670 con 0x7fac980719f0 2024-05-10T03:33:09.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.148+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac98131630 con 0x7fac980719f0 2024-05-10T03:33:09.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.149+0000 7fac8e000640 1 -- 172.21.15.55:0/57521163 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac800370a0 con 0x7fac980719f0 2024-05-10T03:33:09.152 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 msgr2=0x7fac7406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 0x7fac7406d110 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fac88009bd0 tx=0x7fac88008040 comp rx=0 tx=0).stop 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 msgr2=0x7fac98134070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac80009890 tx=0x7fac800046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 shutdown_connections 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac98071e00 0x7fac980b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac980719f0 0x7fac98134070 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7406ac00 0x7fac7406d110 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 --2- 172.21.15.55:0/57521163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac9806b840 0x7fac9815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 >> 172.21.15.55:0/57521163 conn(0x7fac9806c750 msgr2=0x7fac981415e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:09.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 shutdown_connections 2024-05-10T03:33:09.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.152+0000 7fac97200640 1 -- 172.21.15.55:0/57521163 wait complete. 2024-05-10T03:33:09.182 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:09.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.468+0000 7f8d40e00640 1 Processor -- start 2024-05-10T03:33:09.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.469+0000 7f8d40e00640 1 -- start start 2024-05-10T03:33:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.471+0000 7f8d40e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 0x7f8d3c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.471+0000 7f8d40e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.471+0000 7f8d40e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0b4ca0 con 0x7f8d3c161d10 2024-05-10T03:33:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.471+0000 7f8d40e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c0b4f80 con 0x7f8d3c0b4060 2024-05-10T03:33:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.471+0000 7f8d40e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cded0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.472+0000 7f8d33e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.472+0000 7f8d33e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:53376/0 (socket says 172.21.15.55:53376) 2024-05-10T03:33:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.472+0000 7f8d33e00640 1 -- 172.21.15.55:0/1396337340 learned_addr learned my addr 172.21.15.55:0/1396337340 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.472+0000 7f8d3b400640 1 --2- 172.21.15.55:0/1396337340 >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 0x7f8d3c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.473+0000 7f8d33e00640 1 -- 172.21.15.55:0/1396337340 >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 msgr2=0x7f8d3c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.473+0000 7f8d33e00640 1 --2- 172.21.15.55:0/1396337340 >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 0x7f8d3c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.473+0000 7f8d33e00640 1 -- 172.21.15.55:0/1396337340 >> v1:172.21.15.162:6789/0 conn(0x7f8d3c161d10 legacy=0x7f8d3c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.473+0000 7f8d33e00640 1 -- 172.21.15.55:0/1396337340 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0cdbd0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.473+0000 7f8d3b400640 1 --2- 172.21.15.55:0/1396337340 >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 0x7f8d3c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:33:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.474+0000 7f8d33e00640 1 --2- 172.21.15.55:0/1396337340 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009870 tx=0x7f8d2402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0528930b4123b92 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.475+0000 7f8d3aa00640 1 -- 172.21.15.55:0/1396337340 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24032440 con 0x7f8d3c1613c0 2024-05-10T03:33:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.475+0000 7f8d3aa00640 1 -- 172.21.15.55:0/1396337340 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d240325a0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.475+0000 7f8d3aa00640 1 -- 172.21.15.55:0/1396337340 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24032930 con 0x7f8d3c1613c0 2024-05-10T03:33:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 -- 172.21.15.55:0/1396337340 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 msgr2=0x7f8d3c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1396337340 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009870 tx=0x7f8d2402a1e0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 -- 172.21.15.55:0/1396337340 shutdown_connections 2024-05-10T03:33:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1396337340 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c1613c0 0x7f8d3c1617d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1396337340 >> v2:172.21.15.162:3300/0 conn(0x7f8d3c0b4060 0x7f8d3c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.476+0000 7f8d40e00640 1 -- 172.21.15.55:0/1396337340 >> 172.21.15.55:0/1396337340 conn(0x7f8d3c0b2d60 msgr2=0x7f8d3c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.477+0000 7f8d40e00640 1 -- 172.21.15.55:0/1396337340 shutdown_connections 2024-05-10T03:33:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.477+0000 7f8d40e00640 1 -- 172.21.15.55:0/1396337340 wait complete. 2024-05-10T03:33:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.480+0000 7f8d40e00640 1 Processor -- start 2024-05-10T03:33:09.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.480+0000 7f8d40e00640 1 -- start start 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d3c0b4060 0x7f8d3c148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c161d10 0x7f8d3c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cc3e0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c15f3f0 con 0x7f8d3c0b4060 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d40e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cb0f0 con 0x7f8d3c161d10 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53978/0 (socket says 172.21.15.55:53978) 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.482+0000 7f8d33e00640 1 -- 172.21.15.55:0/1998563024 learned_addr learned my addr 172.21.15.55:0/1998563024 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d3b400640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d3c0b4060 0x7f8d3c148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d3be00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c161d10 0x7f8d3c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 -- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c161d10 msgr2=0x7f8d3c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c161d10 0x7f8d3c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 -- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d3c0b4060 msgr2=0x7f8d3c148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d3c0b4060 0x7f8d3c148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 -- 172.21.15.55:0/1998563024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009400 con 0x7f8d3c1613c0 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.483+0000 7f8d33e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8d24000c00 tx=0x7f8d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.484+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24032440 con 0x7f8d3c1613c0 2024-05-10T03:33:09.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.484+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240094d0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.484+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24030980 con 0x7f8d3c1613c0 2024-05-10T03:33:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.484+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24030d10 con 0x7f8d3c1613c0 2024-05-10T03:33:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.484+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c0cbf50 con 0x7f8d3c1613c0 2024-05-10T03:33:09.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.485+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0cc0e0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.486+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d24039040 con 0x7f8d3c1613c0 2024-05-10T03:33:09.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.487+0000 7f8d33400640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 0x7f8d1806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:09.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.487+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8d240a9ee0 con 0x7f8d3c1613c0 2024-05-10T03:33:09.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.488+0000 7f8d3b400640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 0x7f8d1806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:09.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.489+0000 7f8d3b400640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 0x7f8d1806d140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0d0c90 tx=0x7f8d2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:09.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.493+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8d24074e60 con 0x7f8d3c1613c0 2024-05-10T03:33:09.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.691+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d3c04e290 con 0x7f8d3c1613c0 2024-05-10T03:33:09.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.692+0000 7f8d33400640 1 -- 172.21.15.55:0/1998563024 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d24097020 con 0x7f8d3c1613c0 2024-05-10T03:33:09.695 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:09.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 msgr2=0x7f8d1806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 0x7f8d1806d140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0d0c90 tx=0x7f8d2c008040 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 msgr2=0x7f8d3c0ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8d24000c00 tx=0x7f8d240046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 shutdown_connections 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d3c161d10 0x7f8d3c15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d3c1613c0 0x7f8d3c0ca610 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d1806ac30 0x7f8d1806d140 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 --2- 172.21.15.55:0/1998563024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d3c0b4060 0x7f8d3c148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.695+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 >> 172.21.15.55:0/1998563024 conn(0x7f8d3c0b2d60 msgr2=0x7f8d3c155af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.696+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 shutdown_connections 2024-05-10T03:33:09.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:09.696+0000 7f8d40e00640 1 -- 172.21.15.55:0/1998563024 wait complete. 2024-05-10T03:33:09.726 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:10.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.007+0000 7f8ea6800640 1 Processor -- start 2024-05-10T03:33:10.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.008+0000 7f8ea6800640 1 -- start start 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea6800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 0x7f8ea0143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ea006b120 0x7f8ea006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea6800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00cdbe0 con 0x7f8ea01435d0 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea6800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea015e8b0 con 0x7f8ea006b120 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea6800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea00ce140 con 0x7f8ea00b2040 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.010+0000 7f8ea5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ea006b120 0x7f8ea006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ea006b120 0x7f8ea006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53990/0 (socket says 172.21.15.55:53990) 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 -- 172.21.15.55:0/3414707620 learned_addr learned my addr 172.21.15.55:0/3414707620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5400640 1 --2- 172.21.15.55:0/3414707620 >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 0x7f8ea0143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 -- 172.21.15.55:0/3414707620 >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 msgr2=0x7f8ea0143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 --2- 172.21.15.55:0/3414707620 >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 0x7f8ea0143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 -- 172.21.15.55:0/3414707620 >> v1:172.21.15.162:6789/0 conn(0x7f8ea01435d0 legacy=0x7f8ea00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5e00640 1 -- 172.21.15.55:0/3414707620 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea006ba70 con 0x7f8ea006b120 2024-05-10T03:33:10.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.011+0000 7f8ea5400640 1 --2- 172.21.15.55:0/3414707620 >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 0x7f8ea0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.012+0000 7f8ea5e00640 1 --2- 172.21.15.55:0/3414707620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea006b120 0x7f8ea006b530 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009e50 tx=0x7f8e9802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51f4b9a3ebf65ac server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.013+0000 7f8e9fe00640 1 -- 172.21.15.55:0/3414707620 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98031600 con 0x7f8ea006b120 2024-05-10T03:33:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.013+0000 7f8e9fe00640 1 -- 172.21.15.55:0/3414707620 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9802d050 con 0x7f8ea006b120 2024-05-10T03:33:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.013+0000 7f8e9fe00640 1 -- 172.21.15.55:0/3414707620 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98031980 con 0x7f8ea006b120 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 -- 172.21.15.55:0/3414707620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea006b120 msgr2=0x7f8ea006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 --2- 172.21.15.55:0/3414707620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea006b120 0x7f8ea006b530 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009e50 tx=0x7f8e9802a680 comp rx=0 tx=0).stop 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 -- 172.21.15.55:0/3414707620 shutdown_connections 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 --2- 172.21.15.55:0/3414707620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea006b120 0x7f8ea006b530 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 --2- 172.21.15.55:0/3414707620 >> v2:172.21.15.162:3300/0 conn(0x7f8ea00b2040 0x7f8ea0143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 -- 172.21.15.55:0/3414707620 >> 172.21.15.55:0/3414707620 conn(0x7f8ea006c730 msgr2=0x7f8ea00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 -- 172.21.15.55:0/3414707620 shutdown_connections 2024-05-10T03:33:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.014+0000 7f8ea6800640 1 -- 172.21.15.55:0/3414707620 wait complete. 2024-05-10T03:33:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.016+0000 7f8ea6800640 1 Processor -- start 2024-05-10T03:33:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.017+0000 7f8ea6800640 1 -- start start 2024-05-10T03:33:10.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 0x7f8ea013b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ea01435d0 0x7f8ea013dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea014b5a0 con 0x7f8ea00b2040 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0139e80 con 0x7f8ea006b120 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.018+0000 7f8ea6800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea013bb10 con 0x7f8ea01435d0 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54006/0 (socket says 172.21.15.55:54006) 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 -- 172.21.15.55:0/1934291021 learned_addr learned my addr 172.21.15.55:0/1934291021 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea5400640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 0x7f8ea013b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea5e00640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ea01435d0 0x7f8ea013dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 -- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ea01435d0 msgr2=0x7f8ea013dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ea01435d0 0x7f8ea013dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 -- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 msgr2=0x7f8ea013b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 0x7f8ea013b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea4a00640 1 -- 172.21.15.55:0/1934291021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009b30 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.019+0000 7f8ea5400640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 0x7f8ea013b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8ea4a00640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00b3e0 tx=0x7f8e8c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c011830 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c0119b0 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c011da0 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea013a0d0 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.020+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea013a690 con 0x7f8ea00b2040 2024-05-10T03:33:10.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.021+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea00b1680 con 0x7f8ea00b2040 2024-05-10T03:33:10.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.023+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e8c01a410 con 0x7f8ea00b2040 2024-05-10T03:33:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.024+0000 7f8e9d600640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 0x7f8e8006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.024+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e8c0894c0 con 0x7f8ea00b2040 2024-05-10T03:33:10.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.024+0000 7f8ea5400640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 0x7f8e8006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.026+0000 7f8ea5400640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 0x7f8e8006d110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a990 tx=0x7f8e94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.028+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8e8c0544d0 con 0x7f8ea00b2040 2024-05-10T03:33:10.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.236+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ea0143360 con 0x7f8ea00b2040 2024-05-10T03:33:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.237+0000 7f8e9d600640 1 -- 172.21.15.55:0/1934291021 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e8c0150a0 con 0x7f8ea00b2040 2024-05-10T03:33:10.240 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 msgr2=0x7f8e8006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 0x7f8e8006d110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a990 tx=0x7f8e94008040 comp rx=0 tx=0).stop 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 msgr2=0x7f8ea01f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00b3e0 tx=0x7f8e8c00b8a0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 shutdown_connections 2024-05-10T03:33:10.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ea01435d0 0x7f8ea013dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ea00b2040 0x7f8ea01f35c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8e8006ac00 0x7f8e8006d110 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 --2- 172.21.15.55:0/1934291021 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ea006b120 0x7f8ea013b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 >> 172.21.15.55:0/1934291021 conn(0x7f8ea006c730 msgr2=0x7f8ea013f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 shutdown_connections 2024-05-10T03:33:10.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.240+0000 7f8ea6800640 1 -- 172.21.15.55:0/1934291021 wait complete. 2024-05-10T03:33:10.270 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:33:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.553+0000 7f011da00640 1 Processor -- start 2024-05-10T03:33:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.553+0000 7f011da00640 1 -- start start 2024-05-10T03:33:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.555+0000 7f011da00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01180b2040 0x7f0118143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.555+0000 7f011da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01181435d0 0x7f01180ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.555+0000 7f011da00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01180cdbe0 con 0x7f011806b120 2024-05-10T03:33:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.555+0000 7f011da00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011815e8b0 con 0x7f01181435d0 2024-05-10T03:33:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.555+0000 7f011da00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01180ce140 con 0x7f01180b2040 2024-05-10T03:33:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01181435d0 0x7f01180ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01181435d0 0x7f01180ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54012/0 (socket says 172.21.15.55:54012) 2024-05-10T03:33:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 -- 172.21.15.55:0/2744869220 learned_addr learned my addr 172.21.15.55:0/2744869220 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117e00640 1 --2- 172.21.15.55:0/2744869220 >> v2:172.21.15.162:3300/0 conn(0x7f01180b2040 0x7f0118143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 -- 172.21.15.55:0/2744869220 >> v2:172.21.15.162:3300/0 conn(0x7f01180b2040 msgr2=0x7f0118143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 --2- 172.21.15.55:0/2744869220 >> v2:172.21.15.162:3300/0 conn(0x7f01180b2040 0x7f0118143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 -- 172.21.15.55:0/2744869220 >> v1:172.21.15.162:6789/0 conn(0x7f011806b120 legacy=0x7f011806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.556+0000 7f0117400640 1 -- 172.21.15.55:0/2744869220 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011815ede0 con 0x7f01181435d0 2024-05-10T03:33:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.557+0000 7f0117400640 1 --2- 172.21.15.55:0/2744869220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01181435d0 0x7f01180ce5d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f011806cfc0 tx=0x7f010402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73da2a4e23ee4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.558+0000 7f0116a00640 1 -- 172.21.15.55:0/2744869220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010402ae90 con 0x7f01181435d0 2024-05-10T03:33:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.558+0000 7f0116a00640 1 -- 172.21.15.55:0/2744869220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0104004030 con 0x7f01181435d0 2024-05-10T03:33:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.558+0000 7f0116a00640 1 -- 172.21.15.55:0/2744869220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0104004420 con 0x7f01181435d0 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 -- 172.21.15.55:0/2744869220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01181435d0 msgr2=0x7f01180ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 --2- 172.21.15.55:0/2744869220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01181435d0 0x7f01180ce5d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f011806cfc0 tx=0x7f010402a180 comp rx=0 tx=0).stop 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 -- 172.21.15.55:0/2744869220 shutdown_connections 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 --2- 172.21.15.55:0/2744869220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01181435d0 0x7f01180ce5d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 --2- 172.21.15.55:0/2744869220 >> v2:172.21.15.162:3300/0 conn(0x7f01180b2040 0x7f0118143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 -- 172.21.15.55:0/2744869220 >> 172.21.15.55:0/2744869220 conn(0x7f011806c730 msgr2=0x7f01180b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 -- 172.21.15.55:0/2744869220 shutdown_connections 2024-05-10T03:33:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.559+0000 7f011da00640 1 -- 172.21.15.55:0/2744869220 wait complete. 2024-05-10T03:33:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.561+0000 7f011da00640 1 Processor -- start 2024-05-10T03:33:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.562+0000 7f011da00640 1 -- start start 2024-05-10T03:33:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 0x7f01181f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 0x7f01181f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011815b710 con 0x7f011806b120 2024-05-10T03:33:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01181f4a50 con 0x7f01181435d0 2024-05-10T03:33:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f011da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01181f4ee0 con 0x7f01180b2040 2024-05-10T03:33:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f0117e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.563+0000 7f0117e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54014/0 (socket says 172.21.15.55:54014) 2024-05-10T03:33:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 -- 172.21.15.55:0/149085572 learned_addr learned my addr 172.21.15.55:0/149085572 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117400640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 0x7f01181f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f011d000640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 0x7f01181f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 -- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 msgr2=0x7f01181f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 0x7f01181f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 -- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 msgr2=0x7f01181f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 0x7f01181f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 -- 172.21.15.55:0/149085572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104009460 con 0x7f011806b120 2024-05-10T03:33:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117400640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 0x7f01181f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f011d000640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 0x7f01181f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.564+0000 7f0117e00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f010800d5f0 tx=0x7f010800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.565+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01080125f0 con 0x7f011806b120 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.565+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108012750 con 0x7f011806b120 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.565+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108012b00 con 0x7f011806b120 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.565+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01181f4120 con 0x7f011806b120 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.565+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01181f46e0 con 0x7f011806b120 2024-05-10T03:33:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.566+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01180b1680 con 0x7f011806b120 2024-05-10T03:33:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.567+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0108012c60 con 0x7f011806b120 2024-05-10T03:33:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.568+0000 7f010fe00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 0x7f00f806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.568+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f01080899e0 con 0x7f011806b120 2024-05-10T03:33:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.569+0000 7f0117400640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 0x7f00f806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.570+0000 7f0117400640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 0x7f00f806d110 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0104033000 tx=0x7f010402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.573+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f01080549e0 con 0x7f011806b120 2024-05-10T03:33:10.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.787+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f01180ccc40 con 0x7f011806b120 2024-05-10T03:33:10.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.788+0000 7f010fe00640 1 -- 172.21.15.55:0/149085572 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0108058000 con 0x7f011806b120 2024-05-10T03:33:10.791 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.790+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 msgr2=0x7f00f806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.790+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 0x7f00f806d110 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0104033000 tx=0x7f010402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.790+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 msgr2=0x7f011815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.790+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f010800d5f0 tx=0x7f010800dab0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 shutdown_connections 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f00f806ac00 0x7f00f806d110 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01181435d0 0x7f01181f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01180b2040 0x7f01181f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 --2- 172.21.15.55:0/149085572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f011806b120 0x7f011815ca50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 >> 172.21.15.55:0/149085572 conn(0x7f011806c730 msgr2=0x7f011813f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:10.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 shutdown_connections 2024-05-10T03:33:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:10.791+0000 7f011da00640 1 -- 172.21.15.55:0/149085572 wait complete. 2024-05-10T03:33:10.823 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:33:10.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:10.823 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:10.921 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:33:10.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:10.921 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:33:10.974 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:10.974 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:11.031 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:11.031 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:11.088 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:11.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.380+0000 7f000ce00640 1 Processor -- start 2024-05-10T03:33:11.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.380+0000 7f000ce00640 1 -- start start 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.382+0000 7f000ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f000806b6e0 0x7f00080ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.382+0000 7f000ce00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f00080b2700 0x7f00080b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.382+0000 7f000ce00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00080715e0 con 0x7f00081659c0 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.382+0000 7f000ce00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00080711c0 con 0x7f000806b6e0 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.382+0000 7f000ce00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000806baf0 con 0x7f00080b2700 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0005400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f00081659c0 0x7f0008165dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45764/0 (socket says 172.21.15.55:45764) 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0005400640 1 -- 172.21.15.55:0/2380784635 learned_addr learned my addr 172.21.15.55:0/2380784635 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:11.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0005e00640 1 --2- 172.21.15.55:0/2380784635 >> v2:172.21.15.55:3300/0 conn(0x7f000806b6e0 0x7f00080ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0006800640 1 --2- 172.21.15.55:0/2380784635 >> v2:172.21.15.162:3300/0 conn(0x7f00080b2700 0x7f00080b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0004a00640 1 -- 172.21.15.55:0/2380784635 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146359896 0 0) 0x7efff4001110 con 0x7f00081659c0 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.383+0000 7f0004a00640 1 -- 172.21.15.55:0/2380784635 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effec003520 con 0x7f00081659c0 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.384+0000 7f0005e00640 1 -- 172.21.15.55:0/2380784635 >> v2:172.21.15.162:3300/0 conn(0x7f00080b2700 msgr2=0x7f00080b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.384+0000 7f0005e00640 1 --2- 172.21.15.55:0/2380784635 >> v2:172.21.15.162:3300/0 conn(0x7f00080b2700 0x7f00080b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.384+0000 7f0005e00640 1 -- 172.21.15.55:0/2380784635 >> v1:172.21.15.55:6789/0 conn(0x7f00081659c0 legacy=0x7f0008165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.384+0000 7f0005e00640 1 -- 172.21.15.55:0/2380784635 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00081667f0 con 0x7f000806b6e0 2024-05-10T03:33:11.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.385+0000 7f0005e00640 1 --2- 172.21.15.55:0/2380784635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f000806b6e0 0x7f00080ce3b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f00080cd1d0 tx=0x7efff802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc53a4b93d9331d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.385+0000 7f0004a00640 1 -- 172.21.15.55:0/2380784635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8007740 con 0x7f000806b6e0 2024-05-10T03:33:11.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.385+0000 7f0004a00640 1 -- 172.21.15.55:0/2380784635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff802d050 con 0x7f000806b6e0 2024-05-10T03:33:11.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.385+0000 7f0004a00640 1 -- 172.21.15.55:0/2380784635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8007ac0 con 0x7f000806b6e0 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 -- 172.21.15.55:0/2380784635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f000806b6e0 msgr2=0x7f00080ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 --2- 172.21.15.55:0/2380784635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f000806b6e0 0x7f00080ce3b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f00080cd1d0 tx=0x7efff802a5f0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 -- 172.21.15.55:0/2380784635 shutdown_connections 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 --2- 172.21.15.55:0/2380784635 >> v2:172.21.15.162:3300/0 conn(0x7f00080b2700 0x7f00080b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 --2- 172.21.15.55:0/2380784635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f000806b6e0 0x7f00080ce3b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 -- 172.21.15.55:0/2380784635 >> 172.21.15.55:0/2380784635 conn(0x7f00080cbb90 msgr2=0x7f00080cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:11.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 -- 172.21.15.55:0/2380784635 shutdown_connections 2024-05-10T03:33:11.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.386+0000 7f000ce00640 1 -- 172.21.15.55:0/2380784635 wait complete. 2024-05-10T03:33:11.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.389+0000 7f000ce00640 1 Processor -- start 2024-05-10T03:33:11.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.389+0000 7f000ce00640 1 -- start start 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000806b6e0 0x7f0008158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 0x7f00081ec110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0008159d90 con 0x7f00080b2700 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00081ed450 con 0x7f00081659c0 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.390+0000 7f000ce00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00081592a0 con 0x7f000806b6e0 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54026/0 (socket says 172.21.15.55:54026) 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 -- 172.21.15.55:0/1492391482 learned_addr learned my addr 172.21.15.55:0/1492391482 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:11.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005e00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000806b6e0 0x7f0008158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0006800640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 0x7f00081ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 -- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000806b6e0 msgr2=0x7f0008158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000806b6e0 0x7f0008158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 -- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 msgr2=0x7f00081ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 0x7f00081ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0005400640 1 -- 172.21.15.55:0/1492391482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff8009a70 con 0x7f00080b2700 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.391+0000 7f0006800640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 0x7f00081ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7f0005400640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efff400c420 tx=0x7efff400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4005ce0 con 0x7f00080b2700 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4005e40 con 0x7f00080b2700 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40072f0 con 0x7f00080b2700 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00081ebc80 con 0x7f00080b2700 2024-05-10T03:33:11.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.392+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00081594f0 con 0x7f00080b2700 2024-05-10T03:33:11.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.393+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00081eba50 con 0x7f00080b2700 2024-05-10T03:33:11.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.394+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efff4007450 con 0x7f00080b2700 2024-05-10T03:33:11.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.395+0000 7efffe000640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 0x7effe806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.395+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efff4088920 con 0x7f00080b2700 2024-05-10T03:33:11.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.396+0000 7f0005e00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 0x7effe806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.397+0000 7f0005e00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 0x7effe806d110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efff8034000 tx=0x7efff8033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.400+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f00081eba50 con 0x7f00080b2700 2024-05-10T03:33:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.603+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00080715e0 con 0x7f00080b2700 2024-05-10T03:33:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.606+0000 7efffe000640 1 -- 172.21.15.55:0/1492391482 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f00080715e0 con 0x7f00080b2700 2024-05-10T03:33:11.610 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:11.610 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 msgr2=0x7effe806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 0x7effe806d110 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efff8034000 tx=0x7efff8033040 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 msgr2=0x7f0008154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efff400c420 tx=0x7efff400c8e0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 shutdown_connections 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7effe806ac00 0x7effe806d110 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00081659c0 0x7f00081ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00080b2700 0x7f0008154da0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 --2- 172.21.15.55:0/1492391482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000806b6e0 0x7f0008158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 >> 172.21.15.55:0/1492391482 conn(0x7f00080cbb90 msgr2=0x7f000815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.609+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 shutdown_connections 2024-05-10T03:33:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.610+0000 7f000ce00640 1 -- 172.21.15.55:0/1492391482 wait complete. 2024-05-10T03:33:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:54.741096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:55.615018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:11.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:11.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.928+0000 7f5d43000640 1 Processor -- start 2024-05-10T03:33:11.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.928+0000 7f5d43000640 1 -- start start 2024-05-10T03:33:11.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d43000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d43000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5d3c0b66f0 0x7f5d3c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d43000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0d2b70 con 0x7f5d3c0b2320 2024-05-10T03:33:11.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d43000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c0b4500 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d43000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0d2e50 con 0x7f5d3c0d2670 2024-05-10T03:33:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d42600640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5d3c0b2320 0x7f5d3c1613a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45776/0 (socket says 172.21.15.55:45776) 2024-05-10T03:33:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d42600640 1 -- 172.21.15.55:0/3474456022 learned_addr learned my addr 172.21.15.55:0/3474456022 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.931+0000 7f5d41200640 1 --2- 172.21.15.55:0/3474456022 >> v2:172.21.15.55:3300/0 conn(0x7f5d3c0b66f0 0x7f5d3c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41200640 1 -- 172.21.15.55:0/3474456022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 msgr2=0x7f5d3c15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41200640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41200640 1 -- 172.21.15.55:0/3474456022 >> v1:172.21.15.55:6789/0 conn(0x7f5d3c0b2320 legacy=0x7f5d3c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41200640 1 -- 172.21.15.55:0/3474456022 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0b2a10 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.932+0000 7f5d3be00640 1 -- 172.21.15.55:0/3474456022 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980804923 0 0) 0x7f5d3c0d2b70 con 0x7f5d3c0b2320 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.933+0000 7f5d41200640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cdfe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d280097a0 tx=0x7f5d2802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509f9d90fbc61836 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.934+0000 7f5d3be00640 1 -- 172.21.15.55:0/3474456022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28032440 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.934+0000 7f5d3be00640 1 -- 172.21.15.55:0/3474456022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d280325a0 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.934+0000 7f5d3be00640 1 -- 172.21.15.55:0/3474456022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28032930 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.934+0000 7f5d43000640 1 -- 172.21.15.55:0/3474456022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 msgr2=0x7f5d3c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.934+0000 7f5d43000640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cdfe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d280097a0 tx=0x7f5d2802a180 comp rx=0 tx=0).stop 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 -- 172.21.15.55:0/3474456022 shutdown_connections 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cdfe0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 --2- 172.21.15.55:0/3474456022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 -- 172.21.15.55:0/3474456022 >> 172.21.15.55:0/3474456022 conn(0x7f5d3c0d0350 msgr2=0x7f5d3c0d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 -- 172.21.15.55:0/3474456022 shutdown_connections 2024-05-10T03:33:11.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.935+0000 7f5d43000640 1 -- 172.21.15.55:0/3474456022 wait complete. 2024-05-10T03:33:11.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.938+0000 7f5d43000640 1 Processor -- start 2024-05-10T03:33:11.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.938+0000 7f5d43000640 1 -- start start 2024-05-10T03:33:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 0x7f5d3c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c15f640 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c151060 con 0x7f5d3c0b2320 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.940+0000 7f5d43000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c15e770 con 0x7f5d3c0d2670 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54042/0 (socket says 172.21.15.55:54042) 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 -- 172.21.15.55:0/3482582011 learned_addr learned my addr 172.21.15.55:0/3482582011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 0x7f5d3c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 -- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 msgr2=0x7f5d3c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d42600640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 -- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 msgr2=0x7f5d3c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 0x7f5d3c15de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41200640 1 -- 172.21.15.55:0/3482582011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28009400 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.941+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 0x7f5d3c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d41200640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098c0 tx=0x7f5d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28032440 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28006e10 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28030ba0 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0b8c10 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.942+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c15eca0 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.943+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c14fb70 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.945+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d28039040 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.946+0000 7f5d39600640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 0x7f5d1c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.946+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5d280a9d90 con 0x7f5d3c0b66f0 2024-05-10T03:33:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.947+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 0x7f5d1c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.948+0000 7f5d41c00640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 0x7f5d1c06d110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006fd0 tx=0x7f5d2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:11.951+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d28074d00 con 0x7f5d3c0b66f0 2024-05-10T03:33:12.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.153+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d3c04e290 con 0x7f5d3c0b66f0 2024-05-10T03:33:12.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.154+0000 7f5d39600640 1 -- 172.21.15.55:0/3482582011 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d280370a0 con 0x7f5d3c0b66f0 2024-05-10T03:33:12.157 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:12.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 msgr2=0x7f5d1c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 0x7f5d1c06d110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c006fd0 tx=0x7f5d2c008040 comp rx=0 tx=0).stop 2024-05-10T03:33:12.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 msgr2=0x7f5d3c0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5d280098c0 tx=0x7f5d280046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 shutdown_connections 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d3c0d2670 0x7f5d3c158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d3c0b66f0 0x7f5d3c0cc6a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d1c06ac00 0x7f5d1c06d110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 --2- 172.21.15.55:0/3482582011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d3c0b2320 0x7f5d3c15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 >> 172.21.15.55:0/3482582011 conn(0x7f5d3c0d0350 msgr2=0x7f5d3c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.156+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 shutdown_connections 2024-05-10T03:33:12.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.157+0000 7f5d43000640 1 -- 172.21.15.55:0/3482582011 wait complete. 2024-05-10T03:33:12.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.464+0000 7f7c15e00640 1 Processor -- start 2024-05-10T03:33:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.465+0000 7f7c15e00640 1 -- start start 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7c100cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c15e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1015b5e0 con 0x7f7c10140900 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c15e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c100b4380 con 0x7f7c1006b830 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c15e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1015acf0 con 0x7f7c100cc3a0 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c07e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7c10140900 0x7f7c1006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45780/0 (socket says 172.21.15.55:45780) 2024-05-10T03:33:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c07e00640 1 -- 172.21.15.55:0/1266736603 learned_addr learned my addr 172.21.15.55:0/1266736603 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.467+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7c100cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024414310 0 0) 0x7f7c1015b5e0 con 0x7f7c10140900 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf4005730 con 0x7f7c10140900 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1649049421 0 0) 0x7f7bf4005730 con 0x7f7c10140900 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 msgr2=0x7f7c100cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7c100cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 >> v1:172.21.15.162:6789/0 conn(0x7f7c1006b830 legacy=0x7f7c1015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1015af90 con 0x7f7c10140900 2024-05-10T03:33:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7bf8001670 con 0x7f7c10140900 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 >> v1:172.21.15.55:6789/0 conn(0x7f7c10140900 legacy=0x7f7c1006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bf4005bb0 0x7f7bf4006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bf40069d0 0x7f7bf4006de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7bf40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1015acf0 con 0x7f7bf4005bb0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf400dcc0 con 0x7f7bf40069d0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.468+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf400de80 con 0x7f7c100cc3a0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bf4005bb0 0x7f7bf4006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c07e00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bf40069d0 0x7f7bf4006de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0f400640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7bf40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 -- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 msgr2=0x7f7bf40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7bf40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 -- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bf40069d0 msgr2=0x7f7bf4006de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bf40069d0 0x7f7bf4006de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0ea00640 1 -- 172.21.15.55:0/1266736603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4005730 con 0x7f7bf4005bb0 2024-05-10T03:33:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c07e00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bf40069d0 0x7f7bf4006de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.469+0000 7f7c0f400640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c100cc3a0 0x7f7bf40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.470+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bf4005bb0 0x7f7bf4006020 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009a90 tx=0x7f7bf4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33beb2d28aa220b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.470+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00031400 con 0x7f7bf4005bb0 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.470+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0002d050 con 0x7f7bf4005bb0 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.470+0000 7f7c0e000640 1 -- 172.21.15.55:0/1266736603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00031780 con 0x7f7bf4005bb0 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 -- 172.21.15.55:0/1266736603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bf4005bb0 msgr2=0x7f7bf4006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 --2- 172.21.15.55:0/1266736603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bf4005bb0 0x7f7bf4006020 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009a90 tx=0x7f7bf4007a00 comp rx=0 tx=0).stop 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c0f400640 1 -- 172.21.15.55:0/1266736603 reap_dead start 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 -- 172.21.15.55:0/1266736603 shutdown_connections 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 -- 172.21.15.55:0/1266736603 >> 172.21.15.55:0/1266736603 conn(0x7f7c1006c120 msgr2=0x7f7c1006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 -- 172.21.15.55:0/1266736603 shutdown_connections 2024-05-10T03:33:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.471+0000 7f7c15e00640 1 -- 172.21.15.55:0/1266736603 wait complete. 2024-05-10T03:33:12.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.474+0000 7f7c15e00640 1 Processor -- start 2024-05-10T03:33:12.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.474+0000 7f7c15e00640 1 -- start start 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.475+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 0x7f7c100c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1013c850 0x7f7c1006acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c15e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c15e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101f3010 con 0x7f7c1006b210 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c15e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100c8fc0 con 0x7f7c1013c850 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c15e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1006b620 con 0x7f7bf4005bb0 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54062/0 (socket says 172.21.15.55:54062) 2024-05-10T03:33:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 -- 172.21.15.55:0/3199758533 learned_addr learned my addr 172.21.15.55:0/3199758533 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c07e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1013c850 0x7f7c1006acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 0x7f7c100c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 -- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 msgr2=0x7f7c100c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 0x7f7c100c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 -- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1013c850 msgr2=0x7f7c1006acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1013c850 0x7f7c1006acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0f400640 1 -- 172.21.15.55:0/3199758533 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c101f3260 con 0x7f7c1006b210 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.476+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 0x7f7c100c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c0f400640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7c1015b550 tx=0x7f7c0800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08006d50 con 0x7f7c1006b210 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c08003970 con 0x7f7c1006b210 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08003d60 con 0x7f7c1006b210 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c101f3540 con 0x7f7c1006b210 2024-05-10T03:33:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.477+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c100c4940 con 0x7f7c1006b210 2024-05-10T03:33:12.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.478+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c10066f60 con 0x7f7c1006b210 2024-05-10T03:33:12.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.479+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c08005070 con 0x7f7c1006b210 2024-05-10T03:33:12.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.480+0000 7f7c07400640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 0x7f7bec06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.481+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7c08089310 con 0x7f7c1006b210 2024-05-10T03:33:12.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.481+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 0x7f7bec06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.482+0000 7f7c0ea00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 0x7f7bec06d140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c00033000 tx=0x7f7c00008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:12.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.485+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c08054280 con 0x7f7c1006b210 2024-05-10T03:33:12.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.679+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c10067220 con 0x7f7c1006b210 2024-05-10T03:33:12.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.680+0000 7f7c07400640 1 -- 172.21.15.55:0/3199758533 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c0801ce80 con 0x7f7c1006b210 2024-05-10T03:33:12.683 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.682+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 msgr2=0x7f7bec06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.682+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 0x7f7bec06d140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c00033000 tx=0x7f7c00008030 comp rx=0 tx=0).stop 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.682+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 msgr2=0x7f7c1013e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.682+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7c1015b550 tx=0x7f7c0800dc00 comp rx=0 tx=0).stop 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 shutdown_connections 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c1006b210 0x7f7c1013e300 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7bec06ac30 0x7f7bec06d140 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1013c850 0x7f7c1006acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 --2- 172.21.15.55:0/3199758533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bf4005bb0 0x7f7c100c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 >> 172.21.15.55:0/3199758533 conn(0x7f7c1006c120 msgr2=0x7f7c100702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 shutdown_connections 2024-05-10T03:33:12.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.683+0000 7f7c15e00640 1 -- 172.21.15.55:0/3199758533 wait complete. 2024-05-10T03:33:12.712 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:12.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.985+0000 7f1c22600640 1 Processor -- start 2024-05-10T03:33:12.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.986+0000 7f1c22600640 1 -- start start 2024-05-10T03:33:12.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c22600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c1c13e390 0x7f1c1c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c22600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1c1c06b830 0x7f1c1c13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:12.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c22600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1c13eae0 con 0x7f1c1c1414c0 2024-05-10T03:33:12.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c22600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c13e7a0 con 0x7f1c1c13e390 2024-05-10T03:33:12.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c22600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c15c020 con 0x7f1c1c06b830 2024-05-10T03:33:12.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c21200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c1c13e390 0x7f1c1c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c13e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1c1c1414c0 0x7f1c1c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43940/0 (socket says 172.21.15.55:43940) 2024-05-10T03:33:12.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.988+0000 7f1c13e00640 1 -- 172.21.15.55:0/3309233223 learned_addr learned my addr 172.21.15.55:0/3309233223 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:12.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c21c00640 1 --2- 172.21.15.55:0/3309233223 >> v2:172.21.15.162:3300/0 conn(0x7f1c1c06b830 0x7f1c1c13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c21200640 1 -- 172.21.15.55:0/3309233223 >> v2:172.21.15.162:3300/0 conn(0x7f1c1c06b830 msgr2=0x7f1c1c13ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c21200640 1 --2- 172.21.15.55:0/3309233223 >> v2:172.21.15.162:3300/0 conn(0x7f1c1c06b830 0x7f1c1c13ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c21200640 1 -- 172.21.15.55:0/3309233223 >> v1:172.21.15.55:6789/0 conn(0x7f1c1c1414c0 legacy=0x7f1c1c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c21200640 1 -- 172.21.15.55:0/3309233223 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c15c300 con 0x7f1c1c13e390 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.989+0000 7f1c1be00640 1 -- 172.21.15.55:0/3309233223 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596904444 0 0) 0x7f1c1c13eae0 con 0x7f1c1c1414c0 2024-05-10T03:33:12.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.990+0000 7f1c21200640 1 --2- 172.21.15.55:0/3309233223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c13e390 0x7f1c1c15b9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0cd9d0 tx=0x7f1c0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefc9f7c7aa6a72d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:12.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.990+0000 7f1c1be00640 1 -- 172.21.15.55:0/3309233223 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c032440 con 0x7f1c1c13e390 2024-05-10T03:33:12.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.990+0000 7f1c1be00640 1 -- 172.21.15.55:0/3309233223 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0325a0 con 0x7f1c1c13e390 2024-05-10T03:33:12.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.990+0000 7f1c1be00640 1 -- 172.21.15.55:0/3309233223 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c032930 con 0x7f1c1c13e390 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 -- 172.21.15.55:0/3309233223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c13e390 msgr2=0x7f1c1c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 --2- 172.21.15.55:0/3309233223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c13e390 0x7f1c1c15b9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0cd9d0 tx=0x7f1c0c02a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 -- 172.21.15.55:0/3309233223 shutdown_connections 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 --2- 172.21.15.55:0/3309233223 >> v2:172.21.15.162:3300/0 conn(0x7f1c1c06b830 0x7f1c1c13ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 --2- 172.21.15.55:0/3309233223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c13e390 0x7f1c1c15b9d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.991+0000 7f1c22600640 1 -- 172.21.15.55:0/3309233223 >> 172.21.15.55:0/3309233223 conn(0x7f1c1c06c730 msgr2=0x7f1c1c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.992+0000 7f1c22600640 1 -- 172.21.15.55:0/3309233223 shutdown_connections 2024-05-10T03:33:12.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.992+0000 7f1c22600640 1 -- 172.21.15.55:0/3309233223 wait complete. 2024-05-10T03:33:12.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.994+0000 7f1c22600640 1 Processor -- start 2024-05-10T03:33:12.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.994+0000 7f1c22600640 1 -- start start 2024-05-10T03:33:12.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 0x7f1c1c15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 0x7f1c1c130380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c12e200 con 0x7f1c1c070e40 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c12d930 con 0x7f1c1c1414c0 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c22600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0ab250 con 0x7f1c1c13e390 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c21c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c21c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41794/0 (socket says 172.21.15.55:41794) 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c21c00640 1 -- 172.21.15.55:0/1275042840 learned_addr learned my addr 172.21.15.55:0/1275042840 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c21200640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 0x7f1c1c15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.996+0000 7f1c13e00640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 0x7f1c1c130380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 -- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 msgr2=0x7f1c1c15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 0x7f1c1c15a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 -- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 msgr2=0x7f1c1c130380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 0x7f1c1c130380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 -- 172.21.15.55:0/1275042840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009400 con 0x7f1c1c070e40 2024-05-10T03:33:13.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21200640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 0x7f1c1c15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:13.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c13e00640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 0x7f1c1c130380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:13.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c21c00640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007e90 tx=0x7f1c1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14014420 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c14014580 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14014930 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c141a50 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.997+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c0adfc0 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:12.999+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c15a650 con 0x7f1c1c070e40 2024-05-10T03:33:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.000+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1c14014a90 con 0x7f1c1c070e40 2024-05-10T03:33:13.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.001+0000 7f1c19600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 0x7f1bf806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.001+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1c14089e80 con 0x7f1c1c070e40 2024-05-10T03:33:13.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.002+0000 7f1c21200640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 0x7f1bf806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.003+0000 7f1c21200640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 0x7f1bf806d140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c034000 tx=0x7f1c0c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:13.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.006+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1c14054df0 con 0x7f1c1c070e40 2024-05-10T03:33:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.205+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c1c12e340 con 0x7f1c1c070e40 2024-05-10T03:33:13.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.205+0000 7f1c19600640 1 -- 172.21.15.55:0/1275042840 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c1c12e340 con 0x7f1c1c070e40 2024-05-10T03:33:13.209 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.208+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 msgr2=0x7f1bf806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.208+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 0x7f1bf806d140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c034000 tx=0x7f1c0c02e000 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.208+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 msgr2=0x7f1c1c071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.208+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1c14007e90 tx=0x7f1c1400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 shutdown_connections 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c070e40 0x7f1c1c071250 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1bf806ac30 0x7f1bf806d140 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c1c1414c0 0x7f1c1c130380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 --2- 172.21.15.55:0/1275042840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c1c13e390 0x7f1c1c15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 >> 172.21.15.55:0/1275042840 conn(0x7f1c1c06c730 msgr2=0x7f1c1c151ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 shutdown_connections 2024-05-10T03:33:13.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.209+0000 7f1c22600640 1 -- 172.21.15.55:0/1275042840 wait complete. 2024-05-10T03:33:13.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:33:13.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.513+0000 7f686f200640 1 Processor -- start 2024-05-10T03:33:13.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.513+0000 7f686f200640 1 -- start start 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.515+0000 7f686f200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f687015b9d0 0x7f687006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.515+0000 7f686f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f687006b660 0x7f687013ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687015bde0 con 0x7f687013e390 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687013e7a0 con 0x7f687006b660 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686f200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687013eae0 con 0x7f687015b9d0 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686de00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f687013e390 0x7f68701414c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43944/0 (socket says 172.21.15.55:43944) 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686de00640 1 -- 172.21.15.55:0/59060182 learned_addr learned my addr 172.21.15.55:0/59060182 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:13.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686e800640 1 --2- 172.21.15.55:0/59060182 >> v2:172.21.15.55:3300/0 conn(0x7f687006b660 0x7f687013ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686d400640 1 --2- 172.21.15.55:0/59060182 >> v2:172.21.15.162:3300/0 conn(0x7f687015b9d0 0x7f687006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686e800640 1 -- 172.21.15.55:0/59060182 >> v2:172.21.15.162:3300/0 conn(0x7f687015b9d0 msgr2=0x7f687006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686e800640 1 --2- 172.21.15.55:0/59060182 >> v2:172.21.15.162:3300/0 conn(0x7f687015b9d0 0x7f687006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686ca00640 1 -- 172.21.15.55:0/59060182 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372218157 0 0) 0x7f6860001110 con 0x7f687013e390 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.516+0000 7f686e800640 1 -- 172.21.15.55:0/59060182 >> v1:172.21.15.55:6789/0 conn(0x7f687013e390 legacy=0x7f68701414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.517+0000 7f686e800640 1 -- 172.21.15.55:0/59060182 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687006ba70 con 0x7f687006b660 2024-05-10T03:33:13.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.518+0000 7f686e800640 1 --2- 172.21.15.55:0/59060182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687006b660 0x7f687013ef70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f68700ae8f0 tx=0x7f686802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4b5ee2dd5a6807 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:13.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.518+0000 7f686ca00640 1 -- 172.21.15.55:0/59060182 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686802ae60 con 0x7f687006b660 2024-05-10T03:33:13.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.518+0000 7f686ca00640 1 -- 172.21.15.55:0/59060182 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868032440 con 0x7f687006b660 2024-05-10T03:33:13.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.518+0000 7f686ca00640 1 -- 172.21.15.55:0/59060182 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680327d0 con 0x7f687006b660 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 -- 172.21.15.55:0/59060182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687006b660 msgr2=0x7f687013ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 --2- 172.21.15.55:0/59060182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687006b660 0x7f687013ef70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f68700ae8f0 tx=0x7f686802a110 comp rx=0 tx=0).stop 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 -- 172.21.15.55:0/59060182 shutdown_connections 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 --2- 172.21.15.55:0/59060182 >> v2:172.21.15.162:3300/0 conn(0x7f687015b9d0 0x7f687006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 --2- 172.21.15.55:0/59060182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687006b660 0x7f687013ef70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 -- 172.21.15.55:0/59060182 >> 172.21.15.55:0/59060182 conn(0x7f687006c730 msgr2=0x7f68700b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 -- 172.21.15.55:0/59060182 shutdown_connections 2024-05-10T03:33:13.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.519+0000 7f686f200640 1 -- 172.21.15.55:0/59060182 wait complete. 2024-05-10T03:33:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.521+0000 7f686f200640 1 Processor -- start 2024-05-10T03:33:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.521+0000 7f686f200640 1 -- start start 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f687006b660 0x7f687013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 0x7f6870154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870157d70 con 0x7f687015b9d0 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6870154da0 con 0x7f687013e390 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6870155c00 con 0x7f687006b660 2024-05-10T03:33:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41804/0 (socket says 172.21.15.55:41804) 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 -- 172.21.15.55:0/2348062882 learned_addr learned my addr 172.21.15.55:0/2348062882 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686d400640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 0x7f6870154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686de00640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f687006b660 0x7f687013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 -- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f687006b660 msgr2=0x7f687013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f687006b660 0x7f687013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 -- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 msgr2=0x7f6870154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 0x7f6870154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686e800640 1 -- 172.21.15.55:0/2348062882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868009400 con 0x7f687015b9d0 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.523+0000 7f686d400640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 0x7f6870154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:13.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f686e800640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6868009890 tx=0x7f68680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868030b00 con 0x7f687015b9d0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868030c80 con 0x7f687015b9d0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686803d440 con 0x7f687015b9d0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6870154ff0 con 0x7f687015b9d0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.524+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68701555b0 con 0x7f687015b9d0 2024-05-10T03:33:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.525+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6870156750 con 0x7f687015b9d0 2024-05-10T03:33:13.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.526+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6868039040 con 0x7f687015b9d0 2024-05-10T03:33:13.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.527+0000 7f6866000640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 0x7f685006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:13.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.527+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f68680aa270 con 0x7f687015b9d0 2024-05-10T03:33:13.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.528+0000 7f686de00640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 0x7f685006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:13.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.529+0000 7f686de00640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 0x7f685006d110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6860000f60 tx=0x7f6860009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:13.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.532+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6870156750 con 0x7f687015b9d0 2024-05-10T03:33:13.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.742+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6870156750 con 0x7f687015b9d0 2024-05-10T03:33:13.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.742+0000 7f6866000640 1 -- 172.21.15.55:0/2348062882 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6870156750 con 0x7f687015b9d0 2024-05-10T03:33:13.745 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:33:13.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 msgr2=0x7f685006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 0x7f685006d110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6860000f60 tx=0x7f6860009040 comp rx=0 tx=0).stop 2024-05-10T03:33:13.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 msgr2=0x7f6870158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6868009890 tx=0x7f68680046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 shutdown_connections 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f687015b9d0 0x7f6870158ca0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f685006ac00 0x7f685006d110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f687013e390 0x7f6870154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 --2- 172.21.15.55:0/2348062882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f687006b660 0x7f687013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 >> 172.21.15.55:0/2348062882 conn(0x7f687006c730 msgr2=0x7f687015d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 shutdown_connections 2024-05-10T03:33:13.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:13.745+0000 7f686f200640 1 -- 172.21.15.55:0/2348062882 wait complete. 2024-05-10T03:33:13.777 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:33:13.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:13.777 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:13.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:33:13.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:13.879 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:33:13.931 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:13.931 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:13.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:33:13.991 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:13.991 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:14.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:33:14.047 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.337+0000 7fbd50c00640 1 Processor -- start 2024-05-10T03:33:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.337+0000 7fbd50c00640 1 -- start start 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.339+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd4c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd50c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c06ad20 con 0x7fbd4c0b2140 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd50c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c141f50 con 0x7fbd4c0b2550 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd50c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c06a110 con 0x7fbd4c15fe50 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd49400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbd4c0b2140 0x7fbd4c142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43948/0 (socket says 172.21.15.55:43948) 2024-05-10T03:33:14.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd49400640 1 -- 172.21.15.55:0/367137938 learned_addr learned my addr 172.21.15.55:0/367137938 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd49e00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd4c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934281592 0 0) 0x7fbd4c06ad20 con 0x7fbd4c0b2140 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.340+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd2c003520 con 0x7fbd4c0b2140 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 494810087 0 0) 0x7fbd2c003520 con 0x7fbd4c0b2140 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 msgr2=0x7fbd4c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd4c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 >> v1:172.21.15.162:6789/0 conn(0x7fbd4c0b2550 legacy=0x7fbd4c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c06a3f0 con 0x7fbd4c0b2140 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbd340019f0 con 0x7fbd4c0b2140 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 >> v1:172.21.15.55:6789/0 conn(0x7fbd4c0b2140 legacy=0x7fbd4c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd2c0039a0 0x7fbd2c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd2c0047c0 0x7fbd2c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd2c005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c06a110 con 0x7fbd2c0039a0 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c00bbd0 con 0x7fbd2c0047c0 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c00bd90 con 0x7fbd4c15fe50 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.341+0000 7fbd49e00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd2c0039a0 0x7fbd2c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd4a800640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd2c005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd49e00640 1 -- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 msgr2=0x7fbd2c005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd49e00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd2c005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd49e00640 1 -- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd2c0047c0 msgr2=0x7fbd2c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd49e00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd2c0047c0 0x7fbd2c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd49e00640 1 -- 172.21.15.55:0/367137938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c003520 con 0x7fbd2c0039a0 2024-05-10T03:33:14.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.342+0000 7fbd4a800640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd4c15fe50 0x7fbd2c005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:14.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.343+0000 7fbd49e00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd2c0039a0 0x7fbd2c003df0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00ba00 tx=0x7fbd2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bb81213149382b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.343+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c02ada0 con 0x7fbd2c0039a0 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.343+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c02e070 con 0x7fbd2c0039a0 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.343+0000 7fbd48a00640 1 -- 172.21.15.55:0/367137938 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c034620 con 0x7fbd2c0039a0 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 -- 172.21.15.55:0/367137938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd2c0039a0 msgr2=0x7fbd2c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 --2- 172.21.15.55:0/367137938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd2c0039a0 0x7fbd2c003df0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00ba00 tx=0x7fbd2c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd4a800640 1 -- 172.21.15.55:0/367137938 reap_dead start 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 -- 172.21.15.55:0/367137938 shutdown_connections 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 -- 172.21.15.55:0/367137938 >> 172.21.15.55:0/367137938 conn(0x7fbd4c0cbb90 msgr2=0x7fbd4c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 -- 172.21.15.55:0/367137938 shutdown_connections 2024-05-10T03:33:14.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.344+0000 7fbd50c00640 1 -- 172.21.15.55:0/367137938 wait complete. 2024-05-10T03:33:14.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.347+0000 7fbd50c00640 1 Processor -- start 2024-05-10T03:33:14.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.347+0000 7fbd50c00640 1 -- start start 2024-05-10T03:33:14.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 0x7fbd4c13d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd4c070050 0x7fbd4c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1392a0 con 0x7fbd4c071a80 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c15e350 con 0x7fbd4c070050 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.348+0000 7fbd50c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0acf60 con 0x7fbd2c0039a0 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41826/0 (socket says 172.21.15.55:41826) 2024-05-10T03:33:14.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 -- 172.21.15.55:0/379900202 learned_addr learned my addr 172.21.15.55:0/379900202 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:14.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49e00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 0x7fbd4c13d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 -- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 msgr2=0x7fbd4c13d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 0x7fbd4c13d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd4a800640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd4c070050 0x7fbd4c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 -- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd4c070050 msgr2=0x7fbd4c070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd4c070050 0x7fbd4c070460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49400640 1 -- 172.21.15.55:0/379900202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c0acc10 con 0x7fbd4c071a80 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.349+0000 7fbd49e00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 0x7fbd4c13d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd49400640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ca50 tx=0x7fbd3400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400a740 con 0x7fbd4c071a80 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400a8c0 con 0x7fbd4c071a80 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400ac70 con 0x7fbd4c071a80 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c15e5a0 con 0x7fbd4c071a80 2024-05-10T03:33:14.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.350+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c0ae470 con 0x7fbd4c071a80 2024-05-10T03:33:14.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.351+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c15eac0 con 0x7fbd4c071a80 2024-05-10T03:33:14.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.353+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd340070d0 con 0x7fbd4c071a80 2024-05-10T03:33:14.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.354+0000 7fbd42000640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 0x7fbd2806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.354+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbd34088ea0 con 0x7fbd4c071a80 2024-05-10T03:33:14.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.354+0000 7fbd49e00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 0x7fbd2806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.356+0000 7fbd49e00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 0x7fbd2806d110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0057a0 tx=0x7fbd3c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.357+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbd34053f90 con 0x7fbd4c071a80 2024-05-10T03:33:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.595+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd4c066f60 con 0x7fbd4c071a80 2024-05-10T03:33:14.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.597+0000 7fbd42000640 1 -- 172.21.15.55:0/379900202 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fbd34018070 con 0x7fbd4c071a80 2024-05-10T03:33:14.601 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:14.601 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 msgr2=0x7fbd2806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 0x7fbd2806d110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0057a0 tx=0x7fbd3c030000 comp rx=0 tx=0).stop 2024-05-10T03:33:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 msgr2=0x7fbd4c13ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ca50 tx=0x7fbd3400cf10 comp rx=0 tx=0).stop 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 shutdown_connections 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbd2806ac00 0x7fbd2806d110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd4c070050 0x7fbd4c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd4c071a80 0x7fbd4c13ddb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 --2- 172.21.15.55:0/379900202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd2c0039a0 0x7fbd4c13d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 >> 172.21.15.55:0/379900202 conn(0x7fbd4c0cbb90 msgr2=0x7fbd4c0cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.600+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 shutdown_connections 2024-05-10T03:33:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.601+0000 7fbd50c00640 1 -- 172.21.15.55:0/379900202 wait complete. 2024-05-10T03:33:14.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:14.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.920+0000 7fe321a00640 1 Processor -- start 2024-05-10T03:33:14.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.920+0000 7fe321a00640 1 -- start start 2024-05-10T03:33:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.922+0000 7fe321a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c144710 0x7fe31c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.922+0000 7fe321a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe31c06d7e0 0x7fe31c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.922+0000 7fe321a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c06d350 con 0x7fe31c0b2fc0 2024-05-10T03:33:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.922+0000 7fe321a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c06c740 con 0x7fe31c06d7e0 2024-05-10T03:33:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.922+0000 7fe321a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c06ca20 con 0x7fe31c144710 2024-05-10T03:33:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.923+0000 7fe313e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe31c06d7e0 0x7fe31c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.923+0000 7fe313e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe31c06d7e0 0x7fe31c0b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41842/0 (socket says 172.21.15.55:41842) 2024-05-10T03:33:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.923+0000 7fe313e00640 1 -- 172.21.15.55:0/1529250525 learned_addr learned my addr 172.21.15.55:0/1529250525 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.923+0000 7fe31be00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c144710 0x7fe31c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.924+0000 7fe313e00640 1 -- 172.21.15.55:0/1529250525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c144710 msgr2=0x7fe31c144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.924+0000 7fe313e00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c144710 0x7fe31c144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.924+0000 7fe313e00640 1 -- 172.21.15.55:0/1529250525 >> v1:172.21.15.162:6789/0 conn(0x7fe31c0b2fc0 legacy=0x7fe31c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.924+0000 7fe313e00640 1 -- 172.21.15.55:0/1529250525 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c06cd00 con 0x7fe31c06d7e0 2024-05-10T03:33:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.925+0000 7fe313e00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c0b2a80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe304009870 tx=0x7fe30402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1beb875d9ac8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.925+0000 7fe31b400640 1 -- 172.21.15.55:0/1529250525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304032440 con 0x7fe31c06d7e0 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.925+0000 7fe31b400640 1 -- 172.21.15.55:0/1529250525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3040325a0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.925+0000 7fe31b400640 1 -- 172.21.15.55:0/1529250525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304032930 con 0x7fe31c06d7e0 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 -- 172.21.15.55:0/1529250525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 msgr2=0x7fe31c0b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c0b2a80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe304009870 tx=0x7fe30402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 -- 172.21.15.55:0/1529250525 shutdown_connections 2024-05-10T03:33:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c0b2a80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 --2- 172.21.15.55:0/1529250525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c144710 0x7fe31c144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.926+0000 7fe321a00640 1 -- 172.21.15.55:0/1529250525 >> 172.21.15.55:0/1529250525 conn(0x7fe31c06a110 msgr2=0x7fe31c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.927+0000 7fe321a00640 1 -- 172.21.15.55:0/1529250525 shutdown_connections 2024-05-10T03:33:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.927+0000 7fe321a00640 1 -- 172.21.15.55:0/1529250525 wait complete. 2024-05-10T03:33:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.930+0000 7fe321a00640 1 Processor -- start 2024-05-10T03:33:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.930+0000 7fe321a00640 1 -- start start 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c0b2fc0 0x7fe31c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c144710 0x7fe31c0ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c15dfd0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c15abc0 con 0x7fe31c144710 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe321a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c137630 con 0x7fe31c0b2fc0 2024-05-10T03:33:14.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe31be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe31be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41852/0 (socket says 172.21.15.55:41852) 2024-05-10T03:33:14.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.932+0000 7fe31be00640 1 -- 172.21.15.55:0/318553583 learned_addr learned my addr 172.21.15.55:0/318553583 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe313e00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c0b2fc0 0x7fe31c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 -- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c0b2fc0 msgr2=0x7fe31c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c0b2fc0 0x7fe31c15a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe321000640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c144710 0x7fe31c0ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 -- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c144710 msgr2=0x7fe31c0ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c144710 0x7fe31c0ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 -- 172.21.15.55:0/318553583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040094d0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe31be00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e840 tx=0x7fe30c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.933+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0094f0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.934+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c00c940 con 0x7fe31c06d7e0 2024-05-10T03:33:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.934+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00ccd0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.934+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c137910 con 0x7fe31c06d7e0 2024-05-10T03:33:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.934+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c137ed0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.935+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c066ce0 con 0x7fe31c06d7e0 2024-05-10T03:33:14.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.936+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe30c013470 con 0x7fe31c06d7e0 2024-05-10T03:33:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.937+0000 7fe318c00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 0x7fe2f806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:14.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.938+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe30c089640 con 0x7fe31c06d7e0 2024-05-10T03:33:14.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.938+0000 7fe313e00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 0x7fe2f806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:14.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.940+0000 7fe313e00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 0x7fe2f806d140 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe304034000 tx=0x7fe30402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:14.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:14.942+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe30c0545b0 con 0x7fe31c06d7e0 2024-05-10T03:33:15.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.146+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe31c04e290 con 0x7fe31c06d7e0 2024-05-10T03:33:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.148+0000 7fe318c00640 1 -- 172.21.15.55:0/318553583 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fe30c057bd0 con 0x7fe31c06d7e0 2024-05-10T03:33:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:15.152 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 msgr2=0x7fe2f806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 0x7fe2f806d140 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe304034000 tx=0x7fe30402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 msgr2=0x7fe31c071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e840 tx=0x7fe30c00ed00 comp rx=0 tx=0).stop 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 shutdown_connections 2024-05-10T03:33:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2f806ac30 0x7fe2f806d140 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe31c144710 0x7fe31c0ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe31c0b2fc0 0x7fe31c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 --2- 172.21.15.55:0/318553583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe31c06d7e0 0x7fe31c071af0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 >> 172.21.15.55:0/318553583 conn(0x7fe31c06a110 msgr2=0x7fe31c0aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 shutdown_connections 2024-05-10T03:33:15.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.151+0000 7fe321a00640 1 -- 172.21.15.55:0/318553583 wait complete. 2024-05-10T03:33:15.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:32:54.741096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:32:55.615018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:15.180 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:15.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.443+0000 7fe42be00640 1 Processor -- start 2024-05-10T03:33:15.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.444+0000 7fe42be00640 1 -- start start 2024-05-10T03:33:15.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.446+0000 7fe42be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe42c0ce7c0 0x7fe42c0709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.446+0000 7fe42be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe42c0b1290 0x7fe42c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.446+0000 7fe42be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c0cbc10 con 0x7fe42c0cb800 2024-05-10T03:33:15.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.446+0000 7fe42be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c15e640 con 0x7fe42c0b1290 2024-05-10T03:33:15.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.446+0000 7fe42be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0b3e20 con 0x7fe42c0ce7c0 2024-05-10T03:33:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe42c0b1290 0x7fe42c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe42c0b1290 0x7fe42c06b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41856/0 (socket says 172.21.15.55:41856) 2024-05-10T03:33:15.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 -- 172.21.15.55:0/2725612778 learned_addr learned my addr 172.21.15.55:0/2725612778 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42a000640 1 --2- 172.21.15.55:0/2725612778 >> v2:172.21.15.162:3300/0 conn(0x7fe42c0ce7c0 0x7fe42c0709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 -- 172.21.15.55:0/2725612778 >> v2:172.21.15.162:3300/0 conn(0x7fe42c0ce7c0 msgr2=0x7fe42c0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 --2- 172.21.15.55:0/2725612778 >> v2:172.21.15.162:3300/0 conn(0x7fe42c0ce7c0 0x7fe42c0709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 -- 172.21.15.55:0/2725612778 >> v1:172.21.15.162:6789/0 conn(0x7fe42c0cb800 legacy=0x7fe42c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:15.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.447+0000 7fe42b400640 1 -- 172.21.15.55:0/2725612778 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0cebd0 con 0x7fe42c0b1290 2024-05-10T03:33:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.448+0000 7fe42b400640 1 --2- 172.21.15.55:0/2725612778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0b1290 0x7fe42c06b120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe42c072130 tx=0x7fe42402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6cbe40f0cdb66e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.449+0000 7fe429600640 1 -- 172.21.15.55:0/2725612778 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424031600 con 0x7fe42c0b1290 2024-05-10T03:33:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.449+0000 7fe429600640 1 -- 172.21.15.55:0/2725612778 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42402d050 con 0x7fe42c0b1290 2024-05-10T03:33:15.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.449+0000 7fe429600640 1 -- 172.21.15.55:0/2725612778 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424031980 con 0x7fe42c0b1290 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.449+0000 7fe42be00640 1 -- 172.21.15.55:0/2725612778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0b1290 msgr2=0x7fe42c06b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.449+0000 7fe42be00640 1 --2- 172.21.15.55:0/2725612778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0b1290 0x7fe42c06b120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe42c072130 tx=0x7fe42402a6f0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 -- 172.21.15.55:0/2725612778 shutdown_connections 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 --2- 172.21.15.55:0/2725612778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0b1290 0x7fe42c06b120 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 --2- 172.21.15.55:0/2725612778 >> v2:172.21.15.162:3300/0 conn(0x7fe42c0ce7c0 0x7fe42c0709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 -- 172.21.15.55:0/2725612778 >> 172.21.15.55:0/2725612778 conn(0x7fe42c06c750 msgr2=0x7fe42c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 -- 172.21.15.55:0/2725612778 shutdown_connections 2024-05-10T03:33:15.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.450+0000 7fe42be00640 1 -- 172.21.15.55:0/2725612778 wait complete. 2024-05-10T03:33:15.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.452+0000 7fe42be00640 1 Processor -- start 2024-05-10T03:33:15.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.452+0000 7fe42be00640 1 -- start start 2024-05-10T03:33:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42c0b1290 0x7fe42c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe42c0ce7c0 0x7fe42c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c152960 con 0x7fe42c0cb800 2024-05-10T03:33:15.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c1f71a0 con 0x7fe42c0b1290 2024-05-10T03:33:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c151170 con 0x7fe42c0ce7c0 2024-05-10T03:33:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41870/0 (socket says 172.21.15.55:41870) 2024-05-10T03:33:15.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.454+0000 7fe42a000640 1 -- 172.21.15.55:0/1693030861 learned_addr learned my addr 172.21.15.55:0/1693030861 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42b400640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe42c0ce7c0 0x7fe42c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42aa00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42c0b1290 0x7fe42c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 -- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe42c0ce7c0 msgr2=0x7fe42c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe42c0ce7c0 0x7fe42c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 -- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42c0b1290 msgr2=0x7fe42c14ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42c0b1290 0x7fe42c14ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 -- 172.21.15.55:0/1693030861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009b30 con 0x7fe42c0cb800 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe42a000640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d5f0 tx=0x7fe41800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.455+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180125f0 con 0x7fe42c0cb800 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.456+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418012750 con 0x7fe42c0cb800 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.456+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418012b00 con 0x7fe42c0cb800 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.456+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c1f73f0 con 0x7fe42c0cb800 2024-05-10T03:33:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.456+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c1f79b0 con 0x7fe42c0cb800 2024-05-10T03:33:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.457+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c04e290 con 0x7fe42c0cb800 2024-05-10T03:33:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.458+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe418012c60 con 0x7fe42c0cb800 2024-05-10T03:33:15.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.459+0000 7fe422a00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 0x7fe40c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.459+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe418089a70 con 0x7fe42c0cb800 2024-05-10T03:33:15.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.460+0000 7fe42aa00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 0x7fe40c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.461+0000 7fe42aa00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 0x7fe40c06d110 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:15.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.463+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe4180549e0 con 0x7fe42c0cb800 2024-05-10T03:33:15.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.661+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe42c159ce0 con 0x7fe42c0cb800 2024-05-10T03:33:15.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.662+0000 7fe422a00640 1 -- 172.21.15.55:0/1693030861 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe418058000 con 0x7fe42c0cb800 2024-05-10T03:33:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 msgr2=0x7fe40c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 0x7fe40c06d110 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c008040 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 msgr2=0x7fe42c1530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d5f0 tx=0x7fe41800dab0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 shutdown_connections 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe42c0ce7c0 0x7fe42c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe42c0cb800 0x7fe42c1530f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe40c06ac00 0x7fe40c06d110 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 --2- 172.21.15.55:0/1693030861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42c0b1290 0x7fe42c14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 >> 172.21.15.55:0/1693030861 conn(0x7fe42c06c750 msgr2=0x7fe42c0706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 shutdown_connections 2024-05-10T03:33:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.664+0000 7fe42be00640 1 -- 172.21.15.55:0/1693030861 wait complete. 2024-05-10T03:33:15.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:15.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.980+0000 7f9657200640 1 Processor -- start 2024-05-10T03:33:15.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.980+0000 7f9657200640 1 -- start start 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.982+0000 7f9657200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96500719f0 0x7f965006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9657200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f965006b840 0x7f96500700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9657200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f9650070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9657200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965015fdf0 con 0x7f96500719f0 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9657200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965015f1e0 con 0x7f965006b840 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9657200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965015f4c0 con 0x7f9650071e00 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96500719f0 0x7f965006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96500719f0 0x7f965006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41872/0 (socket says 172.21.15.55:41872) 2024-05-10T03:33:15.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 -- 172.21.15.55:0/2567444364 learned_addr learned my addr 172.21.15.55:0/2567444364 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:15.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 -- 172.21.15.55:0/2567444364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 msgr2=0x7f9650070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:15.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 --2- 172.21.15.55:0/2567444364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f9650070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 -- 172.21.15.55:0/2567444364 >> v2:172.21.15.162:3300/0 conn(0x7f965006b840 msgr2=0x7f96500700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:15.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.983+0000 7f9655e00640 1 --2- 172.21.15.55:0/2567444364 >> v2:172.21.15.162:3300/0 conn(0x7f965006b840 0x7f96500700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.984+0000 7f9655e00640 1 -- 172.21.15.55:0/2567444364 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965015f7a0 con 0x7f96500719f0 2024-05-10T03:33:15.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.985+0000 7f9655e00640 1 --2- 172.21.15.55:0/2567444364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f965006b300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f965006ff90 tx=0x7f964802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b847ac0ef1c6338d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:15.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.985+0000 7f9654a00640 1 -- 172.21.15.55:0/2567444364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648031600 con 0x7f96500719f0 2024-05-10T03:33:15.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.985+0000 7f9654a00640 1 -- 172.21.15.55:0/2567444364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964802d050 con 0x7f96500719f0 2024-05-10T03:33:15.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.985+0000 7f9654a00640 1 -- 172.21.15.55:0/2567444364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648031980 con 0x7f96500719f0 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 -- 172.21.15.55:0/2567444364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 msgr2=0x7f965006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 --2- 172.21.15.55:0/2567444364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f965006b300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f965006ff90 tx=0x7f964802a600 comp rx=0 tx=0).stop 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 -- 172.21.15.55:0/2567444364 shutdown_connections 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 --2- 172.21.15.55:0/2567444364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f9650070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 --2- 172.21.15.55:0/2567444364 >> v2:172.21.15.162:3300/0 conn(0x7f965006b840 0x7f96500700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 --2- 172.21.15.55:0/2567444364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f965006b300 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 -- 172.21.15.55:0/2567444364 >> 172.21.15.55:0/2567444364 conn(0x7f965006c750 msgr2=0x7f96500b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 -- 172.21.15.55:0/2567444364 shutdown_connections 2024-05-10T03:33:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.986+0000 7f9657200640 1 -- 172.21.15.55:0/2567444364 wait complete. 2024-05-10T03:33:15.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.988+0000 7f9657200640 1 Processor -- start 2024-05-10T03:33:15.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.989+0000 7f9657200640 1 -- start start 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f965006b840 0x7f96500c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f96500ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650136280 con 0x7f96500719f0 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500c43f0 con 0x7f965006b840 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9657200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965015b7d0 con 0x7f9650071e00 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.990+0000 7f9655400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41882/0 (socket says 172.21.15.55:41882) 2024-05-10T03:33:15.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 -- 172.21.15.55:0/1837431861 learned_addr learned my addr 172.21.15.55:0/1837431861 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655e00640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f965006b840 0x7f96500c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9656800640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f96500ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 -- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 msgr2=0x7f96500ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f96500ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 -- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f965006b840 msgr2=0x7f96500c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f965006b840 0x7f96500c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:15.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.991+0000 7f9655400640 1 -- 172.21.15.55:0/1837431861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009b30 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9655400640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f964000b410 tx=0x7f964000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011830 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96400119b0 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96500c5250 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011d60 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.992+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96500c4920 con 0x7f96500719f0 2024-05-10T03:33:15.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.993+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96500c53e0 con 0x7f96500719f0 2024-05-10T03:33:15.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.994+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f964001a400 con 0x7f96500719f0 2024-05-10T03:33:15.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.995+0000 7f9646000640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 0x7f963406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:15.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.995+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9640089590 con 0x7f96500719f0 2024-05-10T03:33:15.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.996+0000 7f9655e00640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 0x7f963406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:15.997+0000 7f9655e00640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 0x7f963406d140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9648033000 tx=0x7f9648009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:16.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.000+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f96400545a0 con 0x7f96500719f0 2024-05-10T03:33:16.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.200+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f965004e290 con 0x7f96500719f0 2024-05-10T03:33:16.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.202+0000 7f9646000640 1 -- 172.21.15.55:0/1837431861 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9640015020 con 0x7f96500719f0 2024-05-10T03:33:16.205 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:16.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.204+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 msgr2=0x7f963406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.204+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 0x7f963406d140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9648033000 tx=0x7f9648009000 comp rx=0 tx=0).stop 2024-05-10T03:33:16.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.204+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 msgr2=0x7f96500b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.204+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f964000b410 tx=0x7f964000b8d0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 shutdown_connections 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9650071e00 0x7f96500ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96500719f0 0x7f96500b37c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f963406ac30 0x7f963406d140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 --2- 172.21.15.55:0/1837431861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f965006b840 0x7f96500c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 >> 172.21.15.55:0/1837431861 conn(0x7f965006c750 msgr2=0x7f965015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 shutdown_connections 2024-05-10T03:33:16.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.205+0000 7f9657200640 1 -- 172.21.15.55:0/1837431861 wait complete. 2024-05-10T03:33:16.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:16.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.516+0000 7fa96f200640 1 Processor -- start 2024-05-10T03:33:16.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.516+0000 7fa96f200640 1 -- start start 2024-05-10T03:33:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa970143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9700cdbe0 con 0x7fa97006b120 2024-05-10T03:33:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97015e8b0 con 0x7fa9701435d0 2024-05-10T03:33:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700ce140 con 0x7fa9700b2040 2024-05-10T03:33:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96e800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa97006b120 0x7fa97006b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43960/0 (socket says 172.21.15.55:43960) 2024-05-10T03:33:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.519+0000 7fa96e800640 1 -- 172.21.15.55:0/168300908 learned_addr learned my addr 172.21.15.55:0/168300908 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96de00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa970143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941830106 0 0) 0x7fa9700cdbe0 con 0x7fa97006b120 2024-05-10T03:33:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003520 con 0x7fa97006b120 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1877540348 0 0) 0x7fa954003520 con 0x7fa97006b120 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 msgr2=0x7fa970143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.520+0000 7fa96ca00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa970143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 >> v1:172.21.15.162:6789/0 conn(0x7fa9701435d0 legacy=0x7fa9700ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97015ede0 con 0x7fa97006b120 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa9680019f0 con 0x7fa97006b120 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 >> v1:172.21.15.55:6789/0 conn(0x7fa97006b120 legacy=0x7fa97006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9540039a0 0x7fa954003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa9540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa954004d00 0x7fa954005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700ce140 con 0x7fa9540039a0 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400bbb0 con 0x7fa954004d00 2024-05-10T03:33:16.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400bd70 con 0x7fa9700b2040 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96de00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9540039a0 0x7fa954003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96e800640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa954004d00 0x7fa954005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.521+0000 7fa96d400640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa9540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96de00640 1 -- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 msgr2=0x7fa9540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96de00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa9540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96de00640 1 -- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa954004d00 msgr2=0x7fa954005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96de00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa954004d00 0x7fa954005170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96de00640 1 -- 172.21.15.55:0/168300908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954003520 con 0x7fa9540039a0 2024-05-10T03:33:16.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.522+0000 7fa96d400640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9700b2040 0x7fa9540047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:16.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.523+0000 7fa96de00640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9540039a0 0x7fa954003e10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa960009ac0 tx=0x7fa9540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e0e2ecf5244641 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:16.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.523+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960031400 con 0x7fa9540039a0 2024-05-10T03:33:16.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.523+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96002d050 con 0x7fa9540039a0 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.523+0000 7fa96ca00640 1 -- 172.21.15.55:0/168300908 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960031780 con 0x7fa9540039a0 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 -- 172.21.15.55:0/168300908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9540039a0 msgr2=0x7fa954003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 --2- 172.21.15.55:0/168300908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9540039a0 0x7fa954003e10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa960009ac0 tx=0x7fa9540057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96e800640 1 -- 172.21.15.55:0/168300908 reap_dead start 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 -- 172.21.15.55:0/168300908 shutdown_connections 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 -- 172.21.15.55:0/168300908 >> 172.21.15.55:0/168300908 conn(0x7fa97006c730 msgr2=0x7fa9700b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 -- 172.21.15.55:0/168300908 shutdown_connections 2024-05-10T03:33:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.524+0000 7fa96f200640 1 -- 172.21.15.55:0/168300908 wait complete. 2024-05-10T03:33:16.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.527+0000 7fa96f200640 1 Processor -- start 2024-05-10T03:33:16.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.527+0000 7fa96f200640 1 -- start start 2024-05-10T03:33:16.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9540039a0 0x7fa97013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 0x7fa97013c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970159f50 con 0x7fa9701592d0 2024-05-10T03:33:16.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97013d420 con 0x7fa97013fbf0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.528+0000 7fa96f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700b0240 con 0x7fa9540039a0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41892/0 (socket says 172.21.15.55:41892) 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 -- 172.21.15.55:0/2178855699 learned_addr learned my addr 172.21.15.55:0/2178855699 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96de00640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9540039a0 0x7fa97013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96d400640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 0x7fa97013c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 -- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9540039a0 msgr2=0x7fa97013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9540039a0 0x7fa97013f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 -- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 msgr2=0x7fa97013c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 0x7fa97013c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96e800640 1 -- 172.21.15.55:0/2178855699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97013a1d0 con 0x7fa9701592d0 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.529+0000 7fa96d400640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 0x7fa97013c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:16.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa96e800640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b46e0 tx=0x7fa96800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800b8d0 con 0x7fa9701592d0 2024-05-10T03:33:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800ba30 con 0x7fa9701592d0 2024-05-10T03:33:16.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800bde0 con 0x7fa9701592d0 2024-05-10T03:33:16.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9700b0490 con 0x7fa9701592d0 2024-05-10T03:33:16.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.530+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa970139b80 con 0x7fa9701592d0 2024-05-10T03:33:16.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.531+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9700b1680 con 0x7fa9701592d0 2024-05-10T03:33:16.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.532+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa968007070 con 0x7fa9701592d0 2024-05-10T03:33:16.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.533+0000 7fa966000640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 0x7fa95006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:16.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.533+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa968089860 con 0x7fa9701592d0 2024-05-10T03:33:16.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.534+0000 7fa96de00640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 0x7fa95006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:16.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.535+0000 7fa96de00640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 0x7fa95006d140 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa960033000 tx=0x7fa960008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.538+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa968054860 con 0x7fa9701592d0 2024-05-10T03:33:16.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.742+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa97013a360 con 0x7fa9701592d0 2024-05-10T03:33:16.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.743+0000 7fa966000640 1 -- 172.21.15.55:0/2178855699 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa968057e80 con 0x7fa9701592d0 2024-05-10T03:33:16.746 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:16.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.745+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 msgr2=0x7fa95006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.745+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 0x7fa95006d140 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa960033000 tx=0x7fa960008030 comp rx=0 tx=0).stop 2024-05-10T03:33:16.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 msgr2=0x7fa97013c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b46e0 tx=0x7fa96800b0d0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 shutdown_connections 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9701592d0 0x7fa97013c730 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa95006ac30 0x7fa95006d140 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa97013fbf0 0x7fa97013c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 --2- 172.21.15.55:0/2178855699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9540039a0 0x7fa97013f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 >> 172.21.15.55:0/2178855699 conn(0x7fa97006c730 msgr2=0x7fa9700b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 shutdown_connections 2024-05-10T03:33:16.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:16.746+0000 7fa96f200640 1 -- 172.21.15.55:0/2178855699 wait complete. 2024-05-10T03:33:16.775 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:17.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.055+0000 7f3c5a200640 1 Processor -- start 2024-05-10T03:33:17.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.055+0000 7f3c5a200640 1 -- start start 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c5a200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c540b2170 0x7f3c5415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c5a200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c5a200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540d25a0 con 0x7f3c540b6680 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c5a200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540d2b20 con 0x7f3c540b2170 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c5a200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5415db20 con 0x7f3c541618b0 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c53e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c540b2170 0x7f3c5415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c53e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c540b2170 0x7f3c5415d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41904/0 (socket says 172.21.15.55:41904) 2024-05-10T03:33:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.058+0000 7f3c53e00640 1 -- 172.21.15.55:0/1970983211 learned_addr learned my addr 172.21.15.55:0/1970983211 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.059+0000 7f3c53e00640 1 -- 172.21.15.55:0/1970983211 >> v2:172.21.15.162:3300/0 conn(0x7f3c541618b0 msgr2=0x7f3c5406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.059+0000 7f3c53e00640 1 --2- 172.21.15.55:0/1970983211 >> v2:172.21.15.162:3300/0 conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.059+0000 7f3c53e00640 1 -- 172.21.15.55:0/1970983211 >> v1:172.21.15.162:6789/0 conn(0x7f3c540b6680 legacy=0x7f3c541611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.059+0000 7f3c53e00640 1 -- 172.21.15.55:0/1970983211 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540b28d0 con 0x7f3c540b2170 2024-05-10T03:33:17.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.060+0000 7f3c53e00640 1 --2- 172.21.15.55:0/1970983211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b2170 0x7f3c5415d5e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3c54162260 tx=0x7f3c4002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef034955c5a224a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.060+0000 7f3c53400640 1 -- 172.21.15.55:0/1970983211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40031600 con 0x7f3c540b2170 2024-05-10T03:33:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.060+0000 7f3c53400640 1 -- 172.21.15.55:0/1970983211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4002d050 con 0x7f3c540b2170 2024-05-10T03:33:17.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.060+0000 7f3c53400640 1 -- 172.21.15.55:0/1970983211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40031980 con 0x7f3c540b2170 2024-05-10T03:33:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 -- 172.21.15.55:0/1970983211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b2170 msgr2=0x7f3c5415d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1970983211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b2170 0x7f3c5415d5e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3c54162260 tx=0x7f3c4002a6f0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 -- 172.21.15.55:0/1970983211 shutdown_connections 2024-05-10T03:33:17.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1970983211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b2170 0x7f3c5415d5e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1970983211 >> v2:172.21.15.162:3300/0 conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 -- 172.21.15.55:0/1970983211 >> 172.21.15.55:0/1970983211 conn(0x7f3c540b5730 msgr2=0x7f3c540d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.061+0000 7f3c5a200640 1 -- 172.21.15.55:0/1970983211 shutdown_connections 2024-05-10T03:33:17.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.062+0000 7f3c5a200640 1 -- 172.21.15.55:0/1970983211 wait complete. 2024-05-10T03:33:17.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.065+0000 7f3c5a200640 1 Processor -- start 2024-05-10T03:33:17.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.065+0000 7f3c5a200640 1 -- start start 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 0x7f3c54156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54160530 con 0x7f3c540b6680 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540cc740 con 0x7f3c541618b0 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c5a200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54156810 con 0x7f3c540b2170 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c53e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c53e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41906/0 (socket says 172.21.15.55:41906) 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.067+0000 7f3c53e00640 1 -- 172.21.15.55:0/1454803174 learned_addr learned my addr 172.21.15.55:0/1454803174 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c59800640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c58e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 0x7f3c54156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 -- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 msgr2=0x7f3c54156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 0x7f3c54156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 -- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 msgr2=0x7f3c5406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 -- 172.21.15.55:0/1454803174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c40009b30 con 0x7f3c540b6680 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c59800640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c58e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 0x7f3c54156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:17.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.068+0000 7f3c53e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3c400093e0 tx=0x7f3c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.069+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4003b820 con 0x7f3c540b6680 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.069+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4003b980 con 0x7f3c540b6680 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.069+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c540cc990 con 0x7f3c540b6680 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.069+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4003bd30 con 0x7f3c540b6680 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.069+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5415e710 con 0x7f3c540b6680 2024-05-10T03:33:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.070+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54067740 con 0x7f3c540b6680 2024-05-10T03:33:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.071+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c40038050 con 0x7f3c540b6680 2024-05-10T03:33:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.073+0000 7f3c50c00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 0x7f3c3406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.073+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3c400aa200 con 0x7f3c540b6680 2024-05-10T03:33:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.074+0000 7f3c58e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 0x7f3c3406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.075+0000 7f3c58e00640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 0x7f3c3406d110 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400a830 tx=0x7f3c44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.079+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3c40075230 con 0x7f3c540b6680 2024-05-10T03:33:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.322+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c5404e290 con 0x7f3c540b6680 2024-05-10T03:33:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.323+0000 7f3c50c00640 1 -- 172.21.15.55:0/1454803174 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f3c40036090 con 0x7f3c540b6680 2024-05-10T03:33:17.326 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:17.326 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:17.326 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.325+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 msgr2=0x7f3c3406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.325+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 0x7f3c3406d110 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400a830 tx=0x7f3c44008040 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.325+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 msgr2=0x7f3c541562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.325+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3c400093e0 tx=0x7f3c40004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 shutdown_connections 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3c3406ac00 0x7f3c3406d110 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3c541618b0 0x7f3c5406aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c540b6680 0x7f3c541562d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 --2- 172.21.15.55:0/1454803174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3c540b2170 0x7f3c54156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 >> 172.21.15.55:0/1454803174 conn(0x7f3c540b5730 msgr2=0x7f3c540ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:17.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 shutdown_connections 2024-05-10T03:33:17.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.326+0000 7f3c5a200640 1 -- 172.21.15.55:0/1454803174 wait complete. 2024-05-10T03:33:17.355 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:17.592 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:33:17.600 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:33:17.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.630+0000 7fcb77200640 1 Processor -- start 2024-05-10T03:33:17.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.630+0000 7fcb77200640 1 -- start start 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.632+0000 7fcb77200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.632+0000 7fcb77200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb700b3f00 0x7fcb700d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.632+0000 7fcb77200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb700b2aa0 con 0x7fcb700ce3a0 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.632+0000 7fcb77200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb700b6190 con 0x7fcb700b3f00 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.632+0000 7fcb77200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700b6c30 con 0x7fcb7015d970 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb76800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb700b3f00 0x7fcb700d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb75400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcb700ce3a0 0x7fcb700b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43970/0 (socket says 172.21.15.55:43970) 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb75400640 1 -- 172.21.15.55:0/2975486129 learned_addr learned my addr 172.21.15.55:0/2975486129 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb75e00640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb76800640 1 -- 172.21.15.55:0/2975486129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 msgr2=0x7fcb700b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb76800640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb76800640 1 -- 172.21.15.55:0/2975486129 >> v1:172.21.15.55:6789/0 conn(0x7fcb700ce3a0 legacy=0x7fcb700b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb76800640 1 -- 172.21.15.55:0/2975486129 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700b2d10 con 0x7fcb700b3f00 2024-05-10T03:33:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.633+0000 7fcb74a00640 1 -- 172.21.15.55:0/2975486129 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464927158 0 0) 0x7fcb700b2aa0 con 0x7fcb700ce3a0 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.634+0000 7fcb76800640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700d1400 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009d90 tx=0x7fcb6c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e1381591064ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.635+0000 7fcb74a00640 1 -- 172.21.15.55:0/2975486129 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c007740 con 0x7fcb700b3f00 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.635+0000 7fcb74a00640 1 -- 172.21.15.55:0/2975486129 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c02d050 con 0x7fcb700b3f00 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.635+0000 7fcb74a00640 1 -- 172.21.15.55:0/2975486129 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c007ac0 con 0x7fcb700b3f00 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 -- 172.21.15.55:0/2975486129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 msgr2=0x7fcb700d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700d1400 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009d90 tx=0x7fcb6c02a660 comp rx=0 tx=0).stop 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 -- 172.21.15.55:0/2975486129 shutdown_connections 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700d1400 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 --2- 172.21.15.55:0/2975486129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 -- 172.21.15.55:0/2975486129 >> 172.21.15.55:0/2975486129 conn(0x7fcb700d0370 msgr2=0x7fcb700d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 -- 172.21.15.55:0/2975486129 shutdown_connections 2024-05-10T03:33:17.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.636+0000 7fcb77200640 1 -- 172.21.15.55:0/2975486129 wait complete. 2024-05-10T03:33:17.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.638+0000 7fcb77200640 1 Processor -- start 2024-05-10T03:33:17.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.638+0000 7fcb77200640 1 -- start start 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb700ce3a0 0x7fcb70152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb701530c0 con 0x7fcb700b3f00 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700c9040 con 0x7fcb700ce3a0 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb77200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7015c700 con 0x7fcb7015d970 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb75e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb75e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41918/0 (socket says 172.21.15.55:41918) 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.640+0000 7fcb75e00640 1 -- 172.21.15.55:0/1934152517 learned_addr learned my addr 172.21.15.55:0/1934152517 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75400640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb700ce3a0 0x7fcb70152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb76800640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 -- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 msgr2=0x7fcb700cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 -- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb700ce3a0 msgr2=0x7fcb70152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb700ce3a0 0x7fcb70152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 -- 172.21.15.55:0/1934152517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c009a70 con 0x7fcb700b3f00 2024-05-10T03:33:17.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.641+0000 7fcb75e00640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e870 tx=0x7fcb6400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.642+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640094f0 con 0x7fcb700b3f00 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.642+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6400c940 con 0x7fcb700b3f00 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.642+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700c9290 con 0x7fcb700b3f00 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.642+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400ccf0 con 0x7fcb700b3f00 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.642+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7015cc30 con 0x7fcb700b3f00 2024-05-10T03:33:17.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.643+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7004e290 con 0x7fcb700b3f00 2024-05-10T03:33:17.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.644+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb64013470 con 0x7fcb700b3f00 2024-05-10T03:33:17.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.646+0000 7fcb6a000640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 0x7fcb5406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:17.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.646+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb640894b0 con 0x7fcb700b3f00 2024-05-10T03:33:17.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.646+0000 7fcb75400640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 0x7fcb5406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:17.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.647+0000 7fcb75400640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 0x7fcb5406d110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb700c6780 tx=0x7fcb60007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:17.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.650+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcb640545f0 con 0x7fcb700b3f00 2024-05-10T03:33:17.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.844+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb700b3c00 con 0x7fcb700b3f00 2024-05-10T03:33:17.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.846+0000 7fcb6a000640 1 -- 172.21.15.55:0/1934152517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fcb64057c10 con 0x7fcb700b3f00 2024-05-10T03:33:17.850 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:17.850 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:17.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 msgr2=0x7fcb5406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 0x7fcb5406d110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb700c6780 tx=0x7fcb60007de0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 msgr2=0x7fcb700c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e870 tx=0x7fcb6400ed30 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 shutdown_connections 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb7015d970 0x7fcb700cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb5406ac00 0x7fcb5406d110 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb700ce3a0 0x7fcb70152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 --2- 172.21.15.55:0/1934152517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb700b3f00 0x7fcb700c8a40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 >> 172.21.15.55:0/1934152517 conn(0x7fcb700d0370 msgr2=0x7fcb700b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 shutdown_connections 2024-05-10T03:33:17.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:17.849+0000 7fcb77200640 1 -- 172.21.15.55:0/1934152517 wait complete. 2024-05-10T03:33:17.876 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:33:17.876 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:18.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.145+0000 7fd7b5800640 1 Processor -- start 2024-05-10T03:33:18.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.145+0000 7fd7b5800640 1 -- start start 2024-05-10T03:33:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.147+0000 7fd7b5800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144120 0x7fd7b006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.147+0000 7fd7b5800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7b00b3190 0x7fd7b00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.147+0000 7fd7b5800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b006d350 con 0x7fd7b0144710 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.147+0000 7fd7b5800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b0144530 con 0x7fd7b00b3190 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.147+0000 7fd7b5800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b006dd20 con 0x7fd7b0144120 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7afe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd7b0144710 0x7fd7b00b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43984/0 (socket says 172.21.15.55:43984) 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7afe00640 1 -- 172.21.15.55:0/3942611044 learned_addr learned my addr 172.21.15.55:0/3942611044 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7b4e00640 1 --2- 172.21.15.55:0/3942611044 >> v2:172.21.15.55:3300/0 conn(0x7fd7b00b3190 0x7fd7b00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7af400640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144120 0x7fd7b006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7aea00640 1 -- 172.21.15.55:0/3942611044 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272776270 0 0) 0x7fd7b006d350 con 0x7fd7b0144710 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.148+0000 7fd7aea00640 1 -- 172.21.15.55:0/3942611044 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd794003520 con 0x7fd7b0144710 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.149+0000 7fd7b4e00640 1 -- 172.21.15.55:0/3942611044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144120 msgr2=0x7fd7b006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.149+0000 7fd7b4e00640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144120 0x7fd7b006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.149+0000 7fd7b4e00640 1 -- 172.21.15.55:0/3942611044 >> v1:172.21.15.55:6789/0 conn(0x7fd7b0144710 legacy=0x7fd7b00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.149+0000 7fd7b4e00640 1 -- 172.21.15.55:0/3942611044 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b006c740 con 0x7fd7b00b3190 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.150+0000 7fd7b4e00640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b00b3190 0x7fd7b00cd9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009d90 tx=0x7fd7a802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8330c3f3e0c59b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.150+0000 7fd7aea00640 1 -- 172.21.15.55:0/3942611044 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8007740 con 0x7fd7b00b3190 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.150+0000 7fd7aea00640 1 -- 172.21.15.55:0/3942611044 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a802d050 con 0x7fd7b00b3190 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.150+0000 7fd7aea00640 1 -- 172.21.15.55:0/3942611044 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8007aa0 con 0x7fd7b00b3190 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 -- 172.21.15.55:0/3942611044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b00b3190 msgr2=0x7fd7b00cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b00b3190 0x7fd7b00cd9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009d90 tx=0x7fd7a802a540 comp rx=0 tx=0).stop 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 -- 172.21.15.55:0/3942611044 shutdown_connections 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144120 0x7fd7b006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 --2- 172.21.15.55:0/3942611044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b00b3190 0x7fd7b00cd9c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 -- 172.21.15.55:0/3942611044 >> 172.21.15.55:0/3942611044 conn(0x7fd7b006a110 msgr2=0x7fd7b006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 -- 172.21.15.55:0/3942611044 shutdown_connections 2024-05-10T03:33:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.151+0000 7fd7b5800640 1 -- 172.21.15.55:0/3942611044 wait complete. 2024-05-10T03:33:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.153+0000 7fd7b5800640 1 Processor -- start 2024-05-10T03:33:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.153+0000 7fd7b5800640 1 -- start start 2024-05-10T03:33:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b00b3190 0x7fd7b015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144710 0x7fd7b01f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01f6ea0 con 0x7fd7b0144120 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b015d9a0 con 0x7fd7b00b3190 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7b5800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b01f56d0 con 0x7fd7b0144710 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41924/0 (socket says 172.21.15.55:41924) 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 -- 172.21.15.55:0/2510182975 learned_addr learned my addr 172.21.15.55:0/2510182975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 -- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144710 msgr2=0x7fd7b01f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144710 0x7fd7b01f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 -- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b00b3190 msgr2=0x7fd7b015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b00b3190 0x7fd7b015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.155+0000 7fd7af400640 1 -- 172.21.15.55:0/2510182975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a8009a70 con 0x7fd7b0144120 2024-05-10T03:33:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7af400640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d5c0 tx=0x7fd79c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0125f0 con 0x7fd7b0144120 2024-05-10T03:33:18.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c012750 con 0x7fd7b0144120 2024-05-10T03:33:18.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c012b00 con 0x7fd7b0144120 2024-05-10T03:33:18.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b015ded0 con 0x7fd7b0144120 2024-05-10T03:33:18.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.156+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b01f5080 con 0x7fd7b0144120 2024-05-10T03:33:18.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.157+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b0066ce0 con 0x7fd7b0144120 2024-05-10T03:33:18.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.159+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd79c012c60 con 0x7fd7b0144120 2024-05-10T03:33:18.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.160+0000 7fd7a7e00640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 0x7fd79006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.160+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd79c0899c0 con 0x7fd7b0144120 2024-05-10T03:33:18.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.160+0000 7fd7afe00640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 0x7fd79006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.162+0000 7fd7afe00640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 0x7fd79006d140 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f60 tx=0x7fd7a00078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.164+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd79c0549d0 con 0x7fd7b0144120 2024-05-10T03:33:18.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.399+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b0071ed0 con 0x7fd7b0144120 2024-05-10T03:33:18.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.401+0000 7fd7a7e00640 1 -- 172.21.15.55:0/2510182975 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fd79c057ff0 con 0x7fd7b0144120 2024-05-10T03:33:18.404 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:18.404 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:18.404 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.403+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 msgr2=0x7fd79006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.403+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 0x7fd79006d140 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f60 tx=0x7fd7a00078f0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.403+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 msgr2=0x7fd7b015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.403+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d5c0 tx=0x7fd79c00da80 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 shutdown_connections 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b0144710 0x7fd7b01f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7b0144120 0x7fd7b015d370 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd79006ac30 0x7fd79006d140 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 --2- 172.21.15.55:0/2510182975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b00b3190 0x7fd7b015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 >> 172.21.15.55:0/2510182975 conn(0x7fd7b006a110 msgr2=0x7fd7b015aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 shutdown_connections 2024-05-10T03:33:18.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.404+0000 7fd7b5800640 1 -- 172.21.15.55:0/2510182975 wait complete. 2024-05-10T03:33:18.434 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:18.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.720+0000 7f35cfe00640 1 Processor -- start 2024-05-10T03:33:18.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.721+0000 7f35cfe00640 1 -- start start 2024-05-10T03:33:18.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 0x7f35d0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d006d7e0 0x7f35d00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35d00b2fc0 0x7f35d015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d006d350 con 0x7f35d00b2fc0 2024-05-10T03:33:18.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d006c740 con 0x7f35d0144710 2024-05-10T03:33:18.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.723+0000 7f35cfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d006ca20 con 0x7f35d006d7e0 2024-05-10T03:33:18.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35d00b2fc0 0x7f35d015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35d00b2fc0 0x7f35d015eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41930/0 (socket says 172.21.15.55:41930) 2024-05-10T03:33:18.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 -- 172.21.15.55:0/3088935231 learned_addr learned my addr 172.21.15.55:0/3088935231 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35c6000640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d006d7e0 0x7f35d00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cea00640 1 --2- 172.21.15.55:0/3088935231 >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 0x7f35d0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 -- 172.21.15.55:0/3088935231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d006d7e0 msgr2=0x7f35d00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d006d7e0 0x7f35d00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 -- 172.21.15.55:0/3088935231 >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 msgr2=0x7f35d0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 --2- 172.21.15.55:0/3088935231 >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 0x7f35d0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cf400640 1 -- 172.21.15.55:0/3088935231 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d006cd00 con 0x7f35d00b2fc0 2024-05-10T03:33:18.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.724+0000 7f35cea00640 1 --2- 172.21.15.55:0/3088935231 >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 0x7f35d0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:18.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.725+0000 7f35cf400640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d00b2fc0 0x7f35d015eff0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f35d0072240 tx=0x7f35c802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18bf8afd2bb6ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.726+0000 7f35ce000640 1 -- 172.21.15.55:0/3088935231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8031600 con 0x7f35d00b2fc0 2024-05-10T03:33:18.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.726+0000 7f35ce000640 1 -- 172.21.15.55:0/3088935231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c802d050 con 0x7f35d00b2fc0 2024-05-10T03:33:18.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.726+0000 7f35ce000640 1 -- 172.21.15.55:0/3088935231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8031980 con 0x7f35d00b2fc0 2024-05-10T03:33:18.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 -- 172.21.15.55:0/3088935231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d00b2fc0 msgr2=0x7f35d015eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d00b2fc0 0x7f35d015eff0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f35d0072240 tx=0x7f35c802a610 comp rx=0 tx=0).stop 2024-05-10T03:33:18.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 -- 172.21.15.55:0/3088935231 shutdown_connections 2024-05-10T03:33:18.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d00b2fc0 0x7f35d015eff0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 --2- 172.21.15.55:0/3088935231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d006d7e0 0x7f35d00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 --2- 172.21.15.55:0/3088935231 >> v2:172.21.15.162:3300/0 conn(0x7f35d0144710 0x7f35d0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 -- 172.21.15.55:0/3088935231 >> 172.21.15.55:0/3088935231 conn(0x7f35d006a110 msgr2=0x7f35d006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:18.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 -- 172.21.15.55:0/3088935231 shutdown_connections 2024-05-10T03:33:18.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.727+0000 7f35cfe00640 1 -- 172.21.15.55:0/3088935231 wait complete. 2024-05-10T03:33:18.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.729+0000 7f35cfe00640 1 Processor -- start 2024-05-10T03:33:18.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.729+0000 7f35cfe00640 1 -- start start 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d00b2fc0 0x7f35d00afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0144710 0x7f35d00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01e76f0 con 0x7f35d006d7e0 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01e5eb0 con 0x7f35d0144710 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01e7860 con 0x7f35d00b2fc0 2024-05-10T03:33:18.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41936/0 (socket says 172.21.15.55:41936) 2024-05-10T03:33:18.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.731+0000 7f35cea00640 1 -- 172.21.15.55:0/2512785941 learned_addr learned my addr 172.21.15.55:0/2512785941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35c6000640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d00b2fc0 0x7f35d00afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cf400640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0144710 0x7f35d00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 -- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d00b2fc0 msgr2=0x7f35d00afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d00b2fc0 0x7f35d00afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 -- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0144710 msgr2=0x7f35d00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0144710 0x7f35d00b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 -- 172.21.15.55:0/2512785941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009b30 con 0x7f35d006d7e0 2024-05-10T03:33:18.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35cea00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c820 tx=0x7f35c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0007a30 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0007b90 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.732+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000ce10 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.733+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d015db60 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.733+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d015e400 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.734+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d0066ce0 con 0x7f35d006d7e0 2024-05-10T03:33:18.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.735+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f35c00113f0 con 0x7f35d006d7e0 2024-05-10T03:33:18.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.736+0000 7f35c7400640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 0x7f35ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:18.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.736+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f35c00895b0 con 0x7f35d006d7e0 2024-05-10T03:33:18.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.736+0000 7f35c6000640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 0x7f35ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:18.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.738+0000 7f35c6000640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 0x7f35ac06d110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:18.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.741+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f35c0054520 con 0x7f35d006d7e0 2024-05-10T03:33:18.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.980+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35d004e290 con 0x7f35d006d7e0 2024-05-10T03:33:18.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.981+0000 7f35c7400640 1 -- 172.21.15.55:0/2512785941 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f35c0057b40 con 0x7f35d006d7e0 2024-05-10T03:33:18.985 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:18.985 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:32:56.602949+0000","modified":"2024-05-10T03:33:00.044266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/1356084610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1356084610},{"type":"v1","addr":"172.21.15.55:6835","nonce":1356084610}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-05-10T03:33:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 31 2024-05-10T03:33:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 msgr2=0x7f35ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 0x7f35ac06d110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8008040 comp rx=0 tx=0).stop 2024-05-10T03:33:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 msgr2=0x7f35d015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:18.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c820 tx=0x7f35c000cce0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 shutdown_connections 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f35ac06ac00 0x7f35ac06d110 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f35d0144710 0x7f35d00b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35d00b2fc0 0x7f35d00afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 --2- 172.21.15.55:0/2512785941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35d006d7e0 0x7f35d015a0d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 >> 172.21.15.55:0/2512785941 conn(0x7f35d006a110 msgr2=0x7f35d00acfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 shutdown_connections 2024-05-10T03:33:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:18.984+0000 7f35cfe00640 1 -- 172.21.15.55:0/2512785941 wait complete. 2024-05-10T03:33:19.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:19.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.296+0000 7fb6a3e00640 1 Processor -- start 2024-05-10T03:33:19.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.296+0000 7fb6a3e00640 1 -- start start 2024-05-10T03:33:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6a40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a406d350 con 0x7fb6a4144710 2024-05-10T03:33:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a40b36d0 con 0x7fb6a4144120 2024-05-10T03:33:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4144530 con 0x7fb6a406d7e0 2024-05-10T03:33:19.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a2a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb6a4144710 0x7fb6a40b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43990/0 (socket says 172.21.15.55:43990) 2024-05-10T03:33:19.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.298+0000 7fb6a2a00640 1 -- 172.21.15.55:0/228763444 learned_addr learned my addr 172.21.15.55:0/228763444 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:19.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a3400640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6a40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332028288 0 0) 0x7fb6a406d350 con 0x7fb6a4144710 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb688003520 con 0x7fb6a4144710 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 305243382 0 0) 0x7fb688003520 con 0x7fb6a4144710 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 msgr2=0x7fb6a40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6a40b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 >> v1:172.21.15.162:6789/0 conn(0x7fb6a4144120 legacy=0x7fb6a40cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a406c740 con 0x7fb6a4144710 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb6940019f0 con 0x7fb6a4144710 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.299+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 >> v1:172.21.15.55:6789/0 conn(0x7fb6a4144710 legacy=0x7fb6a40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb688003a10 0x7fb688003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb688005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4144530 con 0x7fb688004d70 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb688010090 con 0x7fb688003a10 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688010250 con 0x7fb6a406d7e0 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb688005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a2a00640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb688003a10 0x7fb688003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb69a000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 -- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 msgr2=0x7fb6880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 -- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb688003a10 msgr2=0x7fb688003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb688003a10 0x7fb688003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb6a3400640 1 -- 172.21.15.55:0/228763444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688003520 con 0x7fb688004d70 2024-05-10T03:33:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.300+0000 7fb69a000640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a406d7e0 0x7fb6880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.302+0000 7fb6a3400640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb688005240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009a90 tx=0x7fb6880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352917b3650439d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.302+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c031400 con 0x7fb688004d70 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.302+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c02d050 con 0x7fb688004d70 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.302+0000 7fb6a2000640 1 -- 172.21.15.55:0/228763444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c031780 con 0x7fb688004d70 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 -- 172.21.15.55:0/228763444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 msgr2=0x7fb688005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/228763444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb688005240 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009a90 tx=0x7fb6880057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3400640 1 -- 172.21.15.55:0/228763444 reap_dead start 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 -- 172.21.15.55:0/228763444 shutdown_connections 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 -- 172.21.15.55:0/228763444 >> 172.21.15.55:0/228763444 conn(0x7fb6a406a110 msgr2=0x7fb6a406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 -- 172.21.15.55:0/228763444 shutdown_connections 2024-05-10T03:33:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.303+0000 7fb6a3e00640 1 -- 172.21.15.55:0/228763444 wait complete. 2024-05-10T03:33:19.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.305+0000 7fb6a3e00640 1 Processor -- start 2024-05-10T03:33:19.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.305+0000 7fb6a3e00640 1 -- start start 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6a413cf00 0x7fb6a415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a415df60 0x7fb6a415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a413fbb0 con 0x7fb688004d70 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4140010 con 0x7fb6a413cf00 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a415c940 con 0x7fb6a415df60 2024-05-10T03:33:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41952/0 (socket says 172.21.15.55:41952) 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.307+0000 7fb6a2a00640 1 -- 172.21.15.55:0/497337745 learned_addr learned my addr 172.21.15.55:0/497337745 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb69a000640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6a413cf00 0x7fb6a415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a3400640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a415df60 0x7fb6a415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 -- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a415df60 msgr2=0x7fb6a415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a415df60 0x7fb6a415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 -- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6a413cf00 msgr2=0x7fb6a415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6a413cf00 0x7fb6a415c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 -- 172.21.15.55:0/497337745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a413e5a0 con 0x7fb688004d70 2024-05-10T03:33:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.308+0000 7fb6a2a00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb6940079d0 tx=0x7fb69400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.309+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400ee50 con 0x7fb688004d70 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.309+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694013420 con 0x7fb688004d70 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.309+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940137d0 con 0x7fb688004d70 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.309+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a415e8b0 con 0x7fb688004d70 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.309+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a413d990 con 0x7fb688004d70 2024-05-10T03:33:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.310+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a413e730 con 0x7fb688004d70 2024-05-10T03:33:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.312+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb694013930 con 0x7fb688004d70 2024-05-10T03:33:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.312+0000 7fb69be00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 0x7fb68006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.313+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb694089bf0 con 0x7fb688004d70 2024-05-10T03:33:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.313+0000 7fb69a000640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 0x7fb68006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.315+0000 7fb69a000640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 0x7fb68006d110 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6a415bd50 tx=0x7fb68c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.317+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb694054c90 con 0x7fb688004d70 2024-05-10T03:33:19.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.516+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6a413e730 con 0x7fb688004d70 2024-05-10T03:33:19.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.519+0000 7fb69be00640 1 -- 172.21.15.55:0/497337745 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fb6940582b0 con 0x7fb688004d70 2024-05-10T03:33:19.523 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:19.523 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":66,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:32:57.607329+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:32:54.741096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-10T03:32:55.615018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:19.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 msgr2=0x7fb68006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 0x7fb68006d110 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb6a415bd50 tx=0x7fb68c009040 comp rx=0 tx=0).stop 2024-05-10T03:33:19.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 msgr2=0x7fb6a4155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb6940079d0 tx=0x7fb69400e530 comp rx=0 tx=0).stop 2024-05-10T03:33:19.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 shutdown_connections 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6a415df60 0x7fb6a415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb68006ac00 0x7fb68006d110 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6a413cf00 0x7fb6a415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 --2- 172.21.15.55:0/497337745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb688004d70 0x7fb6a4155ca0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 >> 172.21.15.55:0/497337745 conn(0x7fb6a406a110 msgr2=0x7fb6a40aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 shutdown_connections 2024-05-10T03:33:19.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.522+0000 7fb6a3e00640 1 -- 172.21.15.55:0/497337745 wait complete. 2024-05-10T03:33:19.553 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:33:19.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.831+0000 7f3a8fe00640 1 Processor -- start 2024-05-10T03:33:19.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.832+0000 7f3a8fe00640 1 -- start start 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a9015adc0 0x7f3a900ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9006b830 0x7f3a900b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a900b4330 con 0x7f3a9015e090 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9015a0d0 con 0x7f3a9015adc0 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015a2e0 con 0x7f3a9006b830 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a9015adc0 0x7f3a900ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8e000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3a9015e090 0x7f3a9006b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43994/0 (socket says 172.21.15.55:43994) 2024-05-10T03:33:19.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8e000640 1 -- 172.21.15.55:0/3495246344 learned_addr learned my addr 172.21.15.55:0/3495246344 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.834+0000 7f3a8f400640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9006b830 0x7f3a900b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.835+0000 7f3a8ea00640 1 -- 172.21.15.55:0/3495246344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9006b830 msgr2=0x7f3a900b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.835+0000 7f3a8ea00640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9006b830 0x7f3a900b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.835+0000 7f3a8ea00640 1 -- 172.21.15.55:0/3495246344 >> v1:172.21.15.55:6789/0 conn(0x7f3a9015e090 legacy=0x7f3a9006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.835+0000 7f3a8ea00640 1 -- 172.21.15.55:0/3495246344 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9015a5c0 con 0x7f3a9015adc0 2024-05-10T03:33:19.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.835+0000 7f3a8d600640 1 -- 172.21.15.55:0/3495246344 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719637317 0 0) 0x7f3a900b4330 con 0x7f3a9015e090 2024-05-10T03:33:19.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.836+0000 7f3a8ea00640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9015adc0 0x7f3a900ce880 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3a800097a0 tx=0x7f3a8002b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3dc0110d375d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.836+0000 7f3a8d600640 1 -- 172.21.15.55:0/3495246344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80033440 con 0x7f3a9015adc0 2024-05-10T03:33:19.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8d600640 1 -- 172.21.15.55:0/3495246344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a800335a0 con 0x7f3a9015adc0 2024-05-10T03:33:19.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8d600640 1 -- 172.21.15.55:0/3495246344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80033910 con 0x7f3a9015adc0 2024-05-10T03:33:19.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 -- 172.21.15.55:0/3495246344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9015adc0 msgr2=0x7f3a900ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9015adc0 0x7f3a900ce880 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3a800097a0 tx=0x7f3a8002b290 comp rx=0 tx=0).stop 2024-05-10T03:33:19.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 -- 172.21.15.55:0/3495246344 shutdown_connections 2024-05-10T03:33:19.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9006b830 0x7f3a900b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/3495246344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9015adc0 0x7f3a900ce880 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 -- 172.21.15.55:0/3495246344 >> 172.21.15.55:0/3495246344 conn(0x7f3a9006c730 msgr2=0x7f3a900b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:19.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.837+0000 7f3a8fe00640 1 -- 172.21.15.55:0/3495246344 shutdown_connections 2024-05-10T03:33:19.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.838+0000 7f3a8fe00640 1 -- 172.21.15.55:0/3495246344 wait complete. 2024-05-10T03:33:19.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.840+0000 7f3a8fe00640 1 Processor -- start 2024-05-10T03:33:19.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.840+0000 7f3a8fe00640 1 -- start start 2024-05-10T03:33:19.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9015adc0 0x7f3a900b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a9015e090 0x7f3a9014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900afb70 con 0x7f3a9014fa60 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900a9af0 con 0x7f3a9015e090 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900af640 con 0x7f3a9015adc0 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41972/0 (socket says 172.21.15.55:41972) 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8f400640 1 -- 172.21.15.55:0/1234915010 learned_addr learned my addr 172.21.15.55:0/1234915010 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8ea00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9015adc0 0x7f3a900b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.842+0000 7f3a8e000640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a9015e090 0x7f3a9014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 -- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9015adc0 msgr2=0x7f3a900b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9015adc0 0x7f3a900b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 -- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a9015e090 msgr2=0x7f3a9014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:19.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a9015e090 0x7f3a9014cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:19.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 -- 172.21.15.55:0/1234915010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009400 con 0x7f3a9014fa60 2024-05-10T03:33:19.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8f400640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007d80 tx=0x7f3a8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014420 con 0x7f3a9014fa60 2024-05-10T03:33:19.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88014580 con 0x7f3a9014fa60 2024-05-10T03:33:19.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901503b0 con 0x7f3a9014fa60 2024-05-10T03:33:19.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.843+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014930 con 0x7f3a9014fa60 2024-05-10T03:33:19.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.844+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a900af890 con 0x7f3a9014fa60 2024-05-10T03:33:19.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.845+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9004e290 con 0x7f3a9014fa60 2024-05-10T03:33:19.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.846+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a88014a90 con 0x7f3a9014fa60 2024-05-10T03:33:19.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.847+0000 7f3a86a00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 0x7f3a6c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:19.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.847+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3a88089e80 con 0x7f3a9014fa60 2024-05-10T03:33:19.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.847+0000 7f3a8ea00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 0x7f3a6c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:19.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.849+0000 7f3a8ea00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 0x7f3a6c06d110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3a80035000 tx=0x7f3a8002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:19.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:19.852+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a88054df0 con 0x7f3a9014fa60 2024-05-10T03:33:20.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.088+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3a900cb230 con 0x7f3a9014fa60 2024-05-10T03:33:20.094 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:33:20.091+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:33:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.112+0000 7f3a86a00640 1 -- 172.21.15.55:0/1234915010 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3a88058410 con 0x7f3a9014fa60 2024-05-10T03:33:20.116 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 msgr2=0x7f3a6c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 0x7f3a6c06d110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3a80035000 tx=0x7f3a8002f000 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 msgr2=0x7f3a9014fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007d80 tx=0x7f3a8800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 shutdown_connections 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a9014fa60 0x7f3a9014fe70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a6c06ac00 0x7f3a6c06d110 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a9015e090 0x7f3a9014cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 --2- 172.21.15.55:0/1234915010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a9015adc0 0x7f3a900b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 >> 172.21.15.55:0/1234915010 conn(0x7f3a9006c730 msgr2=0x7f3a90164bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:20.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 shutdown_connections 2024-05-10T03:33:20.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.115+0000 7f3a8fe00640 1 -- 172.21.15.55:0/1234915010 wait complete. 2024-05-10T03:33:20.120 INFO:tasks.ceph.mds.c.smithi055.stderr: -2123> 2024-05-10T03:33:00.943+0000 7f1934a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:33:20.143 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:33:20.251 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:33:20.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.407+0000 7ff01f200640 1 Processor -- start 2024-05-10T03:33:20.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.407+0000 7ff01f200640 1 -- start start 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01f200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff01815a380 0x7ff0180b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff0180b24a0 0x7ff0180ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff018070350 con 0x7ff0180cd940 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff018070fd0 con 0x7ff0180b24a0 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01f200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff018070630 con 0x7ff01815a380 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01de00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff0180cd940 0x7ff0180713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43998/0 (socket says 172.21.15.55:43998) 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01de00640 1 -- 172.21.15.55:0/1072251579 learned_addr learned my addr 172.21.15.55:0/1072251579 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:20.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.410+0000 7ff01e800640 1 --2- 172.21.15.55:0/1072251579 >> v2:172.21.15.55:3300/0 conn(0x7ff0180b24a0 0x7ff0180ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:20.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01d400640 1 --2- 172.21.15.55:0/1072251579 >> v2:172.21.15.162:3300/0 conn(0x7ff01815a380 0x7ff0180b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:20.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01ca00640 1 -- 172.21.15.55:0/1072251579 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022016328 0 0) 0x7ff018070350 con 0x7ff0180cd940 2024-05-10T03:33:20.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01ca00640 1 -- 172.21.15.55:0/1072251579 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff000003520 con 0x7ff0180cd940 2024-05-10T03:33:20.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01e800640 1 -- 172.21.15.55:0/1072251579 >> v2:172.21.15.162:3300/0 conn(0x7ff01815a380 msgr2=0x7ff0180b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01e800640 1 --2- 172.21.15.55:0/1072251579 >> v2:172.21.15.162:3300/0 conn(0x7ff01815a380 0x7ff0180b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01e800640 1 -- 172.21.15.55:0/1072251579 >> v1:172.21.15.55:6789/0 conn(0x7ff0180cd940 legacy=0x7ff0180713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.411+0000 7ff01e800640 1 -- 172.21.15.55:0/1072251579 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018070910 con 0x7ff0180b24a0 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.412+0000 7ff01e800640 1 --2- 172.21.15.55:0/1072251579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180b24a0 0x7ff0180ce1c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cee80 tx=0x7ff01402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f98e4595f2032 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.412+0000 7ff01ca00640 1 -- 172.21.15.55:0/1072251579 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01402aea0 con 0x7ff0180b24a0 2024-05-10T03:33:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.412+0000 7ff01ca00640 1 -- 172.21.15.55:0/1072251579 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01402d050 con 0x7ff0180b24a0 2024-05-10T03:33:20.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.412+0000 7ff01ca00640 1 -- 172.21.15.55:0/1072251579 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014007960 con 0x7ff0180b24a0 2024-05-10T03:33:20.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 -- 172.21.15.55:0/1072251579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180b24a0 msgr2=0x7ff0180ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 --2- 172.21.15.55:0/1072251579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180b24a0 0x7ff0180ce1c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cee80 tx=0x7ff01402a680 comp rx=0 tx=0).stop 2024-05-10T03:33:20.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 -- 172.21.15.55:0/1072251579 shutdown_connections 2024-05-10T03:33:20.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 --2- 172.21.15.55:0/1072251579 >> v2:172.21.15.162:3300/0 conn(0x7ff01815a380 0x7ff0180b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 --2- 172.21.15.55:0/1072251579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180b24a0 0x7ff0180ce1c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 -- 172.21.15.55:0/1072251579 >> 172.21.15.55:0/1072251579 conn(0x7ff0180b1290 msgr2=0x7ff01806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:20.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 -- 172.21.15.55:0/1072251579 shutdown_connections 2024-05-10T03:33:20.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.413+0000 7ff01f200640 1 -- 172.21.15.55:0/1072251579 wait complete. 2024-05-10T03:33:20.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.416+0000 7ff01f200640 1 Processor -- start 2024-05-10T03:33:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.416+0000 7ff01f200640 1 -- start start 2024-05-10T03:33:20.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0180b24a0 0x7ff01814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff01815a380 0x7ff018146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018148680 con 0x7ff0180cd940 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01814d130 con 0x7ff0180b24a0 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.417+0000 7ff01f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180c6c20 con 0x7ff01815a380 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42020/0 (socket says 172.21.15.55:42020) 2024-05-10T03:33:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 -- 172.21.15.55:0/3344946640 learned_addr learned my addr 172.21.15.55:0/3344946640 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 -- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff01815a380 msgr2=0x7ff018146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff01815a380 0x7ff018146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 -- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0180b24a0 msgr2=0x7ff01814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0180b24a0 0x7ff01814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 -- 172.21.15.55:0/3344946640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014009a70 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.418+0000 7ff01d400640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d5f0 tx=0x7ff00800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.419+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080125f0 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.419+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008012750 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.419+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008012b00 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.419+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0180c6f00 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.419+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0180c74c0 con 0x7ff0180cd940 2024-05-10T03:33:20.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.420+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01804e290 con 0x7ff0180cd940 2024-05-10T03:33:20.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.421+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff008012c60 con 0x7ff0180cd940 2024-05-10T03:33:20.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.423+0000 7ff012000640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 0x7feffc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:20.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.423+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6115+0+0 (secure 0 0 0) 0x7ff008089b10 con 0x7ff0180cd940 2024-05-10T03:33:20.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.423+0000 7ff01de00640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 0x7feffc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.425+0000 7ff01de00640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 0x7feffc06d110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cb1d0 tx=0x7ff00c007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:20.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.428+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff008054a20 con 0x7ff0180cd940 2024-05-10T03:33:20.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:20.677+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff01806b5c0 con 0x7ff0180cd940 2024-05-10T03:33:21.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.261+0000 7ff012000640 1 -- 172.21.15.55:0/3344946640 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff008058040 con 0x7ff0180cd940 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.264+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 msgr2=0x7feffc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.264+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 0x7feffc06d110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cb1d0 tx=0x7ff00c007930 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.264+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 msgr2=0x7ff018146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.264+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff00800d5f0 tx=0x7ff00800dab0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 shutdown_connections 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff01815a380 0x7ff018146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0180cd940 0x7ff018146210 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7feffc06ac00 0x7feffc06d110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 --2- 172.21.15.55:0/3344946640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0180b24a0 0x7ff01814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 >> 172.21.15.55:0/3344946640 conn(0x7ff0180b1290 msgr2=0x7ff01815ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:21.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 shutdown_connections 2024-05-10T03:33:21.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.265+0000 7ff01f200640 1 -- 172.21.15.55:0/3344946640 wait complete. 2024-05-10T03:33:21.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:33:21.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.576+0000 7fdbd5e00640 1 Processor -- start 2024-05-10T03:33:21.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.576+0000 7fdbd5e00640 1 -- start start 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbd00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd006a2b0 con 0x7fdbd00cdd40 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd006a590 con 0x7fdbd00b8970 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d2b00 con 0x7fdbd00d12b0 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdbd00cdd40 0x7fdbd00ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44004/0 (socket says 172.21.15.55:44004) 2024-05-10T03:33:21.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd5400640 1 -- 172.21.15.55:0/2065990542 learned_addr learned my addr 172.21.15.55:0/2065990542 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.579+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbd00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925549412 0 0) 0x7fdbd006a2b0 con 0x7fdbd00cdd40 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb4003520 con 0x7fdbd00cdd40 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096427713 0 0) 0x7fdbb4003520 con 0x7fdbd00cdd40 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 msgr2=0x7fdbd00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbd00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 >> v1:172.21.15.162:6789/0 conn(0x7fdbd00b8970 legacy=0x7fdbd00b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd006a870 con 0x7fdbd00cdd40 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fdbc8001670 con 0x7fdbd00cdd40 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 >> v1:172.21.15.55:6789/0 conn(0x7fdbd00cdd40 legacy=0x7fdbd00ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbb40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbb4003f20 0x7fdbb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4004d00 0x7fdbb4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d2b00 con 0x7fdbb4003f20 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400bbc0 con 0x7fdbb4004d00 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.580+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400bd80 con 0x7fdbd00d12b0 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbb4003f20 0x7fdbb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbd5400640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4004d00 0x7fdbb4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbb40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 -- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 msgr2=0x7fdbb40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:21.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00d12b0 0x7fdbb40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 -- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4004d00 msgr2=0x7fdbb4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:21.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4004d00 0x7fdbb4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.581+0000 7fdbcfe00640 1 -- 172.21.15.55:0/2065990542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4003520 con 0x7fdbb4003f20 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.582+0000 7fdbcfe00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbb4003f20 0x7fdbb40047c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc001e50 tx=0x7fdbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cba4a7901f6c747 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc02ae70 con 0x7fdbb4003f20 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc003970 con 0x7fdbb4003f20 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbcf400640 1 -- 172.21.15.55:0/2065990542 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc003d20 con 0x7fdbb4003f20 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbd5e00640 1 -- 172.21.15.55:0/2065990542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbb4003f20 msgr2=0x7fdbb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/2065990542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbb4003f20 0x7fdbb40047c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc001e50 tx=0x7fdbb40057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.583+0000 7fdbd5400640 1 -- 172.21.15.55:0/2065990542 reap_dead start 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.584+0000 7fdbd5e00640 1 -- 172.21.15.55:0/2065990542 shutdown_connections 2024-05-10T03:33:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.584+0000 7fdbd5e00640 1 -- 172.21.15.55:0/2065990542 >> 172.21.15.55:0/2065990542 conn(0x7fdbd00d3100 msgr2=0x7fdbd00d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.584+0000 7fdbd5e00640 1 -- 172.21.15.55:0/2065990542 shutdown_connections 2024-05-10T03:33:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.584+0000 7fdbd5e00640 1 -- 172.21.15.55:0/2065990542 wait complete. 2024-05-10T03:33:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.586+0000 7fdbd5e00640 1 Processor -- start 2024-05-10T03:33:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.586+0000 7fdbd5e00640 1 -- start start 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 0x7fdbd014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00b3570 0x7fdbd00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd014e930 con 0x7fdbd014ac30 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01502d0 con 0x7fdbb4003f20 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd014c770 con 0x7fdbd00b3570 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42024/0 (socket says 172.21.15.55:42024) 2024-05-10T03:33:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.588+0000 7fdbd5400640 1 -- 172.21.15.55:0/1253589520 learned_addr learned my addr 172.21.15.55:0/1253589520 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 0x7fdbd014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbcfe00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00b3570 0x7fdbd00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 -- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00b3570 msgr2=0x7fdbd00b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00b3570 0x7fdbd00b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 -- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 msgr2=0x7fdbd014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 0x7fdbd014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 -- 172.21.15.55:0/1253589520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd014c9c0 con 0x7fdbd014ac30 2024-05-10T03:33:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 0x7fdbd014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5400640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdbd01622a0 tx=0x7fdbc800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800eed0 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800c640 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd014cca0 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800c9f0 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.589+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd014d260 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.590+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd0067220 con 0x7fdbd014ac30 2024-05-10T03:33:21.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.592+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdbc800cb50 con 0x7fdbd014ac30 2024-05-10T03:33:21.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.592+0000 7fdbccc00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 0x7fdbb006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:21.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.593+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fdbc8088720 con 0x7fdbd014ac30 2024-05-10T03:33:21.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.593+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 0x7fdbb006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:21.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.594+0000 7fdbd4a00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 0x7fdbb006d140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbd008bc20 tx=0x7fdbc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:21.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.597+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdbc8053630 con 0x7fdbd014ac30 2024-05-10T03:33:21.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:21.798+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdbd00d3920 con 0x7fdbd014ac30 2024-05-10T03:33:22.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.267+0000 7fdbccc00640 1 -- 172.21.15.55:0/1253589520 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdbc80190d0 con 0x7fdbd014ac30 2024-05-10T03:33:22.272 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:33:22.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 msgr2=0x7fdbb006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 0x7fdbb006d140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbd008bc20 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-05-10T03:33:22.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 msgr2=0x7fdbd014b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdbd01622a0 tx=0x7fdbc800eb90 comp rx=0 tx=0).stop 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 shutdown_connections 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbd014ac30 0x7fdbd014b040 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdbd00b3570 0x7fdbd00b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdbb006ac30 0x7fdbb006d140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.271+0000 7fdbd5e00640 1 --2- 172.21.15.55:0/1253589520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbb4003f20 0x7fdbd014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.272+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 >> 172.21.15.55:0/1253589520 conn(0x7fdbd00d3100 msgr2=0x7fdbd00b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.272+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 shutdown_connections 2024-05-10T03:33:22.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.272+0000 7fdbd5e00640 1 -- 172.21.15.55:0/1253589520 wait complete. 2024-05-10T03:33:22.308 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.588+0000 7fc15a600640 1 Processor -- start 2024-05-10T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.588+0000 7fc15a600640 1 -- start start 2024-05-10T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc15a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 0x7fc1540ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc15a600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1540b2b70 0x7fc15415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc15a600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1540b4050 con 0x7fc1540ce690 2024-05-10T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc15a600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1540ceaa0 con 0x7fc1540b2b70 2024-05-10T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc15a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540b3440 con 0x7fc1540b44e0 2024-05-10T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc152a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc1540ce690 0x7fc1540b2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44006/0 (socket says 172.21.15.55:44006) 2024-05-10T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc152a00640 1 -- 172.21.15.55:0/1040645394 learned_addr learned my addr 172.21.15.55:0/1040645394 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.590+0000 7fc153e00640 1 --2- 172.21.15.55:0/1040645394 >> v2:172.21.15.55:3300/0 conn(0x7fc1540b2b70 0x7fc15415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153400640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 0x7fc1540ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153e00640 1 -- 172.21.15.55:0/1040645394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 msgr2=0x7fc1540ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153e00640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 0x7fc1540ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153e00640 1 -- 172.21.15.55:0/1040645394 >> v1:172.21.15.55:6789/0 conn(0x7fc1540ce690 legacy=0x7fc1540b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153e00640 1 -- 172.21.15.55:0/1040645394 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1540b3720 con 0x7fc1540b2b70 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc152000640 1 -- 172.21.15.55:0/1040645394 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624749974 0 0) 0x7fc1540b4050 con 0x7fc1540ce690 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.591+0000 7fc153400640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 0x7fc1540ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.592+0000 7fc153e00640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b2b70 0x7fc15415cf60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009d90 tx=0x7fc14c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6543735349847f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc152000640 1 -- 172.21.15.55:0/1040645394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c007740 con 0x7fc1540b2b70 2024-05-10T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc152000640 1 -- 172.21.15.55:0/1040645394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c02d050 con 0x7fc1540b2b70 2024-05-10T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc152000640 1 -- 172.21.15.55:0/1040645394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c007ac0 con 0x7fc1540b2b70 2024-05-10T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 -- 172.21.15.55:0/1040645394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b2b70 msgr2=0x7fc15415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b2b70 0x7fc15415cf60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009d90 tx=0x7fc14c02a660 comp rx=0 tx=0).stop 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 -- 172.21.15.55:0/1040645394 shutdown_connections 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b2b70 0x7fc15415cf60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 --2- 172.21.15.55:0/1040645394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540b44e0 0x7fc1540ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 -- 172.21.15.55:0/1040645394 >> 172.21.15.55:0/1040645394 conn(0x7fc1540d34f0 msgr2=0x7fc1540d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 -- 172.21.15.55:0/1040645394 shutdown_connections 2024-05-10T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.593+0000 7fc15a600640 1 -- 172.21.15.55:0/1040645394 wait complete. 2024-05-10T03:33:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.596+0000 7fc15a600640 1 Processor -- start 2024-05-10T03:33:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.596+0000 7fc15a600640 1 -- start start 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.597+0000 7fc15a600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1540b2b70 0x7fc1540cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.597+0000 7fc15a600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.597+0000 7fc15a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540ce690 0x7fc154152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc15a600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154150cb0 con 0x7fc1540b44e0 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc15a600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540cd630 con 0x7fc1540b2b70 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc15a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540ce000 con 0x7fc1540ce690 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42038/0 (socket says 172.21.15.55:42038) 2024-05-10T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 -- 172.21.15.55:0/190646453 learned_addr learned my addr 172.21.15.55:0/190646453 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:22.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 -- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540ce690 msgr2=0x7fc154152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540ce690 0x7fc154152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 -- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1540b2b70 msgr2=0x7fc1540cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1540b2b70 0x7fc1540cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.598+0000 7fc152a00640 1 -- 172.21.15.55:0/190646453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c009a70 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc152a00640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c3e0 tx=0x7fc14000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1400074e0 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc140007660 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140004a20 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1540cc750 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.599+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1540ccd00 con 0x7fc1540b44e0 2024-05-10T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.600+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1540cd7a0 con 0x7fc1540b44e0 2024-05-10T03:33:22.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.602+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc140004b80 con 0x7fc1540b44e0 2024-05-10T03:33:22.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.602+0000 7fc14b400640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 0x7fc13406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:22.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.603+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5677+0+0 (secure 0 0 0) 0x7fc140089110 con 0x7fc1540b44e0 2024-05-10T03:33:22.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.603+0000 7fc153400640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 0x7fc13406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:22.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.605+0000 7fc153400640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 0x7fc13406d110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:22.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.608+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc140054310 con 0x7fc1540b44e0 2024-05-10T03:33:22.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:22.811+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc154067290 con 0x7fc1540b44e0 2024-05-10T03:33:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.282+0000 7fc14b400640 1 -- 172.21.15.55:0/190646453 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc140057930 con 0x7fc1540b44e0 2024-05-10T03:33:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 msgr2=0x7fc13406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 0x7fc13406d110 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc144006fd0 tx=0x7fc144008040 comp rx=0 tx=0).stop 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 msgr2=0x7fc1540cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c3e0 tx=0x7fc14000c8a0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 shutdown_connections 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1540ce690 0x7fc154152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1540b44e0 0x7fc1540cc210 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc13406ac00 0x7fc13406d110 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 --2- 172.21.15.55:0/190646453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1540b2b70 0x7fc1540cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 >> 172.21.15.55:0/190646453 conn(0x7fc1540d34f0 msgr2=0x7fc1540c6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:23.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 shutdown_connections 2024-05-10T03:33:23.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.287+0000 7fc15a600640 1 -- 172.21.15.55:0/190646453 wait complete. 2024-05-10T03:33:23.320 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:33:23.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.598+0000 7fb8f6000640 1 Processor -- start 2024-05-10T03:33:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.599+0000 7fb8f6000640 1 -- start start 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f015a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 0x7fb8f0071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8f00b1f60 0x7fb8f00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f0070fd0 con 0x7fb8f00b1f60 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f015a8c0 con 0x7fb8f00713f0 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f6000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0070350 con 0x7fb8f00cd940 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f5600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8f00b1f60 0x7fb8f00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f5600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8f00b1f60 0x7fb8f00b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51892/0 (socket says 172.21.15.55:51892) 2024-05-10T03:33:23.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.601+0000 7fb8f5600640 1 -- 172.21.15.55:0/3498080335 learned_addr learned my addr 172.21.15.55:0/3498080335 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:23.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8f5600640 1 -- 172.21.15.55:0/3498080335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 msgr2=0x7fb8f015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:23.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8f5600640 1 --2- 172.21.15.55:0/3498080335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8f5600640 1 -- 172.21.15.55:0/3498080335 >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 msgr2=0x7fb8f0071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:23.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8e7e00640 1 --2- 172.21.15.55:0/3498080335 >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 0x7fb8f0071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8f5600640 1 --2- 172.21.15.55:0/3498080335 >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 0x7fb8f0071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8f5600640 1 -- 172.21.15.55:0/3498080335 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f0070630 con 0x7fb8f00b1f60 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.602+0000 7fb8e7e00640 1 --2- 172.21.15.55:0/3498080335 >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 0x7fb8f0071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.603+0000 7fb8f5600640 1 --2- 172.21.15.55:0/3498080335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00b1f60 0x7fb8f00b2370 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006c980 tx=0x7fb8e802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e337924f1df2730a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.603+0000 7fb8efe00640 1 -- 172.21.15.55:0/3498080335 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8031600 con 0x7fb8f00b1f60 2024-05-10T03:33:23.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.603+0000 7fb8efe00640 1 -- 172.21.15.55:0/3498080335 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e802d050 con 0x7fb8f00b1f60 2024-05-10T03:33:23.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.603+0000 7fb8efe00640 1 -- 172.21.15.55:0/3498080335 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8031980 con 0x7fb8f00b1f60 2024-05-10T03:33:23.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 -- 172.21.15.55:0/3498080335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00b1f60 msgr2=0x7fb8f00b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 --2- 172.21.15.55:0/3498080335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00b1f60 0x7fb8f00b2370 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006c980 tx=0x7fb8e802a6c0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 -- 172.21.15.55:0/3498080335 shutdown_connections 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 --2- 172.21.15.55:0/3498080335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00b1f60 0x7fb8f00b2370 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 --2- 172.21.15.55:0/3498080335 >> v2:172.21.15.162:3300/0 conn(0x7fb8f00713f0 0x7fb8f0071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 --2- 172.21.15.55:0/3498080335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 -- 172.21.15.55:0/3498080335 >> 172.21.15.55:0/3498080335 conn(0x7fb8f00b1290 msgr2=0x7fb8f006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 -- 172.21.15.55:0/3498080335 shutdown_connections 2024-05-10T03:33:23.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.604+0000 7fb8f6000640 1 -- 172.21.15.55:0/3498080335 wait complete. 2024-05-10T03:33:23.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.607+0000 7fb8f6000640 1 Processor -- start 2024-05-10T03:33:23.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.607+0000 7fb8f6000640 1 -- start start 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.608+0000 7fb8f6000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f6000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f00b1f60 0x7fb8f0157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f6000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f006a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f6000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00ac8e0 con 0x7fb8f00713f0 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f6000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0157f30 con 0x7fb8f00b1f60 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f6000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0157a60 con 0x7fb8f00cd940 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f4c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51894/0 (socket says 172.21.15.55:51894) 2024-05-10T03:33:23.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f4c00640 1 -- 172.21.15.55:0/2845348837 learned_addr learned my addr 172.21.15.55:0/2845348837 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:23.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f4c00640 1 -- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 msgr2=0x7fb8f006a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8e7e00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f00b1f60 0x7fb8f0157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f5600640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f006a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.609+0000 7fb8f4c00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f006a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f4c00640 1 -- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f00b1f60 msgr2=0x7fb8f0157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f4c00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f00b1f60 0x7fb8f0157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f4c00640 1 -- 172.21.15.55:0/2845348837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8009b30 con 0x7fb8f00713f0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f4c00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b410 tx=0x7fb8e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0011830 con 0x7fb8f00713f0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00119b0 con 0x7fb8f00713f0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0011d60 con 0x7fb8f00713f0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f00acaf0 con 0x7fb8f00713f0 2024-05-10T03:33:23.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.610+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00ace80 con 0x7fb8f00713f0 2024-05-10T03:33:23.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.612+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f004e290 con 0x7fb8f00713f0 2024-05-10T03:33:23.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.613+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb8e00193b0 con 0x7fb8f00713f0 2024-05-10T03:33:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.614+0000 7fb8ed600640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 0x7fb8cc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.614+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb8e0089090 con 0x7fb8f00713f0 2024-05-10T03:33:23.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.615+0000 7fb8e7e00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 0x7fb8cc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:23.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.616+0000 7fb8e7e00640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 0x7fb8cc06d110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006ba50 tx=0x7fb8d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.619+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb8e0054300 con 0x7fb8f00713f0 2024-05-10T03:33:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.821+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb8f0153c00 con 0x7fb8f00713f0 2024-05-10T03:33:23.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.832+0000 7fb8ed600640 1 -- 172.21.15.55:0/2845348837 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8e00150a0 con 0x7fb8f00713f0 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.835+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 msgr2=0x7fb8cc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.835+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 0x7fb8cc06d110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006ba50 tx=0x7fb8d8008040 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.835+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 msgr2=0x7fb8f014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.835+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b410 tx=0x7fb8e0009f90 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 shutdown_connections 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f00cd940 0x7fb8f006a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb8cc06ac00 0x7fb8cc06d110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f00b1f60 0x7fb8f0157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 --2- 172.21.15.55:0/2845348837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8f00713f0 0x7fb8f014cca0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:23.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 >> 172.21.15.55:0/2845348837 conn(0x7fb8f00b1290 msgr2=0x7fb8f00b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 shutdown_connections 2024-05-10T03:33:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:23.836+0000 7fb8f6000640 1 -- 172.21.15.55:0/2845348837 wait complete. 2024-05-10T03:33:23.868 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-05-10T03:33:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.149+0000 7f0e83e00640 1 Processor -- start 2024-05-10T03:33:24.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.149+0000 7f0e83e00640 1 -- start start 2024-05-10T03:33:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.151+0000 7f0e83e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e84147900 0x7f0e84162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.151+0000 7f0e83e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840b1f60 con 0x7f0e840b2d00 2024-05-10T03:33:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.151+0000 7f0e83e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e84162ed0 con 0x7f0e84144b60 2024-05-10T03:33:24.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.151+0000 7f0e83e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e840b2220 con 0x7f0e84147900 2024-05-10T03:33:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e82a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e840b2d00 0x7f0e841471f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45658/0 (socket says 172.21.15.55:45658) 2024-05-10T03:33:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e82a00640 1 -- 172.21.15.55:0/2455507795 learned_addr learned my addr 172.21.15.55:0/2455507795 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e7a000640 1 --2- 172.21.15.55:0/2455507795 >> v2:172.21.15.55:3300/0 conn(0x7f0e84147900 0x7f0e84162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e82000640 1 -- 172.21.15.55:0/2455507795 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882087659 0 0) 0x7f0e840b1f60 con 0x7f0e840b2d00 2024-05-10T03:33:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e82000640 1 -- 172.21.15.55:0/2455507795 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e68003520 con 0x7f0e840b2d00 2024-05-10T03:33:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e7a000640 1 -- 172.21.15.55:0/2455507795 >> v1:172.21.15.162:6789/0 conn(0x7f0e84144b60 legacy=0x7f0e84162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.152+0000 7f0e7a000640 1 -- 172.21.15.55:0/2455507795 >> v1:172.21.15.55:6789/0 conn(0x7f0e840b2d00 legacy=0x7f0e841471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.153+0000 7f0e7a000640 1 -- 172.21.15.55:0/2455507795 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840b2500 con 0x7f0e84147900 2024-05-10T03:33:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.154+0000 7f0e7a000640 1 --2- 172.21.15.55:0/2455507795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84147900 0x7f0e84162130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0097a0 tx=0x7f0e6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e66481a4076d868 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.154+0000 7f0e82000640 1 -- 172.21.15.55:0/2455507795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c02ae50 con 0x7f0e84147900 2024-05-10T03:33:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.154+0000 7f0e82000640 1 -- 172.21.15.55:0/2455507795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c032440 con 0x7f0e84147900 2024-05-10T03:33:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.154+0000 7f0e82000640 1 -- 172.21.15.55:0/2455507795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0327d0 con 0x7f0e84147900 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 -- 172.21.15.55:0/2455507795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84147900 msgr2=0x7f0e84162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2455507795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84147900 0x7f0e84162130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0097a0 tx=0x7f0e6c02a140 comp rx=0 tx=0).stop 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 -- 172.21.15.55:0/2455507795 shutdown_connections 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2455507795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84147900 0x7f0e84162130 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 -- 172.21.15.55:0/2455507795 >> 172.21.15.55:0/2455507795 conn(0x7f0e8406a110 msgr2=0x7f0e8406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 -- 172.21.15.55:0/2455507795 shutdown_connections 2024-05-10T03:33:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.155+0000 7f0e83e00640 1 -- 172.21.15.55:0/2455507795 wait complete. 2024-05-10T03:33:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.158+0000 7f0e83e00640 1 Processor -- start 2024-05-10T03:33:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.158+0000 7f0e83e00640 1 -- start start 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e840b2d00 0x7f0e84152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e84142790 0x7f0e841fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8413ef00 con 0x7f0e84144b60 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8406b3f0 con 0x7f0e84142790 2024-05-10T03:33:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e83e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84158b90 con 0x7f0e840b2d00 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51916/0 (socket says 172.21.15.55:51916) 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e7a000640 1 -- 172.21.15.55:0/2275814421 learned_addr learned my addr 172.21.15.55:0/2275814421 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.161+0000 7f0e82a00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e840b2d00 0x7f0e84152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e83400640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e84142790 0x7f0e841fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 -- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e840b2d00 msgr2=0x7f0e84152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e840b2d00 0x7f0e84152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 -- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e84142790 msgr2=0x7f0e841fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e84142790 0x7f0e841fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 -- 172.21.15.55:0/2275814421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009400 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.162+0000 7f0e7a000640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c000c00 tx=0x7f0e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.163+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03d910 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.163+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c03dab0 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.163+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e841fb780 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.163+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03de60 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.163+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8406b640 con 0x7f0e84144b60 2024-05-10T03:33:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.165+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e841fb910 con 0x7f0e84144b60 2024-05-10T03:33:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.166+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e6c039040 con 0x7f0e84144b60 2024-05-10T03:33:24.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.167+0000 7f0e7be00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 0x7f0e6006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:24.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.167+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0e6c0a98d0 con 0x7f0e84144b60 2024-05-10T03:33:24.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.168+0000 7f0e82a00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 0x7f0e6006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:24.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.170+0000 7f0e82a00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 0x7f0e6006d110 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0e74001560 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:24.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.172+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0e6c074c60 con 0x7f0e84144b60 2024-05-10T03:33:24.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:24.453+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) v1 -- 0x7f0e84066ce0 con 0x7f0e84144b60 2024-05-10T03:33:25.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.286+0000 7f0e7be00640 1 -- 172.21.15.55:0/2275814421 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v237) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0e6c037020 con 0x7f0e84144b60 2024-05-10T03:33:25.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 msgr2=0x7f0e6006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 0x7f0e6006d110 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0e74001560 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2024-05-10T03:33:25.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 msgr2=0x7f0e841fad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c000c00 tx=0x7f0e6c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 shutdown_connections 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0e6006ac00 0x7f0e6006d110 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e84142790 0x7f0e841fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e84144b60 0x7f0e841fad00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 --2- 172.21.15.55:0/2275814421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e840b2d00 0x7f0e84152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 >> 172.21.15.55:0/2275814421 conn(0x7f0e8406a110 msgr2=0x7f0e84143e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 shutdown_connections 2024-05-10T03:33:25.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.290+0000 7f0e83e00640 1 -- 172.21.15.55:0/2275814421 wait complete. 2024-05-10T03:33:25.325 INFO:tasks.cephfs_test_runner:test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) ... skipped 'Requires FUSE client to inject client metadata' 2024-05-10T03:33:25.325 INFO:tasks.cephfs_test_runner:Starting test: test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-05-10T03:33:25.325 INFO:tasks.cephfs_test_runner:test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:33:25.326 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:25.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.610+0000 7f51a9000640 1 Processor -- start 2024-05-10T03:33:25.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.611+0000 7f51a9000640 1 -- start start 2024-05-10T03:33:25.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.613+0000 7f51a9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 0x7f51a40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.613+0000 7f51a9000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51a41659c0 0x7f51a4165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.613+0000 7f51a9000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a40715e0 con 0x7f51a4162380 2024-05-10T03:33:25.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.613+0000 7f51a9000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40711c0 con 0x7f51a41659c0 2024-05-10T03:33:25.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.613+0000 7f51a9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a406baf0 con 0x7f51a406b6e0 2024-05-10T03:33:25.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f51a2800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f51a4162380 0x7f51a4162790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45674/0 (socket says 172.21.15.55:45674) 2024-05-10T03:33:25.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f51a2800640 1 -- 172.21.15.55:0/2971659910 learned_addr learned my addr 172.21.15.55:0/2971659910 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f5199400640 1 --2- 172.21.15.55:0/2971659910 >> v2:172.21.15.162:3300/0 conn(0x7f51a41659c0 0x7f51a4165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f51a1e00640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 0x7f51a40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012315234 0 0) 0x7f519c001490 con 0x7f51a4162380 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.614+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5188003520 con 0x7f51a4162380 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818474338 0 0) 0x7f5188003520 con 0x7f51a4162380 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 msgr2=0x7f51a40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 0x7f51a40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 >> v2:172.21.15.162:3300/0 conn(0x7f51a41659c0 msgr2=0x7f51a4165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 --2- 172.21.15.55:0/2971659910 >> v2:172.21.15.162:3300/0 conn(0x7f51a41659c0 0x7f51a4165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4166310 con 0x7f51a4162380 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f519c001b80 con 0x7f51a4162380 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 >> v1:172.21.15.55:6789/0 conn(0x7f51a4162380 legacy=0x7f51a4162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5188003970 0x7f5188003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 0x7f51880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5188004d00 0x7f5188005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40711c0 con 0x7f5188003970 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a406baf0 con 0x7f5188004d00 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518800ba00 con 0x7f51a406b6e0 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1e00640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5188003970 0x7f5188003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1e00640 1 -- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 msgr2=0x7f51880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.615+0000 7f51a1e00640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51a406b6e0 0x7f51880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.616+0000 7f51a1e00640 1 -- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5188004d00 msgr2=0x7f5188005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.616+0000 7f51a1e00640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5188004d00 0x7f5188005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.616+0000 7f51a1e00640 1 -- 172.21.15.55:0/2971659910 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188003520 con 0x7f5188003970 2024-05-10T03:33:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.617+0000 7f51a1e00640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5188003970 0x7f5188003de0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5194009a90 tx=0x7f51880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520240b2dbc3a776 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.617+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031400 con 0x7f5188003970 2024-05-10T03:33:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.617+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519402d050 con 0x7f5188003970 2024-05-10T03:33:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.617+0000 7f51a1400640 1 -- 172.21.15.55:0/2971659910 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940317a0 con 0x7f5188003970 2024-05-10T03:33:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 -- 172.21.15.55:0/2971659910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5188003970 msgr2=0x7f5188003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 --2- 172.21.15.55:0/2971659910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5188003970 0x7f5188003de0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5194009a90 tx=0x7f51880057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a2800640 1 -- 172.21.15.55:0/2971659910 reap_dead start 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 -- 172.21.15.55:0/2971659910 shutdown_connections 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 -- 172.21.15.55:0/2971659910 >> 172.21.15.55:0/2971659910 conn(0x7f51a40cbb90 msgr2=0x7f51a40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 -- 172.21.15.55:0/2971659910 shutdown_connections 2024-05-10T03:33:25.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.618+0000 7f51a9000640 1 -- 172.21.15.55:0/2971659910 wait complete. 2024-05-10T03:33:25.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.620+0000 7f51a9000640 1 Processor -- start 2024-05-10T03:33:25.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.620+0000 7f51a9000640 1 -- start start 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5188003970 0x7f51a40a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 0x7f51a406c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41507c0 con 0x7f51a41615f0 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4161310 con 0x7f51a406bf10 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f51a9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a415f580 con 0x7f5188003970 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f5199400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f5199400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51932/0 (socket says 172.21.15.55:51932) 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.622+0000 7f5199400640 1 -- 172.21.15.55:0/892351636 learned_addr learned my addr 172.21.15.55:0/892351636 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:25.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f51a1e00640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5188003970 0x7f51a40a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f51a2800640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 0x7f51a406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 -- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5188003970 msgr2=0x7f51a40a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5188003970 0x7f51a40a8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 -- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 msgr2=0x7f51a406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 0x7f51a406c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 -- 172.21.15.55:0/892351636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4161a00 con 0x7f51a41615f0 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f51a2800640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 0x7f51a406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:25.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.623+0000 7f5199400640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e7c0 tx=0x7f518c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.624+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00eed0 con 0x7f51a41615f0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.624+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c00c8e0 con 0x7f51a41615f0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.624+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00cc90 con 0x7f51a41615f0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.624+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a406c860 con 0x7f51a41615f0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.624+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a415e5e0 con 0x7f51a41615f0 2024-05-10T03:33:25.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.625+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4066f60 con 0x7f51a41615f0 2024-05-10T03:33:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.626+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f518c013470 con 0x7f51a41615f0 2024-05-10T03:33:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.627+0000 7f519aa00640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 0x7f518006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.627+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f518c0892f0 con 0x7f51a41615f0 2024-05-10T03:33:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.628+0000 7f51a1e00640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 0x7f518006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:25.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.629+0000 7f51a1e00640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 0x7f518006d110 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5194033000 tx=0x7f5194008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:25.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.632+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f518c0545f0 con 0x7f51a41615f0 2024-05-10T03:33:25.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.825+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51a4067220 con 0x7f51a41615f0 2024-05-10T03:33:25.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.827+0000 7f519aa00640 1 -- 172.21.15.55:0/892351636 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+12724 (secure 0 0 0) 0x7f518c057c10 con 0x7f51a41615f0 2024-05-10T03:33:25.831 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:25.831 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":69,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:23.262697+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6835/1356084610":"2024-05-11T03:33:20.090797+0000","172.21.15.55:6834/1356084610":"2024-05-11T03:33:20.090797+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 msgr2=0x7f518006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 0x7f518006d110 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5194033000 tx=0x7f5194008030 comp rx=0 tx=0).stop 2024-05-10T03:33:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 msgr2=0x7f51a4150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e7c0 tx=0x7f518c00ec80 comp rx=0 tx=0).stop 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 shutdown_connections 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f518006ac00 0x7f518006d110 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51a406bf10 0x7f51a406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51a41615f0 0x7f51a4150280 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 --2- 172.21.15.55:0/892351636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5188003970 0x7f51a40a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.829+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 >> 172.21.15.55:0/892351636 conn(0x7f51a40cbb90 msgr2=0x7f51a415bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.830+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 shutdown_connections 2024-05-10T03:33:25.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:25.830+0000 7f51a9000640 1 -- 172.21.15.55:0/892351636 wait complete. 2024-05-10T03:33:25.861 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:25.862 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:26.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.128+0000 7f619a600640 1 Processor -- start 2024-05-10T03:33:26.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.128+0000 7f619a600640 1 -- start start 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f619a600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61940719f0 0x7f619406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f619a600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f619406b840 0x7f61940700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f619a600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619415fdf0 con 0x7f6194071e00 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f619a600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619415f1e0 con 0x7f619406b840 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f619a600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619415f4c0 con 0x7f61940719f0 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f6193e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f619406b840 0x7f61940700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f6193e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f619406b840 0x7f61940700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51936/0 (socket says 172.21.15.55:51936) 2024-05-10T03:33:26.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f6193e00640 1 -- 172.21.15.55:0/3833930399 learned_addr learned my addr 172.21.15.55:0/3833930399 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:26.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.131+0000 7f6199200640 1 --2- 172.21.15.55:0/3833930399 >> v2:172.21.15.162:3300/0 conn(0x7f61940719f0 0x7f619406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.132+0000 7f6193e00640 1 -- 172.21.15.55:0/3833930399 >> v2:172.21.15.162:3300/0 conn(0x7f61940719f0 msgr2=0x7f619406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:26.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.132+0000 7f6193e00640 1 --2- 172.21.15.55:0/3833930399 >> v2:172.21.15.162:3300/0 conn(0x7f61940719f0 0x7f619406b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.132+0000 7f6193e00640 1 -- 172.21.15.55:0/3833930399 >> v1:172.21.15.162:6789/0 conn(0x7f6194071e00 legacy=0x7f6194070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:26.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.132+0000 7f6193e00640 1 -- 172.21.15.55:0/3833930399 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619415f7a0 con 0x7f619406b840 2024-05-10T03:33:26.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.133+0000 7f6193e00640 1 --2- 172.21.15.55:0/3833930399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61940700e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f61940b41f0 tx=0x7f618002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b891c089efb1380 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.133+0000 7f6193400640 1 -- 172.21.15.55:0/3833930399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180032440 con 0x7f619406b840 2024-05-10T03:33:26.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.133+0000 7f6193400640 1 -- 172.21.15.55:0/3833930399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61800325a0 con 0x7f619406b840 2024-05-10T03:33:26.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.133+0000 7f6193400640 1 -- 172.21.15.55:0/3833930399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180032930 con 0x7f619406b840 2024-05-10T03:33:26.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 -- 172.21.15.55:0/3833930399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 msgr2=0x7f61940700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 --2- 172.21.15.55:0/3833930399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61940700e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f61940b41f0 tx=0x7f618002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 -- 172.21.15.55:0/3833930399 shutdown_connections 2024-05-10T03:33:26.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 --2- 172.21.15.55:0/3833930399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61940700e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 --2- 172.21.15.55:0/3833930399 >> v2:172.21.15.162:3300/0 conn(0x7f61940719f0 0x7f619406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.134+0000 7f619a600640 1 -- 172.21.15.55:0/3833930399 >> 172.21.15.55:0/3833930399 conn(0x7f619406c750 msgr2=0x7f61940b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:26.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.135+0000 7f619a600640 1 -- 172.21.15.55:0/3833930399 shutdown_connections 2024-05-10T03:33:26.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.135+0000 7f619a600640 1 -- 172.21.15.55:0/3833930399 wait complete. 2024-05-10T03:33:26.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.137+0000 7f619a600640 1 Processor -- start 2024-05-10T03:33:26.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.137+0000 7f619a600640 1 -- start start 2024-05-10T03:33:26.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61940719f0 0x7f619413c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6194071e00 0x7f61940adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619415a100 con 0x7f619406b840 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940ac990 con 0x7f61940719f0 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f619a600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6194159c30 con 0x7f6194071e00 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51948/0 (socket says 172.21.15.55:51948) 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 -- 172.21.15.55:0/687285275 learned_addr learned my addr 172.21.15.55:0/687285275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6193e00640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61940719f0 0x7f619413c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 -- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6194071e00 msgr2=0x7f61940adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6194071e00 0x7f61940adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 -- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61940719f0 msgr2=0x7f619413c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61940719f0 0x7f619413c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.139+0000 7f6199200640 1 -- 172.21.15.55:0/687285275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800094d0 con 0x7f619406b840 2024-05-10T03:33:26.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f6199200640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f618800d5f0 tx=0x7f618800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61880125f0 con 0x7f619406b840 2024-05-10T03:33:26.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188012750 con 0x7f619406b840 2024-05-10T03:33:26.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940acbe0 con 0x7f619406b840 2024-05-10T03:33:26.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188012b00 con 0x7f619406b840 2024-05-10T03:33:26.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.140+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61940ad1a0 con 0x7f619406b840 2024-05-10T03:33:26.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.141+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619404e290 con 0x7f619406b840 2024-05-10T03:33:26.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.142+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6188012c60 con 0x7f619406b840 2024-05-10T03:33:26.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.143+0000 7f6190c00640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 0x7f617406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.143+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6188089780 con 0x7f619406b840 2024-05-10T03:33:26.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.144+0000 7f6193e00640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 0x7f617406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.145+0000 7f6193e00640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 0x7f617406d110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6180034000 tx=0x7f618002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.148+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f61880549f0 con 0x7f619406b840 2024-05-10T03:33:26.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.347+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61941434c0 con 0x7f619406b840 2024-05-10T03:33:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.349+0000 7f6190c00640 1 -- 172.21.15.55:0/687285275 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6188058010 con 0x7f619406b840 2024-05-10T03:33:26.353 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 msgr2=0x7f617406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 0x7f617406d110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6180034000 tx=0x7f618002e000 comp rx=0 tx=0).stop 2024-05-10T03:33:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 msgr2=0x7f61941596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f618800d5f0 tx=0x7f618800dab0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 shutdown_connections 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6194071e00 0x7f61940adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f617406ac00 0x7f617406d110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61940719f0 0x7f619413c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 --2- 172.21.15.55:0/687285275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f619406b840 0x7f61941596f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 >> 172.21.15.55:0/687285275 conn(0x7f619406c750 msgr2=0x7f619415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 shutdown_connections 2024-05-10T03:33:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.352+0000 7f619a600640 1 -- 172.21.15.55:0/687285275 wait complete. 2024-05-10T03:33:26.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-05-10T03:33:26.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.665+0000 7f0d1b000640 1 Processor -- start 2024-05-10T03:33:26.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.665+0000 7f0d1b000640 1 -- start start 2024-05-10T03:33:26.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.667+0000 7f0d1b000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0d140ad7b0 0x7f0d14161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d1b000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d141623a0 con 0x7f0d14161c70 2024-05-10T03:33:26.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d1b000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140cdd60 con 0x7f0d140720f0 2024-05-10T03:33:26.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d1b000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140ce040 con 0x7f0d140ad7b0 2024-05-10T03:33:26.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d0b400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0d14161c70 0x7f0d140719e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45688/0 (socket says 172.21.15.55:45688) 2024-05-10T03:33:26.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d0b400640 1 -- 172.21.15.55:0/4090899827 learned_addr learned my addr 172.21.15.55:0/4090899827 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:26.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.668+0000 7f0d13e00640 1 --2- 172.21.15.55:0/4090899827 >> v2:172.21.15.162:3300/0 conn(0x7f0d140ad7b0 0x7f0d14161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626360228 0 0) 0x7f0d141623a0 con 0x7f0d14161c70 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf8003520 con 0x7f0d14161c70 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1494393563 0 0) 0x7f0cf8003520 con 0x7f0d14161c70 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13e00640 1 -- 172.21.15.55:0/4090899827 >> v1:172.21.15.162:6789/0 conn(0x7f0d140720f0 legacy=0x7f0d141541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13e00640 1 -- 172.21.15.55:0/4090899827 >> v1:172.21.15.55:6789/0 conn(0x7f0d14161c70 legacy=0x7f0d140719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.669+0000 7f0d13e00640 1 -- 172.21.15.55:0/4090899827 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140ce320 con 0x7f0d140ad7b0 2024-05-10T03:33:26.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.671+0000 7f0d13e00640 1 --2- 172.21.15.55:0/4090899827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140ad7b0 0x7f0d14161730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009800 tx=0x7f0d0402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1b859fe1563f27 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.671+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc001690 con 0x7f0d140ad7b0 2024-05-10T03:33:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.671+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0402ab90 con 0x7f0d140ad7b0 2024-05-10T03:33:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.671+0000 7f0d13400640 1 -- 172.21.15.55:0/4090899827 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04032440 con 0x7f0d140ad7b0 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 -- 172.21.15.55:0/4090899827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140ad7b0 msgr2=0x7f0d14161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 --2- 172.21.15.55:0/4090899827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140ad7b0 0x7f0d14161730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009800 tx=0x7f0d0402a100 comp rx=0 tx=0).stop 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 -- 172.21.15.55:0/4090899827 shutdown_connections 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 --2- 172.21.15.55:0/4090899827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140ad7b0 0x7f0d14161730 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 -- 172.21.15.55:0/4090899827 >> 172.21.15.55:0/4090899827 conn(0x7f0d140cbb90 msgr2=0x7f0d140cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 -- 172.21.15.55:0/4090899827 shutdown_connections 2024-05-10T03:33:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.672+0000 7f0d1b000640 1 -- 172.21.15.55:0/4090899827 wait complete. 2024-05-10T03:33:26.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.675+0000 7f0d1b000640 1 Processor -- start 2024-05-10T03:33:26.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.675+0000 7f0d1b000640 1 -- start start 2024-05-10T03:33:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140720f0 0x7f0d1415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d140ad7b0 0x7f0d141427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141423a0 con 0x7f0d14161c70 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415da80 con 0x7f0d140720f0 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d1b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141e7760 con 0x7f0d140ad7b0 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d19000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d19000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51950/0 (socket says 172.21.15.55:51950) 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d19000640 1 -- 172.21.15.55:0/1103507800 learned_addr learned my addr 172.21.15.55:0/1103507800 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d13e00640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140720f0 0x7f0d1415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.677+0000 7f0d0b400640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d140ad7b0 0x7f0d141427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 -- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d140ad7b0 msgr2=0x7f0d141427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d140ad7b0 0x7f0d141427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 -- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140720f0 msgr2=0x7f0d1415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140720f0 0x7f0d1415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 -- 172.21.15.55:0/1103507800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04009400 con 0x7f0d14161c70 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d19000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c6b0 tx=0x7f0d0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003b70 con 0x7f0d14161c70 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c003cd0 con 0x7f0d14161c70 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c005310 con 0x7f0d14161c70 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.678+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14141720 con 0x7f0d14161c70 2024-05-10T03:33:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.679+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14141ce0 con 0x7f0d14161c70 2024-05-10T03:33:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.680+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d141424e0 con 0x7f0d14161c70 2024-05-10T03:33:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.682+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d0c005470 con 0x7f0d14161c70 2024-05-10T03:33:26.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.682+0000 7f0d10c00640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 0x7f0cf006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:26.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.683+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0d0c088f30 con 0x7f0d14161c70 2024-05-10T03:33:26.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.683+0000 7f0d13e00640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 0x7f0cf006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:26.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.685+0000 7f0d13e00640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 0x7f0cf006d110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d0402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:26.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.688+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0d141424e0 con 0x7f0d14161c70 2024-05-10T03:33:26.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:26.964+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) v1 -- 0x7f0d141424e0 con 0x7f0d14161c70 2024-05-10T03:33:27.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.305+0000 7f0d10c00640 1 -- 172.21.15.55:0/1103507800 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v239) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0d141424e0 con 0x7f0d14161c70 2024-05-10T03:33:27.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 msgr2=0x7f0cf006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 0x7f0cf006d110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d0402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 msgr2=0x7f0d14142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c6b0 tx=0x7f0d0c00cb70 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 shutdown_connections 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.309+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d14161c70 0x7f0d14142d00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d140ad7b0 0x7f0d141427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf006ac00 0x7f0cf006d110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 --2- 172.21.15.55:0/1103507800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d140720f0 0x7f0d1415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 >> 172.21.15.55:0/1103507800 conn(0x7f0d140cbb90 msgr2=0x7f0d140aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 shutdown_connections 2024-05-10T03:33:27.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.310+0000 7f0d1b000640 1 -- 172.21.15.55:0/1103507800 wait complete. 2024-05-10T03:33:27.344 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:33:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.632+0000 7f5dd7e00640 1 Processor -- start 2024-05-10T03:33:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.632+0000 7f5dd7e00640 1 -- start start 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5dd806b120 0x7f5dd806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dd80710f0 0x7f5dd8071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd815a900 con 0x7f5dd80710f0 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd80b2a30 con 0x7f5dd806b120 2024-05-10T03:33:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815eec0 con 0x7f5dd815e920 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dd80710f0 0x7f5dd8071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dd80710f0 0x7f5dd8071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51964/0 (socket says 172.21.15.55:51964) 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.635+0000 7f5dd7400640 1 -- 172.21.15.55:0/2321446483 learned_addr learned my addr 172.21.15.55:0/2321446483 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd6a00640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dcfe00640 1 --2- 172.21.15.55:0/2321446483 >> v2:172.21.15.162:3300/0 conn(0x7f5dd806b120 0x7f5dd806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd7400640 1 -- 172.21.15.55:0/2321446483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 msgr2=0x7f5dd80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd7400640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd80cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd7400640 1 -- 172.21.15.55:0/2321446483 >> v2:172.21.15.162:3300/0 conn(0x7f5dd806b120 msgr2=0x7f5dd806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd7400640 1 --2- 172.21.15.55:0/2321446483 >> v2:172.21.15.162:3300/0 conn(0x7f5dd806b120 0x7f5dd806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.636+0000 7f5dd7400640 1 -- 172.21.15.55:0/2321446483 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd806ba70 con 0x7f5dd80710f0 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.637+0000 7f5dd7400640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd8071500 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8072210 tx=0x7f5dd002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322844dd1af94d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.637+0000 7f5dd6000640 1 -- 172.21.15.55:0/2321446483 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0031600 con 0x7f5dd80710f0 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.637+0000 7f5dd6000640 1 -- 172.21.15.55:0/2321446483 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd002d050 con 0x7f5dd80710f0 2024-05-10T03:33:27.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.637+0000 7f5dd6000640 1 -- 172.21.15.55:0/2321446483 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0031980 con 0x7f5dd80710f0 2024-05-10T03:33:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 -- 172.21.15.55:0/2321446483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 msgr2=0x7f5dd8071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd8071500 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8072210 tx=0x7f5dd002a6a0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 -- 172.21.15.55:0/2321446483 shutdown_connections 2024-05-10T03:33:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd8071500 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/2321446483 >> v2:172.21.15.162:3300/0 conn(0x7f5dd806b120 0x7f5dd806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/2321446483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 -- 172.21.15.55:0/2321446483 >> 172.21.15.55:0/2321446483 conn(0x7f5dd806c750 msgr2=0x7f5dd80b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 -- 172.21.15.55:0/2321446483 shutdown_connections 2024-05-10T03:33:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.638+0000 7f5dd7e00640 1 -- 172.21.15.55:0/2321446483 wait complete. 2024-05-10T03:33:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.641+0000 7f5dd7e00640 1 Processor -- start 2024-05-10T03:33:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.641+0000 7f5dd7e00640 1 -- start start 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dd806b120 0x7f5dd813c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd8150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd81424b0 con 0x7f5dd80710f0 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd8152850 con 0x7f5dd806b120 2024-05-10T03:33:27.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.642+0000 7f5dd7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd813d1e0 con 0x7f5dd815e920 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51976/0 (socket says 172.21.15.55:51976) 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 -- 172.21.15.55:0/884037192 learned_addr learned my addr 172.21.15.55:0/884037192 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dd6a00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dd806b120 0x7f5dd813c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dd7400640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 -- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 msgr2=0x7f5dd8150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd8150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 -- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dd806b120 msgr2=0x7f5dd813c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dd806b120 0x7f5dd813c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dcfe00640 1 -- 172.21.15.55:0/884037192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009b30 con 0x7f5dd80710f0 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.643+0000 7f5dd7400640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd8150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:27.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dcfe00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000d5c0 tx=0x7f5dc000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00125f0 con 0x7f5dd80710f0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0012750 con 0x7f5dd80710f0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd813cc20 con 0x7f5dd80710f0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0012b00 con 0x7f5dd80710f0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.644+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd8140c50 con 0x7f5dd80710f0 2024-05-10T03:33:27.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.645+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd804e290 con 0x7f5dd80710f0 2024-05-10T03:33:27.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.646+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5dc0012c60 con 0x7f5dd80710f0 2024-05-10T03:33:27.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.647+0000 7f5dcea00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 0x7f5db406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:27.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.647+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5dc0089610 con 0x7f5dd80710f0 2024-05-10T03:33:27.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.648+0000 7f5dd6a00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 0x7f5db406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.649+0000 7f5dd6a00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 0x7f5db406d110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:27.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.652+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5dc00549b0 con 0x7f5dd80710f0 2024-05-10T03:33:27.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.851+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5dd80b2300 con 0x7f5dd80710f0 2024-05-10T03:33:27.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.859+0000 7f5dcea00640 1 -- 172.21.15.55:0/884037192 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5dc0057fd0 con 0x7f5dd80710f0 2024-05-10T03:33:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 msgr2=0x7f5db406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 0x7f5db406d110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-05-10T03:33:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 msgr2=0x7f5dd813c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000d5c0 tx=0x7f5dc000da80 comp rx=0 tx=0).stop 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 shutdown_connections 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dd815e920 0x7f5dd8150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dd80710f0 0x7f5dd813c5c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5db406ac00 0x7f5db406d110 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 --2- 172.21.15.55:0/884037192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dd806b120 0x7f5dd813c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.862+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 >> 172.21.15.55:0/884037192 conn(0x7f5dd806c750 msgr2=0x7f5dd814f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.863+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 shutdown_connections 2024-05-10T03:33:27.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:27.863+0000 7f5dd7e00640 1 -- 172.21.15.55:0/884037192 wait complete. 2024-05-10T03:33:27.895 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:33:27.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:27.999 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:28.106 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:33:28.107 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:28.107 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:28.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:33:28.168 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:28.168 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:28.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:33:28.226 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:28.226 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:28.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:33:28.287 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:28.287 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:28.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:33:28.345 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:33:28.346 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:28.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.633+0000 7f0f2cc00640 1 Processor -- start 2024-05-10T03:33:28.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.633+0000 7f0f2cc00640 1 -- start start 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f2cc00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 0x7f0f28161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f2cc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f28071100 0x7f0f28071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f2cc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f28161c40 con 0x7f0f281549e0 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f2cc00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280b39f0 con 0x7f0f28071100 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f2cc00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280b3cd0 con 0x7f0f281612f0 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f27e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f28071100 0x7f0f28071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f27e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0f28071100 0x7f0f28071510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51978/0 (socket says 172.21.15.55:51978) 2024-05-10T03:33:28.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f27e00640 1 -- 172.21.15.55:0/1883745249 learned_addr learned my addr 172.21.15.55:0/1883745249 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.635+0000 7f0f26a00640 1 --2- 172.21.15.55:0/1883745249 >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 0x7f0f28161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.636+0000 7f0f27e00640 1 -- 172.21.15.55:0/1883745249 >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 msgr2=0x7f0f28161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.636+0000 7f0f27e00640 1 --2- 172.21.15.55:0/1883745249 >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 0x7f0f28161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.636+0000 7f0f27e00640 1 -- 172.21.15.55:0/1883745249 >> v1:172.21.15.162:6789/0 conn(0x7f0f281549e0 legacy=0x7f0f280b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.636+0000 7f0f27e00640 1 -- 172.21.15.55:0/1883745249 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280b3fb0 con 0x7f0f28071100 2024-05-10T03:33:28.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.636+0000 7f0f26a00640 1 --2- 172.21.15.55:0/1883745249 >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 0x7f0f28161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:28.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.637+0000 7f0f27e00640 1 --2- 172.21.15.55:0/1883745249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28071510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d73b9db3bce3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:28.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.637+0000 7f0f26000640 1 -- 172.21.15.55:0/1883745249 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20032440 con 0x7f0f28071100 2024-05-10T03:33:28.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.637+0000 7f0f26000640 1 -- 172.21.15.55:0/1883745249 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f200325a0 con 0x7f0f28071100 2024-05-10T03:33:28.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.637+0000 7f0f26000640 1 -- 172.21.15.55:0/1883745249 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20032930 con 0x7f0f28071100 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1883745249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 msgr2=0x7f0f28071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1883745249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28071510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1883745249 shutdown_connections 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1883745249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28071510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1883745249 >> v2:172.21.15.162:3300/0 conn(0x7f0f281612f0 0x7f0f28161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1883745249 >> 172.21.15.55:0/1883745249 conn(0x7f0f2806a110 msgr2=0x7f0f2806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1883745249 shutdown_connections 2024-05-10T03:33:28.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.638+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1883745249 wait complete. 2024-05-10T03:33:28.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.641+0000 7f0f2cc00640 1 Processor -- start 2024-05-10T03:33:28.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.641+0000 7f0f2cc00640 1 -- start start 2024-05-10T03:33:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f281549e0 0x7f0f281e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f281612f0 0x7f0f2815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280cd6c0 con 0x7f0f28071100 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281f7d40 con 0x7f0f281549e0 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.642+0000 7f0f2cc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28143ff0 con 0x7f0f281612f0 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51988/0 (socket says 172.21.15.55:51988) 2024-05-10T03:33:28.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 -- 172.21.15.55:0/1153611314 learned_addr learned my addr 172.21.15.55:0/1153611314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:28.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27e00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f281612f0 0x7f0f2815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f26a00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f281549e0 0x7f0f281e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 -- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f281612f0 msgr2=0x7f0f2815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f281612f0 0x7f0f2815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 -- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f281549e0 msgr2=0x7f0f281e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f281549e0 0x7f0f281e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.643+0000 7f0f27400640 1 -- 172.21.15.55:0/1153611314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200094d0 con 0x7f0f28071100 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f27400640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800b9c0 tx=0x7f0f1800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800cba0 con 0x7f0f28071100 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1800cd00 con 0x7f0f28071100 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18003bb0 con 0x7f0f28071100 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281f7130 con 0x7f0f28071100 2024-05-10T03:33:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.644+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f281f76f0 con 0x7f0f28071100 2024-05-10T03:33:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.645+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2804e290 con 0x7f0f28071100 2024-05-10T03:33:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.647+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f18014440 con 0x7f0f28071100 2024-05-10T03:33:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.647+0000 7f0f1f400640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 0x7f0f0406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.648+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f0f180890c0 con 0x7f0f28071100 2024-05-10T03:33:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.648+0000 7f0f26a00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 0x7f0f0406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.650+0000 7f0f26a00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 0x7f0f0406d110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ccb90 tx=0x7f0f10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.652+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f18054330 con 0x7f0f28071100 2024-05-10T03:33:28.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.894+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f28160cc0 con 0x7f0f28071100 2024-05-10T03:33:28.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.896+0000 7f0f1f400640 1 -- 172.21.15.55:0/1153611314 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0f18057950 con 0x7f0f28071100 2024-05-10T03:33:28.900 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:28.900 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1697555244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":1697555244},{"type":"v1","addr":"172.21.15.162:6837","nonce":1697555244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2961246553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2961246553},{"type":"v1","addr":"172.21.15.162:6835","nonce":2961246553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6126,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/756940436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":756940436},{"type":"v1","addr":"172.21.15.55:6833","nonce":756940436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6450,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4154339071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":4154339071},{"type":"v1","addr":"172.21.15.55:6835","nonce":4154339071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[]} 2024-05-10T03:33:28.900 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 33 2024-05-10T03:33:28.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 msgr2=0x7f0f0406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:28.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 0x7f0f0406d110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ccb90 tx=0x7f0f10008040 comp rx=0 tx=0).stop 2024-05-10T03:33:28.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 msgr2=0x7f0f28143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:28.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800b9c0 tx=0x7f0f1800be80 comp rx=0 tx=0).stop 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 shutdown_connections 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f281612f0 0x7f0f2815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f0406ac00 0x7f0f0406d110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f281549e0 0x7f0f281e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 --2- 172.21.15.55:0/1153611314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f28071100 0x7f0f28143ab0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.898+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 >> 172.21.15.55:0/1153611314 conn(0x7f0f2806a110 msgr2=0x7f0f280b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.899+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 shutdown_connections 2024-05-10T03:33:28.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:28.899+0000 7f0f2cc00640 1 -- 172.21.15.55:0/1153611314 wait complete. 2024-05-10T03:33:28.931 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:33:28.932 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:33:28.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:33:28.932 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:33:28.932 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:33:28.932 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:33:28.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:33:28.932 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:33:28.933 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:33:28.933 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:33:28.933 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:33:28.933 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:33:28.933 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:33:28.933 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:33:28.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:33:28.933 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:33:28.957 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:33:28.957 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:33:28.970 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:33:28.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:33:28.973 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:33:28.973 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:33:28.979 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:33:28.979 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:33:29.017 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:33:29.017 DEBUG:teuthology.parallel:result is None 2024-05-10T03:33:29.024 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:33:29.024 DEBUG:teuthology.parallel:result is None 2024-05-10T03:33:29.026 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:33:29.026 DEBUG:teuthology.parallel:result is None 2024-05-10T03:33:29.032 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:33:29.032 DEBUG:teuthology.parallel:result is None 2024-05-10T03:33:29.032 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:33:29.242 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:33:29.243 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:33:29.253 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:33:29.255 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:33:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.289+0000 7f7d33000640 1 Processor -- start 2024-05-10T03:33:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.289+0000 7f7d33000640 1 -- start start 2024-05-10T03:33:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d33000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d2c071db0 0x7f7d2c158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d33000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d33000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0708b0 con 0x7f7d2c0b3860 2024-05-10T03:33:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d33000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0b2490 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d33000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c070050 con 0x7f7d2c071db0 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d23e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.291+0000 7f7d23e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52006/0 (socket says 172.21.15.55:52006) 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d23e00640 1 -- 172.21.15.55:0/84497399 learned_addr learned my addr 172.21.15.55:0/84497399 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d2be00640 1 --2- 172.21.15.55:0/84497399 >> v2:172.21.15.162:3300/0 conn(0x7f7d2c071db0 0x7f7d2c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d23e00640 1 -- 172.21.15.55:0/84497399 >> v2:172.21.15.162:3300/0 conn(0x7f7d2c071db0 msgr2=0x7f7d2c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d23e00640 1 --2- 172.21.15.55:0/84497399 >> v2:172.21.15.162:3300/0 conn(0x7f7d2c071db0 0x7f7d2c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d23e00640 1 -- 172.21.15.55:0/84497399 >> v1:172.21.15.162:6789/0 conn(0x7f7d2c0b3860 legacy=0x7f7d2c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.292+0000 7f7d23e00640 1 -- 172.21.15.55:0/84497399 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c15b940 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.293+0000 7f7d23e00640 1 --2- 172.21.15.55:0/84497399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ceb80 tx=0x7f7d1402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886e8593fe4fb045 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.293+0000 7f7d2b400640 1 -- 172.21.15.55:0/84497399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14032710 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.293+0000 7f7d2b400640 1 -- 172.21.15.55:0/84497399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14030050 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.293+0000 7f7d2b400640 1 -- 172.21.15.55:0/84497399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14032ab0 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 -- 172.21.15.55:0/84497399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 msgr2=0x7f7d2c0ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 --2- 172.21.15.55:0/84497399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0ceb80 tx=0x7f7d1402c810 comp rx=0 tx=0).stop 2024-05-10T03:33:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 -- 172.21.15.55:0/84497399 shutdown_connections 2024-05-10T03:33:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 --2- 172.21.15.55:0/84497399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c0ccf90 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 --2- 172.21.15.55:0/84497399 >> v2:172.21.15.162:3300/0 conn(0x7f7d2c071db0 0x7f7d2c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 -- 172.21.15.55:0/84497399 >> 172.21.15.55:0/84497399 conn(0x7f7d2c0cbb90 msgr2=0x7f7d2c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 -- 172.21.15.55:0/84497399 shutdown_connections 2024-05-10T03:33:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.294+0000 7f7d33000640 1 -- 172.21.15.55:0/84497399 wait complete. 2024-05-10T03:33:29.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.297+0000 7f7d33000640 1 Processor -- start 2024-05-10T03:33:29.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.297+0000 7f7d33000640 1 -- start start 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2c071db0 0x7f7d2c13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 0x7f7d2c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1efa30 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c141930 con 0x7f7d2c0b3860 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.298+0000 7f7d33000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0a9400 con 0x7f7d2c071db0 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52016/0 (socket says 172.21.15.55:52016) 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 -- 172.21.15.55:0/3903186329 learned_addr learned my addr 172.21.15.55:0/3903186329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d23e00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2c071db0 0x7f7d2c13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d31000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 0x7f7d2c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 -- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2c071db0 msgr2=0x7f7d2c13c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2c071db0 0x7f7d2c13c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 -- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 msgr2=0x7f7d2c0afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 0x7f7d2c0afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d2be00640 1 -- 172.21.15.55:0/3903186329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1400b770 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.299+0000 7f7d31000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 0x7f7d2c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d2be00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e8a0 tx=0x7f7d1c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ee90 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00c940 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c140160 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00cd10 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.300+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c0a9930 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.301+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c066f60 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.302+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7d1c013470 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.303+0000 7f7d28c00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 0x7f7d0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:29.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.303+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7d1c089340 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.304+0000 7f7d23e00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 0x7f7d0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:29.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.304+0000 7f7d23e00640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 0x7f7d0806d140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7d14037000 tx=0x7f7d14032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:29.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.308+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7d1c0545b0 con 0x7f7d2c06a4b0 2024-05-10T03:33:29.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:29.515+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7d2c067220 con 0x7f7d2c06a4b0 2024-05-10T03:33:30.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.339+0000 7f7d28c00640 1 -- 172.21.15.55:0/3903186329 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7d1c057bd0 con 0x7f7d2c06a4b0 2024-05-10T03:33:30.343 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:33:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 msgr2=0x7f7d0806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 0x7f7d0806d140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7d14037000 tx=0x7f7d14032000 comp rx=0 tx=0).stop 2024-05-10T03:33:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 msgr2=0x7f7d2c1e4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e8a0 tx=0x7f7d1c00ed60 comp rx=0 tx=0).stop 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 shutdown_connections 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7d0806ac30 0x7f7d0806d140 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d2c0b3860 0x7f7d2c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d2c071db0 0x7f7d2c13c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 --2- 172.21.15.55:0/3903186329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d2c06a4b0 0x7f7d2c1e4d20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 >> 172.21.15.55:0/3903186329 conn(0x7f7d2c0cbb90 msgr2=0x7f7d2c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 shutdown_connections 2024-05-10T03:33:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.343+0000 7f7d33000640 1 -- 172.21.15.55:0/3903186329 wait complete. 2024-05-10T03:33:30.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:33:30.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.651+0000 7ff4e0c00640 1 Processor -- start 2024-05-10T03:33:30.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.651+0000 7ff4e0c00640 1 -- start start 2024-05-10T03:33:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.653+0000 7ff4e0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.653+0000 7ff4e0c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff4dc1618b0 0x7ff4dc06aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.653+0000 7ff4e0c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0d25a0 con 0x7ff4dc0b2170 2024-05-10T03:33:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.653+0000 7ff4e0c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0d2b20 con 0x7ff4dc1618b0 2024-05-10T03:33:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.653+0000 7ff4e0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc15db20 con 0x7ff4dc0b6680 2024-05-10T03:33:30.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.654+0000 7ff4db400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.654+0000 7ff4db400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:58576/0 (socket says 172.21.15.55:58576) 2024-05-10T03:33:30.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.654+0000 7ff4daa00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff4dc0b2170 0x7ff4dc1611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:46374/0 (socket says 172.21.15.55:46374) 2024-05-10T03:33:30.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.654+0000 7ff4db400640 1 -- 172.21.15.55:0/4090603730 learned_addr learned my addr 172.21.15.55:0/4090603730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:30.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.654+0000 7ff4dbe00640 1 --2- 172.21.15.55:0/4090603730 >> v2:172.21.15.162:3300/0 conn(0x7ff4dc1618b0 0x7ff4dc06aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.655+0000 7ff4db400640 1 -- 172.21.15.55:0/4090603730 >> v2:172.21.15.162:3300/0 conn(0x7ff4dc1618b0 msgr2=0x7ff4dc06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:30.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.655+0000 7ff4db400640 1 --2- 172.21.15.55:0/4090603730 >> v2:172.21.15.162:3300/0 conn(0x7ff4dc1618b0 0x7ff4dc06aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.655+0000 7ff4db400640 1 -- 172.21.15.55:0/4090603730 >> v1:172.21.15.162:6789/0 conn(0x7ff4dc0b2170 legacy=0x7ff4dc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.655+0000 7ff4db400640 1 -- 172.21.15.55:0/4090603730 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0b28d0 con 0x7ff4dc0b6680 2024-05-10T03:33:30.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.656+0000 7ff4db400640 1 --2- 172.21.15.55:0/4090603730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009870 tx=0x7ff4cc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7236521ecfbb575c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.657+0000 7ff4da000640 1 -- 172.21.15.55:0/4090603730 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032440 con 0x7ff4dc0b6680 2024-05-10T03:33:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.657+0000 7ff4da000640 1 -- 172.21.15.55:0/4090603730 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0325a0 con 0x7ff4dc0b6680 2024-05-10T03:33:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.657+0000 7ff4da000640 1 -- 172.21.15.55:0/4090603730 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032930 con 0x7ff4dc0b6680 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 -- 172.21.15.55:0/4090603730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 msgr2=0x7ff4dc15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/4090603730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009870 tx=0x7ff4cc02a270 comp rx=0 tx=0).stop 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 -- 172.21.15.55:0/4090603730 shutdown_connections 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/4090603730 >> v2:172.21.15.162:3300/0 conn(0x7ff4dc1618b0 0x7ff4dc06aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/4090603730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc15d5e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 -- 172.21.15.55:0/4090603730 >> 172.21.15.55:0/4090603730 conn(0x7ff4dc0b5730 msgr2=0x7ff4dc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 -- 172.21.15.55:0/4090603730 shutdown_connections 2024-05-10T03:33:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.658+0000 7ff4e0c00640 1 -- 172.21.15.55:0/4090603730 wait complete. 2024-05-10T03:33:30.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.661+0000 7ff4e0c00640 1 Processor -- start 2024-05-10T03:33:30.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.661+0000 7ff4e0c00640 1 -- start start 2024-05-10T03:33:30.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4dc1618b0 0x7ff4dc1e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc158d90 con 0x7ff4dc0b2170 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1e9120 con 0x7ff4dc1618b0 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.663+0000 7ff4e0c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc158f00 con 0x7ff4dc0b6680 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52028/0 (socket says 172.21.15.55:52028) 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 -- 172.21.15.55:0/96153319 learned_addr learned my addr 172.21.15.55:0/96153319 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4dbe00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4dc1618b0 0x7ff4dc1e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4daa00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 -- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 msgr2=0x7ff4dc1e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc1e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 -- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4dc1618b0 msgr2=0x7ff4dc1e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4dc1618b0 0x7ff4dc1e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.664+0000 7ff4db400640 1 -- 172.21.15.55:0/96153319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0094d0 con 0x7ff4dc0b2170 2024-05-10T03:33:30.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4db400640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:30.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032440 con 0x7ff4dc0b2170 2024-05-10T03:33:30.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc030980 con 0x7ff4dc0b2170 2024-05-10T03:33:30.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc15a1f0 con 0x7ff4dc0b2170 2024-05-10T03:33:30.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc030d30 con 0x7ff4dc0b2170 2024-05-10T03:33:30.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.665+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc159430 con 0x7ff4dc0b2170 2024-05-10T03:33:30.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.666+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc067740 con 0x7ff4dc0b2170 2024-05-10T03:33:30.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.667+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4cc039040 con 0x7ff4dc0b2170 2024-05-10T03:33:30.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.668+0000 7ff4d3400640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 0x7ff4bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:30.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.669+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff4cc0aa1d0 con 0x7ff4dc0b2170 2024-05-10T03:33:30.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.669+0000 7ff4daa00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 0x7ff4bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:30.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.670+0000 7ff4daa00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 0x7ff4bc06d140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc06b380 tx=0x7ff4c8007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:30.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.672+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff4cc075630 con 0x7ff4dc0b2170 2024-05-10T03:33:30.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.947+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff4dc04e290 con 0x7ff4dc0b2170 2024-05-10T03:33:30.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.949+0000 7ff4d3400640 1 -- 172.21.15.55:0/96153319 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff4cc078c50 con 0x7ff4dc0b2170 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:33:30.953 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.954 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.955 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.957 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.959 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.960 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-10T03:33:30.961 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.962 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:33:30.963 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:33:30.964 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.965 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:30.966 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:33:30.966 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:33:30.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 msgr2=0x7ff4bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 0x7ff4bc06d140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc06b380 tx=0x7ff4c8007ca0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 msgr2=0x7ff4dc149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 shutdown_connections 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff4bc06ac30 0x7ff4bc06d140 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4dc1618b0 0x7ff4dc1e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4dc0b6680 0x7ff4dc1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 --2- 172.21.15.55:0/96153319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4dc0b2170 0x7ff4dc149420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 >> 172.21.15.55:0/96153319 conn(0x7ff4dc0b5730 msgr2=0x7ff4dc0b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 shutdown_connections 2024-05-10T03:33:30.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:30.952+0000 7ff4e0c00640 1 -- 172.21.15.55:0/96153319 wait complete. 2024-05-10T03:33:30.983 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.259+0000 7f620f200640 1 Processor -- start 2024-05-10T03:33:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.259+0000 7f620f200640 1 -- start start 2024-05-10T03:33:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.261+0000 7f620f200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62080ce5f0 0x7f6208157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.261+0000 7f620f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 0x7f62080b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.261+0000 7f620f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620815edc0 con 0x7f620815ddd0 2024-05-10T03:33:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.261+0000 7f620f200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62081623d0 con 0x7f62080ce5f0 2024-05-10T03:33:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.261+0000 7f620f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620815e1e0 con 0x7f62081613c0 2024-05-10T03:33:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.262+0000 7f620d400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f62080ce5f0 0x7f6208157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.262+0000 7f620de00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f620815ddd0 0x7f62080cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:46382/0 (socket says 172.21.15.55:46382) 2024-05-10T03:33:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.262+0000 7f620de00640 1 -- 172.21.15.55:0/359837683 learned_addr learned my addr 172.21.15.55:0/359837683 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.262+0000 7f620e800640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 0x7f62080b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620ca00640 1 -- 172.21.15.55:0/359837683 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765884967 0 0) 0x7f620815edc0 con 0x7f620815ddd0 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620ca00640 1 -- 172.21.15.55:0/359837683 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f4003520 con 0x7f620815ddd0 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620d400640 1 -- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 msgr2=0x7f62080b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620d400640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 0x7f62080b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620d400640 1 -- 172.21.15.55:0/359837683 >> v1:172.21.15.162:6789/0 conn(0x7f620815ddd0 legacy=0x7f62080cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620d400640 1 -- 172.21.15.55:0/359837683 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080cecd0 con 0x7f62080ce5f0 2024-05-10T03:33:31.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.263+0000 7f620e800640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 0x7f62080b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:33:31.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.265+0000 7f620d400640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62080ce5f0 0x7f6208157e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097a0 tx=0x7f61f802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156c7ff2b6d6acdf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.265+0000 7f620ca00640 1 -- 172.21.15.55:0/359837683 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f802ae50 con 0x7f62080ce5f0 2024-05-10T03:33:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.265+0000 7f620ca00640 1 -- 172.21.15.55:0/359837683 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f8032440 con 0x7f62080ce5f0 2024-05-10T03:33:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.265+0000 7f620ca00640 1 -- 172.21.15.55:0/359837683 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80327b0 con 0x7f62080ce5f0 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 -- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62080ce5f0 msgr2=0x7f6208157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62080ce5f0 0x7f6208157e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097a0 tx=0x7f61f802a100 comp rx=0 tx=0).stop 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 -- 172.21.15.55:0/359837683 shutdown_connections 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f62081613c0 0x7f62080b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 --2- 172.21.15.55:0/359837683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62080ce5f0 0x7f6208157e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 -- 172.21.15.55:0/359837683 >> 172.21.15.55:0/359837683 conn(0x7f62080d0370 msgr2=0x7f62080d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:31.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 -- 172.21.15.55:0/359837683 shutdown_connections 2024-05-10T03:33:31.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.266+0000 7f620f200640 1 -- 172.21.15.55:0/359837683 wait complete. 2024-05-10T03:33:31.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.269+0000 7f620f200640 1 Processor -- start 2024-05-10T03:33:31.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.269+0000 7f620f200640 1 -- start start 2024-05-10T03:33:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 0x7f620815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62081613c0 0x7f6208141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208141dd0 con 0x7f62081584c0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208141b90 con 0x7f62081613c0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.270+0000 7f620f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6208142020 con 0x7f620815ddd0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52030/0 (socket says 172.21.15.55:52030) 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 -- 172.21.15.55:0/1911085617 learned_addr learned my addr 172.21.15.55:0/1911085617 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620de00640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 0x7f620815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620d400640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62081613c0 0x7f6208141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 -- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 msgr2=0x7f620815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 0x7f620815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 -- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62081613c0 msgr2=0x7f6208141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62081613c0 0x7f6208141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.271+0000 7f620e800640 1 -- 172.21.15.55:0/1911085617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204009400 con 0x7f62081584c0 2024-05-10T03:33:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f620de00640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 0x7f620815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f620e800640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f620400e880 tx=0x7f620400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204009660 con 0x7f62081584c0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6204004020 con 0x7f62081584c0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040043b0 con 0x7f62081584c0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8009400 con 0x7f62081584c0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.272+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620813f400 con 0x7f62081584c0 2024-05-10T03:33:31.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.273+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620804e290 con 0x7f62081584c0 2024-05-10T03:33:31.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.275+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f620400c940 con 0x7f62081584c0 2024-05-10T03:33:31.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.276+0000 7f61fe000640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 0x7f61ec06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.276+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f6204089070 con 0x7f62081584c0 2024-05-10T03:33:31.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.277+0000 7f620de00640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 0x7f61ec06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.278+0000 7f620de00640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 0x7f61ec06d110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6200001560 tx=0x7f6200009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.281+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f62040bc900 con 0x7f62081584c0 2024-05-10T03:33:31.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.483+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62080b27e0 con 0x7f62081584c0 2024-05-10T03:33:31.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.485+0000 7f61fe000640 1 -- 172.21.15.55:0/1911085617 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+12595 (secure 0 0 0) 0x7f62040543c0 con 0x7f62081584c0 2024-05-10T03:33:31.490 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:31.490 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":74,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:30.317502+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 msgr2=0x7f61ec06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 0x7f61ec06d110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6200001560 tx=0x7f6200009040 comp rx=0 tx=0).stop 2024-05-10T03:33:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 msgr2=0x7f6208141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f620400e880 tx=0x7f620400ed40 comp rx=0 tx=0).stop 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 shutdown_connections 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62081584c0 0x7f6208141650 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f61ec06ac00 0x7f61ec06d110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62081613c0 0x7f6208141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 --2- 172.21.15.55:0/1911085617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f620815ddd0 0x7f620815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 >> 172.21.15.55:0/1911085617 conn(0x7f62080d0370 msgr2=0x7f62080b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 shutdown_connections 2024-05-10T03:33:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.488+0000 7f620f200640 1 -- 172.21.15.55:0/1911085617 wait complete. 2024-05-10T03:33:31.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:31.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:31.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.804+0000 7fe1f1400640 1 Processor -- start 2024-05-10T03:33:31.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.805+0000 7fe1f1400640 1 -- start start 2024-05-10T03:33:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f1400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1ec15e920 0x7fe1ec0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec06b120 0x7fe1ec06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f1400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0b2a30 con 0x7fe1ec0710f0 2024-05-10T03:33:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f1400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec15a900 con 0x7fe1ec15e920 2024-05-10T03:33:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f1400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec15eec0 con 0x7fe1ec06b120 2024-05-10T03:33:31.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1ebe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1ec15e920 0x7fe1ec0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f0a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1ec0710f0 0x7fe1ec071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45702/0 (socket says 172.21.15.55:45702) 2024-05-10T03:33:31.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.807+0000 7fe1f0a00640 1 -- 172.21.15.55:0/1895214574 learned_addr learned my addr 172.21.15.55:0/1895214574 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:31.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1e3e00640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec06b120 0x7fe1ec06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1eb400640 1 -- 172.21.15.55:0/1895214574 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935744186 0 0) 0x7fe1ec0b2a30 con 0x7fe1ec0710f0 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1eb400640 1 -- 172.21.15.55:0/1895214574 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d0003520 con 0x7fe1ec0710f0 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1895214574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec06b120 msgr2=0x7fe1ec06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1ebe00640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec06b120 0x7fe1ec06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1895214574 >> v1:172.21.15.55:6789/0 conn(0x7fe1ec0710f0 legacy=0x7fe1ec071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.808+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1895214574 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec06ba70 con 0x7fe1ec15e920 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.809+0000 7fe1ebe00640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec15e920 0x7fe1ec0cc240 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0097a0 tx=0x7fe1dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb0a6cb646e204c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.810+0000 7fe1eb400640 1 -- 172.21.15.55:0/1895214574 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032440 con 0x7fe1ec15e920 2024-05-10T03:33:31.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.810+0000 7fe1eb400640 1 -- 172.21.15.55:0/1895214574 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0325a0 con 0x7fe1ec15e920 2024-05-10T03:33:31.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.810+0000 7fe1eb400640 1 -- 172.21.15.55:0/1895214574 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032910 con 0x7fe1ec15e920 2024-05-10T03:33:31.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.810+0000 7fe1f1400640 1 -- 172.21.15.55:0/1895214574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec15e920 msgr2=0x7fe1ec0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.810+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec15e920 0x7fe1ec0cc240 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0097a0 tx=0x7fe1dc02a180 comp rx=0 tx=0).stop 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 -- 172.21.15.55:0/1895214574 shutdown_connections 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec06b120 0x7fe1ec06b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1895214574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec15e920 0x7fe1ec0cc240 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 -- 172.21.15.55:0/1895214574 >> 172.21.15.55:0/1895214574 conn(0x7fe1ec06c750 msgr2=0x7fe1ec0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 -- 172.21.15.55:0/1895214574 shutdown_connections 2024-05-10T03:33:31.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.811+0000 7fe1f1400640 1 -- 172.21.15.55:0/1895214574 wait complete. 2024-05-10T03:33:31.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.813+0000 7fe1f1400640 1 Processor -- start 2024-05-10T03:33:31.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.813+0000 7fe1f1400640 1 -- start start 2024-05-10T03:33:31.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1ec0710f0 0x7fe1ec12e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec15e920 0x7fe1ec154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec152c20 con 0x7fe1ec06b120 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec141850 con 0x7fe1ec0710f0 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1f1400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec13fba0 con 0x7fe1ec15e920 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52050/0 (socket says 172.21.15.55:52050) 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1936498948 learned_addr learned my addr 172.21.15.55:0/1936498948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec15e920 msgr2=0x7fe1ec154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1ebe00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec15e920 0x7fe1ec154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.815+0000 7fe1e3e00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1ec0710f0 0x7fe1ec12e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1ec0710f0 msgr2=0x7fe1ec12e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1ebe00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1ec0710f0 0x7fe1ec12e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:31.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1ebe00640 1 -- 172.21.15.55:0/1936498948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc009400 con 0x7fe1ec06b120 2024-05-10T03:33:31.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1ebe00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc02a1c0 tx=0x7fe1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc006e10 con 0x7fe1ec06b120 2024-05-10T03:33:31.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0044c0 con 0x7fe1ec06b120 2024-05-10T03:33:31.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc003810 con 0x7fe1ec06b120 2024-05-10T03:33:31.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec12e860 con 0x7fe1ec06b120 2024-05-10T03:33:31.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.816+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec12ed50 con 0x7fe1ec06b120 2024-05-10T03:33:31.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.818+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec151600 con 0x7fe1ec06b120 2024-05-10T03:33:31.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.819+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe1dc003970 con 0x7fe1ec06b120 2024-05-10T03:33:31.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.820+0000 7fe1e8c00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 0x7fe1c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:31.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.820+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe1dc0a9c50 con 0x7fe1ec06b120 2024-05-10T03:33:31.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.820+0000 7fe1e3e00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 0x7fe1c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:31.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.822+0000 7fe1e3e00640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 0x7fe1c806d140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4006fd0 tx=0x7fe1d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:31.824+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe1ec151600 con 0x7fe1ec06b120 2024-05-10T03:33:32.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.025+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1ec151600 con 0x7fe1ec06b120 2024-05-10T03:33:32.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.026+0000 7fe1e8c00640 1 -- 172.21.15.55:0/1936498948 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1ec151600 con 0x7fe1ec06b120 2024-05-10T03:33:32.030 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.028+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 msgr2=0x7fe1c806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.028+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 0x7fe1c806d140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4006fd0 tx=0x7fe1d4008040 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.028+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 msgr2=0x7fe1ec13f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.028+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc02a1c0 tx=0x7fe1dc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 shutdown_connections 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1ec15e920 0x7fe1ec154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe1c806ac30 0x7fe1c806d140 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1ec0710f0 0x7fe1ec12e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 --2- 172.21.15.55:0/1936498948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1ec06b120 0x7fe1ec13f660 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 >> 172.21.15.55:0/1936498948 conn(0x7fe1ec06c750 msgr2=0x7fe1ec157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 shutdown_connections 2024-05-10T03:33:32.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.029+0000 7fe1f1400640 1 -- 172.21.15.55:0/1936498948 wait complete. 2024-05-10T03:33:32.060 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:33:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.338+0000 7fb035200640 1 Processor -- start 2024-05-10T03:33:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.338+0000 7fb035200640 1 -- start start 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0300d1ba0 0x7fb0300b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03015d700 0x7fb03015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0300cdf70 0x7fb0300ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03015e050 con 0x7fb0300d1ba0 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0300b7b50 con 0x7fb0300cdf70 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb035200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300ce8c0 con 0x7fb03015d700 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb02f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0300d1ba0 0x7fb0300b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb02f400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb0300d1ba0 0x7fb0300b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52052/0 (socket says 172.21.15.55:52052) 2024-05-10T03:33:32.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.340+0000 7fb02f400640 1 -- 172.21.15.55:0/1710699 learned_addr learned my addr 172.21.15.55:0/1710699 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb026a00640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03015d700 0x7fb03015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02fe00640 1 --2- 172.21.15.55:0/1710699 >> v2:172.21.15.162:3300/0 conn(0x7fb0300cdf70 0x7fb0300ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02f400640 1 -- 172.21.15.55:0/1710699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03015d700 msgr2=0x7fb03015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02f400640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03015d700 0x7fb03015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02f400640 1 -- 172.21.15.55:0/1710699 >> v2:172.21.15.162:3300/0 conn(0x7fb0300cdf70 msgr2=0x7fb0300ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02f400640 1 --2- 172.21.15.55:0/1710699 >> v2:172.21.15.162:3300/0 conn(0x7fb0300cdf70 0x7fb0300ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.341+0000 7fb02f400640 1 -- 172.21.15.55:0/1710699 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300b6f40 con 0x7fb0300d1ba0 2024-05-10T03:33:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.342+0000 7fb02f400640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0300d1ba0 0x7fb0300b65f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb030162430 tx=0x7fb02002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e269a2cf842cd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.343+0000 7fb02ea00640 1 -- 172.21.15.55:0/1710699 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020032440 con 0x7fb0300d1ba0 2024-05-10T03:33:32.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.343+0000 7fb02ea00640 1 -- 172.21.15.55:0/1710699 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0200325a0 con 0x7fb0300d1ba0 2024-05-10T03:33:32.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.343+0000 7fb02ea00640 1 -- 172.21.15.55:0/1710699 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020032930 con 0x7fb0300d1ba0 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.343+0000 7fb035200640 1 -- 172.21.15.55:0/1710699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0300d1ba0 msgr2=0x7fb0300b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0300d1ba0 0x7fb0300b65f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb030162430 tx=0x7fb02002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 -- 172.21.15.55:0/1710699 shutdown_connections 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 --2- 172.21.15.55:0/1710699 >> v2:172.21.15.162:3300/0 conn(0x7fb0300cdf70 0x7fb0300ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb03015d700 0x7fb03015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 --2- 172.21.15.55:0/1710699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0300d1ba0 0x7fb0300b65f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 -- 172.21.15.55:0/1710699 >> 172.21.15.55:0/1710699 conn(0x7fb0300d0620 msgr2=0x7fb0300d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 -- 172.21.15.55:0/1710699 shutdown_connections 2024-05-10T03:33:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.344+0000 7fb035200640 1 -- 172.21.15.55:0/1710699 wait complete. 2024-05-10T03:33:32.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.346+0000 7fb035200640 1 Processor -- start 2024-05-10T03:33:32.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.346+0000 7fb035200640 1 -- start start 2024-05-10T03:33:32.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0300cdf70 0x7fb0300c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 0x7fb0301f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300aa710 con 0x7fb03015d700 2024-05-10T03:33:32.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300c8a70 con 0x7fb0300cdf70 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb035200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300c98d0 con 0x7fb0300d1ba0 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52056/0 (socket says 172.21.15.55:52056) 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb02fe00640 1 -- 172.21.15.55:0/958216947 learned_addr learned my addr 172.21.15.55:0/958216947 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.348+0000 7fb026a00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 0x7fb0301f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02f400640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0300cdf70 0x7fb0300c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 -- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 msgr2=0x7fb0301f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 0x7fb0301f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 -- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0300cdf70 msgr2=0x7fb0300c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0300cdf70 0x7fb0300c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 -- 172.21.15.55:0/958216947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200094d0 con 0x7fb03015d700 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb026a00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 0x7fb0301f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:32.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb02fe00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb028007d40 tx=0x7fb02800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028014420 con 0x7fb03015d700 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb028014580 con 0x7fb03015d700 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0300c8cc0 con 0x7fb03015d700 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.349+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028014930 con 0x7fb03015d700 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.350+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0300c9280 con 0x7fb03015d700 2024-05-10T03:33:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.351+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03004e290 con 0x7fb03015d700 2024-05-10T03:33:32.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.352+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb028014a90 con 0x7fb03015d700 2024-05-10T03:33:32.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.353+0000 7fb027e00640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 0x7fb00c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.354+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb0280591a0 con 0x7fb03015d700 2024-05-10T03:33:32.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.354+0000 7fb02f400640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 0x7fb00c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.356+0000 7fb02f400640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 0x7fb00c06d110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb020034000 tx=0x7fb02002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.359+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb028054de0 con 0x7fb03015d700 2024-05-10T03:33:32.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.561+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb0300c9a10 con 0x7fb03015d700 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.562+0000 7fb027e00640 1 -- 172.21.15.55:0/958216947 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb0280098c0 con 0x7fb03015d700 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:33:32.566 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:33:32.567 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:33:32.567 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:33:32.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 msgr2=0x7fb00c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 0x7fb00c06d110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb020034000 tx=0x7fb02002e000 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 msgr2=0x7fb0301f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb028007d40 tx=0x7fb02800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 shutdown_connections 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb03015d700 0x7fb0301f73e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0300d1ba0 0x7fb0301f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb00c06ac00 0x7fb00c06d110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 --2- 172.21.15.55:0/958216947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0300cdf70 0x7fb0300c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 >> 172.21.15.55:0/958216947 conn(0x7fb0300d0620 msgr2=0x7fb03006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 shutdown_connections 2024-05-10T03:33:32.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.565+0000 7fb035200640 1 -- 172.21.15.55:0/958216947 wait complete. 2024-05-10T03:33:32.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:33:32.599 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:33:32.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.883+0000 7f8a2e600640 1 Processor -- start 2024-05-10T03:33:32.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.883+0000 7f8a2e600640 1 -- start start 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a2e600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a280d1330 0x7f8a2815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a2e600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a280b2240 con 0x7f8a280ce520 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a2e600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a280b2490 con 0x7f8a280b1cf0 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a2e600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a280d2480 con 0x7f8a280d1330 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a27e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8a280ce520 0x7f8a2806a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37460/0 (socket says 172.21.15.55:37460) 2024-05-10T03:33:32.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.886+0000 7f8a27e00640 1 -- 172.21.15.55:0/2458105327 learned_addr learned my addr 172.21.15.55:0/2458105327 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:32.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a2dc00640 1 --2- 172.21.15.55:0/2458105327 >> v2:172.21.15.162:3300/0 conn(0x7f8a280d1330 0x7f8a2815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208834921 0 0) 0x7f8a280b2240 con 0x7f8a280ce520 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a0c003520 con 0x7f8a280ce520 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283260430 0 0) 0x7f8a0c003520 con 0x7f8a280ce520 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 >> v2:172.21.15.162:3300/0 conn(0x7f8a280d1330 msgr2=0x7f8a2815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 --2- 172.21.15.55:0/2458105327 >> v2:172.21.15.162:3300/0 conn(0x7f8a280d1330 0x7f8a2815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 >> v1:172.21.15.162:6789/0 conn(0x7f8a280b1cf0 legacy=0x7f8a280cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280d2760 con 0x7f8a280ce520 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8a14001670 con 0x7f8a280ce520 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 >> v1:172.21.15.55:6789/0 conn(0x7f8a280ce520 legacy=0x7f8a2806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c003970 0x7f8a0c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 0x7f8a0c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0c005180 0x7f8a0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280d2480 con 0x7f8a0c0047c0 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c00c120 con 0x7f8a0c003970 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.887+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c00c2e0 con 0x7f8a0c005180 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 0x7f8a0c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a2d200640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c003970 0x7f8a0c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a2dc00640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0c005180 0x7f8a0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 -- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0c005180 msgr2=0x7f8a0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a0c005180 0x7f8a0c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 -- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c003970 msgr2=0x7f8a0c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:32.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c003970 0x7f8a0c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a27e00640 1 -- 172.21.15.55:0/2458105327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c003520 con 0x7f8a0c0047c0 2024-05-10T03:33:32.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.888+0000 7f8a2dc00640 1 -- 172.21.15.55:0/2458105327 reap_dead start 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.889+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 0x7f8a0c004bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c210 tx=0x7f8a0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b7448ac50f5545 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.890+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14005ad0 con 0x7f8a0c0047c0 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.890+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1402d070 con 0x7f8a0c0047c0 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.890+0000 7f8a27400640 1 -- 172.21.15.55:0/2458105327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14005e90 con 0x7f8a0c0047c0 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.890+0000 7f8a2e600640 1 -- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 msgr2=0x7f8a0c004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.890+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 0x7f8a0c004bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c210 tx=0x7f8a0c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.891+0000 7f8a2e600640 1 -- 172.21.15.55:0/2458105327 shutdown_connections 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.891+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2458105327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a0c0047c0 0x7f8a0c004bd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.891+0000 7f8a2e600640 1 -- 172.21.15.55:0/2458105327 >> 172.21.15.55:0/2458105327 conn(0x7f8a280d0850 msgr2=0x7f8a280d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.891+0000 7f8a2e600640 1 -- 172.21.15.55:0/2458105327 shutdown_connections 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.891+0000 7f8a2e600640 1 -- 172.21.15.55:0/2458105327 wait complete. 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.893+0000 7f8a2e600640 1 Processor -- start 2024-05-10T03:33:32.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.893+0000 7f8a2e600640 1 -- start start 2024-05-10T03:33:32.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 0x7f8a2815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a280cb3f0 0x7f8a280cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a281f37b0 con 0x7f8a280cc520 2024-05-10T03:33:32.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a281f3310 con 0x7f8a0c0047c0 2024-05-10T03:33:32.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a2e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280ca390 con 0x7f8a280cb3f0 2024-05-10T03:33:32.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53804/0 (socket says 172.21.15.55:53804) 2024-05-10T03:33:32.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.895+0000 7f8a27e00640 1 -- 172.21.15.55:0/2160915046 learned_addr learned my addr 172.21.15.55:0/2160915046 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a2d200640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 0x7f8a2815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 -- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a280cb3f0 msgr2=0x7f8a280cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a280cb3f0 0x7f8a280cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 -- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 msgr2=0x7f8a2815ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 0x7f8a2815ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 -- 172.21.15.55:0/2160915046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280ca630 con 0x7f8a280cc520 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a2d200640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 0x7f8a2815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.896+0000 7f8a27e00640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001930 tx=0x7f8a14001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.897+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140049e0 con 0x7f8a280cc520 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.897+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14004b60 con 0x7f8a280cc520 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.897+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14007070 con 0x7f8a280cc520 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.897+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a280ca910 con 0x7f8a280cc520 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.897+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a280caed0 con 0x7f8a280cc520 2024-05-10T03:33:32.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.898+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a280cb060 con 0x7f8a280cc520 2024-05-10T03:33:32.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.899+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8a1402f030 con 0x7f8a280cc520 2024-05-10T03:33:32.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.900+0000 7f8a24c00640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 0x7f8a0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:32.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.900+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8a140a89e0 con 0x7f8a280cc520 2024-05-10T03:33:32.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.901+0000 7f8a2d200640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 0x7f8a0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:32.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.902+0000 7f8a2d200640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 0x7f8a0806d140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000f80 tx=0x7f8a20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:32.906+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8a14073d30 con 0x7f8a280cc520 2024-05-10T03:33:33.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.107+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8a280cb060 con 0x7f8a280cc520 2024-05-10T03:33:33.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.381+0000 7f8a24c00640 1 -- 172.21.15.55:0/2160915046 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8a14007ce0 con 0x7f8a280cc520 2024-05-10T03:33:33.386 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:33:33.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.386+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 msgr2=0x7f8a0806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:33.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.386+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 0x7f8a0806d140 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8a20000f80 tx=0x7f8a20009040 comp rx=0 tx=0).stop 2024-05-10T03:33:33.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.386+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 msgr2=0x7f8a281f2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:33.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.386+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001930 tx=0x7f8a14001a00 comp rx=0 tx=0).stop 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 shutdown_connections 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a280cb3f0 0x7f8a280cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a280cc520 0x7f8a281f2dd0 secure :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001930 tx=0x7f8a14001a00 comp rx=0 tx=0).stop 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8a0806ac30 0x7f8a0806d140 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 --2- 172.21.15.55:0/2160915046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a0c0047c0 0x7f8a2815ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.387+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 >> 172.21.15.55:0/2160915046 conn(0x7f8a280d0850 msgr2=0x7f8a2815cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:33.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.389+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 shutdown_connections 2024-05-10T03:33:33.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.390+0000 7f8a2e600640 1 -- 172.21.15.55:0/2160915046 wait complete. 2024-05-10T03:33:33.428 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:33:33.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.710+0000 7f0f03200640 1 Processor -- start 2024-05-10T03:33:33.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.710+0000 7f0f03200640 1 -- start start 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0efc161a60 0x7f0efc0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0efc0ce800 0x7f0efc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0b2e40 0x7f0efc1657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc071520 con 0x7f0efc161a60 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0efc0cec10 con 0x7f0efc0ce800 2024-05-10T03:33:33.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.712+0000 7f0f03200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc0b3250 con 0x7f0efc0b2e40 2024-05-10T03:33:33.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0efc161a60 0x7f0efc0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0efc161a60 0x7f0efc0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53816/0 (socket says 172.21.15.55:53816) 2024-05-10T03:33:33.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 -- 172.21.15.55:0/2345008004 learned_addr learned my addr 172.21.15.55:0/2345008004 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:33.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01400640 1 --2- 172.21.15.55:0/2345008004 >> v2:172.21.15.162:3300/0 conn(0x7f0efc0ce800 0x7f0efc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f02800640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0b2e40 0x7f0efc1657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 -- 172.21.15.55:0/2345008004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0b2e40 msgr2=0x7f0efc1657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0b2e40 0x7f0efc1657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 -- 172.21.15.55:0/2345008004 >> v2:172.21.15.162:3300/0 conn(0x7f0efc0ce800 msgr2=0x7f0efc0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 --2- 172.21.15.55:0/2345008004 >> v2:172.21.15.162:3300/0 conn(0x7f0efc0ce800 0x7f0efc0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.713+0000 7f0f01e00640 1 -- 172.21.15.55:0/2345008004 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc071770 con 0x7f0efc161a60 2024-05-10T03:33:33.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.714+0000 7f0f01e00640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc0ce2c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06fd40 tx=0x7f0ef402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903db20743f1ac2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:33.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.715+0000 7f0f00a00640 1 -- 172.21.15.55:0/2345008004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4032440 con 0x7f0efc161a60 2024-05-10T03:33:33.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.715+0000 7f0f00a00640 1 -- 172.21.15.55:0/2345008004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef40325a0 con 0x7f0efc161a60 2024-05-10T03:33:33.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.715+0000 7f0f00a00640 1 -- 172.21.15.55:0/2345008004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4032930 con 0x7f0efc161a60 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.715+0000 7f0f03200640 1 -- 172.21.15.55:0/2345008004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 msgr2=0x7f0efc0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.715+0000 7f0f03200640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc0ce2c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0efc06fd40 tx=0x7f0ef402a270 comp rx=0 tx=0).stop 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 -- 172.21.15.55:0/2345008004 shutdown_connections 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0b2e40 0x7f0efc1657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 --2- 172.21.15.55:0/2345008004 >> v2:172.21.15.162:3300/0 conn(0x7f0efc0ce800 0x7f0efc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 --2- 172.21.15.55:0/2345008004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc0ce2c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 -- 172.21.15.55:0/2345008004 >> 172.21.15.55:0/2345008004 conn(0x7f0efc0b1810 msgr2=0x7f0efc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 -- 172.21.15.55:0/2345008004 shutdown_connections 2024-05-10T03:33:33.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.716+0000 7f0f03200640 1 -- 172.21.15.55:0/2345008004 wait complete. 2024-05-10T03:33:33.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.718+0000 7f0f03200640 1 Processor -- start 2024-05-10T03:33:33.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.718+0000 7f0f03200640 1 -- start start 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0efc0b2e40 0x7f0efc160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0ce800 0x7f0efc153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc1f88f0 con 0x7f0efc161a60 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc159c80 con 0x7f0efc0b2e40 2024-05-10T03:33:33.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f03200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc159ed0 con 0x7f0efc0ce800 2024-05-10T03:33:33.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f02800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f02800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53820/0 (socket says 172.21.15.55:53820) 2024-05-10T03:33:33.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.720+0000 7f0f02800640 1 -- 172.21.15.55:0/1935185830 learned_addr learned my addr 172.21.15.55:0/1935185830 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:33.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f01e00640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0efc0b2e40 0x7f0efc160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f01400640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0ce800 0x7f0efc153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 -- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0ce800 msgr2=0x7f0efc153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0ce800 0x7f0efc153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 -- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0efc0b2e40 msgr2=0x7f0efc160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0efc0b2e40 0x7f0efc160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 -- 172.21.15.55:0/1935185830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40094d0 con 0x7f0efc161a60 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.721+0000 7f0f02800640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8007d20 tx=0x7f0ef800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.722+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8014420 con 0x7f0efc161a60 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.722+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef80145a0 con 0x7f0efc161a60 2024-05-10T03:33:33.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.722+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8014950 con 0x7f0efc161a60 2024-05-10T03:33:33.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.722+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc15a1b0 con 0x7f0efc161a60 2024-05-10T03:33:33.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.722+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc156330 con 0x7f0efc161a60 2024-05-10T03:33:33.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.723+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc06d170 con 0x7f0efc161a60 2024-05-10T03:33:33.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.724+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ef8014ab0 con 0x7f0efc161a60 2024-05-10T03:33:33.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.725+0000 7f0ef2000640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 0x7f0ee006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:33.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.725+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f0ef808aa20 con 0x7f0efc161a60 2024-05-10T03:33:33.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.726+0000 7f0f01e00640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 0x7f0ee006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:33.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.727+0000 7f0f01e00640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 0x7f0ee006d110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4034000 tx=0x7f0ef402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:33.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.730+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ef8055cf0 con 0x7f0efc161a60 2024-05-10T03:33:33.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:33.933+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0efc04e370 con 0x7f0efc161a60 2024-05-10T03:33:34.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.389+0000 7f0ef2000640 1 -- 172.21.15.55:0/1935185830 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0ef8059310 con 0x7f0efc161a60 2024-05-10T03:33:34.394 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:33:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.396+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 msgr2=0x7f0ee006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.396+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 0x7f0ee006d110 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4034000 tx=0x7f0ef402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.396+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 msgr2=0x7f0efc15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.396+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8007d20 tx=0x7f0ef800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 shutdown_connections 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0efc161a60 0x7f0efc15e540 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0efc0ce800 0x7f0efc153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ee006ac00 0x7f0ee006d110 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 --2- 172.21.15.55:0/1935185830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0efc0b2e40 0x7f0efc160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.398+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 >> 172.21.15.55:0/1935185830 conn(0x7f0efc0b1810 msgr2=0x7f0efc0b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.402+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 shutdown_connections 2024-05-10T03:33:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.402+0000 7f0f03200640 1 -- 172.21.15.55:0/1935185830 wait complete. 2024-05-10T03:33:34.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:33:34.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.698+0000 7fe602e00640 1 Processor -- start 2024-05-10T03:33:34.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.698+0000 7fe602e00640 1 -- start start 2024-05-10T03:33:34.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.700+0000 7fe602e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5fc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.700+0000 7fe602e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc15fdf0 con 0x7fe5fc071e00 2024-05-10T03:33:34.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.700+0000 7fe602e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc15f1e0 con 0x7fe5fc06b840 2024-05-10T03:33:34.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.700+0000 7fe602e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc141d10 con 0x7fe5fc0719f0 2024-05-10T03:33:34.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.701+0000 7fe602400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe5fc071e00 0x7fe5fc141600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37464/0 (socket says 172.21.15.55:37464) 2024-05-10T03:33:34.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.701+0000 7fe602400640 1 -- 172.21.15.55:0/2639064369 learned_addr learned my addr 172.21.15.55:0/2639064369 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:34.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.701+0000 7fe601a00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5fc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.701+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791384694 0 0) 0x7fe5fc15fdf0 con 0x7fe5fc071e00 2024-05-10T03:33:34.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e4003520 con 0x7fe5fc071e00 2024-05-10T03:33:34.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4069568566 0 0) 0x7fe5e4003520 con 0x7fe5fc071e00 2024-05-10T03:33:34.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 msgr2=0x7fe5fc06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:34.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5fc06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 >> v1:172.21.15.162:6789/0 conn(0x7fe5fc06b840 legacy=0x7fe5fc0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc15f4c0 con 0x7fe5fc071e00 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe5f40019f0 con 0x7fe5fc071e00 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 >> v1:172.21.15.55:6789/0 conn(0x7fe5fc071e00 legacy=0x7fe5fc141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5e4003a10 0x7fe5e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e40047c0 0x7fe5e4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc141d10 con 0x7fe5e4003a10 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40054c0 con 0x7fe5e40047c0 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4005680 con 0x7fe5fc0719f0 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.702+0000 7fe601a00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5e4003a10 0x7fe5e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe602400640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601a00640 1 -- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 msgr2=0x7fe5e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601000640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e40047c0 0x7fe5e4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601a00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5fc0719f0 0x7fe5e400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601a00640 1 -- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e40047c0 msgr2=0x7fe5e4004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:34.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601a00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5e40047c0 0x7fe5e4004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.703+0000 7fe601a00640 1 -- 172.21.15.55:0/2639064369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e4003520 con 0x7fe5e4003a10 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.704+0000 7fe601a00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5e4003a10 0x7fe5e4003e80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009ac0 tx=0x7fe5e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeed10055fbd394e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.704+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0031400 con 0x7fe5e4003a10 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.704+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f002d050 con 0x7fe5e4003a10 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.704+0000 7fe5fbe00640 1 -- 172.21.15.55:0/2639064369 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0031780 con 0x7fe5e4003a10 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 -- 172.21.15.55:0/2639064369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5e4003a10 msgr2=0x7fe5e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 --2- 172.21.15.55:0/2639064369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5e4003a10 0x7fe5e4003e80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009ac0 tx=0x7fe5e40057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602400640 1 -- 172.21.15.55:0/2639064369 reap_dead start 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 -- 172.21.15.55:0/2639064369 shutdown_connections 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 -- 172.21.15.55:0/2639064369 >> 172.21.15.55:0/2639064369 conn(0x7fe5fc06c750 msgr2=0x7fe5fc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 -- 172.21.15.55:0/2639064369 shutdown_connections 2024-05-10T03:33:34.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.705+0000 7fe602e00640 1 -- 172.21.15.55:0/2639064369 wait complete. 2024-05-10T03:33:34.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.708+0000 7fe602e00640 1 Processor -- start 2024-05-10T03:33:34.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.708+0000 7fe602e00640 1 -- start start 2024-05-10T03:33:34.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e4003a10 0x7fe5fc1379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 0x7fe5fc0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1f5640 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc139820 con 0x7fe5fc135f80 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.709+0000 7fe602e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc1f4770 con 0x7fe5e4003a10 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53834/0 (socket says 172.21.15.55:53834) 2024-05-10T03:33:34.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 -- 172.21.15.55:0/3312364261 learned_addr learned my addr 172.21.15.55:0/3312364261 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:34.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe601000640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 0x7fe5fc0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe601a00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e4003a10 0x7fe5fc1379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 -- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e4003a10 msgr2=0x7fe5fc1379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e4003a10 0x7fe5fc1379a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 -- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 msgr2=0x7fe5fc0c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 0x7fe5fc0c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 -- 172.21.15.55:0/3312364261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc1f49c0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe601000640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 0x7fe5fc0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.710+0000 7fe602400640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4007c40 tx=0x7fe5f400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.711+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004d60 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.711+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f400bb80 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.711+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4007070 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.711+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc1f4ca0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.711+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc1f5260 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.712+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc1f53f0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.714+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe5f40073e0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.715+0000 7fe5f9600640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 0x7fe5dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:34.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.715+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fe5f408a180 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.716+0000 7fe601a00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 0x7fe5dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:34.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.717+0000 7fe601a00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 0x7fe5dc06d140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0033000 tx=0x7fe5f0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:34.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.719+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe5f40552d0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.956+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5fc04e290 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.965+0000 7fe5f9600640 1 -- 172.21.15.55:0/3312364261 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe5f40588f0 con 0x7fe5fc15e1c0 2024-05-10T03:33:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 11 and data pool 12 2024-05-10T03:33:34.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.970+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 msgr2=0x7fe5dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.970+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 0x7fe5dc06d140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0033000 tx=0x7fe5f0008030 comp rx=0 tx=0).stop 2024-05-10T03:33:34.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.970+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 msgr2=0x7fe5fc0c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:34.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.970+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4007c40 tx=0x7fe5f400b4a0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 shutdown_connections 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5fc15e1c0 0x7fe5fc0c4e40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe5dc06ac30 0x7fe5dc06d140 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0033000 tx=0x7fe5f0008030 comp rx=0 tx=0).stop 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5fc135f80 0x7fe5fc0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 --2- 172.21.15.55:0/3312364261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5e4003a10 0x7fe5fc1379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.971+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 >> 172.21.15.55:0/3312364261 conn(0x7fe5fc06c750 msgr2=0x7fe5fc142290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:34.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.978+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 shutdown_connections 2024-05-10T03:33:34.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:34.979+0000 7fe602e00640 1 -- 172.21.15.55:0/3312364261 wait complete. 2024-05-10T03:33:35.024 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.268+0000 7f2a9b000640 1 Processor -- start 2024-05-10T03:33:35.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.268+0000 7f2a9b000640 1 -- start start 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.270+0000 7f2a9b000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a940ce8f0 0x7f2a9406b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.270+0000 7f2a9b000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a940d1ab0 0x7f2a940d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.270+0000 7f2a9b000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9415efb0 con 0x7f2a9406b070 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.270+0000 7f2a9b000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9406c0b0 con 0x7f2a940ce8f0 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.270+0000 7f2a9b000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9406baf0 con 0x7f2a940d1ab0 2024-05-10T03:33:35.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a940ce8f0 0x7f2a9406b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a940ce8f0 0x7f2a9406b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53842/0 (socket says 172.21.15.55:53842) 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 -- 172.21.15.55:0/4219250359 learned_addr learned my addr 172.21.15.55:0/4219250359 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a99000640 1 --2- 172.21.15.55:0/4219250359 >> v2:172.21.15.162:3300/0 conn(0x7f2a940d1ab0 0x7f2a940d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 -- 172.21.15.55:0/4219250359 >> v2:172.21.15.162:3300/0 conn(0x7f2a940d1ab0 msgr2=0x7f2a940d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 --2- 172.21.15.55:0/4219250359 >> v2:172.21.15.162:3300/0 conn(0x7f2a940d1ab0 0x7f2a940d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 -- 172.21.15.55:0/4219250359 >> v1:172.21.15.162:6789/0 conn(0x7f2a9406b070 legacy=0x7f2a940ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:35.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.271+0000 7f2a93400640 1 -- 172.21.15.55:0/4219250359 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9406bdd0 con 0x7f2a940ce8f0 2024-05-10T03:33:35.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.273+0000 7f2a93400640 1 --2- 172.21.15.55:0/4219250359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940ce8f0 0x7f2a9406b480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ca2f0 tx=0x7f2a7c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127fd708c3fe183b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.273+0000 7f2a92a00640 1 -- 172.21.15.55:0/4219250359 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c031600 con 0x7f2a940ce8f0 2024-05-10T03:33:35.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.273+0000 7f2a92a00640 1 -- 172.21.15.55:0/4219250359 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c02d050 con 0x7f2a940ce8f0 2024-05-10T03:33:35.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.273+0000 7f2a92a00640 1 -- 172.21.15.55:0/4219250359 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c031980 con 0x7f2a940ce8f0 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 -- 172.21.15.55:0/4219250359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940ce8f0 msgr2=0x7f2a9406b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 --2- 172.21.15.55:0/4219250359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940ce8f0 0x7f2a9406b480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ca2f0 tx=0x7f2a7c02a5d0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 -- 172.21.15.55:0/4219250359 shutdown_connections 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 --2- 172.21.15.55:0/4219250359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940ce8f0 0x7f2a9406b480 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 --2- 172.21.15.55:0/4219250359 >> v2:172.21.15.162:3300/0 conn(0x7f2a940d1ab0 0x7f2a940d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 -- 172.21.15.55:0/4219250359 >> 172.21.15.55:0/4219250359 conn(0x7f2a940d32a0 msgr2=0x7f2a940d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 -- 172.21.15.55:0/4219250359 shutdown_connections 2024-05-10T03:33:35.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.274+0000 7f2a9b000640 1 -- 172.21.15.55:0/4219250359 wait complete. 2024-05-10T03:33:35.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.276+0000 7f2a9b000640 1 Processor -- start 2024-05-10T03:33:35.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.276+0000 7f2a9b000640 1 -- start start 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 0x7f2a9415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a94142680 0x7f2a94142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941406c0 con 0x7f2a940d1ab0 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94141e50 con 0x7f2a9406b070 2024-05-10T03:33:35.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a9b000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9415d9f0 con 0x7f2a94142680 2024-05-10T03:33:35.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a93400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a93400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53848/0 (socket says 172.21.15.55:53848) 2024-05-10T03:33:35.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.278+0000 7f2a93400640 1 -- 172.21.15.55:0/1383983110 learned_addr learned my addr 172.21.15.55:0/1383983110 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:35.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a99000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a94142680 0x7f2a94142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93e00640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 0x7f2a9415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 -- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a94142680 msgr2=0x7f2a94142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a94142680 0x7f2a94142a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 -- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 msgr2=0x7f2a9415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 0x7f2a9415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 -- 172.21.15.55:0/1383983110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009b30 con 0x7f2a940d1ab0 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93e00640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 0x7f2a9415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.279+0000 7f2a93400640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c034040 tx=0x7f2a7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.280+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c03d860 con 0x7f2a940d1ab0 2024-05-10T03:33:35.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.280+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c03d9c0 con 0x7f2a940d1ab0 2024-05-10T03:33:35.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.280+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c03dd70 con 0x7f2a940d1ab0 2024-05-10T03:33:35.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.280+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9415d730 con 0x7f2a940d1ab0 2024-05-10T03:33:35.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.280+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a94153820 con 0x7f2a940d1ab0 2024-05-10T03:33:35.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.281+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a94140830 con 0x7f2a940d1ab0 2024-05-10T03:33:35.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.282+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2a7c03b400 con 0x7f2a940d1ab0 2024-05-10T03:33:35.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.283+0000 7f2a8be00640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 0x7f2a7006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.283+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2a7c0aa4b0 con 0x7f2a940d1ab0 2024-05-10T03:33:35.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.284+0000 7f2a93e00640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 0x7f2a7006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.285+0000 7f2a93e00640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 0x7f2a7006d110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b6500 tx=0x7f2a84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.290+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2a94140830 con 0x7f2a940d1ab0 2024-05-10T03:33:35.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.492+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a94140800 con 0x7f2a940d1ab0 2024-05-10T03:33:35.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.495+0000 7f2a8be00640 1 -- 172.21.15.55:0/1383983110 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16322 (secure 0 0 0) 0x7f2a7c0754b0 con 0x7f2a940d1ab0 2024-05-10T03:33:35.500 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:35.500 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":77,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:34.958370+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 msgr2=0x7f2a7006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 0x7f2a7006d110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b6500 tx=0x7f2a84008040 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 msgr2=0x7f2a9415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c034040 tx=0x7f2a7c004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 shutdown_connections 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a94142680 0x7f2a94142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a940d1ab0 0x7f2a9415b560 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.498+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2a7006ac00 0x7f2a7006d110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.499+0000 7f2a9b000640 1 --2- 172.21.15.55:0/1383983110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a9406b070 0x7f2a9415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.499+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 >> 172.21.15.55:0/1383983110 conn(0x7f2a940d32a0 msgr2=0x7f2a940b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.499+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 shutdown_connections 2024-05-10T03:33:35.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.499+0000 7f2a9b000640 1 -- 172.21.15.55:0/1383983110 wait complete. 2024-05-10T03:33:35.527 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:35.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.798+0000 7f13d3200640 1 Processor -- start 2024-05-10T03:33:35.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.799+0000 7f13d3200640 1 -- start start 2024-05-10T03:33:35.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.801+0000 7f13d3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41612f0 0x7f13d4161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.801+0000 7f13d3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13d4071100 0x7f13d4071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.801+0000 7f13d3200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d4161c40 con 0x7f13d41549e0 2024-05-10T03:33:35.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.801+0000 7f13d3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40b39f0 con 0x7f13d4071100 2024-05-10T03:33:35.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.801+0000 7f13d3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40b3cd0 con 0x7f13d41612f0 2024-05-10T03:33:35.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13d4071100 0x7f13d4071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13d4071100 0x7f13d4071510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53854/0 (socket says 172.21.15.55:53854) 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 -- 172.21.15.55:0/1875536602 learned_addr learned my addr 172.21.15.55:0/1875536602 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13c9400640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41612f0 0x7f13d4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 -- 172.21.15.55:0/1875536602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41612f0 msgr2=0x7f13d4161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41612f0 0x7f13d4161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 -- 172.21.15.55:0/1875536602 >> v1:172.21.15.162:6789/0 conn(0x7f13d41549e0 legacy=0x7f13d40b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.802+0000 7f13d2800640 1 -- 172.21.15.55:0/1875536602 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40b3fb0 con 0x7f13d4071100 2024-05-10T03:33:35.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.804+0000 7f13d2800640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d4071100 0x7f13d4071510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009e50 tx=0x7f13cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5496607f6705d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.804+0000 7f13d1400640 1 -- 172.21.15.55:0/1875536602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031600 con 0x7f13d4071100 2024-05-10T03:33:35.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.804+0000 7f13d1400640 1 -- 172.21.15.55:0/1875536602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc02d050 con 0x7f13d4071100 2024-05-10T03:33:35.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.804+0000 7f13d1400640 1 -- 172.21.15.55:0/1875536602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031980 con 0x7f13d4071100 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 -- 172.21.15.55:0/1875536602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d4071100 msgr2=0x7f13d4071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d4071100 0x7f13d4071510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009e50 tx=0x7f13cc02a6a0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 -- 172.21.15.55:0/1875536602 shutdown_connections 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d4071100 0x7f13d4071510 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 --2- 172.21.15.55:0/1875536602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41612f0 0x7f13d4161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 -- 172.21.15.55:0/1875536602 >> 172.21.15.55:0/1875536602 conn(0x7f13d406a110 msgr2=0x7f13d406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:35.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 -- 172.21.15.55:0/1875536602 shutdown_connections 2024-05-10T03:33:35.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.805+0000 7f13d3200640 1 -- 172.21.15.55:0/1875536602 wait complete. 2024-05-10T03:33:35.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.809+0000 7f13d3200640 1 Processor -- start 2024-05-10T03:33:35.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.809+0000 7f13d3200640 1 -- start start 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13d4071100 0x7f13d414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41549e0 0x7f13d4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d414ac10 con 0x7f13d41612f0 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41e7480 con 0x7f13d4071100 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.811+0000 7f13d3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4155b40 con 0x7f13d41549e0 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53856/0 (socket says 172.21.15.55:53856) 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 -- 172.21.15.55:0/1904248020 learned_addr learned my addr 172.21.15.55:0/1904248020 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:35.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13c9400640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41549e0 0x7f13d4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d1e00640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13d4071100 0x7f13d414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 -- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41549e0 msgr2=0x7f13d4155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41549e0 0x7f13d4155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 -- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13d4071100 msgr2=0x7f13d414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13d4071100 0x7f13d414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:35.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.812+0000 7f13d2800640 1 -- 172.21.15.55:0/1904248020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc009b30 con 0x7f13d41612f0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13d2800640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009e50 tx=0x7f13cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc03b820 con 0x7f13d41612f0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc03b9a0 con 0x7f13d41612f0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc03bd50 con 0x7f13d41612f0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d4155880 con 0x7f13d41612f0 2024-05-10T03:33:35.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.813+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d4156070 con 0x7f13d41612f0 2024-05-10T03:33:35.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.814+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d404e290 con 0x7f13d41612f0 2024-05-10T03:33:35.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.815+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f13cc038050 con 0x7f13d41612f0 2024-05-10T03:33:35.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.816+0000 7f13caa00640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 0x7f13b006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:35.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.817+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f13cc0a9fa0 con 0x7f13d41612f0 2024-05-10T03:33:35.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.817+0000 7f13d1e00640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 0x7f13b006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:35.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.819+0000 7f13d1e00640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 0x7f13b006d110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f13c4007fd0 tx=0x7f13c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:35.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:35.821+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f13cc074fa0 con 0x7f13d41612f0 2024-05-10T03:33:36.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.018+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13d40b2360 con 0x7f13d41612f0 2024-05-10T03:33:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.021+0000 7f13caa00640 1 -- 172.21.15.55:0/1904248020 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f13cc04b090 con 0x7f13d41612f0 2024-05-10T03:33:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:36.025 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:36.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 msgr2=0x7f13b006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 0x7f13b006d110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f13c4007fd0 tx=0x7f13c4009040 comp rx=0 tx=0).stop 2024-05-10T03:33:36.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 msgr2=0x7f13d41e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009e50 tx=0x7f13cc004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 shutdown_connections 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13d41612f0 0x7f13d41e6b30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13d41549e0 0x7f13d4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f13b006ac00 0x7f13b006d110 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 --2- 172.21.15.55:0/1904248020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13d4071100 0x7f13d414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 >> 172.21.15.55:0/1904248020 conn(0x7f13d406a110 msgr2=0x7f13d40b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 shutdown_connections 2024-05-10T03:33:36.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.024+0000 7f13d3200640 1 -- 172.21.15.55:0/1904248020 wait complete. 2024-05-10T03:33:36.055 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:33:36.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.339+0000 7f29dea00640 1 Processor -- start 2024-05-10T03:33:36.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.339+0000 7f29dea00640 1 -- start start 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29dea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29d8158090 0x7f29d815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29dea00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f29d806b830 0x7f29d8161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29dea00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d815e430 con 0x7f29d8161430 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29dea00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d815de40 con 0x7f29d8158090 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29dea00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d815ee00 con 0x7f29d806b830 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29d7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29d8158090 0x7f29d815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29d7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29d8158090 0x7f29d815e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53864/0 (socket says 172.21.15.55:53864) 2024-05-10T03:33:36.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.341+0000 7f29d7e00640 1 -- 172.21.15.55:0/3227193558 learned_addr learned my addr 172.21.15.55:0/3227193558 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:36.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.342+0000 7f29d7e00640 1 -- 172.21.15.55:0/3227193558 >> v2:172.21.15.162:3300/0 conn(0x7f29d806b830 msgr2=0x7f29d8161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:36.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.342+0000 7f29dca00640 1 --2- 172.21.15.55:0/3227193558 >> v2:172.21.15.162:3300/0 conn(0x7f29d806b830 0x7f29d8161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.342+0000 7f29d7e00640 1 --2- 172.21.15.55:0/3227193558 >> v2:172.21.15.162:3300/0 conn(0x7f29d806b830 0x7f29d8161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.342+0000 7f29d7e00640 1 -- 172.21.15.55:0/3227193558 >> v1:172.21.15.55:6789/0 conn(0x7f29d8161430 legacy=0x7f29d806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.342+0000 7f29d7e00640 1 -- 172.21.15.55:0/3227193558 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8161ef0 con 0x7f29d8158090 2024-05-10T03:33:36.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.343+0000 7f29d7e00640 1 --2- 172.21.15.55:0/3227193558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8158090 0x7f29d815e8c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f29d806da40 tx=0x7f29c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1726e19016aa3d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:36.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.343+0000 7f29d6a00640 1 -- 172.21.15.55:0/3227193558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8031600 con 0x7f29d8158090 2024-05-10T03:33:36.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.343+0000 7f29d6a00640 1 -- 172.21.15.55:0/3227193558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c802d050 con 0x7f29d8158090 2024-05-10T03:33:36.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.343+0000 7f29d6a00640 1 -- 172.21.15.55:0/3227193558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8031980 con 0x7f29d8158090 2024-05-10T03:33:36.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 -- 172.21.15.55:0/3227193558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8158090 msgr2=0x7f29d815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 --2- 172.21.15.55:0/3227193558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8158090 0x7f29d815e8c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f29d806da40 tx=0x7f29c802a6a0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 -- 172.21.15.55:0/3227193558 shutdown_connections 2024-05-10T03:33:36.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 --2- 172.21.15.55:0/3227193558 >> v2:172.21.15.162:3300/0 conn(0x7f29d806b830 0x7f29d8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 --2- 172.21.15.55:0/3227193558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8158090 0x7f29d815e8c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 -- 172.21.15.55:0/3227193558 >> 172.21.15.55:0/3227193558 conn(0x7f29d8070900 msgr2=0x7f29d8070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:36.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 -- 172.21.15.55:0/3227193558 shutdown_connections 2024-05-10T03:33:36.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.344+0000 7f29dea00640 1 -- 172.21.15.55:0/3227193558 wait complete. 2024-05-10T03:33:36.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.347+0000 7f29dea00640 1 Processor -- start 2024-05-10T03:33:36.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.347+0000 7f29dea00640 1 -- start start 2024-05-10T03:33:36.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d8158090 0x7f29d80c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 0x7f29d81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d815f310 con 0x7f29d8160600 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d815f560 con 0x7f29d8158090 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.348+0000 7f29dea00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d815f840 con 0x7f29d8161430 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53878/0 (socket says 172.21.15.55:53878) 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 -- 172.21.15.55:0/1957452147 learned_addr learned my addr 172.21.15.55:0/1957452147 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29d7e00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d8158090 0x7f29d80c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29d7400640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 0x7f29d81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 -- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 msgr2=0x7f29d81f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 0x7f29d81f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 -- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d8158090 msgr2=0x7f29d80c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d8158090 0x7f29d80c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29dca00640 1 -- 172.21.15.55:0/1957452147 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8009b30 con 0x7f29d8160600 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.349+0000 7f29d7400640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 0x7f29d81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29dca00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f29d000c5c0 tx=0x7f29d000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0004220 con 0x7f29d8160600 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0004380 con 0x7f29d8160600 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0003830 con 0x7f29d8160600 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d815fb20 con 0x7f29d8160600 2024-05-10T03:33:36.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.350+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d81600e0 con 0x7f29d8160600 2024-05-10T03:33:36.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.351+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d8067220 con 0x7f29d8160600 2024-05-10T03:33:36.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.352+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f29d0003990 con 0x7f29d8160600 2024-05-10T03:33:36.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.353+0000 7f29cfe00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 0x7f29b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:36.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.353+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f29d0089220 con 0x7f29d8160600 2024-05-10T03:33:36.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.354+0000 7f29d7e00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 0x7f29b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:36.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.355+0000 7f29d7e00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 0x7f29b406d110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f29c8033000 tx=0x7f29c8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:36.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.358+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f29d00542f0 con 0x7f29d8160600 2024-05-10T03:33:36.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:36.598+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f29d80c4d40 con 0x7f29d8160600 2024-05-10T03:33:37.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.429+0000 7f29cfe00640 1 -- 172.21.15.55:0/1957452147 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f29d0057910 con 0x7f29d8160600 2024-05-10T03:33:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 msgr2=0x7f29b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 0x7f29b406d110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f29c8033000 tx=0x7f29c8009000 comp rx=0 tx=0).stop 2024-05-10T03:33:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 msgr2=0x7f29d8143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f29d000c5c0 tx=0x7f29d000ca80 comp rx=0 tx=0).stop 2024-05-10T03:33:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 shutdown_connections 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29d8160600 0x7f29d8143f00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29d8161430 0x7f29d81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f29b406ac00 0x7f29b406d110 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 --2- 172.21.15.55:0/1957452147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29d8158090 0x7f29d80c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.432+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 >> 172.21.15.55:0/1957452147 conn(0x7f29d8070900 msgr2=0x7f29d806af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.433+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 shutdown_connections 2024-05-10T03:33:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.433+0000 7f29dea00640 1 -- 172.21.15.55:0/1957452147 wait complete. 2024-05-10T03:33:37.468 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:33:37.468 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:33:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.755+0000 7fa5a0e00640 1 Processor -- start 2024-05-10T03:33:37.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.755+0000 7fa5a0e00640 1 -- start start 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa5a0e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa59c070050 0x7fa59c141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa5a0e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 0x7fa59c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa5a0e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c071ce0 con 0x7fa59c15fed0 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa5a0e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c070f40 con 0x7fa59c070050 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa5a0e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c15f280 con 0x7fa59c0cd750 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa599e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa59c070050 0x7fa59c141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa599e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa59c070050 0x7fa59c141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53886/0 (socket says 172.21.15.55:53886) 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.758+0000 7fa599e00640 1 -- 172.21.15.55:0/4010086206 learned_addr learned my addr 172.21.15.55:0/4010086206 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa593e00640 1 --2- 172.21.15.55:0/4010086206 >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 0x7fa59c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa599e00640 1 -- 172.21.15.55:0/4010086206 >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 msgr2=0x7fa59c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:37.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa599e00640 1 --2- 172.21.15.55:0/4010086206 >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 0x7fa59c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa599e00640 1 -- 172.21.15.55:0/4010086206 >> v1:172.21.15.162:6789/0 conn(0x7fa59c15fed0 legacy=0x7fa59c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:37.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa599e00640 1 -- 172.21.15.55:0/4010086206 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c070460 con 0x7fa59c070050 2024-05-10T03:33:37.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.759+0000 7fa593e00640 1 --2- 172.21.15.55:0/4010086206 >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 0x7fa59c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:37.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.760+0000 7fa599e00640 1 --2- 172.21.15.55:0/4010086206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c141a30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa59c066c50 tx=0x7fa58c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f9f8f6a2aac4dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:37.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.760+0000 7fa599400640 1 -- 172.21.15.55:0/4010086206 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c032440 con 0x7fa59c070050 2024-05-10T03:33:37.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.760+0000 7fa599400640 1 -- 172.21.15.55:0/4010086206 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c0325a0 con 0x7fa59c070050 2024-05-10T03:33:37.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.760+0000 7fa599400640 1 -- 172.21.15.55:0/4010086206 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c032930 con 0x7fa59c070050 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.761+0000 7fa5a0e00640 1 -- 172.21.15.55:0/4010086206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 msgr2=0x7fa59c141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.761+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/4010086206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c141a30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa59c066c50 tx=0x7fa58c02a230 comp rx=0 tx=0).stop 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 -- 172.21.15.55:0/4010086206 shutdown_connections 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/4010086206 >> v2:172.21.15.162:3300/0 conn(0x7fa59c0cd750 0x7fa59c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/4010086206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c141a30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 -- 172.21.15.55:0/4010086206 >> 172.21.15.55:0/4010086206 conn(0x7fa59c06c120 msgr2=0x7fa59c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 -- 172.21.15.55:0/4010086206 shutdown_connections 2024-05-10T03:33:37.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.762+0000 7fa5a0e00640 1 -- 172.21.15.55:0/4010086206 wait complete. 2024-05-10T03:33:37.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.765+0000 7fa5a0e00640 1 Processor -- start 2024-05-10T03:33:37.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.765+0000 7fa5a0e00640 1 -- start start 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa59c0cd750 0x7fa59c0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 0x7fa59c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0adce0 con 0x7fa59c070050 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0b2670 con 0x7fa59c0cd750 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.767+0000 7fa5a0e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0adf30 con 0x7fa59c15fed0 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53888/0 (socket says 172.21.15.55:53888) 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 -- 172.21.15.55:0/1341754724 learned_addr learned my addr 172.21.15.55:0/1341754724 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa59a800640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 0x7fa59c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 -- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 msgr2=0x7fa59c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa593e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa59c0cd750 0x7fa59c0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 0x7fa59c0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 -- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa59c0cd750 msgr2=0x7fa59c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa59c0cd750 0x7fa59c0b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa599e00640 1 -- 172.21.15.55:0/1341754724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0094d0 con 0x7fa59c070050 2024-05-10T03:33:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.768+0000 7fa59a800640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 0x7fa59c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa599e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000c00 tx=0x7fa58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c032440 con 0x7fa59c070050 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c030b80 con 0x7fa59c070050 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c03d420 con 0x7fa59c070050 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c0ae210 con 0x7fa59c070050 2024-05-10T03:33:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.769+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c0b2ba0 con 0x7fa59c070050 2024-05-10T03:33:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.771+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c0b2d30 con 0x7fa59c070050 2024-05-10T03:33:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.772+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa58c039040 con 0x7fa59c070050 2024-05-10T03:33:37.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.773+0000 7fa592000640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 0x7fa57806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:37.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.773+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa58c0a9ee0 con 0x7fa59c070050 2024-05-10T03:33:37.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.773+0000 7fa593e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 0x7fa57806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:37.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.775+0000 7fa593e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 0x7fa57806d140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa59c1e55c0 tx=0x7fa584008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:37.778+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa58c074f70 con 0x7fa59c070050 2024-05-10T03:33:38.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.016+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa59c071ce0 con 0x7fa59c070050 2024-05-10T03:33:38.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.439+0000 7fa592000640 1 -- 172.21.15.55:0/1341754724 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa59c071ce0 con 0x7fa59c070050 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 msgr2=0x7fa57806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 0x7fa57806d140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa59c1e55c0 tx=0x7fa584008040 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 msgr2=0x7fa59c133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa58c000c00 tx=0x7fa58c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 shutdown_connections 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa59c15fed0 0x7fa59c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa57806ac30 0x7fa57806d140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa59c0cd750 0x7fa59c0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 --2- 172.21.15.55:0/1341754724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c070050 0x7fa59c133690 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 >> 172.21.15.55:0/1341754724 conn(0x7fa59c06c120 msgr2=0x7fa59c1410c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:38.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 shutdown_connections 2024-05-10T03:33:38.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.443+0000 7fa5a0e00640 1 -- 172.21.15.55:0/1341754724 wait complete. 2024-05-10T03:33:38.476 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:33:38.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:38.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.760+0000 7f838be00640 1 Processor -- start 2024-05-10T03:33:38.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.760+0000 7f838be00640 1 -- start start 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 0x7f838c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 0x7f838c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b46b0 0x7f838c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0b4ac0 con 0x7f838c0b46b0 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c15ad30 con 0x7f838c15a5d0 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c15b010 con 0x7f838c0cb730 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b46b0 0x7f838c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f838c0b46b0 0x7f838c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53894/0 (socket says 172.21.15.55:53894) 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.762+0000 7f838b400640 1 -- 172.21.15.55:0/3017793661 learned_addr learned my addr 172.21.15.55:0/3017793661 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838b400640 1 -- 172.21.15.55:0/3017793661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 msgr2=0x7f838c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838aa00640 1 --2- 172.21.15.55:0/3017793661 >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 0x7f838c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f8383e00640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 0x7f838c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838b400640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 0x7f838c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838b400640 1 -- 172.21.15.55:0/3017793661 >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 msgr2=0x7f838c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838b400640 1 --2- 172.21.15.55:0/3017793661 >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 0x7f838c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838b400640 1 -- 172.21.15.55:0/3017793661 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c15b2f0 con 0x7f838c0b46b0 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f838aa00640 1 --2- 172.21.15.55:0/3017793661 >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 0x7f838c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.763+0000 7f8383e00640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 0x7f838c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.764+0000 7f838b400640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b46b0 0x7f838c15df40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f838c06deb0 tx=0x7f838402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498cec195f8e9dea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.764+0000 7f838a000640 1 -- 172.21.15.55:0/3017793661 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384031600 con 0x7f838c0b46b0 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.764+0000 7f838a000640 1 -- 172.21.15.55:0/3017793661 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838402d050 con 0x7f838c0b46b0 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.764+0000 7f838a000640 1 -- 172.21.15.55:0/3017793661 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384031980 con 0x7f838c0b46b0 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 -- 172.21.15.55:0/3017793661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b46b0 msgr2=0x7f838c15df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b46b0 0x7f838c15df40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f838c06deb0 tx=0x7f838402a690 comp rx=0 tx=0).stop 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 -- 172.21.15.55:0/3017793661 shutdown_connections 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c0b46b0 0x7f838c15df40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 --2- 172.21.15.55:0/3017793661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0cb730 0x7f838c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 --2- 172.21.15.55:0/3017793661 >> v2:172.21.15.162:3300/0 conn(0x7f838c15a5d0 0x7f838c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 -- 172.21.15.55:0/3017793661 >> 172.21.15.55:0/3017793661 conn(0x7f838c06a110 msgr2=0x7f838c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 -- 172.21.15.55:0/3017793661 shutdown_connections 2024-05-10T03:33:38.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.765+0000 7f838be00640 1 -- 172.21.15.55:0/3017793661 wait complete. 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.767+0000 7f838be00640 1 Processor -- start 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.768+0000 7f838be00640 1 -- start start 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0b46b0 0x7f838c141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 0x7f838c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c149cb0 con 0x7f838c15a5d0 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c14d550 con 0x7f838c0cb730 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c149650 con 0x7f838c0b46b0 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53902/0 (socket says 172.21.15.55:53902) 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.769+0000 7f838b400640 1 -- 172.21.15.55:0/2156936759 learned_addr learned my addr 172.21.15.55:0/2156936759 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f8383e00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 0x7f838c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 -- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0b46b0 msgr2=0x7f838c141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0b46b0 0x7f838c141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 -- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 msgr2=0x7f838c14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 0x7f838c14ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 -- 172.21.15.55:0/2156936759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384009b30 con 0x7f838c15a5d0 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f8383e00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 0x7f838c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.770+0000 7f838b400640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8384009f40 tx=0x7f8384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.771+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838403d860 con 0x7f838c15a5d0 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.771+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838403d9c0 con 0x7f838c15a5d0 2024-05-10T03:33:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.771+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c1421f0 con 0x7f838c15a5d0 2024-05-10T03:33:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.771+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838403dd70 con 0x7f838c15a5d0 2024-05-10T03:33:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.771+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c1498d0 con 0x7f838c15a5d0 2024-05-10T03:33:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.772+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c149df0 con 0x7f838c15a5d0 2024-05-10T03:33:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.773+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8384046410 con 0x7f838c15a5d0 2024-05-10T03:33:38.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.774+0000 7f8383400640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 0x7f836806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:38.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.774+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f83840aa300 con 0x7f838c15a5d0 2024-05-10T03:33:38.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.775+0000 7f838aa00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 0x7f836806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:38.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.776+0000 7f838aa00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 0x7f836806d140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:38.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:38.779+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f838c149df0 con 0x7f838c15a5d0 2024-05-10T03:33:39.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.018+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838c149df0 con 0x7f838c15a5d0 2024-05-10T03:33:39.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.019+0000 7f8383400640 1 -- 172.21.15.55:0/2156936759 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f838c149df0 con 0x7f838c15a5d0 2024-05-10T03:33:39.023 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:39.023 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:39.023 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.021+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 msgr2=0x7f836806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.021+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 0x7f836806d140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c008040 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.021+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 msgr2=0x7f838c14a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.021+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8384009f40 tx=0x7f8384004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 shutdown_connections 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f838c15a5d0 0x7f838c14a0d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f836806ac30 0x7f836806d140 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f838c0cb730 0x7f838c14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 --2- 172.21.15.55:0/2156936759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f838c0b46b0 0x7f838c141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 >> 172.21.15.55:0/2156936759 conn(0x7f838c06a110 msgr2=0x7f838c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:39.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 shutdown_connections 2024-05-10T03:33:39.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.022+0000 7f838be00640 1 -- 172.21.15.55:0/2156936759 wait complete. 2024-05-10T03:33:39.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:39.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.311+0000 7fe132e00640 1 Processor -- start 2024-05-10T03:33:39.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.312+0000 7fe132e00640 1 -- start start 2024-05-10T03:33:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe132e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c0b2d00 0x7fe12c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe132e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe12c162840 0x7fe12c147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe132e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c162c50 con 0x7fe12c144b60 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe132e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12c0b1f60 con 0x7fe12c162840 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe132e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c162ed0 con 0x7fe12c0b2d00 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.314+0000 7fe131000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe12c144b60 0x7fe12c162130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37488/0 (socket says 172.21.15.55:37488) 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe131000640 1 -- 172.21.15.55:0/3221430334 learned_addr learned my addr 172.21.15.55:0/3221430334 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe132400640 1 --2- 172.21.15.55:0/3221430334 >> v2:172.21.15.55:3300/0 conn(0x7fe12c162840 0x7fe12c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe131a00640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c0b2d00 0x7fe12c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe12be00640 1 -- 172.21.15.55:0/3221430334 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626507474 0 0) 0x7fe12c162c50 con 0x7fe12c144b60 2024-05-10T03:33:39.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe12be00640 1 -- 172.21.15.55:0/3221430334 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe114003520 con 0x7fe12c144b60 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe132400640 1 -- 172.21.15.55:0/3221430334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c0b2d00 msgr2=0x7fe12c1471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe132400640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c0b2d00 0x7fe12c1471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe132400640 1 -- 172.21.15.55:0/3221430334 >> v1:172.21.15.55:6789/0 conn(0x7fe12c144b60 legacy=0x7fe12c162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.315+0000 7fe132400640 1 -- 172.21.15.55:0/3221430334 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0b2220 con 0x7fe12c162840 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.316+0000 7fe132400640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c162840 0x7fe12c147730 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe124009d90 tx=0x7fe12402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56021b5bf09e621 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe12be00640 1 -- 172.21.15.55:0/3221430334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124031600 con 0x7fe12c162840 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe12be00640 1 -- 172.21.15.55:0/3221430334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12402d050 con 0x7fe12c162840 2024-05-10T03:33:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe12be00640 1 -- 172.21.15.55:0/3221430334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124031960 con 0x7fe12c162840 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe132e00640 1 -- 172.21.15.55:0/3221430334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c162840 msgr2=0x7fe12c147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe132e00640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c162840 0x7fe12c147730 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe124009d90 tx=0x7fe12402a5d0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.317+0000 7fe132e00640 1 -- 172.21.15.55:0/3221430334 shutdown_connections 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.318+0000 7fe132e00640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c162840 0x7fe12c147730 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.318+0000 7fe132e00640 1 --2- 172.21.15.55:0/3221430334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c0b2d00 0x7fe12c1471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.318+0000 7fe132e00640 1 -- 172.21.15.55:0/3221430334 >> 172.21.15.55:0/3221430334 conn(0x7fe12c06a110 msgr2=0x7fe12c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.318+0000 7fe132e00640 1 -- 172.21.15.55:0/3221430334 shutdown_connections 2024-05-10T03:33:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.318+0000 7fe132e00640 1 -- 172.21.15.55:0/3221430334 wait complete. 2024-05-10T03:33:39.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.320+0000 7fe132e00640 1 Processor -- start 2024-05-10T03:33:39.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.320+0000 7fe132e00640 1 -- start start 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c144b60 0x7fe12c163ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe12c162840 0x7fe12c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c0ae230 con 0x7fe12c0b2d00 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1649e0 con 0x7fe12c162840 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe132e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1fb170 con 0x7fe12c144b60 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53922/0 (socket says 172.21.15.55:53922) 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 -- 172.21.15.55:0/250919882 learned_addr learned my addr 172.21.15.55:0/250919882 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 -- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c144b60 msgr2=0x7fe12c163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c144b60 0x7fe12c163ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 -- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe12c162840 msgr2=0x7fe12c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe12c162840 0x7fe12c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.322+0000 7fe131a00640 1 -- 172.21.15.55:0/250919882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124009a70 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe131a00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe12000e870 tx=0x7fe12000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1200094f0 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12000c940 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000ccf0 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c1fb450 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.323+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c1fba10 con 0x7fe12c0b2d00 2024-05-10T03:33:39.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.324+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c066ce0 con 0x7fe12c0b2d00 2024-05-10T03:33:39.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.325+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe120013470 con 0x7fe12c0b2d00 2024-05-10T03:33:39.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.326+0000 7fe129600640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 0x7fe10c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.326+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe120089610 con 0x7fe12c0b2d00 2024-05-10T03:33:39.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.327+0000 7fe131000640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 0x7fe10c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.331 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-05-10T03:33:39.328+0000 7feedf000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:33:39.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.328+0000 7fe131000640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 0x7fe10c06d110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe12c06dc70 tx=0x7fe118007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.331+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe120054610 con 0x7fe12c0b2d00 2024-05-10T03:33:39.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.530+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe12c04e290 con 0x7fe12c0b2d00 2024-05-10T03:33:39.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.532+0000 7fe129600640 1 -- 172.21.15.55:0/250919882 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7fe120057c30 con 0x7fe12c0b2d00 2024-05-10T03:33:39.537 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:39.537 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.535+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 msgr2=0x7fe10c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.535+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 0x7fe10c06d110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe12c06dc70 tx=0x7fe118007930 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.535+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 msgr2=0x7fe12c143570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.535+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe12000e870 tx=0x7fe12000ed30 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 shutdown_connections 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe10c06ac00 0x7fe10c06d110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe12c162840 0x7fe12c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe12c144b60 0x7fe12c163ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 --2- 172.21.15.55:0/250919882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe12c0b2d00 0x7fe12c143570 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 >> 172.21.15.55:0/250919882 conn(0x7fe12c06a110 msgr2=0x7fe12c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 shutdown_connections 2024-05-10T03:33:39.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.536+0000 7fe132e00640 1 -- 172.21.15.55:0/250919882 wait complete. 2024-05-10T03:33:39.567 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:33:39.567 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:33:39.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.845+0000 7f3cfe200640 1 Processor -- start 2024-05-10T03:33:39.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.845+0000 7f3cfe200640 1 -- start start 2024-05-10T03:33:39.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cfe200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3cf815fe50 0x7f3cf8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cfe200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2140 0x7f3cf8142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cfe200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf806ad20 con 0x7f3cf80b2550 2024-05-10T03:33:39.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cfe200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf8141f50 con 0x7f3cf815fe50 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cfe200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf806a110 con 0x7f3cf80b2140 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3cf815fe50 0x7f3cf8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3cf80b2550 0x7f3cf806b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37494/0 (socket says 172.21.15.55:37494) 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7e00640 1 -- 172.21.15.55:0/1269535349 learned_addr learned my addr 172.21.15.55:0/1269535349 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cefe00640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2140 0x7f3cf8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7400640 1 -- 172.21.15.55:0/1269535349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2140 msgr2=0x7f3cf8142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7400640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2140 0x7f3cf8142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.847+0000 7f3cf7400640 1 -- 172.21.15.55:0/1269535349 >> v1:172.21.15.55:6789/0 conn(0x7f3cf80b2550 legacy=0x7f3cf806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.848+0000 7f3cf7400640 1 -- 172.21.15.55:0/1269535349 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf806a3f0 con 0x7f3cf815fe50 2024-05-10T03:33:39.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.848+0000 7f3cf6a00640 1 -- 172.21.15.55:0/1269535349 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269766925 0 0) 0x7f3cf806ad20 con 0x7f3cf80b2550 2024-05-10T03:33:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.849+0000 7f3cf7400640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf815fe50 0x7f3cf8141a10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2a07d91c2e6524 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.849+0000 7f3cf6a00640 1 -- 172.21.15.55:0/1269535349 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8032440 con 0x7f3cf815fe50 2024-05-10T03:33:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.849+0000 7f3cf6a00640 1 -- 172.21.15.55:0/1269535349 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce80325a0 con 0x7f3cf815fe50 2024-05-10T03:33:39.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.849+0000 7f3cf6a00640 1 -- 172.21.15.55:0/1269535349 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8032930 con 0x7f3cf815fe50 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 -- 172.21.15.55:0/1269535349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf815fe50 msgr2=0x7f3cf8141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf815fe50 0x7f3cf8141a10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce802a210 comp rx=0 tx=0).stop 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 -- 172.21.15.55:0/1269535349 shutdown_connections 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2140 0x7f3cf8142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 --2- 172.21.15.55:0/1269535349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf815fe50 0x7f3cf8141a10 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 -- 172.21.15.55:0/1269535349 >> 172.21.15.55:0/1269535349 conn(0x7f3cf80cbb90 msgr2=0x7f3cf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 -- 172.21.15.55:0/1269535349 shutdown_connections 2024-05-10T03:33:39.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.850+0000 7f3cfe200640 1 -- 172.21.15.55:0/1269535349 wait complete. 2024-05-10T03:33:39.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.852+0000 7f3cfe200640 1 Processor -- start 2024-05-10T03:33:39.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.853+0000 7f3cfe200640 1 -- start start 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2550 0x7f3cf815e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cf815fe50 0x7f3cf81e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81f3c90 con 0x7f3cf80b2140 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81e4f70 con 0x7f3cf815fe50 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cfe200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81e51c0 con 0x7f3cf80b2550 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53948/0 (socket says 172.21.15.55:53948) 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.854+0000 7f3cf7400640 1 -- 172.21.15.55:0/4000924684 learned_addr learned my addr 172.21.15.55:0/4000924684 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 -- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2550 msgr2=0x7f3cf815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2550 0x7f3cf815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7e00640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cf815fe50 0x7f3cf81e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 -- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cf815fe50 msgr2=0x7f3cf81e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:39.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cf815fe50 0x7f3cf81e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:39.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 -- 172.21.15.55:0/4000924684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce8009400 con 0x7f3cf80b2140 2024-05-10T03:33:39.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cf7400640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8004230 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce8004390 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8030ba0 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.855+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf81e54a0 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.856+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf81e5a60 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.857+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8066f60 con 0x7f3cf80b2140 2024-05-10T03:33:39.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.858+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3ce8039050 con 0x7f3cf80b2140 2024-05-10T03:33:39.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.859+0000 7f3cef400640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 0x7f3cd406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:39.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.859+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3ce80a9f60 con 0x7f3cf80b2140 2024-05-10T03:33:39.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.859+0000 7f3cefe00640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 0x7f3cd406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:39.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.860+0000 7f3cefe00640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 0x7f3cd406d110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8141290 tx=0x7f3ce0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:39.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:39.864+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ce8074ff0 con 0x7f3cf80b2140 2024-05-10T03:33:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.052+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3cf8067220 con 0x7f3cd406ac00 2024-05-10T03:33:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.077+0000 7f3cef400640 1 -- 172.21.15.55:0/4000924684 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286937 (secure 0 0 0) 0x7f3cf8067220 con 0x7f3cd406ac00 2024-05-10T03:33:40.085 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:40.087 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:33:40.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 msgr2=0x7f3cd406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 0x7f3cd406d110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8141290 tx=0x7f3ce0008040 comp rx=0 tx=0).stop 2024-05-10T03:33:40.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 msgr2=0x7f3cf8159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce80046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 shutdown_connections 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3cd406ac00 0x7f3cd406d110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3cf815fe50 0x7f3cf81e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3cf80b2550 0x7f3cf815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 --2- 172.21.15.55:0/4000924684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3cf80b2140 0x7f3cf8159fc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 >> 172.21.15.55:0/4000924684 conn(0x7f3cf80cbb90 msgr2=0x7f3cf815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 shutdown_connections 2024-05-10T03:33:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.082+0000 7f3cfe200640 1 -- 172.21.15.55:0/4000924684 wait complete. 2024-05-10T03:33:40.114 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-05-10T03:33:39.039703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253516,"kb_used_data":8620,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754721204,"statfs":{"total":773094113280,"available":772834512896,"internally_reserved":0,"allocated":8826880,"data_stored":4584208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.663477"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987791+0000","last_change":"2024-05-10T03:33:35.008997+0000","last_active":"2024-05-10T03:33:35.987791+0000","last_peered":"2024-05-10T03:33:35.987791+0000","last_clean":"2024-05-10T03:33:35.987791+0000","last_became_active":"2024-05-10T03:33:35.008633+0000","last_became_peered":"2024-05-10T03:33:35.008633+0000","last_unstale":"2024-05-10T03:33:35.987791+0000","last_undegraded":"2024-05-10T03:33:35.987791+0000","last_fullsized":"2024-05-10T03:33:35.987791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:26.860633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.016263+0000","last_change":"2024-05-10T03:33:35.015123+0000","last_active":"2024-05-10T03:33:35.016263+0000","last_peered":"2024-05-10T03:33:35.016263+0000","last_clean":"2024-05-10T03:33:35.016263+0000","last_became_active":"2024-05-10T03:33:35.014849+0000","last_became_peered":"2024-05-10T03:33:35.014849+0000","last_unstale":"2024-05-10T03:33:35.016263+0000","last_undegraded":"2024-05-10T03:33:35.016263+0000","last_fullsized":"2024-05-10T03:33:35.016263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:16:41.013194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757402+0000","last_change":"2024-05-10T03:33:35.011072+0000","last_active":"2024-05-10T03:33:36.757402+0000","last_peered":"2024-05-10T03:33:36.757402+0000","last_clean":"2024-05-10T03:33:36.757402+0000","last_became_active":"2024-05-10T03:33:35.010038+0000","last_became_peered":"2024-05-10T03:33:35.010038+0000","last_unstale":"2024-05-10T03:33:36.757402+0000","last_undegraded":"2024-05-10T03:33:36.757402+0000","last_fullsized":"2024-05-10T03:33:36.757402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:49.002716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.996002+0000","last_change":"2024-05-10T03:33:34.991271+0000","last_active":"2024-05-10T03:33:34.996002+0000","last_peered":"2024-05-10T03:33:34.996002+0000","last_clean":"2024-05-10T03:33:34.996002+0000","last_became_active":"2024-05-10T03:33:34.990921+0000","last_became_peered":"2024-05-10T03:33:34.990921+0000","last_unstale":"2024-05-10T03:33:34.996002+0000","last_undegraded":"2024-05-10T03:33:34.996002+0000","last_fullsized":"2024-05-10T03:33:34.996002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:12.821437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983852+0000","last_change":"2024-05-10T03:33:35.004776+0000","last_active":"2024-05-10T03:33:35.983852+0000","last_peered":"2024-05-10T03:33:35.983852+0000","last_clean":"2024-05-10T03:33:35.983852+0000","last_became_active":"2024-05-10T03:33:35.004268+0000","last_became_peered":"2024-05-10T03:33:35.004268+0000","last_unstale":"2024-05-10T03:33:35.983852+0000","last_undegraded":"2024-05-10T03:33:35.983852+0000","last_fullsized":"2024-05-10T03:33:35.983852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:26:34.565163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002165+0000","last_change":"2024-05-10T03:33:35.001195+0000","last_active":"2024-05-10T03:33:35.002165+0000","last_peered":"2024-05-10T03:33:35.002165+0000","last_clean":"2024-05-10T03:33:35.002165+0000","last_became_active":"2024-05-10T03:33:34.999824+0000","last_became_peered":"2024-05-10T03:33:34.999824+0000","last_unstale":"2024-05-10T03:33:35.002165+0000","last_undegraded":"2024-05-10T03:33:35.002165+0000","last_fullsized":"2024-05-10T03:33:35.002165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:53.525047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031116+0000","last_change":"2024-05-10T03:33:34.982379+0000","last_active":"2024-05-10T03:33:37.031116+0000","last_peered":"2024-05-10T03:33:37.031116+0000","last_clean":"2024-05-10T03:33:37.031116+0000","last_became_active":"2024-05-10T03:33:34.981828+0000","last_became_peered":"2024-05-10T03:33:34.981828+0000","last_unstale":"2024-05-10T03:33:37.031116+0000","last_undegraded":"2024-05-10T03:33:37.031116+0000","last_fullsized":"2024-05-10T03:33:37.031116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:35.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983347+0000","last_change":"2024-05-10T03:33:34.998037+0000","last_active":"2024-05-10T03:33:35.983347+0000","last_peered":"2024-05-10T03:33:35.983347+0000","last_clean":"2024-05-10T03:33:35.983347+0000","last_became_active":"2024-05-10T03:33:34.996201+0000","last_became_peered":"2024-05-10T03:33:34.996201+0000","last_unstale":"2024-05-10T03:33:35.983347+0000","last_undegraded":"2024-05-10T03:33:35.983347+0000","last_fullsized":"2024-05-10T03:33:35.983347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:55.725572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984702+0000","last_change":"2024-05-10T03:33:35.003546+0000","last_active":"2024-05-10T03:33:35.984702+0000","last_peered":"2024-05-10T03:33:35.984702+0000","last_clean":"2024-05-10T03:33:35.984702+0000","last_became_active":"2024-05-10T03:33:35.003200+0000","last_became_peered":"2024-05-10T03:33:35.003200+0000","last_unstale":"2024-05-10T03:33:35.984702+0000","last_undegraded":"2024-05-10T03:33:35.984702+0000","last_fullsized":"2024-05-10T03:33:35.984702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:31.268989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.999334+0000","last_change":"2024-05-10T03:33:34.992466+0000","last_active":"2024-05-10T03:33:34.999334+0000","last_peered":"2024-05-10T03:33:34.999334+0000","last_clean":"2024-05-10T03:33:34.999334+0000","last_became_active":"2024-05-10T03:33:34.991075+0000","last_became_peered":"2024-05-10T03:33:34.991075+0000","last_unstale":"2024-05-10T03:33:34.999334+0000","last_undegraded":"2024-05-10T03:33:34.999334+0000","last_fullsized":"2024-05-10T03:33:34.999334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:30:39.551149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.017431+0000","last_change":"2024-05-10T03:33:35.016684+0000","last_active":"2024-05-10T03:33:35.017431+0000","last_peered":"2024-05-10T03:33:35.017431+0000","last_clean":"2024-05-10T03:33:35.017431+0000","last_became_active":"2024-05-10T03:33:35.016145+0000","last_became_peered":"2024-05-10T03:33:35.016145+0000","last_unstale":"2024-05-10T03:33:35.017431+0000","last_undegraded":"2024-05-10T03:33:35.017431+0000","last_fullsized":"2024-05-10T03:33:35.017431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:36:24.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.010437+0000","last_change":"2024-05-10T03:33:35.008690+0000","last_active":"2024-05-10T03:33:35.010437+0000","last_peered":"2024-05-10T03:33:35.010437+0000","last_clean":"2024-05-10T03:33:35.010437+0000","last_became_active":"2024-05-10T03:33:35.007181+0000","last_became_peered":"2024-05-10T03:33:35.007181+0000","last_unstale":"2024-05-10T03:33:35.010437+0000","last_undegraded":"2024-05-10T03:33:35.010437+0000","last_fullsized":"2024-05-10T03:33:35.010437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:26.860633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984495+0000","last_change":"2024-05-10T03:33:34.985653+0000","last_active":"2024-05-10T03:33:35.984495+0000","last_peered":"2024-05-10T03:33:35.984495+0000","last_clean":"2024-05-10T03:33:35.984495+0000","last_became_active":"2024-05-10T03:33:34.985318+0000","last_became_peered":"2024-05-10T03:33:34.985318+0000","last_unstale":"2024-05-10T03:33:35.984495+0000","last_undegraded":"2024-05-10T03:33:35.984495+0000","last_fullsized":"2024-05-10T03:33:35.984495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:05.528501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.000249+0000","last_change":"2024-05-10T03:33:34.998497+0000","last_active":"2024-05-10T03:33:35.000249+0000","last_peered":"2024-05-10T03:33:35.000249+0000","last_clean":"2024-05-10T03:33:35.000249+0000","last_became_active":"2024-05-10T03:33:34.998183+0000","last_became_peered":"2024-05-10T03:33:34.998183+0000","last_unstale":"2024-05-10T03:33:35.000249+0000","last_undegraded":"2024-05-10T03:33:35.000249+0000","last_fullsized":"2024-05-10T03:33:35.000249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.777952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.005116+0000","last_change":"2024-05-10T03:33:34.990118+0000","last_active":"2024-05-10T03:33:35.005116+0000","last_peered":"2024-05-10T03:33:35.005116+0000","last_clean":"2024-05-10T03:33:35.005116+0000","last_became_active":"2024-05-10T03:33:34.989845+0000","last_became_peered":"2024-05-10T03:33:34.989845+0000","last_unstale":"2024-05-10T03:33:35.005116+0000","last_undegraded":"2024-05-10T03:33:35.005116+0000","last_fullsized":"2024-05-10T03:33:35.005116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:21.502387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986377+0000","last_change":"2024-05-10T03:33:34.997825+0000","last_active":"2024-05-10T03:33:35.986377+0000","last_peered":"2024-05-10T03:33:35.986377+0000","last_clean":"2024-05-10T03:33:35.986377+0000","last_became_active":"2024-05-10T03:33:34.997508+0000","last_became_peered":"2024-05-10T03:33:34.997508+0000","last_unstale":"2024-05-10T03:33:35.986377+0000","last_undegraded":"2024-05-10T03:33:35.986377+0000","last_fullsized":"2024-05-10T03:33:35.986377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:35.249332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984861+0000","last_change":"2024-05-10T03:33:34.998627+0000","last_active":"2024-05-10T03:33:35.984861+0000","last_peered":"2024-05-10T03:33:35.984861+0000","last_clean":"2024-05-10T03:33:35.984861+0000","last_became_active":"2024-05-10T03:33:34.998180+0000","last_became_peered":"2024-05-10T03:33:34.998180+0000","last_unstale":"2024-05-10T03:33:35.984861+0000","last_undegraded":"2024-05-10T03:33:35.984861+0000","last_fullsized":"2024-05-10T03:33:35.984861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:21.371547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757227+0000","last_change":"2024-05-10T03:33:34.997776+0000","last_active":"2024-05-10T03:33:36.757227+0000","last_peered":"2024-05-10T03:33:36.757227+0000","last_clean":"2024-05-10T03:33:36.757227+0000","last_became_active":"2024-05-10T03:33:34.997554+0000","last_became_peered":"2024-05-10T03:33:34.997554+0000","last_unstale":"2024-05-10T03:33:36.757227+0000","last_undegraded":"2024-05-10T03:33:36.757227+0000","last_fullsized":"2024-05-10T03:33:36.757227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.989496+0000","last_change":"2024-05-10T03:33:34.986048+0000","last_active":"2024-05-10T03:33:34.989496+0000","last_peered":"2024-05-10T03:33:34.989496+0000","last_clean":"2024-05-10T03:33:34.989496+0000","last_became_active":"2024-05-10T03:33:34.985746+0000","last_became_peered":"2024-05-10T03:33:34.985746+0000","last_unstale":"2024-05-10T03:33:34.989496+0000","last_undegraded":"2024-05-10T03:33:34.989496+0000","last_fullsized":"2024-05-10T03:33:34.989496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:38:18.760157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984628+0000","last_change":"2024-05-10T03:33:34.993934+0000","last_active":"2024-05-10T03:33:35.984628+0000","last_peered":"2024-05-10T03:33:35.984628+0000","last_clean":"2024-05-10T03:33:35.984628+0000","last_became_active":"2024-05-10T03:33:34.989698+0000","last_became_peered":"2024-05-10T03:33:34.989698+0000","last_unstale":"2024-05-10T03:33:35.984628+0000","last_undegraded":"2024-05-10T03:33:35.984628+0000","last_fullsized":"2024-05-10T03:33:35.984628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:00.767846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984925+0000","last_change":"2024-05-10T03:33:35.005881+0000","last_active":"2024-05-10T03:33:35.984925+0000","last_peered":"2024-05-10T03:33:35.984925+0000","last_clean":"2024-05-10T03:33:35.984925+0000","last_became_active":"2024-05-10T03:33:35.004077+0000","last_became_peered":"2024-05-10T03:33:35.004077+0000","last_unstale":"2024-05-10T03:33:35.984925+0000","last_undegraded":"2024-05-10T03:33:35.984925+0000","last_fullsized":"2024-05-10T03:33:35.984925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983799+0000","last_change":"2024-05-10T03:33:34.997703+0000","last_active":"2024-05-10T03:33:35.983799+0000","last_peered":"2024-05-10T03:33:35.983799+0000","last_clean":"2024-05-10T03:33:35.983799+0000","last_became_active":"2024-05-10T03:33:34.996161+0000","last_became_peered":"2024-05-10T03:33:34.996161+0000","last_unstale":"2024-05-10T03:33:35.983799+0000","last_undegraded":"2024-05-10T03:33:35.983799+0000","last_fullsized":"2024-05-10T03:33:35.983799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:38:33.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983543+0000","last_change":"2024-05-10T03:33:34.994993+0000","last_active":"2024-05-10T03:33:35.983543+0000","last_peered":"2024-05-10T03:33:35.983543+0000","last_clean":"2024-05-10T03:33:35.983543+0000","last_became_active":"2024-05-10T03:33:34.994473+0000","last_became_peered":"2024-05-10T03:33:34.994473+0000","last_unstale":"2024-05-10T03:33:35.983543+0000","last_undegraded":"2024-05-10T03:33:35.983543+0000","last_fullsized":"2024-05-10T03:33:35.983543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.004889+0000","last_change":"2024-05-10T03:33:34.988398+0000","last_active":"2024-05-10T03:33:35.004889+0000","last_peered":"2024-05-10T03:33:35.004889+0000","last_clean":"2024-05-10T03:33:35.004889+0000","last_became_active":"2024-05-10T03:33:34.988119+0000","last_became_peered":"2024-05-10T03:33:34.988119+0000","last_unstale":"2024-05-10T03:33:35.004889+0000","last_undegraded":"2024-05-10T03:33:35.004889+0000","last_fullsized":"2024-05-10T03:33:35.004889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:39.537388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.012746+0000","last_change":"2024-05-10T03:33:35.008087+0000","last_active":"2024-05-10T03:33:35.012746+0000","last_peered":"2024-05-10T03:33:35.012746+0000","last_clean":"2024-05-10T03:33:35.012746+0000","last_became_active":"2024-05-10T03:33:35.007759+0000","last_became_peered":"2024-05-10T03:33:35.007759+0000","last_unstale":"2024-05-10T03:33:35.012746+0000","last_undegraded":"2024-05-10T03:33:35.012746+0000","last_fullsized":"2024-05-10T03:33:35.012746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:57.019193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.010448+0000","last_change":"2024-05-10T03:33:35.008413+0000","last_active":"2024-05-10T03:33:35.010448+0000","last_peered":"2024-05-10T03:33:35.010448+0000","last_clean":"2024-05-10T03:33:35.010448+0000","last_became_active":"2024-05-10T03:33:35.007924+0000","last_became_peered":"2024-05-10T03:33:35.007924+0000","last_unstale":"2024-05-10T03:33:35.010448+0000","last_undegraded":"2024-05-10T03:33:35.010448+0000","last_fullsized":"2024-05-10T03:33:35.010448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:20.689486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986852+0000","last_change":"2024-05-10T03:33:34.997936+0000","last_active":"2024-05-10T03:33:35.986852+0000","last_peered":"2024-05-10T03:33:35.986852+0000","last_clean":"2024-05-10T03:33:35.986852+0000","last_became_active":"2024-05-10T03:33:34.997576+0000","last_became_peered":"2024-05-10T03:33:34.997576+0000","last_unstale":"2024-05-10T03:33:35.986852+0000","last_undegraded":"2024-05-10T03:33:35.986852+0000","last_fullsized":"2024-05-10T03:33:35.986852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:16.832262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984964+0000","last_change":"2024-05-10T03:33:34.994113+0000","last_active":"2024-05-10T03:33:35.984964+0000","last_peered":"2024-05-10T03:33:35.984964+0000","last_clean":"2024-05-10T03:33:35.984964+0000","last_became_active":"2024-05-10T03:33:34.993615+0000","last_became_peered":"2024-05-10T03:33:34.993615+0000","last_unstale":"2024-05-10T03:33:35.984964+0000","last_undegraded":"2024-05-10T03:33:35.984964+0000","last_fullsized":"2024-05-10T03:33:35.984964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:18.917135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.007437+0000","last_change":"2024-05-10T03:33:35.002816+0000","last_active":"2024-05-10T03:33:35.007437+0000","last_peered":"2024-05-10T03:33:35.007437+0000","last_clean":"2024-05-10T03:33:35.007437+0000","last_became_active":"2024-05-10T03:33:35.002476+0000","last_became_peered":"2024-05-10T03:33:35.002476+0000","last_unstale":"2024-05-10T03:33:35.007437+0000","last_undegraded":"2024-05-10T03:33:35.007437+0000","last_fullsized":"2024-05-10T03:33:35.007437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986570+0000","last_change":"2024-05-10T03:33:35.009782+0000","last_active":"2024-05-10T03:33:35.986570+0000","last_peered":"2024-05-10T03:33:35.986570+0000","last_clean":"2024-05-10T03:33:35.986570+0000","last_became_active":"2024-05-10T03:33:35.009501+0000","last_became_peered":"2024-05-10T03:33:35.009501+0000","last_unstale":"2024-05-10T03:33:35.986570+0000","last_undegraded":"2024-05-10T03:33:35.986570+0000","last_fullsized":"2024-05-10T03:33:35.986570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:30:39.551149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756475+0000","last_change":"2024-05-10T03:33:34.981266+0000","last_active":"2024-05-10T03:33:36.756475+0000","last_peered":"2024-05-10T03:33:36.756475+0000","last_clean":"2024-05-10T03:33:36.756475+0000","last_became_active":"2024-05-10T03:33:34.979560+0000","last_became_peered":"2024-05-10T03:33:34.979560+0000","last_unstale":"2024-05-10T03:33:36.756475+0000","last_undegraded":"2024-05-10T03:33:36.756475+0000","last_fullsized":"2024-05-10T03:33:36.756475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:35.893196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986504+0000","last_change":"2024-05-10T03:33:35.008153+0000","last_active":"2024-05-10T03:33:35.986504+0000","last_peered":"2024-05-10T03:33:35.986504+0000","last_clean":"2024-05-10T03:33:35.986504+0000","last_became_active":"2024-05-10T03:33:35.007685+0000","last_became_peered":"2024-05-10T03:33:35.007685+0000","last_unstale":"2024-05-10T03:33:35.986504+0000","last_undegraded":"2024-05-10T03:33:35.986504+0000","last_fullsized":"2024-05-10T03:33:35.986504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756642+0000","last_change":"2024-05-10T03:33:35.000426+0000","last_active":"2024-05-10T03:33:36.756642+0000","last_peered":"2024-05-10T03:33:36.756642+0000","last_clean":"2024-05-10T03:33:36.756642+0000","last_became_active":"2024-05-10T03:33:34.998665+0000","last_became_peered":"2024-05-10T03:33:34.998665+0000","last_unstale":"2024-05-10T03:33:36.756642+0000","last_undegraded":"2024-05-10T03:33:36.756642+0000","last_fullsized":"2024-05-10T03:33:36.756642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:34.186248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984968+0000","last_change":"2024-05-10T03:33:34.419061+0000","last_active":"2024-05-10T03:33:35.984968+0000","last_peered":"2024-05-10T03:33:35.984968+0000","last_clean":"2024-05-10T03:33:35.984968+0000","last_became_active":"2024-05-10T03:33:34.415943+0000","last_became_peered":"2024-05-10T03:33:34.415943+0000","last_unstale":"2024-05-10T03:33:35.984968+0000","last_undegraded":"2024-05-10T03:33:35.984968+0000","last_fullsized":"2024-05-10T03:33:35.984968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:02:01.592527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.009211+0000","last_change":"2024-05-10T03:33:35.000067+0000","last_active":"2024-05-10T03:33:35.009211+0000","last_peered":"2024-05-10T03:33:35.009211+0000","last_clean":"2024-05-10T03:33:35.009211+0000","last_became_active":"2024-05-10T03:33:34.999770+0000","last_became_peered":"2024-05-10T03:33:34.999770+0000","last_unstale":"2024-05-10T03:33:35.009211+0000","last_undegraded":"2024-05-10T03:33:35.009211+0000","last_fullsized":"2024-05-10T03:33:35.009211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:31:14.362791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987699+0000","last_change":"2024-05-10T03:33:34.424101+0000","last_active":"2024-05-10T03:33:35.987699+0000","last_peered":"2024-05-10T03:33:35.987699+0000","last_clean":"2024-05-10T03:33:35.987699+0000","last_became_active":"2024-05-10T03:33:34.423276+0000","last_became_peered":"2024-05-10T03:33:34.423276+0000","last_unstale":"2024-05-10T03:33:35.987699+0000","last_undegraded":"2024-05-10T03:33:35.987699+0000","last_fullsized":"2024-05-10T03:33:35.987699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:49.359571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984065+0000","last_change":"2024-05-10T03:33:35.005740+0000","last_active":"2024-05-10T03:33:35.984065+0000","last_peered":"2024-05-10T03:33:35.984065+0000","last_clean":"2024-05-10T03:33:35.984065+0000","last_became_active":"2024-05-10T03:33:35.001720+0000","last_became_peered":"2024-05-10T03:33:35.001720+0000","last_unstale":"2024-05-10T03:33:35.984065+0000","last_undegraded":"2024-05-10T03:33:35.984065+0000","last_fullsized":"2024-05-10T03:33:35.984065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:35:58.939759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987195+0000","last_change":"2024-05-10T03:33:34.416288+0000","last_active":"2024-05-10T03:33:35.987195+0000","last_peered":"2024-05-10T03:33:35.987195+0000","last_clean":"2024-05-10T03:33:35.987195+0000","last_became_active":"2024-05-10T03:33:34.415939+0000","last_became_peered":"2024-05-10T03:33:34.415939+0000","last_unstale":"2024-05-10T03:33:35.987195+0000","last_undegraded":"2024-05-10T03:33:35.987195+0000","last_fullsized":"2024-05-10T03:33:35.987195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:03.799869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983587+0000","last_change":"2024-05-10T03:33:35.005874+0000","last_active":"2024-05-10T03:33:35.983587+0000","last_peered":"2024-05-10T03:33:35.983587+0000","last_clean":"2024-05-10T03:33:35.983587+0000","last_became_active":"2024-05-10T03:33:35.004074+0000","last_became_peered":"2024-05-10T03:33:35.004074+0000","last_unstale":"2024-05-10T03:33:35.983587+0000","last_undegraded":"2024-05-10T03:33:35.983587+0000","last_fullsized":"2024-05-10T03:33:35.983587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:34.186248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031752+0000","last_change":"2024-05-10T03:33:34.412661+0000","last_active":"2024-05-10T03:33:37.031752+0000","last_peered":"2024-05-10T03:33:37.031752+0000","last_clean":"2024-05-10T03:33:37.031752+0000","last_became_active":"2024-05-10T03:33:34.412343+0000","last_became_peered":"2024-05-10T03:33:34.412343+0000","last_unstale":"2024-05-10T03:33:37.031752+0000","last_undegraded":"2024-05-10T03:33:37.031752+0000","last_fullsized":"2024-05-10T03:33:37.031752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:42.034522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013571+0000","last_change":"2024-05-10T03:33:35.010558+0000","last_active":"2024-05-10T03:33:35.013571+0000","last_peered":"2024-05-10T03:33:35.013571+0000","last_clean":"2024-05-10T03:33:35.013571+0000","last_became_active":"2024-05-10T03:33:35.009966+0000","last_became_peered":"2024-05-10T03:33:35.009966+0000","last_unstale":"2024-05-10T03:33:35.013571+0000","last_undegraded":"2024-05-10T03:33:35.013571+0000","last_fullsized":"2024-05-10T03:33:35.013571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:26:24.907160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983802+0000","last_change":"2024-05-10T03:33:34.425479+0000","last_active":"2024-05-10T03:33:35.983802+0000","last_peered":"2024-05-10T03:33:35.983802+0000","last_clean":"2024-05-10T03:33:35.983802+0000","last_became_active":"2024-05-10T03:33:34.425167+0000","last_became_peered":"2024-05-10T03:33:34.425167+0000","last_unstale":"2024-05-10T03:33:35.983802+0000","last_undegraded":"2024-05-10T03:33:35.983802+0000","last_fullsized":"2024-05-10T03:33:35.983802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:28.640085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.016753+0000","last_change":"2024-05-10T03:33:35.014909+0000","last_active":"2024-05-10T03:33:35.016753+0000","last_peered":"2024-05-10T03:33:35.016753+0000","last_clean":"2024-05-10T03:33:35.016753+0000","last_became_active":"2024-05-10T03:33:35.014624+0000","last_became_peered":"2024-05-10T03:33:35.014624+0000","last_unstale":"2024-05-10T03:33:35.016753+0000","last_undegraded":"2024-05-10T03:33:35.016753+0000","last_fullsized":"2024-05-10T03:33:35.016753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:23.023094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.996050+0000","last_change":"2024-05-10T03:33:34.414756+0000","last_active":"2024-05-10T03:33:34.996050+0000","last_peered":"2024-05-10T03:33:34.996050+0000","last_clean":"2024-05-10T03:33:34.996050+0000","last_became_active":"2024-05-10T03:33:34.413339+0000","last_became_peered":"2024-05-10T03:33:34.413339+0000","last_unstale":"2024-05-10T03:33:34.996050+0000","last_undegraded":"2024-05-10T03:33:34.996050+0000","last_fullsized":"2024-05-10T03:33:34.996050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984471+0000","last_change":"2024-05-10T03:33:35.005510+0000","last_active":"2024-05-10T03:33:35.984471+0000","last_peered":"2024-05-10T03:33:35.984471+0000","last_clean":"2024-05-10T03:33:35.984471+0000","last_became_active":"2024-05-10T03:33:35.004049+0000","last_became_peered":"2024-05-10T03:33:35.004049+0000","last_unstale":"2024-05-10T03:33:35.984471+0000","last_undegraded":"2024-05-10T03:33:35.984471+0000","last_fullsized":"2024-05-10T03:33:35.984471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:49.002716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.989600+0000","last_change":"2024-05-10T03:33:34.411385+0000","last_active":"2024-05-10T03:33:34.989600+0000","last_peered":"2024-05-10T03:33:34.989600+0000","last_clean":"2024-05-10T03:33:34.989600+0000","last_became_active":"2024-05-10T03:33:34.411069+0000","last_became_peered":"2024-05-10T03:33:34.411069+0000","last_unstale":"2024-05-10T03:33:34.989600+0000","last_undegraded":"2024-05-10T03:33:34.989600+0000","last_fullsized":"2024-05-10T03:33:34.989600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:25.471769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032162+0000","last_change":"2024-05-10T03:33:34.991775+0000","last_active":"2024-05-10T03:33:37.032162+0000","last_peered":"2024-05-10T03:33:37.032162+0000","last_clean":"2024-05-10T03:33:37.032162+0000","last_became_active":"2024-05-10T03:33:34.991459+0000","last_became_peered":"2024-05-10T03:33:34.991459+0000","last_unstale":"2024-05-10T03:33:37.032162+0000","last_undegraded":"2024-05-10T03:33:37.032162+0000","last_fullsized":"2024-05-10T03:33:37.032162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:12.749579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983523+0000","last_change":"2024-05-10T03:33:34.419971+0000","last_active":"2024-05-10T03:33:35.983523+0000","last_peered":"2024-05-10T03:33:35.983523+0000","last_clean":"2024-05-10T03:33:35.983523+0000","last_became_active":"2024-05-10T03:33:34.419636+0000","last_became_peered":"2024-05-10T03:33:34.419636+0000","last_unstale":"2024-05-10T03:33:35.983523+0000","last_undegraded":"2024-05-10T03:33:35.983523+0000","last_fullsized":"2024-05-10T03:33:35.983523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:31.221544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013797+0000","last_change":"2024-05-10T03:33:35.010438+0000","last_active":"2024-05-10T03:33:35.013797+0000","last_peered":"2024-05-10T03:33:35.013797+0000","last_clean":"2024-05-10T03:33:35.013797+0000","last_became_active":"2024-05-10T03:33:35.010104+0000","last_became_peered":"2024-05-10T03:33:35.010104+0000","last_unstale":"2024-05-10T03:33:35.013797+0000","last_undegraded":"2024-05-10T03:33:35.013797+0000","last_fullsized":"2024-05-10T03:33:35.013797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:00.339870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987733+0000","last_change":"2024-05-10T03:33:34.424568+0000","last_active":"2024-05-10T03:33:35.987733+0000","last_peered":"2024-05-10T03:33:35.987733+0000","last_clean":"2024-05-10T03:33:35.987733+0000","last_became_active":"2024-05-10T03:33:34.423766+0000","last_became_peered":"2024-05-10T03:33:34.423766+0000","last_unstale":"2024-05-10T03:33:35.987733+0000","last_undegraded":"2024-05-10T03:33:35.987733+0000","last_fullsized":"2024-05-10T03:33:35.987733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:47.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002159+0000","last_change":"2024-05-10T03:33:34.998134+0000","last_active":"2024-05-10T03:33:35.002159+0000","last_peered":"2024-05-10T03:33:35.002159+0000","last_clean":"2024-05-10T03:33:35.002159+0000","last_became_active":"2024-05-10T03:33:34.996681+0000","last_became_peered":"2024-05-10T03:33:34.996681+0000","last_unstale":"2024-05-10T03:33:35.002159+0000","last_undegraded":"2024-05-10T03:33:35.002159+0000","last_fullsized":"2024-05-10T03:33:35.002159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:16.455983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.974623+0000","last_change":"2024-05-10T03:33:34.412464+0000","last_active":"2024-05-10T03:33:34.974623+0000","last_peered":"2024-05-10T03:33:34.974623+0000","last_clean":"2024-05-10T03:33:34.974623+0000","last_became_active":"2024-05-10T03:33:34.412081+0000","last_became_peered":"2024-05-10T03:33:34.412081+0000","last_unstale":"2024-05-10T03:33:34.974623+0000","last_undegraded":"2024-05-10T03:33:34.974623+0000","last_fullsized":"2024-05-10T03:33:34.974623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:34.648723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.991205+0000","last_change":"2024-05-10T03:33:34.983307+0000","last_active":"2024-05-10T03:33:34.991205+0000","last_peered":"2024-05-10T03:33:34.991205+0000","last_clean":"2024-05-10T03:33:34.991205+0000","last_became_active":"2024-05-10T03:33:34.982446+0000","last_became_peered":"2024-05-10T03:33:34.982446+0000","last_unstale":"2024-05-10T03:33:34.991205+0000","last_undegraded":"2024-05-10T03:33:34.991205+0000","last_fullsized":"2024-05-10T03:33:34.991205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:09.811426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984399+0000","last_change":"2024-05-10T03:33:34.418191+0000","last_active":"2024-05-10T03:33:35.984399+0000","last_peered":"2024-05-10T03:33:35.984399+0000","last_clean":"2024-05-10T03:33:35.984399+0000","last_became_active":"2024-05-10T03:33:34.417848+0000","last_became_peered":"2024-05-10T03:33:34.417848+0000","last_unstale":"2024-05-10T03:33:35.984399+0000","last_undegraded":"2024-05-10T03:33:35.984399+0000","last_fullsized":"2024-05-10T03:33:35.984399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:40.832314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756937+0000","last_change":"2024-05-10T03:33:35.010407+0000","last_active":"2024-05-10T03:33:36.756937+0000","last_peered":"2024-05-10T03:33:36.756937+0000","last_clean":"2024-05-10T03:33:36.756937+0000","last_became_active":"2024-05-10T03:33:35.009961+0000","last_became_peered":"2024-05-10T03:33:35.009961+0000","last_unstale":"2024-05-10T03:33:36.756937+0000","last_undegraded":"2024-05-10T03:33:36.756937+0000","last_fullsized":"2024-05-10T03:33:36.756937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:07:36.236493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031868+0000","last_change":"2024-05-10T03:33:34.419502+0000","last_active":"2024-05-10T03:33:37.031868+0000","last_peered":"2024-05-10T03:33:37.031868+0000","last_clean":"2024-05-10T03:33:37.031868+0000","last_became_active":"2024-05-10T03:33:34.419166+0000","last_became_peered":"2024-05-10T03:33:34.419166+0000","last_unstale":"2024-05-10T03:33:37.031868+0000","last_undegraded":"2024-05-10T03:33:37.031868+0000","last_fullsized":"2024-05-10T03:33:37.031868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:15.639144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.990814+0000","last_change":"2024-05-10T03:33:34.980846+0000","last_active":"2024-05-10T03:33:34.990814+0000","last_peered":"2024-05-10T03:33:34.990814+0000","last_clean":"2024-05-10T03:33:34.990814+0000","last_became_active":"2024-05-10T03:33:34.980561+0000","last_became_peered":"2024-05-10T03:33:34.980561+0000","last_unstale":"2024-05-10T03:33:34.990814+0000","last_undegraded":"2024-05-10T03:33:34.990814+0000","last_fullsized":"2024-05-10T03:33:34.990814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002274+0000","last_change":"2024-05-10T03:33:34.420380+0000","last_active":"2024-05-10T03:33:35.002274+0000","last_peered":"2024-05-10T03:33:35.002274+0000","last_clean":"2024-05-10T03:33:35.002274+0000","last_became_active":"2024-05-10T03:33:34.419103+0000","last_became_peered":"2024-05-10T03:33:34.419103+0000","last_unstale":"2024-05-10T03:33:35.002274+0000","last_undegraded":"2024-05-10T03:33:35.002274+0000","last_fullsized":"2024-05-10T03:33:35.002274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:16.475920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031670+0000","last_change":"2024-05-10T03:33:34.989465+0000","last_active":"2024-05-10T03:33:37.031670+0000","last_peered":"2024-05-10T03:33:37.031670+0000","last_clean":"2024-05-10T03:33:37.031670+0000","last_became_active":"2024-05-10T03:33:34.989125+0000","last_became_peered":"2024-05-10T03:33:34.989125+0000","last_unstale":"2024-05-10T03:33:37.031670+0000","last_undegraded":"2024-05-10T03:33:37.031670+0000","last_fullsized":"2024-05-10T03:33:37.031670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986597+0000","last_change":"2024-05-10T03:33:34.421311+0000","last_active":"2024-05-10T03:33:35.986597+0000","last_peered":"2024-05-10T03:33:35.986597+0000","last_clean":"2024-05-10T03:33:35.986597+0000","last_became_active":"2024-05-10T03:33:34.420917+0000","last_became_peered":"2024-05-10T03:33:34.420917+0000","last_unstale":"2024-05-10T03:33:35.986597+0000","last_undegraded":"2024-05-10T03:33:35.986597+0000","last_fullsized":"2024-05-10T03:33:35.986597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:25.471769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032042+0000","last_change":"2024-05-10T03:33:34.999343+0000","last_active":"2024-05-10T03:33:37.032042+0000","last_peered":"2024-05-10T03:33:37.032042+0000","last_clean":"2024-05-10T03:33:37.032042+0000","last_became_active":"2024-05-10T03:33:34.998493+0000","last_became_peered":"2024-05-10T03:33:34.998493+0000","last_unstale":"2024-05-10T03:33:37.032042+0000","last_undegraded":"2024-05-10T03:33:37.032042+0000","last_fullsized":"2024-05-10T03:33:37.032042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:36.786332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983299+0000","last_change":"2024-05-10T03:33:34.416526+0000","last_active":"2024-05-10T03:33:35.983299+0000","last_peered":"2024-05-10T03:33:35.983299+0000","last_clean":"2024-05-10T03:33:35.983299+0000","last_became_active":"2024-05-10T03:33:34.416153+0000","last_became_peered":"2024-05-10T03:33:34.416153+0000","last_unstale":"2024-05-10T03:33:35.983299+0000","last_undegraded":"2024-05-10T03:33:35.983299+0000","last_fullsized":"2024-05-10T03:33:35.983299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:23:25.613525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032208+0000","last_change":"2024-05-10T03:33:34.981766+0000","last_active":"2024-05-10T03:33:37.032208+0000","last_peered":"2024-05-10T03:33:37.032208+0000","last_clean":"2024-05-10T03:33:37.032208+0000","last_became_active":"2024-05-10T03:33:34.981445+0000","last_became_peered":"2024-05-10T03:33:34.981445+0000","last_unstale":"2024-05-10T03:33:37.032208+0000","last_undegraded":"2024-05-10T03:33:37.032208+0000","last_fullsized":"2024-05-10T03:33:37.032208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:57:44.131089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.015286+0000","last_change":"2024-05-10T03:33:34.409915+0000","last_active":"2024-05-10T03:33:35.015286+0000","last_peered":"2024-05-10T03:33:35.015286+0000","last_clean":"2024-05-10T03:33:35.015286+0000","last_became_active":"2024-05-10T03:33:34.409613+0000","last_became_peered":"2024-05-10T03:33:34.409613+0000","last_unstale":"2024-05-10T03:33:35.015286+0000","last_undegraded":"2024-05-10T03:33:35.015286+0000","last_fullsized":"2024-05-10T03:33:35.015286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.988976+0000","last_change":"2024-05-10T03:33:34.986079+0000","last_active":"2024-05-10T03:33:34.988976+0000","last_peered":"2024-05-10T03:33:34.988976+0000","last_clean":"2024-05-10T03:33:34.988976+0000","last_became_active":"2024-05-10T03:33:34.985769+0000","last_became_peered":"2024-05-10T03:33:34.985769+0000","last_unstale":"2024-05-10T03:33:34.988976+0000","last_undegraded":"2024-05-10T03:33:34.988976+0000","last_fullsized":"2024-05-10T03:33:34.988976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:52.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987822+0000","last_change":"2024-05-10T03:33:34.421333+0000","last_active":"2024-05-10T03:33:35.987822+0000","last_peered":"2024-05-10T03:33:35.987822+0000","last_clean":"2024-05-10T03:33:35.987822+0000","last_became_active":"2024-05-10T03:33:34.420986+0000","last_became_peered":"2024-05-10T03:33:34.420986+0000","last_unstale":"2024-05-10T03:33:35.987822+0000","last_undegraded":"2024-05-10T03:33:35.987822+0000","last_fullsized":"2024-05-10T03:33:35.987822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:33.167351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983401+0000","last_change":"2024-05-10T03:33:34.997695+0000","last_active":"2024-05-10T03:33:35.983401+0000","last_peered":"2024-05-10T03:33:35.983401+0000","last_clean":"2024-05-10T03:33:35.983401+0000","last_became_active":"2024-05-10T03:33:34.996347+0000","last_became_peered":"2024-05-10T03:33:34.996347+0000","last_unstale":"2024-05-10T03:33:35.983401+0000","last_undegraded":"2024-05-10T03:33:35.983401+0000","last_fullsized":"2024-05-10T03:33:35.983401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:36:26.281740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984712+0000","last_change":"2024-05-10T03:33:34.424818+0000","last_active":"2024-05-10T03:33:35.984712+0000","last_peered":"2024-05-10T03:33:35.984712+0000","last_clean":"2024-05-10T03:33:35.984712+0000","last_became_active":"2024-05-10T03:33:34.423908+0000","last_became_peered":"2024-05-10T03:33:34.423908+0000","last_unstale":"2024-05-10T03:33:35.984712+0000","last_undegraded":"2024-05-10T03:33:35.984712+0000","last_fullsized":"2024-05-10T03:33:35.984712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:20.663035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984284+0000","last_change":"2024-05-10T03:33:35.011590+0000","last_active":"2024-05-10T03:33:35.984284+0000","last_peered":"2024-05-10T03:33:35.984284+0000","last_clean":"2024-05-10T03:33:35.984284+0000","last_became_active":"2024-05-10T03:33:35.011296+0000","last_became_peered":"2024-05-10T03:33:35.011296+0000","last_unstale":"2024-05-10T03:33:35.984284+0000","last_undegraded":"2024-05-10T03:33:35.984284+0000","last_fullsized":"2024-05-10T03:33:35.984284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:07.159229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984709+0000","last_change":"2024-05-10T03:33:34.423531+0000","last_active":"2024-05-10T03:33:35.984709+0000","last_peered":"2024-05-10T03:33:35.984709+0000","last_clean":"2024-05-10T03:33:35.984709+0000","last_became_active":"2024-05-10T03:33:34.423100+0000","last_became_peered":"2024-05-10T03:33:34.423100+0000","last_unstale":"2024-05-10T03:33:35.984709+0000","last_undegraded":"2024-05-10T03:33:35.984709+0000","last_fullsized":"2024-05-10T03:33:35.984709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:03:20.778564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.997481+0000","last_change":"2024-05-10T03:33:34.994585+0000","last_active":"2024-05-10T03:33:34.997481+0000","last_peered":"2024-05-10T03:33:34.997481+0000","last_clean":"2024-05-10T03:33:34.997481+0000","last_became_active":"2024-05-10T03:33:34.994218+0000","last_became_peered":"2024-05-10T03:33:34.994218+0000","last_unstale":"2024-05-10T03:33:34.997481+0000","last_undegraded":"2024-05-10T03:33:34.997481+0000","last_fullsized":"2024-05-10T03:33:34.997481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:31.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756961+0000","last_change":"2024-05-10T03:33:34.419095+0000","last_active":"2024-05-10T03:33:36.756961+0000","last_peered":"2024-05-10T03:33:36.756961+0000","last_clean":"2024-05-10T03:33:36.756961+0000","last_became_active":"2024-05-10T03:33:34.418799+0000","last_became_peered":"2024-05-10T03:33:34.418799+0000","last_unstale":"2024-05-10T03:33:36.756961+0000","last_undegraded":"2024-05-10T03:33:36.756961+0000","last_fullsized":"2024-05-10T03:33:36.756961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:29.120089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.011873+0000","last_change":"2024-05-10T03:33:35.005673+0000","last_active":"2024-05-10T03:33:35.011873+0000","last_peered":"2024-05-10T03:33:35.011873+0000","last_clean":"2024-05-10T03:33:35.011873+0000","last_became_active":"2024-05-10T03:33:35.005294+0000","last_became_peered":"2024-05-10T03:33:35.005294+0000","last_unstale":"2024-05-10T03:33:35.011873+0000","last_undegraded":"2024-05-10T03:33:35.011873+0000","last_fullsized":"2024-05-10T03:33:35.011873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:16.455983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.003718+0000","last_change":"2024-05-10T03:33:34.415418+0000","last_active":"2024-05-10T03:33:35.003718+0000","last_peered":"2024-05-10T03:33:35.003718+0000","last_clean":"2024-05-10T03:33:35.003718+0000","last_became_active":"2024-05-10T03:33:34.415087+0000","last_became_peered":"2024-05-10T03:33:34.415087+0000","last_unstale":"2024-05-10T03:33:35.003718+0000","last_undegraded":"2024-05-10T03:33:35.003718+0000","last_fullsized":"2024-05-10T03:33:35.003718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:20.483491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.011818+0000","last_change":"2024-05-10T03:33:35.007265+0000","last_active":"2024-05-10T03:33:35.011818+0000","last_peered":"2024-05-10T03:33:35.011818+0000","last_clean":"2024-05-10T03:33:35.011818+0000","last_became_active":"2024-05-10T03:33:35.006919+0000","last_became_peered":"2024-05-10T03:33:35.006919+0000","last_unstale":"2024-05-10T03:33:35.011818+0000","last_undegraded":"2024-05-10T03:33:35.011818+0000","last_fullsized":"2024-05-10T03:33:35.011818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:55.333376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.970682+0000","last_change":"2024-05-10T03:33:34.412690+0000","last_active":"2024-05-10T03:33:34.970682+0000","last_peered":"2024-05-10T03:33:34.970682+0000","last_clean":"2024-05-10T03:33:34.970682+0000","last_became_active":"2024-05-10T03:33:34.412291+0000","last_became_peered":"2024-05-10T03:33:34.412291+0000","last_unstale":"2024-05-10T03:33:34.970682+0000","last_undegraded":"2024-05-10T03:33:34.970682+0000","last_fullsized":"2024-05-10T03:33:34.970682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:22.004198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032325+0000","last_change":"2024-05-10T03:33:34.988056+0000","last_active":"2024-05-10T03:33:37.032325+0000","last_peered":"2024-05-10T03:33:37.032325+0000","last_clean":"2024-05-10T03:33:37.032325+0000","last_became_active":"2024-05-10T03:33:34.987667+0000","last_became_peered":"2024-05-10T03:33:34.987667+0000","last_unstale":"2024-05-10T03:33:37.032325+0000","last_undegraded":"2024-05-10T03:33:37.032325+0000","last_fullsized":"2024-05-10T03:33:37.032325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:47.350922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.970562+0000","last_change":"2024-05-10T03:33:34.423714+0000","last_active":"2024-05-10T03:33:34.970562+0000","last_peered":"2024-05-10T03:33:34.970562+0000","last_clean":"2024-05-10T03:33:34.970562+0000","last_became_active":"2024-05-10T03:33:34.423249+0000","last_became_peered":"2024-05-10T03:33:34.423249+0000","last_unstale":"2024-05-10T03:33:34.970562+0000","last_undegraded":"2024-05-10T03:33:34.970562+0000","last_fullsized":"2024-05-10T03:33:34.970562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:59.320973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983903+0000","last_change":"2024-05-10T03:33:34.999906+0000","last_active":"2024-05-10T03:33:35.983903+0000","last_peered":"2024-05-10T03:33:35.983903+0000","last_clean":"2024-05-10T03:33:35.983903+0000","last_became_active":"2024-05-10T03:33:34.999478+0000","last_became_peered":"2024-05-10T03:33:34.999478+0000","last_unstale":"2024-05-10T03:33:35.983903+0000","last_undegraded":"2024-05-10T03:33:35.983903+0000","last_fullsized":"2024-05-10T03:33:35.983903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:50.378468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.994363+0000","last_change":"2024-05-10T03:33:34.417010+0000","last_active":"2024-05-10T03:33:34.994363+0000","last_peered":"2024-05-10T03:33:34.994363+0000","last_clean":"2024-05-10T03:33:34.994363+0000","last_became_active":"2024-05-10T03:33:34.415398+0000","last_became_peered":"2024-05-10T03:33:34.415398+0000","last_unstale":"2024-05-10T03:33:34.994363+0000","last_undegraded":"2024-05-10T03:33:34.994363+0000","last_fullsized":"2024-05-10T03:33:34.994363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031239+0000","last_change":"2024-05-10T03:33:34.987778+0000","last_active":"2024-05-10T03:33:37.031239+0000","last_peered":"2024-05-10T03:33:37.031239+0000","last_clean":"2024-05-10T03:33:37.031239+0000","last_became_active":"2024-05-10T03:33:34.987466+0000","last_became_peered":"2024-05-10T03:33:34.987466+0000","last_unstale":"2024-05-10T03:33:37.031239+0000","last_undegraded":"2024-05-10T03:33:37.031239+0000","last_fullsized":"2024-05-10T03:33:37.031239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:17.494816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.994550+0000","last_change":"2024-05-10T03:33:34.414901+0000","last_active":"2024-05-10T03:33:34.994550+0000","last_peered":"2024-05-10T03:33:34.994550+0000","last_clean":"2024-05-10T03:33:34.994550+0000","last_became_active":"2024-05-10T03:33:34.414455+0000","last_became_peered":"2024-05-10T03:33:34.414455+0000","last_unstale":"2024-05-10T03:33:34.994550+0000","last_undegraded":"2024-05-10T03:33:34.994550+0000","last_fullsized":"2024-05-10T03:33:34.994550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984386+0000","last_change":"2024-05-10T03:33:35.005976+0000","last_active":"2024-05-10T03:33:35.984386+0000","last_peered":"2024-05-10T03:33:35.984386+0000","last_clean":"2024-05-10T03:33:35.984386+0000","last_became_active":"2024-05-10T03:33:35.004542+0000","last_became_peered":"2024-05-10T03:33:35.004542+0000","last_unstale":"2024-05-10T03:33:35.984386+0000","last_undegraded":"2024-05-10T03:33:35.984386+0000","last_fullsized":"2024-05-10T03:33:35.984386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:14.049410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987448+0000","last_change":"2024-05-10T03:33:34.413210+0000","last_active":"2024-05-10T03:33:35.987448+0000","last_peered":"2024-05-10T03:33:35.987448+0000","last_clean":"2024-05-10T03:33:35.987448+0000","last_became_active":"2024-05-10T03:33:34.412896+0000","last_became_peered":"2024-05-10T03:33:34.412896+0000","last_unstale":"2024-05-10T03:33:35.987448+0000","last_undegraded":"2024-05-10T03:33:35.987448+0000","last_fullsized":"2024-05-10T03:33:35.987448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013697+0000","last_change":"2024-05-10T03:33:35.008122+0000","last_active":"2024-05-10T03:33:35.013697+0000","last_peered":"2024-05-10T03:33:35.013697+0000","last_clean":"2024-05-10T03:33:35.013697+0000","last_became_active":"2024-05-10T03:33:35.007819+0000","last_became_peered":"2024-05-10T03:33:35.007819+0000","last_unstale":"2024-05-10T03:33:35.013697+0000","last_undegraded":"2024-05-10T03:33:35.013697+0000","last_fullsized":"2024-05-10T03:33:35.013697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:03:01.336252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986808+0000","last_change":"2024-05-10T03:33:34.422356+0000","last_active":"2024-05-10T03:33:35.986808+0000","last_peered":"2024-05-10T03:33:35.986808+0000","last_clean":"2024-05-10T03:33:35.986808+0000","last_became_active":"2024-05-10T03:33:34.421897+0000","last_became_peered":"2024-05-10T03:33:34.421897+0000","last_unstale":"2024-05-10T03:33:35.986808+0000","last_undegraded":"2024-05-10T03:33:35.986808+0000","last_fullsized":"2024-05-10T03:33:35.986808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031662+0000","last_change":"2024-05-10T03:33:34.980819+0000","last_active":"2024-05-10T03:33:37.031662+0000","last_peered":"2024-05-10T03:33:37.031662+0000","last_clean":"2024-05-10T03:33:37.031662+0000","last_became_active":"2024-05-10T03:33:34.980502+0000","last_became_peered":"2024-05-10T03:33:34.980502+0000","last_unstale":"2024-05-10T03:33:37.031662+0000","last_undegraded":"2024-05-10T03:33:37.031662+0000","last_fullsized":"2024-05-10T03:33:37.031662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:43.321219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031484+0000","last_change":"2024-05-10T03:33:34.412686+0000","last_active":"2024-05-10T03:33:37.031484+0000","last_peered":"2024-05-10T03:33:37.031484+0000","last_clean":"2024-05-10T03:33:37.031484+0000","last_became_active":"2024-05-10T03:33:34.412339+0000","last_became_peered":"2024-05-10T03:33:34.412339+0000","last_unstale":"2024-05-10T03:33:37.031484+0000","last_undegraded":"2024-05-10T03:33:37.031484+0000","last_fullsized":"2024-05-10T03:33:37.031484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:50.473172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031689+0000","last_change":"2024-05-10T03:33:34.988856+0000","last_active":"2024-05-10T03:33:37.031689+0000","last_peered":"2024-05-10T03:33:37.031689+0000","last_clean":"2024-05-10T03:33:37.031689+0000","last_became_active":"2024-05-10T03:33:34.982820+0000","last_became_peered":"2024-05-10T03:33:34.982820+0000","last_unstale":"2024-05-10T03:33:37.031689+0000","last_undegraded":"2024-05-10T03:33:37.031689+0000","last_fullsized":"2024-05-10T03:33:37.031689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:45.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984052+0000","last_change":"2024-05-10T03:33:34.418631+0000","last_active":"2024-05-10T03:33:35.984052+0000","last_peered":"2024-05-10T03:33:35.984052+0000","last_clean":"2024-05-10T03:33:35.984052+0000","last_became_active":"2024-05-10T03:33:34.418297+0000","last_became_peered":"2024-05-10T03:33:34.418297+0000","last_unstale":"2024-05-10T03:33:35.984052+0000","last_undegraded":"2024-05-10T03:33:35.984052+0000","last_fullsized":"2024-05-10T03:33:35.984052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:02.710467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757417+0000","last_change":"2024-05-10T03:33:35.011267+0000","last_active":"2024-05-10T03:33:36.757417+0000","last_peered":"2024-05-10T03:33:36.757417+0000","last_clean":"2024-05-10T03:33:36.757417+0000","last_became_active":"2024-05-10T03:33:35.010630+0000","last_became_peered":"2024-05-10T03:33:35.010630+0000","last_unstale":"2024-05-10T03:33:36.757417+0000","last_undegraded":"2024-05-10T03:33:36.757417+0000","last_fullsized":"2024-05-10T03:33:36.757417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:50.378468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031329+0000","last_change":"2024-05-10T03:33:34.415240+0000","last_active":"2024-05-10T03:33:37.031329+0000","last_peered":"2024-05-10T03:33:37.031329+0000","last_clean":"2024-05-10T03:33:37.031329+0000","last_became_active":"2024-05-10T03:33:34.414049+0000","last_became_peered":"2024-05-10T03:33:34.414049+0000","last_unstale":"2024-05-10T03:33:37.031329+0000","last_undegraded":"2024-05-10T03:33:37.031329+0000","last_fullsized":"2024-05-10T03:33:37.031329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:20.855730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757320+0000","last_change":"2024-05-10T03:33:34.992848+0000","last_active":"2024-05-10T03:33:36.757320+0000","last_peered":"2024-05-10T03:33:36.757320+0000","last_clean":"2024-05-10T03:33:36.757320+0000","last_became_active":"2024-05-10T03:33:34.992559+0000","last_became_peered":"2024-05-10T03:33:34.992559+0000","last_unstale":"2024-05-10T03:33:36.757320+0000","last_undegraded":"2024-05-10T03:33:36.757320+0000","last_fullsized":"2024-05-10T03:33:36.757320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756397+0000","last_change":"2024-05-10T03:33:34.417467+0000","last_active":"2024-05-10T03:33:36.756397+0000","last_peered":"2024-05-10T03:33:36.756397+0000","last_clean":"2024-05-10T03:33:36.756397+0000","last_became_active":"2024-05-10T03:33:34.416166+0000","last_became_peered":"2024-05-10T03:33:34.416166+0000","last_unstale":"2024-05-10T03:33:36.756397+0000","last_undegraded":"2024-05-10T03:33:36.756397+0000","last_fullsized":"2024-05-10T03:33:36.756397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:20.663035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987301+0000","last_change":"2024-05-10T03:33:35.002956+0000","last_active":"2024-05-10T03:33:35.987301+0000","last_peered":"2024-05-10T03:33:35.987301+0000","last_clean":"2024-05-10T03:33:35.987301+0000","last_became_active":"2024-05-10T03:33:35.002577+0000","last_became_peered":"2024-05-10T03:33:35.002577+0000","last_unstale":"2024-05-10T03:33:35.987301+0000","last_undegraded":"2024-05-10T03:33:35.987301+0000","last_fullsized":"2024-05-10T03:33:35.987301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:20.689486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.971576+0000","last_change":"2024-05-10T03:33:34.422502+0000","last_active":"2024-05-10T03:33:34.971576+0000","last_peered":"2024-05-10T03:33:34.971576+0000","last_clean":"2024-05-10T03:33:34.971576+0000","last_became_active":"2024-05-10T03:33:34.422198+0000","last_became_peered":"2024-05-10T03:33:34.422198+0000","last_unstale":"2024-05-10T03:33:34.971576+0000","last_undegraded":"2024-05-10T03:33:34.971576+0000","last_fullsized":"2024-05-10T03:33:34.971576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:03:00.317356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":213,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986872+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:33:35.986872+0000","last_peered":"2024-05-10T03:33:35.986872+0000","last_clean":"2024-05-10T03:33:35.986872+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:33:35.986872+0000","last_undegraded":"2024-05-10T03:33:35.986872+0000","last_fullsized":"2024-05-10T03:33:35.986872+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1552384,"data_stored":1019371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.153},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7490000000000001,"5min":1.877,"15min":1.877},"last":1.2150000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1439999999999999,"15min":1.1419999999999999},"min":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.853},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2450000000000001,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9039999999999999,"5min":1.925,"15min":1.925},"last":1.518}]},{"osd":2,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.363,"15min":1.383},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.77,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.347},{"interface":"front","average":{"1min":1.196,"5min":1.3899999999999999,"15min":1.4319999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8460000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.4590000000000001}]},{"osd":3,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3720000000000001,"15min":1.3859999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":2.222,"15min":2.222},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.353,"15min":1.3340000000000001},"min":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9550000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.101}]},{"osd":4,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.446,"15min":1.4690000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9790000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5189999999999999,"15min":1.5069999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2,"5min":2.323,"15min":2.323},"last":1.6990000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.528,"15min":1.51},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.619,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.3},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6879999999999999,"15min":1.696},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1709999999999998,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.762}]},{"osd":6,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.7130000000000001,"15min":1.7250000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.635},{"interface":"front","average":{"1min":1.631,"5min":1.6519999999999999,"15min":1.6399999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.097,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.5760000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":423925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2849999999999999,"15min":1.337},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7689999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.4550000000000001}]},{"osd":1,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.282,"15min":1.3080000000000001},"min":{"1min":0.84199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.556,"5min":3.113,"15min":3.113},"last":1.569},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3720000000000001,"15min":1.4630000000000001},"min":{"1min":0.74099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0350000000000001,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":2.0350000000000001}]},{"osd":2,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.319,"15min":1.2569999999999999},"min":{"1min":0.80400000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7909999999999999,"5min":2.02,"15min":2.02},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.52,"5min":1.4259999999999999,"15min":1.502},"min":{"1min":0.98299999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.202,"5min":3.569,"15min":3.569},"last":1.8220000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1230000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":2.1230000000000002},{"interface":"front","average":{"1min":1.538,"5min":1.367,"15min":1.3240000000000001},"min":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.6910000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.92000000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.766,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.036},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.552,"15min":1.581},"min":{"1min":0.89100000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0579999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.964}]},{"osd":5,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.4590000000000001,"15min":1.4099999999999999},"min":{"1min":1.0089999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.617,"15min":1.621},"min":{"1min":1.1759999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8799999999999999,"5min":2.472,"15min":2.472},"last":1.288}]},{"osd":7,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.1919999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4590000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.841,"5min":1.663,"15min":1.659},"min":{"1min":1.05,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.7569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":946176,"data_stored":422395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0920000000000001,"15min":1.071},"min":{"1min":0.72999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6950000000000001,"5min":1.851,"15min":1.851},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.145,"15min":1.141},"min":{"1min":0.72899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7190000000000001,"5min":1.855,"15min":1.855},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.702,"5min":1.954,"15min":1.954},"last":1.476}]},{"osd":2,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.232,"15min":1.222},"min":{"1min":0.749,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.669,"5min":2.359,"15min":2.359},"last":1.669},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.331},"min":{"1min":1.0069999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.353}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.361,"15min":1.357},"min":{"1min":0.98799999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7210000000000001,"5min":1.881,"15min":1.881},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.357,"15min":1.3049999999999999},"min":{"1min":1.0489999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.909,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.5309999999999999}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9470000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.5920000000000001,"15min":1.603},"min":{"1min":1.246,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1600000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.9990000000000001}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.534,"15min":1.518},"min":{"1min":1.3480000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0190000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.796},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5600000000000001,"15min":1.5549999999999999},"min":{"1min":1.129,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.7829999999999999,"15min":1.8069999999999999},"min":{"1min":1.2330000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.3969999999999998,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":2.125},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.6459999999999999,"15min":1.6220000000000001},"min":{"1min":0.96599999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":2.1680000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":974848,"data_stored":430649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.159,"15min":1.173},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6919999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1259999999999999,"15min":1.1359999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.591,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1779999999999999,"15min":1.1599999999999999},"min":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.613,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.139,"5min":1.248,"15min":1.2729999999999999},"min":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.72,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.79700000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5289999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.236,"5min":1.369,"15min":1.397},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.726,"5min":2.29,"15min":2.29},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.87,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.379,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.464,"5min":1.5649999999999999,"15min":1.581},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.782,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9079999999999999,"5min":2.512,"15min":2.512},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.504,"15min":1.552},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5510000000000002,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.5569999999999999,"15min":1.583},"min":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4769999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.724,"15min":1.7769999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.093,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":0.879}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.6519999999999999,"15min":1.6899999999999999},"min":{"1min":0.63700000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.6160000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.623,"5min":1.736,"15min":1.7430000000000001},"min":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6349999999999998,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":946176,"data_stored":422943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95399999999999996,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.96999999999999997},"min":{"1min":0.54500000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.575,"5min":1.708,"15min":1.708},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.393,"5min":1.603,"15min":1.603},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.202,"5min":1.228,"15min":1.25},"min":{"1min":0.623,"5min":0.377,"15min":0.377},"max":{"1min":1.5329999999999999,"5min":1.964,"15min":1.964},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7450000000000001,"5min":1.99,"15min":1.99},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.238,"5min":1.3140000000000001,"15min":1.3600000000000001},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.679,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.9019999999999999,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3999999999999999,"15min":1.3819999999999999},"min":{"1min":0.90200000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.948,"5min":2.464,"15min":2.464},"last":1.135}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.613,"15min":1.6739999999999999},"min":{"1min":1.012,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2709999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0449999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.5640000000000001,"15min":1.5760000000000001},"min":{"1min":1.123,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.113,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.008},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":1.7949999999999999},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4470000000000001,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.0529999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5840000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1139999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6359999999999999,"15min":1.657},"min":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.0419999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.0680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":966656,"data_stored":429723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93400000000000005,"15min":0.98299999999999998},"min":{"1min":0.504,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.389,"5min":1.772,"15min":1.772},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.97899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.635,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98199999999999998}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.56399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.425,"5min":1.746,"15min":1.746},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6879999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.099}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.085,"15min":1.117},"min":{"1min":0.63600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.526,"5min":2.169,"15min":2.169},"last":1.26},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.21,"15min":1.25},"min":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8520000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.3200000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.5269999999999999},"min":{"1min":0.92600000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.024,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.444},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8600000000000001,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.619}]},{"osd":5,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.464,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.173,"5min":2.508,"15min":2.508},"last":1.786},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5900000000000001,"15min":1.617},"min":{"1min":0.97999999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.0859999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0049999999999999}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.512,"15min":1.5449999999999999},"min":{"1min":0.90700000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1339999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5489999999999999,"15min":1.587},"min":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.331,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.6970000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.607,"15min":1.649},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2749999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.679},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.619,"15min":1.665},"min":{"1min":0.95599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.266,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.907}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":950272,"data_stored":422473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.92100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5089999999999999,"5min":1.607,"15min":1.607},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0469999999999999,"15min":1.085},"min":{"1min":0.52100000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.7030000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.98099999999999998,"15min":1.004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.26},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.986,"5min":2.444,"15min":2.444},"last":1.298}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.319},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.879,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2729999999999999,"15min":1.3009999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9399999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.772}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.98499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3159999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":2.2650000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5329999999999999,"15min":1.585},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.101,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.879}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5209999999999999,"15min":1.585},"min":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2450000000000001,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":2.0539999999999998},{"interface":"front","average":{"1min":1.425,"5min":1.573,"15min":1.631},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0190000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.6459999999999999,"15min":1.7190000000000001},"min":{"1min":1.004,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4470000000000001,"5min":3.48,"15min":3.48},"last":2.133},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.6599999999999999,"15min":1.73},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.004,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.73}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.6839999999999999,"15min":1.7330000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.1859999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.825},{"interface":"front","average":{"1min":1.603,"5min":1.72,"15min":1.782},"min":{"1min":1.113,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.5449999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":2.4140000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1536000,"data_stored":1012729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.95699999999999996,"15min":1.022},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4370000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.107},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.089,"15min":1.1870000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2589999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.997}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.1499999999999999,"15min":1.22},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.593,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.149,"15min":1.2170000000000001},"min":{"1min":0.56299999999999994,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5680000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.056,"15min":1.103},"min":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6160000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.278,"15min":1.3240000000000001},"min":{"1min":0.69799999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.78,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.522}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5409999999999999,"15min":1.633},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.958,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.23},{"interface":"front","average":{"1min":1.302,"5min":1.401,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.768,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.5780000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.4239999999999999,"15min":1.4730000000000001},"min":{"1min":0.80700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7110000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.371},{"interface":"front","average":{"1min":1.373,"5min":1.5489999999999999,"15min":1.627},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.873,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.726}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.593,"15min":1.6419999999999999},"min":{"1min":0.98099999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.9410000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.625},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.647,"15min":1.74},"min":{"1min":1.069,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.831,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.5489999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6879999999999999,"15min":1.792},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2.0840000000000001,"5min":3.149,"15min":3.149},"last":1.661},{"interface":"front","average":{"1min":1.472,"5min":1.5369999999999999,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9019999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.6910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:33:40.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:33:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.437+0000 7fad8ca00640 1 Processor -- start 2024-05-10T03:33:40.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.437+0000 7fad8ca00640 1 -- start start 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad8ca00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad8806d7e0 0x7fad880b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad8ca00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8806d350 con 0x7fad88144120 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad8ca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad880b36d0 con 0x7fad88144710 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad8ca00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad88144530 con 0x7fad8806d7e0 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad86a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fad88144120 0x7fad880cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37498/0 (socket says 172.21.15.55:37498) 2024-05-10T03:33:40.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.439+0000 7fad86a00640 1 -- 172.21.15.55:0/3511317838 learned_addr learned my addr 172.21.15.55:0/3511317838 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:40.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad87e00640 1 --2- 172.21.15.55:0/3511317838 >> v2:172.21.15.162:3300/0 conn(0x7fad8806d7e0 0x7fad880b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898166566 0 0) 0x7fad8806d350 con 0x7fad88144120 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c003520 con 0x7fad88144120 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430639468 0 0) 0x7fad6c003520 con 0x7fad88144120 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 >> v2:172.21.15.162:3300/0 conn(0x7fad8806d7e0 msgr2=0x7fad880b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 --2- 172.21.15.55:0/3511317838 >> v2:172.21.15.162:3300/0 conn(0x7fad8806d7e0 0x7fad880b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 >> v1:172.21.15.162:6789/0 conn(0x7fad88144710 legacy=0x7fad880b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.440+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8806c740 con 0x7fad88144120 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fad74001670 con 0x7fad88144120 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 >> v1:172.21.15.55:6789/0 conn(0x7fad88144120 legacy=0x7fad880cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad6c0039e0 0x7fad6c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c0047c0 0x7fad6c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 0x7fad6c00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88144530 con 0x7fad6c005360 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00c2c0 con 0x7fad6c0047c0 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00c480 con 0x7fad6c0039e0 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad87e00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 0x7fad6c00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad86a00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c0047c0 0x7fad6c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.441+0000 7fad87400640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad6c0039e0 0x7fad6c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.442+0000 7fad87e00640 1 -- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad6c0039e0 msgr2=0x7fad6c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.442+0000 7fad87e00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad6c0039e0 0x7fad6c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.442+0000 7fad87e00640 1 -- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c0047c0 msgr2=0x7fad6c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.442+0000 7fad87e00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c0047c0 0x7fad6c004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.442+0000 7fad87e00640 1 -- 172.21.15.55:0/3511317838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c003520 con 0x7fad6c005360 2024-05-10T03:33:40.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.443+0000 7fad87e00640 1 -- 172.21.15.55:0/3511317838 reap_dead start 2024-05-10T03:33:40.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.443+0000 7fad87e00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 0x7fad6c00b9e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fad80009fc0 tx=0x7fad6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc8763cbc75e062 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.443+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80003b90 con 0x7fad6c005360 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.443+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8000f040 con 0x7fad6c005360 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.443+0000 7fad86000640 1 -- 172.21.15.55:0/3511317838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80004020 con 0x7fad6c005360 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 -- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 msgr2=0x7fad6c00b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 0x7fad6c00b9e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fad80009fc0 tx=0x7fad6c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 -- 172.21.15.55:0/3511317838 shutdown_connections 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 --2- 172.21.15.55:0/3511317838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad6c005360 0x7fad6c00b9e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 -- 172.21.15.55:0/3511317838 >> 172.21.15.55:0/3511317838 conn(0x7fad8806a110 msgr2=0x7fad8806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 -- 172.21.15.55:0/3511317838 shutdown_connections 2024-05-10T03:33:40.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.444+0000 7fad8ca00640 1 -- 172.21.15.55:0/3511317838 wait complete. 2024-05-10T03:33:40.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.446+0000 7fad8ca00640 1 Processor -- start 2024-05-10T03:33:40.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.447+0000 7fad8ca00640 1 -- start start 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c005360 0x7fad8815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad8806ace0 0x7fad8806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88142380 con 0x7fad881f3670 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad881f43a0 con 0x7fad6c005360 2024-05-10T03:33:40.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad8ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806b630 con 0x7fad8806ace0 2024-05-10T03:33:40.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.448+0000 7fad87e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53954/0 (socket says 172.21.15.55:53954) 2024-05-10T03:33:40.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 -- 172.21.15.55:0/1669723423 learned_addr learned my addr 172.21.15.55:0/1669723423 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87400640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c005360 0x7fad8815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad86a00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad8806ace0 0x7fad8806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 -- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad8806ace0 msgr2=0x7fad8806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad8806ace0 0x7fad8806b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 -- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c005360 msgr2=0x7fad8815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c005360 0x7fad8815ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.449+0000 7fad87e00640 1 -- 172.21.15.55:0/1669723423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8806ffd0 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad87e00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fad8000e230 tx=0x7fad80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8003c470 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8003c5f0 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8003c9a0 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad880702b0 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.450+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad88070870 con 0x7fad881f3670 2024-05-10T03:33:40.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.451+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88070a00 con 0x7fad881f3670 2024-05-10T03:33:40.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.453+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fad8003cb00 con 0x7fad881f3670 2024-05-10T03:33:40.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.453+0000 7fad7f400640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 0x7fad6806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:40.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.454+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fad800aa4b0 con 0x7fad881f3670 2024-05-10T03:33:40.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.454+0000 7fad87400640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 0x7fad6806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:40.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.456+0000 7fad87400640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 0x7fad6806d140 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:40.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.460+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fad80075540 con 0x7fad881f3670 2024-05-10T03:33:40.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.657+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad8806df30 con 0x7fad6806ac30 2024-05-10T03:33:40.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.681+0000 7fad7f400640 1 -- 172.21.15.55:0/1669723423 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286937 (secure 0 0 0) 0x7fad8806df30 con 0x7fad6806ac30 2024-05-10T03:33:40.688 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:40.690 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:33:40.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 msgr2=0x7fad6806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 0x7fad6806d140 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad78000c00 tx=0x7fad78009040 comp rx=0 tx=0).stop 2024-05-10T03:33:40.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 msgr2=0x7fad881f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fad8000e230 tx=0x7fad80009750 comp rx=0 tx=0).stop 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 shutdown_connections 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad881f3670 0x7fad881f3a80 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad8806ace0 0x7fad8806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fad6806ac30 0x7fad6806d140 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 --2- 172.21.15.55:0/1669723423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad6c005360 0x7fad8815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 >> 172.21.15.55:0/1669723423 conn(0x7fad8806a110 msgr2=0x7fad880b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 shutdown_connections 2024-05-10T03:33:40.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:40.687+0000 7fad8ca00640 1 -- 172.21.15.55:0/1669723423 wait complete. 2024-05-10T03:33:40.716 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-05-10T03:33:39.039703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253516,"kb_used_data":8620,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754721204,"statfs":{"total":773094113280,"available":772834512896,"internally_reserved":0,"allocated":8826880,"data_stored":4584208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":250596920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.663477"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987791+0000","last_change":"2024-05-10T03:33:35.008997+0000","last_active":"2024-05-10T03:33:35.987791+0000","last_peered":"2024-05-10T03:33:35.987791+0000","last_clean":"2024-05-10T03:33:35.987791+0000","last_became_active":"2024-05-10T03:33:35.008633+0000","last_became_peered":"2024-05-10T03:33:35.008633+0000","last_unstale":"2024-05-10T03:33:35.987791+0000","last_undegraded":"2024-05-10T03:33:35.987791+0000","last_fullsized":"2024-05-10T03:33:35.987791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:26.860633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.016263+0000","last_change":"2024-05-10T03:33:35.015123+0000","last_active":"2024-05-10T03:33:35.016263+0000","last_peered":"2024-05-10T03:33:35.016263+0000","last_clean":"2024-05-10T03:33:35.016263+0000","last_became_active":"2024-05-10T03:33:35.014849+0000","last_became_peered":"2024-05-10T03:33:35.014849+0000","last_unstale":"2024-05-10T03:33:35.016263+0000","last_undegraded":"2024-05-10T03:33:35.016263+0000","last_fullsized":"2024-05-10T03:33:35.016263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:16:41.013194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757402+0000","last_change":"2024-05-10T03:33:35.011072+0000","last_active":"2024-05-10T03:33:36.757402+0000","last_peered":"2024-05-10T03:33:36.757402+0000","last_clean":"2024-05-10T03:33:36.757402+0000","last_became_active":"2024-05-10T03:33:35.010038+0000","last_became_peered":"2024-05-10T03:33:35.010038+0000","last_unstale":"2024-05-10T03:33:36.757402+0000","last_undegraded":"2024-05-10T03:33:36.757402+0000","last_fullsized":"2024-05-10T03:33:36.757402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:49.002716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.996002+0000","last_change":"2024-05-10T03:33:34.991271+0000","last_active":"2024-05-10T03:33:34.996002+0000","last_peered":"2024-05-10T03:33:34.996002+0000","last_clean":"2024-05-10T03:33:34.996002+0000","last_became_active":"2024-05-10T03:33:34.990921+0000","last_became_peered":"2024-05-10T03:33:34.990921+0000","last_unstale":"2024-05-10T03:33:34.996002+0000","last_undegraded":"2024-05-10T03:33:34.996002+0000","last_fullsized":"2024-05-10T03:33:34.996002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:56:12.821437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983852+0000","last_change":"2024-05-10T03:33:35.004776+0000","last_active":"2024-05-10T03:33:35.983852+0000","last_peered":"2024-05-10T03:33:35.983852+0000","last_clean":"2024-05-10T03:33:35.983852+0000","last_became_active":"2024-05-10T03:33:35.004268+0000","last_became_peered":"2024-05-10T03:33:35.004268+0000","last_unstale":"2024-05-10T03:33:35.983852+0000","last_undegraded":"2024-05-10T03:33:35.983852+0000","last_fullsized":"2024-05-10T03:33:35.983852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:26:34.565163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002165+0000","last_change":"2024-05-10T03:33:35.001195+0000","last_active":"2024-05-10T03:33:35.002165+0000","last_peered":"2024-05-10T03:33:35.002165+0000","last_clean":"2024-05-10T03:33:35.002165+0000","last_became_active":"2024-05-10T03:33:34.999824+0000","last_became_peered":"2024-05-10T03:33:34.999824+0000","last_unstale":"2024-05-10T03:33:35.002165+0000","last_undegraded":"2024-05-10T03:33:35.002165+0000","last_fullsized":"2024-05-10T03:33:35.002165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:53.525047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031116+0000","last_change":"2024-05-10T03:33:34.982379+0000","last_active":"2024-05-10T03:33:37.031116+0000","last_peered":"2024-05-10T03:33:37.031116+0000","last_clean":"2024-05-10T03:33:37.031116+0000","last_became_active":"2024-05-10T03:33:34.981828+0000","last_became_peered":"2024-05-10T03:33:34.981828+0000","last_unstale":"2024-05-10T03:33:37.031116+0000","last_undegraded":"2024-05-10T03:33:37.031116+0000","last_fullsized":"2024-05-10T03:33:37.031116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:35.667619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983347+0000","last_change":"2024-05-10T03:33:34.998037+0000","last_active":"2024-05-10T03:33:35.983347+0000","last_peered":"2024-05-10T03:33:35.983347+0000","last_clean":"2024-05-10T03:33:35.983347+0000","last_became_active":"2024-05-10T03:33:34.996201+0000","last_became_peered":"2024-05-10T03:33:34.996201+0000","last_unstale":"2024-05-10T03:33:35.983347+0000","last_undegraded":"2024-05-10T03:33:35.983347+0000","last_fullsized":"2024-05-10T03:33:35.983347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:57:55.725572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984702+0000","last_change":"2024-05-10T03:33:35.003546+0000","last_active":"2024-05-10T03:33:35.984702+0000","last_peered":"2024-05-10T03:33:35.984702+0000","last_clean":"2024-05-10T03:33:35.984702+0000","last_became_active":"2024-05-10T03:33:35.003200+0000","last_became_peered":"2024-05-10T03:33:35.003200+0000","last_unstale":"2024-05-10T03:33:35.984702+0000","last_undegraded":"2024-05-10T03:33:35.984702+0000","last_fullsized":"2024-05-10T03:33:35.984702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:31.268989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.999334+0000","last_change":"2024-05-10T03:33:34.992466+0000","last_active":"2024-05-10T03:33:34.999334+0000","last_peered":"2024-05-10T03:33:34.999334+0000","last_clean":"2024-05-10T03:33:34.999334+0000","last_became_active":"2024-05-10T03:33:34.991075+0000","last_became_peered":"2024-05-10T03:33:34.991075+0000","last_unstale":"2024-05-10T03:33:34.999334+0000","last_undegraded":"2024-05-10T03:33:34.999334+0000","last_fullsized":"2024-05-10T03:33:34.999334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:30:39.551149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.017431+0000","last_change":"2024-05-10T03:33:35.016684+0000","last_active":"2024-05-10T03:33:35.017431+0000","last_peered":"2024-05-10T03:33:35.017431+0000","last_clean":"2024-05-10T03:33:35.017431+0000","last_became_active":"2024-05-10T03:33:35.016145+0000","last_became_peered":"2024-05-10T03:33:35.016145+0000","last_unstale":"2024-05-10T03:33:35.017431+0000","last_undegraded":"2024-05-10T03:33:35.017431+0000","last_fullsized":"2024-05-10T03:33:35.017431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:36:24.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.010437+0000","last_change":"2024-05-10T03:33:35.008690+0000","last_active":"2024-05-10T03:33:35.010437+0000","last_peered":"2024-05-10T03:33:35.010437+0000","last_clean":"2024-05-10T03:33:35.010437+0000","last_became_active":"2024-05-10T03:33:35.007181+0000","last_became_peered":"2024-05-10T03:33:35.007181+0000","last_unstale":"2024-05-10T03:33:35.010437+0000","last_undegraded":"2024-05-10T03:33:35.010437+0000","last_fullsized":"2024-05-10T03:33:35.010437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:04:26.860633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984495+0000","last_change":"2024-05-10T03:33:34.985653+0000","last_active":"2024-05-10T03:33:35.984495+0000","last_peered":"2024-05-10T03:33:35.984495+0000","last_clean":"2024-05-10T03:33:35.984495+0000","last_became_active":"2024-05-10T03:33:34.985318+0000","last_became_peered":"2024-05-10T03:33:34.985318+0000","last_unstale":"2024-05-10T03:33:35.984495+0000","last_undegraded":"2024-05-10T03:33:35.984495+0000","last_fullsized":"2024-05-10T03:33:35.984495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:05.528501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.000249+0000","last_change":"2024-05-10T03:33:34.998497+0000","last_active":"2024-05-10T03:33:35.000249+0000","last_peered":"2024-05-10T03:33:35.000249+0000","last_clean":"2024-05-10T03:33:35.000249+0000","last_became_active":"2024-05-10T03:33:34.998183+0000","last_became_peered":"2024-05-10T03:33:34.998183+0000","last_unstale":"2024-05-10T03:33:35.000249+0000","last_undegraded":"2024-05-10T03:33:35.000249+0000","last_fullsized":"2024-05-10T03:33:35.000249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:16:14.777952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.005116+0000","last_change":"2024-05-10T03:33:34.990118+0000","last_active":"2024-05-10T03:33:35.005116+0000","last_peered":"2024-05-10T03:33:35.005116+0000","last_clean":"2024-05-10T03:33:35.005116+0000","last_became_active":"2024-05-10T03:33:34.989845+0000","last_became_peered":"2024-05-10T03:33:34.989845+0000","last_unstale":"2024-05-10T03:33:35.005116+0000","last_undegraded":"2024-05-10T03:33:35.005116+0000","last_fullsized":"2024-05-10T03:33:35.005116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:21.502387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986377+0000","last_change":"2024-05-10T03:33:34.997825+0000","last_active":"2024-05-10T03:33:35.986377+0000","last_peered":"2024-05-10T03:33:35.986377+0000","last_clean":"2024-05-10T03:33:35.986377+0000","last_became_active":"2024-05-10T03:33:34.997508+0000","last_became_peered":"2024-05-10T03:33:34.997508+0000","last_unstale":"2024-05-10T03:33:35.986377+0000","last_undegraded":"2024-05-10T03:33:35.986377+0000","last_fullsized":"2024-05-10T03:33:35.986377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:20:35.249332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984861+0000","last_change":"2024-05-10T03:33:34.998627+0000","last_active":"2024-05-10T03:33:35.984861+0000","last_peered":"2024-05-10T03:33:35.984861+0000","last_clean":"2024-05-10T03:33:35.984861+0000","last_became_active":"2024-05-10T03:33:34.998180+0000","last_became_peered":"2024-05-10T03:33:34.998180+0000","last_unstale":"2024-05-10T03:33:35.984861+0000","last_undegraded":"2024-05-10T03:33:35.984861+0000","last_fullsized":"2024-05-10T03:33:35.984861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:05:21.371547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757227+0000","last_change":"2024-05-10T03:33:34.997776+0000","last_active":"2024-05-10T03:33:36.757227+0000","last_peered":"2024-05-10T03:33:36.757227+0000","last_clean":"2024-05-10T03:33:36.757227+0000","last_became_active":"2024-05-10T03:33:34.997554+0000","last_became_peered":"2024-05-10T03:33:34.997554+0000","last_unstale":"2024-05-10T03:33:36.757227+0000","last_undegraded":"2024-05-10T03:33:36.757227+0000","last_fullsized":"2024-05-10T03:33:36.757227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.989496+0000","last_change":"2024-05-10T03:33:34.986048+0000","last_active":"2024-05-10T03:33:34.989496+0000","last_peered":"2024-05-10T03:33:34.989496+0000","last_clean":"2024-05-10T03:33:34.989496+0000","last_became_active":"2024-05-10T03:33:34.985746+0000","last_became_peered":"2024-05-10T03:33:34.985746+0000","last_unstale":"2024-05-10T03:33:34.989496+0000","last_undegraded":"2024-05-10T03:33:34.989496+0000","last_fullsized":"2024-05-10T03:33:34.989496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:38:18.760157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984628+0000","last_change":"2024-05-10T03:33:34.993934+0000","last_active":"2024-05-10T03:33:35.984628+0000","last_peered":"2024-05-10T03:33:35.984628+0000","last_clean":"2024-05-10T03:33:35.984628+0000","last_became_active":"2024-05-10T03:33:34.989698+0000","last_became_peered":"2024-05-10T03:33:34.989698+0000","last_unstale":"2024-05-10T03:33:35.984628+0000","last_undegraded":"2024-05-10T03:33:35.984628+0000","last_fullsized":"2024-05-10T03:33:35.984628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:00.767846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984925+0000","last_change":"2024-05-10T03:33:35.005881+0000","last_active":"2024-05-10T03:33:35.984925+0000","last_peered":"2024-05-10T03:33:35.984925+0000","last_clean":"2024-05-10T03:33:35.984925+0000","last_became_active":"2024-05-10T03:33:35.004077+0000","last_became_peered":"2024-05-10T03:33:35.004077+0000","last_unstale":"2024-05-10T03:33:35.984925+0000","last_undegraded":"2024-05-10T03:33:35.984925+0000","last_fullsized":"2024-05-10T03:33:35.984925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983799+0000","last_change":"2024-05-10T03:33:34.997703+0000","last_active":"2024-05-10T03:33:35.983799+0000","last_peered":"2024-05-10T03:33:35.983799+0000","last_clean":"2024-05-10T03:33:35.983799+0000","last_became_active":"2024-05-10T03:33:34.996161+0000","last_became_peered":"2024-05-10T03:33:34.996161+0000","last_unstale":"2024-05-10T03:33:35.983799+0000","last_undegraded":"2024-05-10T03:33:35.983799+0000","last_fullsized":"2024-05-10T03:33:35.983799+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:38:33.843772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983543+0000","last_change":"2024-05-10T03:33:34.994993+0000","last_active":"2024-05-10T03:33:35.983543+0000","last_peered":"2024-05-10T03:33:35.983543+0000","last_clean":"2024-05-10T03:33:35.983543+0000","last_became_active":"2024-05-10T03:33:34.994473+0000","last_became_peered":"2024-05-10T03:33:34.994473+0000","last_unstale":"2024-05-10T03:33:35.983543+0000","last_undegraded":"2024-05-10T03:33:35.983543+0000","last_fullsized":"2024-05-10T03:33:35.983543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.004889+0000","last_change":"2024-05-10T03:33:34.988398+0000","last_active":"2024-05-10T03:33:35.004889+0000","last_peered":"2024-05-10T03:33:35.004889+0000","last_clean":"2024-05-10T03:33:35.004889+0000","last_became_active":"2024-05-10T03:33:34.988119+0000","last_became_peered":"2024-05-10T03:33:34.988119+0000","last_unstale":"2024-05-10T03:33:35.004889+0000","last_undegraded":"2024-05-10T03:33:35.004889+0000","last_fullsized":"2024-05-10T03:33:35.004889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:39.537388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.012746+0000","last_change":"2024-05-10T03:33:35.008087+0000","last_active":"2024-05-10T03:33:35.012746+0000","last_peered":"2024-05-10T03:33:35.012746+0000","last_clean":"2024-05-10T03:33:35.012746+0000","last_became_active":"2024-05-10T03:33:35.007759+0000","last_became_peered":"2024-05-10T03:33:35.007759+0000","last_unstale":"2024-05-10T03:33:35.012746+0000","last_undegraded":"2024-05-10T03:33:35.012746+0000","last_fullsized":"2024-05-10T03:33:35.012746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:54:57.019193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.010448+0000","last_change":"2024-05-10T03:33:35.008413+0000","last_active":"2024-05-10T03:33:35.010448+0000","last_peered":"2024-05-10T03:33:35.010448+0000","last_clean":"2024-05-10T03:33:35.010448+0000","last_became_active":"2024-05-10T03:33:35.007924+0000","last_became_peered":"2024-05-10T03:33:35.007924+0000","last_unstale":"2024-05-10T03:33:35.010448+0000","last_undegraded":"2024-05-10T03:33:35.010448+0000","last_fullsized":"2024-05-10T03:33:35.010448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:20.689486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986852+0000","last_change":"2024-05-10T03:33:34.997936+0000","last_active":"2024-05-10T03:33:35.986852+0000","last_peered":"2024-05-10T03:33:35.986852+0000","last_clean":"2024-05-10T03:33:35.986852+0000","last_became_active":"2024-05-10T03:33:34.997576+0000","last_became_peered":"2024-05-10T03:33:34.997576+0000","last_unstale":"2024-05-10T03:33:35.986852+0000","last_undegraded":"2024-05-10T03:33:35.986852+0000","last_fullsized":"2024-05-10T03:33:35.986852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:32:16.832262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984964+0000","last_change":"2024-05-10T03:33:34.994113+0000","last_active":"2024-05-10T03:33:35.984964+0000","last_peered":"2024-05-10T03:33:35.984964+0000","last_clean":"2024-05-10T03:33:35.984964+0000","last_became_active":"2024-05-10T03:33:34.993615+0000","last_became_peered":"2024-05-10T03:33:34.993615+0000","last_unstale":"2024-05-10T03:33:35.984964+0000","last_undegraded":"2024-05-10T03:33:35.984964+0000","last_fullsized":"2024-05-10T03:33:35.984964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:18.917135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.007437+0000","last_change":"2024-05-10T03:33:35.002816+0000","last_active":"2024-05-10T03:33:35.007437+0000","last_peered":"2024-05-10T03:33:35.007437+0000","last_clean":"2024-05-10T03:33:35.007437+0000","last_became_active":"2024-05-10T03:33:35.002476+0000","last_became_peered":"2024-05-10T03:33:35.002476+0000","last_unstale":"2024-05-10T03:33:35.007437+0000","last_undegraded":"2024-05-10T03:33:35.007437+0000","last_fullsized":"2024-05-10T03:33:35.007437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986570+0000","last_change":"2024-05-10T03:33:35.009782+0000","last_active":"2024-05-10T03:33:35.986570+0000","last_peered":"2024-05-10T03:33:35.986570+0000","last_clean":"2024-05-10T03:33:35.986570+0000","last_became_active":"2024-05-10T03:33:35.009501+0000","last_became_peered":"2024-05-10T03:33:35.009501+0000","last_unstale":"2024-05-10T03:33:35.986570+0000","last_undegraded":"2024-05-10T03:33:35.986570+0000","last_fullsized":"2024-05-10T03:33:35.986570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:30:39.551149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756475+0000","last_change":"2024-05-10T03:33:34.981266+0000","last_active":"2024-05-10T03:33:36.756475+0000","last_peered":"2024-05-10T03:33:36.756475+0000","last_clean":"2024-05-10T03:33:36.756475+0000","last_became_active":"2024-05-10T03:33:34.979560+0000","last_became_peered":"2024-05-10T03:33:34.979560+0000","last_unstale":"2024-05-10T03:33:36.756475+0000","last_undegraded":"2024-05-10T03:33:36.756475+0000","last_fullsized":"2024-05-10T03:33:36.756475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:06:35.893196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986504+0000","last_change":"2024-05-10T03:33:35.008153+0000","last_active":"2024-05-10T03:33:35.986504+0000","last_peered":"2024-05-10T03:33:35.986504+0000","last_clean":"2024-05-10T03:33:35.986504+0000","last_became_active":"2024-05-10T03:33:35.007685+0000","last_became_peered":"2024-05-10T03:33:35.007685+0000","last_unstale":"2024-05-10T03:33:35.986504+0000","last_undegraded":"2024-05-10T03:33:35.986504+0000","last_fullsized":"2024-05-10T03:33:35.986504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756642+0000","last_change":"2024-05-10T03:33:35.000426+0000","last_active":"2024-05-10T03:33:36.756642+0000","last_peered":"2024-05-10T03:33:36.756642+0000","last_clean":"2024-05-10T03:33:36.756642+0000","last_became_active":"2024-05-10T03:33:34.998665+0000","last_became_peered":"2024-05-10T03:33:34.998665+0000","last_unstale":"2024-05-10T03:33:36.756642+0000","last_undegraded":"2024-05-10T03:33:36.756642+0000","last_fullsized":"2024-05-10T03:33:36.756642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:34.186248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984968+0000","last_change":"2024-05-10T03:33:34.419061+0000","last_active":"2024-05-10T03:33:35.984968+0000","last_peered":"2024-05-10T03:33:35.984968+0000","last_clean":"2024-05-10T03:33:35.984968+0000","last_became_active":"2024-05-10T03:33:34.415943+0000","last_became_peered":"2024-05-10T03:33:34.415943+0000","last_unstale":"2024-05-10T03:33:35.984968+0000","last_undegraded":"2024-05-10T03:33:35.984968+0000","last_fullsized":"2024-05-10T03:33:35.984968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:02:01.592527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.009211+0000","last_change":"2024-05-10T03:33:35.000067+0000","last_active":"2024-05-10T03:33:35.009211+0000","last_peered":"2024-05-10T03:33:35.009211+0000","last_clean":"2024-05-10T03:33:35.009211+0000","last_became_active":"2024-05-10T03:33:34.999770+0000","last_became_peered":"2024-05-10T03:33:34.999770+0000","last_unstale":"2024-05-10T03:33:35.009211+0000","last_undegraded":"2024-05-10T03:33:35.009211+0000","last_fullsized":"2024-05-10T03:33:35.009211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:31:14.362791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987699+0000","last_change":"2024-05-10T03:33:34.424101+0000","last_active":"2024-05-10T03:33:35.987699+0000","last_peered":"2024-05-10T03:33:35.987699+0000","last_clean":"2024-05-10T03:33:35.987699+0000","last_became_active":"2024-05-10T03:33:34.423276+0000","last_became_peered":"2024-05-10T03:33:34.423276+0000","last_unstale":"2024-05-10T03:33:35.987699+0000","last_undegraded":"2024-05-10T03:33:35.987699+0000","last_fullsized":"2024-05-10T03:33:35.987699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:49.359571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984065+0000","last_change":"2024-05-10T03:33:35.005740+0000","last_active":"2024-05-10T03:33:35.984065+0000","last_peered":"2024-05-10T03:33:35.984065+0000","last_clean":"2024-05-10T03:33:35.984065+0000","last_became_active":"2024-05-10T03:33:35.001720+0000","last_became_peered":"2024-05-10T03:33:35.001720+0000","last_unstale":"2024-05-10T03:33:35.984065+0000","last_undegraded":"2024-05-10T03:33:35.984065+0000","last_fullsized":"2024-05-10T03:33:35.984065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T03:35:58.939759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987195+0000","last_change":"2024-05-10T03:33:34.416288+0000","last_active":"2024-05-10T03:33:35.987195+0000","last_peered":"2024-05-10T03:33:35.987195+0000","last_clean":"2024-05-10T03:33:35.987195+0000","last_became_active":"2024-05-10T03:33:34.415939+0000","last_became_peered":"2024-05-10T03:33:34.415939+0000","last_unstale":"2024-05-10T03:33:35.987195+0000","last_undegraded":"2024-05-10T03:33:35.987195+0000","last_fullsized":"2024-05-10T03:33:35.987195+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:03.799869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983587+0000","last_change":"2024-05-10T03:33:35.005874+0000","last_active":"2024-05-10T03:33:35.983587+0000","last_peered":"2024-05-10T03:33:35.983587+0000","last_clean":"2024-05-10T03:33:35.983587+0000","last_became_active":"2024-05-10T03:33:35.004074+0000","last_became_peered":"2024-05-10T03:33:35.004074+0000","last_unstale":"2024-05-10T03:33:35.983587+0000","last_undegraded":"2024-05-10T03:33:35.983587+0000","last_fullsized":"2024-05-10T03:33:35.983587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:34.186248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031752+0000","last_change":"2024-05-10T03:33:34.412661+0000","last_active":"2024-05-10T03:33:37.031752+0000","last_peered":"2024-05-10T03:33:37.031752+0000","last_clean":"2024-05-10T03:33:37.031752+0000","last_became_active":"2024-05-10T03:33:34.412343+0000","last_became_peered":"2024-05-10T03:33:34.412343+0000","last_unstale":"2024-05-10T03:33:37.031752+0000","last_undegraded":"2024-05-10T03:33:37.031752+0000","last_fullsized":"2024-05-10T03:33:37.031752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:42.034522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013571+0000","last_change":"2024-05-10T03:33:35.010558+0000","last_active":"2024-05-10T03:33:35.013571+0000","last_peered":"2024-05-10T03:33:35.013571+0000","last_clean":"2024-05-10T03:33:35.013571+0000","last_became_active":"2024-05-10T03:33:35.009966+0000","last_became_peered":"2024-05-10T03:33:35.009966+0000","last_unstale":"2024-05-10T03:33:35.013571+0000","last_undegraded":"2024-05-10T03:33:35.013571+0000","last_fullsized":"2024-05-10T03:33:35.013571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:26:24.907160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983802+0000","last_change":"2024-05-10T03:33:34.425479+0000","last_active":"2024-05-10T03:33:35.983802+0000","last_peered":"2024-05-10T03:33:35.983802+0000","last_clean":"2024-05-10T03:33:35.983802+0000","last_became_active":"2024-05-10T03:33:34.425167+0000","last_became_peered":"2024-05-10T03:33:34.425167+0000","last_unstale":"2024-05-10T03:33:35.983802+0000","last_undegraded":"2024-05-10T03:33:35.983802+0000","last_fullsized":"2024-05-10T03:33:35.983802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:35:28.640085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.016753+0000","last_change":"2024-05-10T03:33:35.014909+0000","last_active":"2024-05-10T03:33:35.016753+0000","last_peered":"2024-05-10T03:33:35.016753+0000","last_clean":"2024-05-10T03:33:35.016753+0000","last_became_active":"2024-05-10T03:33:35.014624+0000","last_became_peered":"2024-05-10T03:33:35.014624+0000","last_unstale":"2024-05-10T03:33:35.016753+0000","last_undegraded":"2024-05-10T03:33:35.016753+0000","last_fullsized":"2024-05-10T03:33:35.016753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:23.023094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.996050+0000","last_change":"2024-05-10T03:33:34.414756+0000","last_active":"2024-05-10T03:33:34.996050+0000","last_peered":"2024-05-10T03:33:34.996050+0000","last_clean":"2024-05-10T03:33:34.996050+0000","last_became_active":"2024-05-10T03:33:34.413339+0000","last_became_peered":"2024-05-10T03:33:34.413339+0000","last_unstale":"2024-05-10T03:33:34.996050+0000","last_undegraded":"2024-05-10T03:33:34.996050+0000","last_fullsized":"2024-05-10T03:33:34.996050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984471+0000","last_change":"2024-05-10T03:33:35.005510+0000","last_active":"2024-05-10T03:33:35.984471+0000","last_peered":"2024-05-10T03:33:35.984471+0000","last_clean":"2024-05-10T03:33:35.984471+0000","last_became_active":"2024-05-10T03:33:35.004049+0000","last_became_peered":"2024-05-10T03:33:35.004049+0000","last_unstale":"2024-05-10T03:33:35.984471+0000","last_undegraded":"2024-05-10T03:33:35.984471+0000","last_fullsized":"2024-05-10T03:33:35.984471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:49.002716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.989600+0000","last_change":"2024-05-10T03:33:34.411385+0000","last_active":"2024-05-10T03:33:34.989600+0000","last_peered":"2024-05-10T03:33:34.989600+0000","last_clean":"2024-05-10T03:33:34.989600+0000","last_became_active":"2024-05-10T03:33:34.411069+0000","last_became_peered":"2024-05-10T03:33:34.411069+0000","last_unstale":"2024-05-10T03:33:34.989600+0000","last_undegraded":"2024-05-10T03:33:34.989600+0000","last_fullsized":"2024-05-10T03:33:34.989600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:25.471769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032162+0000","last_change":"2024-05-10T03:33:34.991775+0000","last_active":"2024-05-10T03:33:37.032162+0000","last_peered":"2024-05-10T03:33:37.032162+0000","last_clean":"2024-05-10T03:33:37.032162+0000","last_became_active":"2024-05-10T03:33:34.991459+0000","last_became_peered":"2024-05-10T03:33:34.991459+0000","last_unstale":"2024-05-10T03:33:37.032162+0000","last_undegraded":"2024-05-10T03:33:37.032162+0000","last_fullsized":"2024-05-10T03:33:37.032162+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:12.749579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983523+0000","last_change":"2024-05-10T03:33:34.419971+0000","last_active":"2024-05-10T03:33:35.983523+0000","last_peered":"2024-05-10T03:33:35.983523+0000","last_clean":"2024-05-10T03:33:35.983523+0000","last_became_active":"2024-05-10T03:33:34.419636+0000","last_became_peered":"2024-05-10T03:33:34.419636+0000","last_unstale":"2024-05-10T03:33:35.983523+0000","last_undegraded":"2024-05-10T03:33:35.983523+0000","last_fullsized":"2024-05-10T03:33:35.983523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:20:31.221544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013797+0000","last_change":"2024-05-10T03:33:35.010438+0000","last_active":"2024-05-10T03:33:35.013797+0000","last_peered":"2024-05-10T03:33:35.013797+0000","last_clean":"2024-05-10T03:33:35.013797+0000","last_became_active":"2024-05-10T03:33:35.010104+0000","last_became_peered":"2024-05-10T03:33:35.010104+0000","last_unstale":"2024-05-10T03:33:35.013797+0000","last_undegraded":"2024-05-10T03:33:35.013797+0000","last_fullsized":"2024-05-10T03:33:35.013797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:00.339870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987733+0000","last_change":"2024-05-10T03:33:34.424568+0000","last_active":"2024-05-10T03:33:35.987733+0000","last_peered":"2024-05-10T03:33:35.987733+0000","last_clean":"2024-05-10T03:33:35.987733+0000","last_became_active":"2024-05-10T03:33:34.423766+0000","last_became_peered":"2024-05-10T03:33:34.423766+0000","last_unstale":"2024-05-10T03:33:35.987733+0000","last_undegraded":"2024-05-10T03:33:35.987733+0000","last_fullsized":"2024-05-10T03:33:35.987733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:27:47.983820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002159+0000","last_change":"2024-05-10T03:33:34.998134+0000","last_active":"2024-05-10T03:33:35.002159+0000","last_peered":"2024-05-10T03:33:35.002159+0000","last_clean":"2024-05-10T03:33:35.002159+0000","last_became_active":"2024-05-10T03:33:34.996681+0000","last_became_peered":"2024-05-10T03:33:34.996681+0000","last_unstale":"2024-05-10T03:33:35.002159+0000","last_undegraded":"2024-05-10T03:33:35.002159+0000","last_fullsized":"2024-05-10T03:33:35.002159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:16.455983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.974623+0000","last_change":"2024-05-10T03:33:34.412464+0000","last_active":"2024-05-10T03:33:34.974623+0000","last_peered":"2024-05-10T03:33:34.974623+0000","last_clean":"2024-05-10T03:33:34.974623+0000","last_became_active":"2024-05-10T03:33:34.412081+0000","last_became_peered":"2024-05-10T03:33:34.412081+0000","last_unstale":"2024-05-10T03:33:34.974623+0000","last_undegraded":"2024-05-10T03:33:34.974623+0000","last_fullsized":"2024-05-10T03:33:34.974623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:38:34.648723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.991205+0000","last_change":"2024-05-10T03:33:34.983307+0000","last_active":"2024-05-10T03:33:34.991205+0000","last_peered":"2024-05-10T03:33:34.991205+0000","last_clean":"2024-05-10T03:33:34.991205+0000","last_became_active":"2024-05-10T03:33:34.982446+0000","last_became_peered":"2024-05-10T03:33:34.982446+0000","last_unstale":"2024-05-10T03:33:34.991205+0000","last_undegraded":"2024-05-10T03:33:34.991205+0000","last_fullsized":"2024-05-10T03:33:34.991205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:44:09.811426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984399+0000","last_change":"2024-05-10T03:33:34.418191+0000","last_active":"2024-05-10T03:33:35.984399+0000","last_peered":"2024-05-10T03:33:35.984399+0000","last_clean":"2024-05-10T03:33:35.984399+0000","last_became_active":"2024-05-10T03:33:34.417848+0000","last_became_peered":"2024-05-10T03:33:34.417848+0000","last_unstale":"2024-05-10T03:33:35.984399+0000","last_undegraded":"2024-05-10T03:33:35.984399+0000","last_fullsized":"2024-05-10T03:33:35.984399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:40.832314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756937+0000","last_change":"2024-05-10T03:33:35.010407+0000","last_active":"2024-05-10T03:33:36.756937+0000","last_peered":"2024-05-10T03:33:36.756937+0000","last_clean":"2024-05-10T03:33:36.756937+0000","last_became_active":"2024-05-10T03:33:35.009961+0000","last_became_peered":"2024-05-10T03:33:35.009961+0000","last_unstale":"2024-05-10T03:33:36.756937+0000","last_undegraded":"2024-05-10T03:33:36.756937+0000","last_fullsized":"2024-05-10T03:33:36.756937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:07:36.236493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031868+0000","last_change":"2024-05-10T03:33:34.419502+0000","last_active":"2024-05-10T03:33:37.031868+0000","last_peered":"2024-05-10T03:33:37.031868+0000","last_clean":"2024-05-10T03:33:37.031868+0000","last_became_active":"2024-05-10T03:33:34.419166+0000","last_became_peered":"2024-05-10T03:33:34.419166+0000","last_unstale":"2024-05-10T03:33:37.031868+0000","last_undegraded":"2024-05-10T03:33:37.031868+0000","last_fullsized":"2024-05-10T03:33:37.031868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:15.639144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.990814+0000","last_change":"2024-05-10T03:33:34.980846+0000","last_active":"2024-05-10T03:33:34.990814+0000","last_peered":"2024-05-10T03:33:34.990814+0000","last_clean":"2024-05-10T03:33:34.990814+0000","last_became_active":"2024-05-10T03:33:34.980561+0000","last_became_peered":"2024-05-10T03:33:34.980561+0000","last_unstale":"2024-05-10T03:33:34.990814+0000","last_undegraded":"2024-05-10T03:33:34.990814+0000","last_fullsized":"2024-05-10T03:33:34.990814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:30.535866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.002274+0000","last_change":"2024-05-10T03:33:34.420380+0000","last_active":"2024-05-10T03:33:35.002274+0000","last_peered":"2024-05-10T03:33:35.002274+0000","last_clean":"2024-05-10T03:33:35.002274+0000","last_became_active":"2024-05-10T03:33:34.419103+0000","last_became_peered":"2024-05-10T03:33:34.419103+0000","last_unstale":"2024-05-10T03:33:35.002274+0000","last_undegraded":"2024-05-10T03:33:35.002274+0000","last_fullsized":"2024-05-10T03:33:35.002274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:16.475920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031670+0000","last_change":"2024-05-10T03:33:34.989465+0000","last_active":"2024-05-10T03:33:37.031670+0000","last_peered":"2024-05-10T03:33:37.031670+0000","last_clean":"2024-05-10T03:33:37.031670+0000","last_became_active":"2024-05-10T03:33:34.989125+0000","last_became_peered":"2024-05-10T03:33:34.989125+0000","last_unstale":"2024-05-10T03:33:37.031670+0000","last_undegraded":"2024-05-10T03:33:37.031670+0000","last_fullsized":"2024-05-10T03:33:37.031670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:26.490665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986597+0000","last_change":"2024-05-10T03:33:34.421311+0000","last_active":"2024-05-10T03:33:35.986597+0000","last_peered":"2024-05-10T03:33:35.986597+0000","last_clean":"2024-05-10T03:33:35.986597+0000","last_became_active":"2024-05-10T03:33:34.420917+0000","last_became_peered":"2024-05-10T03:33:34.420917+0000","last_unstale":"2024-05-10T03:33:35.986597+0000","last_undegraded":"2024-05-10T03:33:35.986597+0000","last_fullsized":"2024-05-10T03:33:35.986597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:49:25.471769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032042+0000","last_change":"2024-05-10T03:33:34.999343+0000","last_active":"2024-05-10T03:33:37.032042+0000","last_peered":"2024-05-10T03:33:37.032042+0000","last_clean":"2024-05-10T03:33:37.032042+0000","last_became_active":"2024-05-10T03:33:34.998493+0000","last_became_peered":"2024-05-10T03:33:34.998493+0000","last_unstale":"2024-05-10T03:33:37.032042+0000","last_undegraded":"2024-05-10T03:33:37.032042+0000","last_fullsized":"2024-05-10T03:33:37.032042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:36.786332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983299+0000","last_change":"2024-05-10T03:33:34.416526+0000","last_active":"2024-05-10T03:33:35.983299+0000","last_peered":"2024-05-10T03:33:35.983299+0000","last_clean":"2024-05-10T03:33:35.983299+0000","last_became_active":"2024-05-10T03:33:34.416153+0000","last_became_peered":"2024-05-10T03:33:34.416153+0000","last_unstale":"2024-05-10T03:33:35.983299+0000","last_undegraded":"2024-05-10T03:33:35.983299+0000","last_fullsized":"2024-05-10T03:33:35.983299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:23:25.613525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032208+0000","last_change":"2024-05-10T03:33:34.981766+0000","last_active":"2024-05-10T03:33:37.032208+0000","last_peered":"2024-05-10T03:33:37.032208+0000","last_clean":"2024-05-10T03:33:37.032208+0000","last_became_active":"2024-05-10T03:33:34.981445+0000","last_became_peered":"2024-05-10T03:33:34.981445+0000","last_unstale":"2024-05-10T03:33:37.032208+0000","last_undegraded":"2024-05-10T03:33:37.032208+0000","last_fullsized":"2024-05-10T03:33:37.032208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:57:44.131089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.015286+0000","last_change":"2024-05-10T03:33:34.409915+0000","last_active":"2024-05-10T03:33:35.015286+0000","last_peered":"2024-05-10T03:33:35.015286+0000","last_clean":"2024-05-10T03:33:35.015286+0000","last_became_active":"2024-05-10T03:33:34.409613+0000","last_became_peered":"2024-05-10T03:33:34.409613+0000","last_unstale":"2024-05-10T03:33:35.015286+0000","last_undegraded":"2024-05-10T03:33:35.015286+0000","last_fullsized":"2024-05-10T03:33:35.015286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.988976+0000","last_change":"2024-05-10T03:33:34.986079+0000","last_active":"2024-05-10T03:33:34.988976+0000","last_peered":"2024-05-10T03:33:34.988976+0000","last_clean":"2024-05-10T03:33:34.988976+0000","last_became_active":"2024-05-10T03:33:34.985769+0000","last_became_peered":"2024-05-10T03:33:34.985769+0000","last_unstale":"2024-05-10T03:33:34.988976+0000","last_undegraded":"2024-05-10T03:33:34.988976+0000","last_fullsized":"2024-05-10T03:33:34.988976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:59:52.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987822+0000","last_change":"2024-05-10T03:33:34.421333+0000","last_active":"2024-05-10T03:33:35.987822+0000","last_peered":"2024-05-10T03:33:35.987822+0000","last_clean":"2024-05-10T03:33:35.987822+0000","last_became_active":"2024-05-10T03:33:34.420986+0000","last_became_peered":"2024-05-10T03:33:34.420986+0000","last_unstale":"2024-05-10T03:33:35.987822+0000","last_undegraded":"2024-05-10T03:33:35.987822+0000","last_fullsized":"2024-05-10T03:33:35.987822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:33.167351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983401+0000","last_change":"2024-05-10T03:33:34.997695+0000","last_active":"2024-05-10T03:33:35.983401+0000","last_peered":"2024-05-10T03:33:35.983401+0000","last_clean":"2024-05-10T03:33:35.983401+0000","last_became_active":"2024-05-10T03:33:34.996347+0000","last_became_peered":"2024-05-10T03:33:34.996347+0000","last_unstale":"2024-05-10T03:33:35.983401+0000","last_undegraded":"2024-05-10T03:33:35.983401+0000","last_fullsized":"2024-05-10T03:33:35.983401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:36:26.281740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984712+0000","last_change":"2024-05-10T03:33:34.424818+0000","last_active":"2024-05-10T03:33:35.984712+0000","last_peered":"2024-05-10T03:33:35.984712+0000","last_clean":"2024-05-10T03:33:35.984712+0000","last_became_active":"2024-05-10T03:33:34.423908+0000","last_became_peered":"2024-05-10T03:33:34.423908+0000","last_unstale":"2024-05-10T03:33:35.984712+0000","last_undegraded":"2024-05-10T03:33:35.984712+0000","last_fullsized":"2024-05-10T03:33:35.984712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:20.663035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984284+0000","last_change":"2024-05-10T03:33:35.011590+0000","last_active":"2024-05-10T03:33:35.984284+0000","last_peered":"2024-05-10T03:33:35.984284+0000","last_clean":"2024-05-10T03:33:35.984284+0000","last_became_active":"2024-05-10T03:33:35.011296+0000","last_became_peered":"2024-05-10T03:33:35.011296+0000","last_unstale":"2024-05-10T03:33:35.984284+0000","last_undegraded":"2024-05-10T03:33:35.984284+0000","last_fullsized":"2024-05-10T03:33:35.984284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:54:07.159229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984709+0000","last_change":"2024-05-10T03:33:34.423531+0000","last_active":"2024-05-10T03:33:35.984709+0000","last_peered":"2024-05-10T03:33:35.984709+0000","last_clean":"2024-05-10T03:33:35.984709+0000","last_became_active":"2024-05-10T03:33:34.423100+0000","last_became_peered":"2024-05-10T03:33:34.423100+0000","last_unstale":"2024-05-10T03:33:35.984709+0000","last_undegraded":"2024-05-10T03:33:35.984709+0000","last_fullsized":"2024-05-10T03:33:35.984709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:03:20.778564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.997481+0000","last_change":"2024-05-10T03:33:34.994585+0000","last_active":"2024-05-10T03:33:34.997481+0000","last_peered":"2024-05-10T03:33:34.997481+0000","last_clean":"2024-05-10T03:33:34.997481+0000","last_became_active":"2024-05-10T03:33:34.994218+0000","last_became_peered":"2024-05-10T03:33:34.994218+0000","last_unstale":"2024-05-10T03:33:34.997481+0000","last_undegraded":"2024-05-10T03:33:34.997481+0000","last_fullsized":"2024-05-10T03:33:34.997481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:31.655605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756961+0000","last_change":"2024-05-10T03:33:34.419095+0000","last_active":"2024-05-10T03:33:36.756961+0000","last_peered":"2024-05-10T03:33:36.756961+0000","last_clean":"2024-05-10T03:33:36.756961+0000","last_became_active":"2024-05-10T03:33:34.418799+0000","last_became_peered":"2024-05-10T03:33:34.418799+0000","last_unstale":"2024-05-10T03:33:36.756961+0000","last_undegraded":"2024-05-10T03:33:36.756961+0000","last_fullsized":"2024-05-10T03:33:36.756961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:43:29.120089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.011873+0000","last_change":"2024-05-10T03:33:35.005673+0000","last_active":"2024-05-10T03:33:35.011873+0000","last_peered":"2024-05-10T03:33:35.011873+0000","last_clean":"2024-05-10T03:33:35.011873+0000","last_became_active":"2024-05-10T03:33:35.005294+0000","last_became_peered":"2024-05-10T03:33:35.005294+0000","last_unstale":"2024-05-10T03:33:35.011873+0000","last_undegraded":"2024-05-10T03:33:35.011873+0000","last_fullsized":"2024-05-10T03:33:35.011873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:16.455983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.003718+0000","last_change":"2024-05-10T03:33:34.415418+0000","last_active":"2024-05-10T03:33:35.003718+0000","last_peered":"2024-05-10T03:33:35.003718+0000","last_clean":"2024-05-10T03:33:35.003718+0000","last_became_active":"2024-05-10T03:33:34.415087+0000","last_became_peered":"2024-05-10T03:33:34.415087+0000","last_unstale":"2024-05-10T03:33:35.003718+0000","last_undegraded":"2024-05-10T03:33:35.003718+0000","last_fullsized":"2024-05-10T03:33:35.003718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:20.483491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.011818+0000","last_change":"2024-05-10T03:33:35.007265+0000","last_active":"2024-05-10T03:33:35.011818+0000","last_peered":"2024-05-10T03:33:35.011818+0000","last_clean":"2024-05-10T03:33:35.011818+0000","last_became_active":"2024-05-10T03:33:35.006919+0000","last_became_peered":"2024-05-10T03:33:35.006919+0000","last_unstale":"2024-05-10T03:33:35.011818+0000","last_undegraded":"2024-05-10T03:33:35.011818+0000","last_fullsized":"2024-05-10T03:33:35.011818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:39:55.333376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.970682+0000","last_change":"2024-05-10T03:33:34.412690+0000","last_active":"2024-05-10T03:33:34.970682+0000","last_peered":"2024-05-10T03:33:34.970682+0000","last_clean":"2024-05-10T03:33:34.970682+0000","last_became_active":"2024-05-10T03:33:34.412291+0000","last_became_peered":"2024-05-10T03:33:34.412291+0000","last_unstale":"2024-05-10T03:33:34.970682+0000","last_undegraded":"2024-05-10T03:33:34.970682+0000","last_fullsized":"2024-05-10T03:33:34.970682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:02:22.004198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.032325+0000","last_change":"2024-05-10T03:33:34.988056+0000","last_active":"2024-05-10T03:33:37.032325+0000","last_peered":"2024-05-10T03:33:37.032325+0000","last_clean":"2024-05-10T03:33:37.032325+0000","last_became_active":"2024-05-10T03:33:34.987667+0000","last_became_peered":"2024-05-10T03:33:34.987667+0000","last_unstale":"2024-05-10T03:33:37.032325+0000","last_undegraded":"2024-05-10T03:33:37.032325+0000","last_fullsized":"2024-05-10T03:33:37.032325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:47.350922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.970562+0000","last_change":"2024-05-10T03:33:34.423714+0000","last_active":"2024-05-10T03:33:34.970562+0000","last_peered":"2024-05-10T03:33:34.970562+0000","last_clean":"2024-05-10T03:33:34.970562+0000","last_became_active":"2024-05-10T03:33:34.423249+0000","last_became_peered":"2024-05-10T03:33:34.423249+0000","last_unstale":"2024-05-10T03:33:34.970562+0000","last_undegraded":"2024-05-10T03:33:34.970562+0000","last_fullsized":"2024-05-10T03:33:34.970562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:23:59.320973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.983903+0000","last_change":"2024-05-10T03:33:34.999906+0000","last_active":"2024-05-10T03:33:35.983903+0000","last_peered":"2024-05-10T03:33:35.983903+0000","last_clean":"2024-05-10T03:33:35.983903+0000","last_became_active":"2024-05-10T03:33:34.999478+0000","last_became_peered":"2024-05-10T03:33:34.999478+0000","last_unstale":"2024-05-10T03:33:35.983903+0000","last_undegraded":"2024-05-10T03:33:35.983903+0000","last_fullsized":"2024-05-10T03:33:35.983903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:50.378468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.994363+0000","last_change":"2024-05-10T03:33:34.417010+0000","last_active":"2024-05-10T03:33:34.994363+0000","last_peered":"2024-05-10T03:33:34.994363+0000","last_clean":"2024-05-10T03:33:34.994363+0000","last_became_active":"2024-05-10T03:33:34.415398+0000","last_became_peered":"2024-05-10T03:33:34.415398+0000","last_unstale":"2024-05-10T03:33:34.994363+0000","last_undegraded":"2024-05-10T03:33:34.994363+0000","last_fullsized":"2024-05-10T03:33:34.994363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031239+0000","last_change":"2024-05-10T03:33:34.987778+0000","last_active":"2024-05-10T03:33:37.031239+0000","last_peered":"2024-05-10T03:33:37.031239+0000","last_clean":"2024-05-10T03:33:37.031239+0000","last_became_active":"2024-05-10T03:33:34.987466+0000","last_became_peered":"2024-05-10T03:33:34.987466+0000","last_unstale":"2024-05-10T03:33:37.031239+0000","last_undegraded":"2024-05-10T03:33:37.031239+0000","last_fullsized":"2024-05-10T03:33:37.031239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:13:17.494816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.994550+0000","last_change":"2024-05-10T03:33:34.414901+0000","last_active":"2024-05-10T03:33:34.994550+0000","last_peered":"2024-05-10T03:33:34.994550+0000","last_clean":"2024-05-10T03:33:34.994550+0000","last_became_active":"2024-05-10T03:33:34.414455+0000","last_became_peered":"2024-05-10T03:33:34.414455+0000","last_unstale":"2024-05-10T03:33:34.994550+0000","last_undegraded":"2024-05-10T03:33:34.994550+0000","last_fullsized":"2024-05-10T03:33:34.994550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984386+0000","last_change":"2024-05-10T03:33:35.005976+0000","last_active":"2024-05-10T03:33:35.984386+0000","last_peered":"2024-05-10T03:33:35.984386+0000","last_clean":"2024-05-10T03:33:35.984386+0000","last_became_active":"2024-05-10T03:33:35.004542+0000","last_became_peered":"2024-05-10T03:33:35.004542+0000","last_unstale":"2024-05-10T03:33:35.984386+0000","last_undegraded":"2024-05-10T03:33:35.984386+0000","last_fullsized":"2024-05-10T03:33:35.984386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:47:14.049410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987448+0000","last_change":"2024-05-10T03:33:34.413210+0000","last_active":"2024-05-10T03:33:35.987448+0000","last_peered":"2024-05-10T03:33:35.987448+0000","last_clean":"2024-05-10T03:33:35.987448+0000","last_became_active":"2024-05-10T03:33:34.412896+0000","last_became_peered":"2024-05-10T03:33:34.412896+0000","last_unstale":"2024-05-10T03:33:35.987448+0000","last_undegraded":"2024-05-10T03:33:35.987448+0000","last_fullsized":"2024-05-10T03:33:35.987448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:53:30.250093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.013697+0000","last_change":"2024-05-10T03:33:35.008122+0000","last_active":"2024-05-10T03:33:35.013697+0000","last_peered":"2024-05-10T03:33:35.013697+0000","last_clean":"2024-05-10T03:33:35.013697+0000","last_became_active":"2024-05-10T03:33:35.007819+0000","last_became_peered":"2024-05-10T03:33:35.007819+0000","last_unstale":"2024-05-10T03:33:35.013697+0000","last_undegraded":"2024-05-10T03:33:35.013697+0000","last_fullsized":"2024-05-10T03:33:35.013697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:03:01.336252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986808+0000","last_change":"2024-05-10T03:33:34.422356+0000","last_active":"2024-05-10T03:33:35.986808+0000","last_peered":"2024-05-10T03:33:35.986808+0000","last_clean":"2024-05-10T03:33:35.986808+0000","last_became_active":"2024-05-10T03:33:34.421897+0000","last_became_peered":"2024-05-10T03:33:34.421897+0000","last_unstale":"2024-05-10T03:33:35.986808+0000","last_undegraded":"2024-05-10T03:33:35.986808+0000","last_fullsized":"2024-05-10T03:33:35.986808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:13:46.332026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031662+0000","last_change":"2024-05-10T03:33:34.980819+0000","last_active":"2024-05-10T03:33:37.031662+0000","last_peered":"2024-05-10T03:33:37.031662+0000","last_clean":"2024-05-10T03:33:37.031662+0000","last_became_active":"2024-05-10T03:33:34.980502+0000","last_became_peered":"2024-05-10T03:33:34.980502+0000","last_unstale":"2024-05-10T03:33:37.031662+0000","last_undegraded":"2024-05-10T03:33:37.031662+0000","last_fullsized":"2024-05-10T03:33:37.031662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:43.321219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031484+0000","last_change":"2024-05-10T03:33:34.412686+0000","last_active":"2024-05-10T03:33:37.031484+0000","last_peered":"2024-05-10T03:33:37.031484+0000","last_clean":"2024-05-10T03:33:37.031484+0000","last_became_active":"2024-05-10T03:33:34.412339+0000","last_became_peered":"2024-05-10T03:33:34.412339+0000","last_unstale":"2024-05-10T03:33:37.031484+0000","last_undegraded":"2024-05-10T03:33:37.031484+0000","last_fullsized":"2024-05-10T03:33:37.031484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:07:50.473172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031689+0000","last_change":"2024-05-10T03:33:34.988856+0000","last_active":"2024-05-10T03:33:37.031689+0000","last_peered":"2024-05-10T03:33:37.031689+0000","last_clean":"2024-05-10T03:33:37.031689+0000","last_became_active":"2024-05-10T03:33:34.982820+0000","last_became_peered":"2024-05-10T03:33:34.982820+0000","last_unstale":"2024-05-10T03:33:37.031689+0000","last_undegraded":"2024-05-10T03:33:37.031689+0000","last_fullsized":"2024-05-10T03:33:37.031689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:45.694257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.984052+0000","last_change":"2024-05-10T03:33:34.418631+0000","last_active":"2024-05-10T03:33:35.984052+0000","last_peered":"2024-05-10T03:33:35.984052+0000","last_clean":"2024-05-10T03:33:35.984052+0000","last_became_active":"2024-05-10T03:33:34.418297+0000","last_became_peered":"2024-05-10T03:33:34.418297+0000","last_unstale":"2024-05-10T03:33:35.984052+0000","last_undegraded":"2024-05-10T03:33:35.984052+0000","last_fullsized":"2024-05-10T03:33:35.984052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:34:02.710467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757417+0000","last_change":"2024-05-10T03:33:35.011267+0000","last_active":"2024-05-10T03:33:36.757417+0000","last_peered":"2024-05-10T03:33:36.757417+0000","last_clean":"2024-05-10T03:33:36.757417+0000","last_became_active":"2024-05-10T03:33:35.010630+0000","last_became_peered":"2024-05-10T03:33:35.010630+0000","last_unstale":"2024-05-10T03:33:36.757417+0000","last_undegraded":"2024-05-10T03:33:36.757417+0000","last_fullsized":"2024-05-10T03:33:36.757417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:11:50.378468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:37.031329+0000","last_change":"2024-05-10T03:33:34.415240+0000","last_active":"2024-05-10T03:33:37.031329+0000","last_peered":"2024-05-10T03:33:37.031329+0000","last_clean":"2024-05-10T03:33:37.031329+0000","last_became_active":"2024-05-10T03:33:34.414049+0000","last_became_peered":"2024-05-10T03:33:34.414049+0000","last_unstale":"2024-05-10T03:33:37.031329+0000","last_undegraded":"2024-05-10T03:33:37.031329+0000","last_fullsized":"2024-05-10T03:33:37.031329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:20.855730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.757320+0000","last_change":"2024-05-10T03:33:34.992848+0000","last_active":"2024-05-10T03:33:36.757320+0000","last_peered":"2024-05-10T03:33:36.757320+0000","last_clean":"2024-05-10T03:33:36.757320+0000","last_became_active":"2024-05-10T03:33:34.992559+0000","last_became_peered":"2024-05-10T03:33:34.992559+0000","last_unstale":"2024-05-10T03:33:36.757320+0000","last_undegraded":"2024-05-10T03:33:36.757320+0000","last_fullsized":"2024-05-10T03:33:36.757320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:37:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:36.756397+0000","last_change":"2024-05-10T03:33:34.417467+0000","last_active":"2024-05-10T03:33:36.756397+0000","last_peered":"2024-05-10T03:33:36.756397+0000","last_clean":"2024-05-10T03:33:36.756397+0000","last_became_active":"2024-05-10T03:33:34.416166+0000","last_became_peered":"2024-05-10T03:33:34.416166+0000","last_unstale":"2024-05-10T03:33:36.756397+0000","last_undegraded":"2024-05-10T03:33:36.756397+0000","last_fullsized":"2024-05-10T03:33:36.756397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:07:20.663035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.987301+0000","last_change":"2024-05-10T03:33:35.002956+0000","last_active":"2024-05-10T03:33:35.987301+0000","last_peered":"2024-05-10T03:33:35.987301+0000","last_clean":"2024-05-10T03:33:35.987301+0000","last_became_active":"2024-05-10T03:33:35.002577+0000","last_became_peered":"2024-05-10T03:33:35.002577+0000","last_unstale":"2024-05-10T03:33:35.987301+0000","last_undegraded":"2024-05-10T03:33:35.987301+0000","last_fullsized":"2024-05-10T03:33:35.987301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:34.372468+0000","last_clean_scrub_stamp":"2024-05-10T03:33:34.372468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:39:20.689486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-10T03:33:34.971576+0000","last_change":"2024-05-10T03:33:34.422502+0000","last_active":"2024-05-10T03:33:34.971576+0000","last_peered":"2024-05-10T03:33:34.971576+0000","last_clean":"2024-05-10T03:33:34.971576+0000","last_became_active":"2024-05-10T03:33:34.422198+0000","last_became_peered":"2024-05-10T03:33:34.422198+0000","last_unstale":"2024-05-10T03:33:34.971576+0000","last_undegraded":"2024-05-10T03:33:34.971576+0000","last_fullsized":"2024-05-10T03:33:34.971576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:33:33.353572+0000","last_clean_scrub_stamp":"2024-05-10T03:33:33.353572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:03:00.317356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":213,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-10T03:33:35.986872+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:33:35.986872+0000","last_peered":"2024-05-10T03:33:35.986872+0000","last_clean":"2024-05-10T03:33:35.986872+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:33:35.986872+0000","last_undegraded":"2024-05-10T03:33:35.986872+0000","last_fullsized":"2024-05-10T03:33:35.986872+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1552384,"data_stored":1019371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.153},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7490000000000001,"5min":1.877,"15min":1.877},"last":1.2150000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1439999999999999,"15min":1.1419999999999999},"min":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":1.853,"15min":1.853},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2450000000000001,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.9039999999999999,"5min":1.925,"15min":1.925},"last":1.518}]},{"osd":2,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.363,"15min":1.383},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.77,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.347},{"interface":"front","average":{"1min":1.196,"5min":1.3899999999999999,"15min":1.4319999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8460000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.4590000000000001}]},{"osd":3,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3720000000000001,"15min":1.3859999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":2.222,"15min":2.222},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.353,"15min":1.3340000000000001},"min":{"1min":0.71899999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9550000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.101}]},{"osd":4,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.446,"15min":1.4690000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9790000000000001,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5189999999999999,"15min":1.5069999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2,"5min":2.323,"15min":2.323},"last":1.6990000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.528,"15min":1.51},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.619,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.3},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6879999999999999,"15min":1.696},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1709999999999998,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.762}]},{"osd":6,"last update":"Fri May 10 03:32:40 2024","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.7130000000000001,"15min":1.7250000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.173,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.635},{"interface":"front","average":{"1min":1.631,"5min":1.6519999999999999,"15min":1.6399999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.097,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.5760000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":423925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2849999999999999,"15min":1.337},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7689999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.4550000000000001}]},{"osd":1,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.282,"15min":1.3080000000000001},"min":{"1min":0.84199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.556,"5min":3.113,"15min":3.113},"last":1.569},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3720000000000001,"15min":1.4630000000000001},"min":{"1min":0.74099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0350000000000001,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":2.0350000000000001}]},{"osd":2,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.319,"15min":1.2569999999999999},"min":{"1min":0.80400000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7909999999999999,"5min":2.02,"15min":2.02},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.52,"5min":1.4259999999999999,"15min":1.502},"min":{"1min":0.98299999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.202,"5min":3.569,"15min":3.569},"last":1.8220000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1230000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":2.1230000000000002},{"interface":"front","average":{"1min":1.538,"5min":1.367,"15min":1.3240000000000001},"min":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.6910000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.92000000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.766,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.036},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.552,"15min":1.581},"min":{"1min":0.89100000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0579999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.964}]},{"osd":5,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.4590000000000001,"15min":1.4099999999999999},"min":{"1min":1.0089999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.617,"15min":1.621},"min":{"1min":1.1759999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8799999999999999,"5min":2.472,"15min":2.472},"last":1.288}]},{"osd":7,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.1919999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4590000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.841,"5min":1.663,"15min":1.659},"min":{"1min":1.05,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.7569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":946176,"data_stored":422395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0920000000000001,"15min":1.071},"min":{"1min":0.72999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6950000000000001,"5min":1.851,"15min":1.851},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.145,"15min":1.141},"min":{"1min":0.72899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7190000000000001,"5min":1.855,"15min":1.855},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.702,"5min":1.954,"15min":1.954},"last":1.476}]},{"osd":2,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.232,"15min":1.222},"min":{"1min":0.749,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.669,"5min":2.359,"15min":2.359},"last":1.669},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.331},"min":{"1min":1.0069999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.353}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.361,"15min":1.357},"min":{"1min":0.98799999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7210000000000001,"5min":1.881,"15min":1.881},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.357,"15min":1.3049999999999999},"min":{"1min":1.0489999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.909,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.5309999999999999}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9470000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.5920000000000001,"15min":1.603},"min":{"1min":1.246,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1600000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.9990000000000001}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.534,"15min":1.518},"min":{"1min":1.3480000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0190000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.796},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5600000000000001,"15min":1.5549999999999999},"min":{"1min":1.129,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.7829999999999999,"15min":1.8069999999999999},"min":{"1min":1.2330000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.3969999999999998,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":2.125},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.6459999999999999,"15min":1.6220000000000001},"min":{"1min":0.96599999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":2.1680000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":974848,"data_stored":430649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.159,"15min":1.173},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.6919999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1259999999999999,"15min":1.1359999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.591,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1779999999999999,"15min":1.1599999999999999},"min":{"1min":0.74399999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.613,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.139,"5min":1.248,"15min":1.2729999999999999},"min":{"1min":0.84399999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.72,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.79700000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5289999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.236,"5min":1.369,"15min":1.397},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.726,"5min":2.29,"15min":2.29},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.87,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.379,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.464,"5min":1.5649999999999999,"15min":1.581},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.782,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.9079999999999999,"5min":2.512,"15min":2.512},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.504,"15min":1.552},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5510000000000002,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.5569999999999999,"15min":1.583},"min":{"1min":0.65500000000000003,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4769999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.724,"15min":1.7769999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.093,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":0.879}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.6519999999999999,"15min":1.6899999999999999},"min":{"1min":0.63700000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.6160000000000001,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.623,"5min":1.736,"15min":1.7430000000000001},"min":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6349999999999998,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":946176,"data_stored":422943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95399999999999996,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93700000000000006,"15min":0.96999999999999997},"min":{"1min":0.54500000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.575,"5min":1.708,"15min":1.708},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.393,"5min":1.603,"15min":1.603},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.202,"5min":1.228,"15min":1.25},"min":{"1min":0.623,"5min":0.377,"15min":0.377},"max":{"1min":1.5329999999999999,"5min":1.964,"15min":1.964},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7450000000000001,"5min":1.99,"15min":1.99},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.238,"5min":1.3140000000000001,"15min":1.3600000000000001},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.679,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3620000000000001,"15min":1.3740000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.9019999999999999,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3999999999999999,"15min":1.3819999999999999},"min":{"1min":0.90200000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.948,"5min":2.464,"15min":2.464},"last":1.135}]},{"osd":5,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.613,"15min":1.6739999999999999},"min":{"1min":1.012,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2709999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":0.84799999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0449999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.5640000000000001,"15min":1.5760000000000001},"min":{"1min":1.123,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.113,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.008},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":1.7949999999999999},"min":{"1min":0.88800000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4470000000000001,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.0529999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5449999999999999,"15min":1.5840000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1139999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6359999999999999,"15min":1.657},"min":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":2.0419999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.0680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":966656,"data_stored":429723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93400000000000005,"15min":0.98299999999999998},"min":{"1min":0.504,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.389,"5min":1.772,"15min":1.772},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.97899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.635,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.98199999999999998}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.56399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.425,"5min":1.746,"15min":1.746},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6879999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.099}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.085,"15min":1.117},"min":{"1min":0.63600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.526,"5min":2.169,"15min":2.169},"last":1.26},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.21,"15min":1.25},"min":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8520000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.3200000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.5269999999999999},"min":{"1min":0.92600000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.024,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.444},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8600000000000001,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.619}]},{"osd":5,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.464,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.173,"5min":2.508,"15min":2.508},"last":1.786},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5900000000000001,"15min":1.617},"min":{"1min":0.97999999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.0859999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":2.0049999999999999}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.512,"15min":1.5449999999999999},"min":{"1min":0.90700000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1339999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5489999999999999,"15min":1.587},"min":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.331,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.6970000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.607,"15min":1.649},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2749999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.679},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.619,"15min":1.665},"min":{"1min":0.95599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.266,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.907}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":950272,"data_stored":422473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.90700000000000003,"15min":0.92100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5089999999999999,"5min":1.607,"15min":1.607},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0469999999999999,"15min":1.085},"min":{"1min":0.52100000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.7030000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.98099999999999998,"15min":1.004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.26},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.1850000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.986,"5min":2.444,"15min":2.444},"last":1.298}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.319},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.879,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2729999999999999,"15min":1.3009999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9399999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.772}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4830000000000001,"15min":1.5169999999999999},"min":{"1min":0.98499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3159999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":2.2650000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.5329999999999999,"15min":1.585},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.101,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.879}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.5209999999999999,"15min":1.585},"min":{"1min":0.89800000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.2450000000000001,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":2.0539999999999998},{"interface":"front","average":{"1min":1.425,"5min":1.573,"15min":1.631},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0190000000000001,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.6459999999999999,"15min":1.7190000000000001},"min":{"1min":1.004,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4470000000000001,"5min":3.48,"15min":3.48},"last":2.133},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.6599999999999999,"15min":1.73},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.004,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.73}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.6839999999999999,"15min":1.7330000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.1859999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.825},{"interface":"front","average":{"1min":1.603,"5min":1.72,"15min":1.782},"min":{"1min":1.113,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.5449999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":2.4140000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1536000,"data_stored":1012729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.95699999999999996,"15min":1.022},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4370000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.107},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.089,"15min":1.1870000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2589999999999999,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.997}]},{"osd":2,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.1499999999999999,"15min":1.22},"min":{"1min":0.53800000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.593,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.149,"15min":1.2170000000000001},"min":{"1min":0.56299999999999994,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5680000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.056,"15min":1.103},"min":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6160000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.278,"15min":1.3240000000000001},"min":{"1min":0.69799999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.78,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.522}]},{"osd":4,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5409999999999999,"15min":1.633},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.958,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.23},{"interface":"front","average":{"1min":1.302,"5min":1.401,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.768,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.5780000000000001}]},{"osd":5,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.4239999999999999,"15min":1.4730000000000001},"min":{"1min":0.80700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7110000000000001,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.371},{"interface":"front","average":{"1min":1.373,"5min":1.5489999999999999,"15min":1.627},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.873,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.726}]},{"osd":6,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.593,"15min":1.6419999999999999},"min":{"1min":0.98099999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.9410000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.625},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.647,"15min":1.74},"min":{"1min":1.069,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.831,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.5489999999999999}]},{"osd":7,"last update":"Fri May 10 03:32:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.6879999999999999,"15min":1.792},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2.0840000000000001,"5min":3.149,"15min":3.149},"last":1.661},{"interface":"front","average":{"1min":1.472,"5min":1.5369999999999999,"15min":1.573},"min":{"1min":0.90200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9019999999999999,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.6910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:33:40.720 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:33:40.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:33:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.047+0000 7f4801200640 1 Processor -- start 2024-05-10T03:33:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.047+0000 7f4801200640 1 -- start start 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f4801200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47fc166270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f4801200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0723b0 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f4801200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0cf030 con 0x7f47fc071c10 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f4801200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1600a0 con 0x7f47fc15c550 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f47faa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f47fc0cf4c0 0x7f47fc06ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37502/0 (socket says 172.21.15.55:37502) 2024-05-10T03:33:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.050+0000 7f47faa00640 1 -- 172.21.15.55:0/694979793 learned_addr learned my addr 172.21.15.55:0/694979793 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47fc166270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473989577 0 0) 0x7f47fc0723b0 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0003520 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172154096 0 0) 0x7f47e0003520 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 msgr2=0x7f47fc166270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47fc166270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 >> v1:172.21.15.162:6789/0 conn(0x7f47fc071c10 legacy=0x7f47fc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc15c960 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f47e4001670 con 0x7f47fc0cf4c0 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47fc166270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 >> v1:172.21.15.55:6789/0 conn(0x7f47fc0cf4c0 legacy=0x7f47fc06ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e00039a0 0x7f47e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47e0004d70 0x7f47e0005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1600a0 con 0x7f47e0004d70 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000bc50 con 0x7f47e00039a0 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.051+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000be10 con 0x7f47fc15c550 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47e0004d70 0x7f47e0005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47faa00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fb400640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e00039a0 0x7f47e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 -- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 msgr2=0x7f47e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47fc15c550 0x7f47e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 -- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e00039a0 msgr2=0x7f47e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e00039a0 0x7f47e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.052+0000 7f47fbe00640 1 -- 172.21.15.55:0/694979793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0003520 con 0x7f47e0004d70 2024-05-10T03:33:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.053+0000 7f47fbe00640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47e0004d70 0x7f47e0005240 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374b4fe385a909e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f402ac50 con 0x7f47e0004d70 2024-05-10T03:33:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f402adb0 con 0x7f47e0004d70 2024-05-10T03:33:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f47fa000640 1 -- 172.21.15.55:0/694979793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4004250 con 0x7f47e0004d70 2024-05-10T03:33:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f4801200640 1 -- 172.21.15.55:0/694979793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47e0004d70 msgr2=0x7f47e0005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f4801200640 1 --2- 172.21.15.55:0/694979793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47e0004d70 0x7f47e0005240 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006fd0 tx=0x7f47e00057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f47fbe00640 1 -- 172.21.15.55:0/694979793 reap_dead start 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f4801200640 1 -- 172.21.15.55:0/694979793 shutdown_connections 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f4801200640 1 -- 172.21.15.55:0/694979793 >> 172.21.15.55:0/694979793 conn(0x7f47fc0710f0 msgr2=0x7f47fc071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.054+0000 7f4801200640 1 -- 172.21.15.55:0/694979793 shutdown_connections 2024-05-10T03:33:41.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.055+0000 7f4801200640 1 -- 172.21.15.55:0/694979793 wait complete. 2024-05-10T03:33:41.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.057+0000 7f4801200640 1 Processor -- start 2024-05-10T03:33:41.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.057+0000 7f4801200640 1 -- start start 2024-05-10T03:33:41.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 0x7f47fc15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 0x7f47fc15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc15fb50 con 0x7f47fc142820 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1440c0 con 0x7f47fc15caf0 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f4801200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc15cf80 con 0x7f47e0004d70 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53974/0 (socket says 172.21.15.55:53974) 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 -- 172.21.15.55:0/1879414487 learned_addr learned my addr 172.21.15.55:0/1879414487 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47faa00640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 0x7f47fc15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fb400640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 0x7f47fc15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 -- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 msgr2=0x7f47fc15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 0x7f47fc15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 -- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 msgr2=0x7f47fc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 0x7f47fc15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fbe00640 1 -- 172.21.15.55:0/1879414487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc15d1d0 con 0x7f47fc142820 2024-05-10T03:33:41.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47fb400640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 0x7f47fc15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:41.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.059+0000 7f47faa00640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 0x7f47fc15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:41.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f47fbe00640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0cfee0 tx=0x7f47f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003820 con 0x7f47fc142820 2024-05-10T03:33:41.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f40039a0 con 0x7f47fc142820 2024-05-10T03:33:41.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc15d4b0 con 0x7f47fc142820 2024-05-10T03:33:41.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003d50 con 0x7f47fc142820 2024-05-10T03:33:41.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.060+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc15da70 con 0x7f47fc142820 2024-05-10T03:33:41.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.061+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc067930 con 0x7f47fc142820 2024-05-10T03:33:41.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.062+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f47f403d430 con 0x7f47fc142820 2024-05-10T03:33:41.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.063+0000 7f47f3400640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 0x7f47d806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.064+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f47f40a9960 con 0x7f47fc142820 2024-05-10T03:33:41.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.064+0000 7f47fb400640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 0x7f47d806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.065+0000 7f47fb400640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 0x7f47d806d140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.068+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f47f40748d0 con 0x7f47fc142820 2024-05-10T03:33:41.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.337+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f47fc0cdac0 con 0x7f47fc142820 2024-05-10T03:33:41.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.346+0000 7f47f3400640 1 -- 172.21.15.55:0/1879414487 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f47f4077ef0 con 0x7f47fc142820 2024-05-10T03:33:41.351 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:33:41.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 msgr2=0x7f47d806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 0x7f47d806d140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009040 comp rx=0 tx=0).stop 2024-05-10T03:33:41.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 msgr2=0x7f47fc143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0cfee0 tx=0x7f47f4004730 comp rx=0 tx=0).stop 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 shutdown_connections 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47fc142820 0x7f47fc143300 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f47d806ac30 0x7f47d806d140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47fc15caf0 0x7f47fc15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 --2- 172.21.15.55:0/1879414487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e0004d70 0x7f47fc15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 >> 172.21.15.55:0/1879414487 conn(0x7f47fc0710f0 msgr2=0x7f47fc0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 shutdown_connections 2024-05-10T03:33:41.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.350+0000 7f4801200640 1 -- 172.21.15.55:0/1879414487 wait complete. 2024-05-10T03:33:41.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:41.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.658+0000 7eff3c000640 1 Processor -- start 2024-05-10T03:33:41.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.658+0000 7eff3c000640 1 -- start start 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3c000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff340ce200 0x7eff340ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3c000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff34165b80 0x7eff34165f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3c000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340b3ca0 con 0x7eff34161f90 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3c000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff341664d0 con 0x7eff340ce200 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3c000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b3090 con 0x7eff34165b80 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff38c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff340ce200 0x7eff340ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff38c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff340ce200 0x7eff340ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53980/0 (socket says 172.21.15.55:53980) 2024-05-10T03:33:41.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff38c00640 1 -- 172.21.15.55:0/2496137633 learned_addr learned my addr 172.21.15.55:0/2496137633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.661+0000 7eff3a000640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff34165b80 0x7eff34165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.662+0000 7eff38c00640 1 -- 172.21.15.55:0/2496137633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff34165b80 msgr2=0x7eff34165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.662+0000 7eff38c00640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff34165b80 0x7eff34165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.662+0000 7eff38c00640 1 -- 172.21.15.55:0/2496137633 >> v1:172.21.15.162:6789/0 conn(0x7eff34161f90 legacy=0x7eff340b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.662+0000 7eff38c00640 1 -- 172.21.15.55:0/2496137633 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340b3370 con 0x7eff340ce200 2024-05-10T03:33:41.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.663+0000 7eff38c00640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff340ce200 0x7eff340ce610 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7eff20009850 tx=0x7eff2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56701fc85cae28b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.664+0000 7eff33e00640 1 -- 172.21.15.55:0/2496137633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20004030 con 0x7eff340ce200 2024-05-10T03:33:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.664+0000 7eff33e00640 1 -- 172.21.15.55:0/2496137633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20004190 con 0x7eff340ce200 2024-05-10T03:33:41.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.664+0000 7eff33e00640 1 -- 172.21.15.55:0/2496137633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20003610 con 0x7eff340ce200 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 -- 172.21.15.55:0/2496137633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff340ce200 msgr2=0x7eff340ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff340ce200 0x7eff340ce610 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7eff20009850 tx=0x7eff2002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 -- 172.21.15.55:0/2496137633 shutdown_connections 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff34165b80 0x7eff34165f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 --2- 172.21.15.55:0/2496137633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff340ce200 0x7eff340ce610 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 -- 172.21.15.55:0/2496137633 >> 172.21.15.55:0/2496137633 conn(0x7eff340d0720 msgr2=0x7eff340d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 -- 172.21.15.55:0/2496137633 shutdown_connections 2024-05-10T03:33:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.665+0000 7eff3c000640 1 -- 172.21.15.55:0/2496137633 wait complete. 2024-05-10T03:33:41.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.668+0000 7eff3c000640 1 Processor -- start 2024-05-10T03:33:41.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.668+0000 7eff3c000640 1 -- start start 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff340ce200 0x7eff34155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 0x7eff340caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340c8970 con 0x7eff34161f90 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340ca5e0 con 0x7eff34165b80 2024-05-10T03:33:41.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.669+0000 7eff3c000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340cbd40 con 0x7eff340ce200 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff3a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 0x7eff340caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53984/0 (socket says 172.21.15.55:53984) 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 -- 172.21.15.55:0/197638872 learned_addr learned my addr 172.21.15.55:0/197638872 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff39600640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff340ce200 0x7eff34155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 -- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff340ce200 msgr2=0x7eff34155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff340ce200 0x7eff34155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 -- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 msgr2=0x7eff340caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 0x7eff340caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff38c00640 1 -- 172.21.15.55:0/197638872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009460 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.670+0000 7eff3a000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 0x7eff340caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff38c00640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7eff20000c00 tx=0x7eff200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20004030 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff20003c50 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff341615a0 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20006040 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.671+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff340ccf40 con 0x7eff34161f90 2024-05-10T03:33:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.673+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34067930 con 0x7eff34161f90 2024-05-10T03:33:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.675+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7eff200061a0 con 0x7eff34161f90 2024-05-10T03:33:41.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.675+0000 7eff31600640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 0x7eff1406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:41.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.675+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7eff200aa530 con 0x7eff34161f90 2024-05-10T03:33:41.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.676+0000 7eff39600640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 0x7eff1406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.677+0000 7eff39600640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 0x7eff1406d110 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff24000f60 tx=0x7eff24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.680+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7eff200756e0 con 0x7eff34161f90 2024-05-10T03:33:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.916+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff341623a0 con 0x7eff34161f90 2024-05-10T03:33:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.917+0000 7eff31600640 1 -- 172.21.15.55:0/197638872 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7eff20078d00 con 0x7eff34161f90 2024-05-10T03:33:41.923 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:41.923 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 msgr2=0x7eff1406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 0x7eff1406d110 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff24000f60 tx=0x7eff24009040 comp rx=0 tx=0).stop 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 msgr2=0x7eff340caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7eff20000c00 tx=0x7eff200046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 shutdown_connections 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff1406ac00 0x7eff1406d110 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff34165b80 0x7eff340caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff34161f90 0x7eff340caa00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 --2- 172.21.15.55:0/197638872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff340ce200 0x7eff34155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 >> 172.21.15.55:0/197638872 conn(0x7eff340d0720 msgr2=0x7eff3415c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 shutdown_connections 2024-05-10T03:33:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:41.920+0000 7eff3c000640 1 -- 172.21.15.55:0/197638872 wait complete. 2024-05-10T03:33:42.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:33:34.958997+0000', 'modified': '2024-05-10T03:33:38.424470+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'd', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6837/3533670538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6835', 'nonce': 3533670538}, {'type': 'v1', 'addr': '172.21.15.162:6837', 'nonce': 3533670538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:33:42.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:33:42.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-10T03:33:42.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.649+0000 7f0867e00640 1 Processor -- start 2024-05-10T03:33:42.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.649+0000 7f0867e00640 1 -- start start 2024-05-10T03:33:42.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.652+0000 7f0867e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f0860166600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.652+0000 7f0867e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086006ad20 con 0x7f086015fef0 2024-05-10T03:33:42.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.652+0000 7f0867e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086015f410 con 0x7f08600b5c80 2024-05-10T03:33:42.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.652+0000 7f0867e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086006b000 con 0x7f086006a820 2024-05-10T03:33:42.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f0865400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f086015fef0 0x7f08600b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37504/0 (socket says 172.21.15.55:37504) 2024-05-10T03:33:42.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f0865400640 1 -- 172.21.15.55:0/1360837917 learned_addr learned my addr 172.21.15.55:0/1360837917 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:42.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f0865e00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f0860166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436848046 0 0) 0x7f086006ad20 con 0x7f086015fef0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0848003520 con 0x7f086015fef0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3631521480 0 0) 0x7f0848003520 con 0x7f086015fef0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.653+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 msgr2=0x7f0860166600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f0860166600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 >> v1:172.21.15.162:6789/0 conn(0x7f08600b5c80 legacy=0x7f086006a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086015f6f0 con 0x7f086015fef0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0854001690 con 0x7f086015fef0 2024-05-10T03:33:42.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 >> v1:172.21.15.55:6789/0 conn(0x7f086015fef0 legacy=0x7f08600b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f08480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0848003f20 0x7f08480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0848004d00 0x7f0848005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086006b000 con 0x7f0848003f20 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084800bba0 con 0x7f0848004d00 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084800bd60 con 0x7f086006a820 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f0864a00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0848003f20 0x7f08480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f0865400640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f08480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.654+0000 7f0865e00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0848004d00 0x7f0848005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0864a00640 1 -- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 msgr2=0x7f08480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0864a00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086006a820 0x7f08480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0864a00640 1 -- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0848004d00 msgr2=0x7f0848005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0864a00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0848004d00 0x7f0848005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0864a00640 1 -- 172.21.15.55:0/1360837917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0848003520 con 0x7f0848003f20 2024-05-10T03:33:42.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.655+0000 7f0865e00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0848004d00 0x7f0848005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.656+0000 7f0864a00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0848003f20 0x7f08480047c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0850001750 tx=0x7f08480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab2a0381fa09d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.656+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500052d0 con 0x7f0848003f20 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.656+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850005430 con 0x7f0848003f20 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.656+0000 7f085fe00640 1 -- 172.21.15.55:0/1360837917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085002abe0 con 0x7f0848003f20 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 -- 172.21.15.55:0/1360837917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0848003f20 msgr2=0x7f08480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 --2- 172.21.15.55:0/1360837917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0848003f20 0x7f08480047c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0850001750 tx=0x7f08480057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0865e00640 1 -- 172.21.15.55:0/1360837917 reap_dead start 2024-05-10T03:33:42.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 -- 172.21.15.55:0/1360837917 shutdown_connections 2024-05-10T03:33:42.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 -- 172.21.15.55:0/1360837917 >> 172.21.15.55:0/1360837917 conn(0x7f08600cd2f0 msgr2=0x7f08600cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 -- 172.21.15.55:0/1360837917 shutdown_connections 2024-05-10T03:33:42.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.657+0000 7f0867e00640 1 -- 172.21.15.55:0/1360837917 wait complete. 2024-05-10T03:33:42.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.659+0000 7f0867e00640 1 Processor -- start 2024-05-10T03:33:42.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.660+0000 7f0867e00640 1 -- start start 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0848003f20 0x7f08600cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0860157a30 0x7f08601423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600b4da0 con 0x7f0860157e40 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08601583c0 con 0x7f0860157a30 2024-05-10T03:33:42.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0867e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08600b32f0 con 0x7f0848003f20 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.661+0000 7f0865e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54008/0 (socket says 172.21.15.55:54008) 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 -- 172.21.15.55:0/946536992 learned_addr learned my addr 172.21.15.55:0/946536992 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0864a00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0860157a30 0x7f08601423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865400640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0848003f20 0x7f08600cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 -- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0848003f20 msgr2=0x7f08600cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0848003f20 0x7f08600cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 -- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0860157a30 msgr2=0x7f08601423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0860157a30 0x7f08601423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 -- 172.21.15.55:0/946536992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860140f20 con 0x7f0860157e40 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.662+0000 7f0865e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0860159870 tx=0x7f085800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.663+0000 7f085d600640 1 -- 172.21.15.55:0/946536992 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858017400 con 0x7f0860157e40 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.663+0000 7f085d600640 1 -- 172.21.15.55:0/946536992 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858011040 con 0x7f0860157e40 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.663+0000 7f085d600640 1 -- 172.21.15.55:0/946536992 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858017780 con 0x7f0860157e40 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.663+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08600cb9c0 con 0x7f0860157e40 2024-05-10T03:33:42.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.663+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08601588f0 con 0x7f0860157e40 2024-05-10T03:33:42.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.666+0000 7f085d600640 1 -- 172.21.15.55:0/946536992 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08580203b0 con 0x7f0860157e40 2024-05-10T03:33:42.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.667+0000 7f085d600640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 0x7f084406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.667+0000 7f085d600640 1 -- 172.21.15.55:0/946536992 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0858089880 con 0x7f0860157e40 2024-05-10T03:33:42.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.668+0000 7f0865400640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 0x7f084406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.668+0000 7f086ee0e740 1 Processor -- start 2024-05-10T03:33:42.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.668+0000 7f086ee0e740 1 -- start start 2024-05-10T03:33:42.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.669+0000 7f0865400640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 0x7f084406d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0854001eb0 tx=0x7f0854009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45d83b0 0x5598a49e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 0x5598a49e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a4909a70 con 0x5598a4601270 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a4907200 con 0x5598a45e0590 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a4902af0 con 0x5598a45d83b0 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54016/0 (socket says 172.21.15.55:54016) 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f0865400640 1 -- 172.21.15.55:0/3678732995 learned_addr learned my addr 172.21.15.55:0/3678732995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.670+0000 7f0864a00640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45d83b0 0x5598a49e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865e00640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 0x5598a49e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 -- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45d83b0 msgr2=0x5598a49e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45d83b0 0x5598a49e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 -- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 msgr2=0x5598a49e5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 0x5598a49e5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 -- 172.21.15.55:0/3678732995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598a48cdb90 con 0x5598a4601270 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865e00640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 0x5598a49e5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f0865400640 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f08540109f0 tx=0x7f085400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb2f2273ba59eae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f084dc00640 1 -- 172.21.15.55:0/3678732995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854014b80 con 0x5598a4601270 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f084dc00640 1 -- 172.21.15.55:0/3678732995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854014ce0 con 0x5598a4601270 2024-05-10T03:33:42.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.671+0000 7f084dc00640 1 -- 172.21.15.55:0/3678732995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854013560 con 0x5598a4601270 2024-05-10T03:33:42.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 -- 172.21.15.55:0/3678732995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 msgr2=0x5598a49cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f08540109f0 tx=0x7f085400c000 comp rx=0 tx=0).stop 2024-05-10T03:33:42.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 -- 172.21.15.55:0/3678732995 shutdown_connections 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a45e0590 0x5598a49e5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45d83b0 0x5598a49e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3678732995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a4601270 0x5598a49cab80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 -- 172.21.15.55:0/3678732995 >> 172.21.15.55:0/3678732995 conn(0x5598a45f53c0 msgr2=0x5598a49a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 -- 172.21.15.55:0/3678732995 shutdown_connections 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.672+0000 7f086ee0e740 1 -- 172.21.15.55:0/3678732995 wait complete. 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.673+0000 7f086ee0e740 1 -- start start 2024-05-10T03:33:42.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.674+0000 7f086ee0e740 1 Processor -- start 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a48a4220 con 0x5598a45d83b0 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a48a3ec0 con 0x5598a4601270 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a4669f20 con 0x5598a45e0590 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.675+0000 7f086ee0e740 10 client.0 _ll_register_callbacks cb 0x5598a48fc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54032/0 (socket says 172.21.15.55:54032) 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 -- 172.21.15.55:0/3580364854 learned_addr learned my addr 172.21.15.55:0/3580364854 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0864a00640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865e00640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 -- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 msgr2=0x5598a49cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 -- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 msgr2=0x5598a49c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865400640 1 -- 172.21.15.55:0/3580364854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085400f040 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.676+0000 7f0865e00640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.677+0000 7f0865400640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x5598a499c580 tx=0x7f08600ce560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f44410e372d3a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.677+0000 7f0842000640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.677+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854022c70 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.677+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598a492cc90 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.677+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f085401c050 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0854014e50 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 10 client.6663 fetch_fsmap learned FSMap version 39 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5598a4901170 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598a492cc90 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f085402c440 con 0x5598a45d83b0 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 10 client.6663 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 10 client.6663 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:33:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 20 client.6663 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a45bfe40 0x5598a49ddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 4 client.6663 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.678+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 --> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598a45f63c0 con 0x5598a45bfe40 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.679+0000 7f0864a00640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a45bfe40 0x5598a49ddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.680+0000 7f0864a00640 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a45bfe40 0x5598a49ddfd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2921c96c2dffaf3 server_cookie=1a480b3bf6934ff4 in_seq=0 out_seq=0 2024-05-10T03:33:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.680+0000 7f0842000640 10 client.6663 ms_handle_connect on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.682+0000 7f0842000640 1 -- 172.21.15.55:0/3580364854 <== mds.0 v2:172.21.15.162:6835/3533670538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598a45f63c0 con 0x5598a45bfe40 2024-05-10T03:33:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.682+0000 7f0842000640 10 client.6663 handle_command_reply: tid=0 2024-05-10T03:33:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.682+0000 7f086ee0e740 1 client.6663 shutdown 2024-05-10T03:33:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 msgr2=0x5598a49c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x5598a499c580 tx=0x7f08600ce560 comp rx=0 tx=0).stop 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 shutdown_connections 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a45bfe40 0x5598a49ddfd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3580364854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c3dc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f0842000640 0 client.6663 ms_handle_reset on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:42.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 >> 172.21.15.55:0/3580364854 conn(0x5598a45f53c0 msgr2=0x5598a49a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 shutdown_connections 2024-05-10T03:33:42.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 1 -- 172.21.15.55:0/3580364854 wait complete. 2024-05-10T03:33:42.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.683+0000 7f086ee0e740 20 client.6663 trim_cache size 0 max 16384 2024-05-10T03:33:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.704+0000 7f086ee0e740 1 Processor -- start 2024-05-10T03:33:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.704+0000 7f086ee0e740 1 -- start start 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a49e7060 con 0x5598a45d83b0 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a49dd3c0 con 0x5598a4601270 2024-05-10T03:33:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a49cb370 con 0x5598a45e0590 2024-05-10T03:33:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.706+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54034/0 (socket says 172.21.15.55:54034) 2024-05-10T03:33:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 -- 172.21.15.55:0/1655247243 learned_addr learned my addr 172.21.15.55:0/1655247243 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865e00640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0864a00640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 -- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 msgr2=0x5598a49cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 -- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 msgr2=0x5598a49e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 -- 172.21.15.55:0/1655247243 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598a4936130 con 0x5598a45d83b0 2024-05-10T03:33:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865e00640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:42.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.707+0000 7f0865400640 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f085402cca0 tx=0x7f085402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb64425c78f4583d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.708+0000 7f084e600640 1 -- 172.21.15.55:0/1655247243 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540147d0 con 0x5598a45d83b0 2024-05-10T03:33:42.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.708+0000 7f084e600640 1 -- 172.21.15.55:0/1655247243 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085402c950 con 0x5598a45d83b0 2024-05-10T03:33:42.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.708+0000 7f084e600640 1 -- 172.21.15.55:0/1655247243 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854022c70 con 0x5598a45d83b0 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 -- 172.21.15.55:0/1655247243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 msgr2=0x5598a49c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f085402cca0 tx=0x7f085402b000 comp rx=0 tx=0).stop 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 -- 172.21.15.55:0/1655247243 shutdown_connections 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 --2- 172.21.15.55:0/1655247243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49c75c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 -- 172.21.15.55:0/1655247243 >> 172.21.15.55:0/1655247243 conn(0x5598a45f53c0 msgr2=0x5598a499ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 -- 172.21.15.55:0/1655247243 shutdown_connections 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.709+0000 7f086ee0e740 1 -- 172.21.15.55:0/1655247243 wait complete. 2024-05-10T03:33:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.710+0000 7f086ee0e740 1 -- start start 2024-05-10T03:33:42.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.710+0000 7f086ee0e740 1 Processor -- start 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.712+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.712+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f086ee0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f086ee0e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a49c4f50 con 0x5598a45d83b0 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598a49c3130 con 0x5598a4601270 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f086ee0e740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598a49cd460 con 0x5598a45e0590 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f086ee0e740 10 client.0 _ll_register_callbacks cb 0x7f0854010eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54048/0 (socket says 172.21.15.55:54048) 2024-05-10T03:33:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 -- 172.21.15.55:0/3824627470 learned_addr learned my addr 172.21.15.55:0/3824627470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:42.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 -- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 msgr2=0x5598a49c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 -- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 msgr2=0x5598a49c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:42.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.713+0000 7f0865400640 1 -- 172.21.15.55:0/3824627470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598a4901170 con 0x5598a45d83b0 2024-05-10T03:33:42.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f0865400640 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0854024040 tx=0x7f0854007030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4432b576298522a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f0842000640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854022e90 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f085401c040 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0854014ce0 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08540049f0 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f086ee0e740 10 client.6669 fetch_fsmap learned FSMap version 39 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.714+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5598a48a3430 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854014ce0 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0854004b50 con 0x5598a45d83b0 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 10 client.6669 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 10 client.6669 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 20 client.6669 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:42.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a49cbf80 0x5598a49cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:42.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 4 client.6669 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:33:42.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.715+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 --> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598a46076c0 con 0x5598a49cbf80 2024-05-10T03:33:42.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.716+0000 7f0864a00640 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a49cbf80 0x5598a49cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.717+0000 7f0864a00640 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a49cbf80 0x5598a49cc390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165a6ad73acf8160 server_cookie=3ab96b31db7451fd in_seq=0 out_seq=0 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.717+0000 7f0842000640 10 client.6669 ms_handle_connect on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.717+0000 7f0842000640 1 -- 172.21.15.55:0/3824627470 <== mds.0 v2:172.21.15.162:6835/3533670538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5598a46076c0 con 0x5598a49cbf80 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.717+0000 7f0842000640 10 client.6669 handle_command_reply: tid=0 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.717+0000 7f086ee0e740 1 client.6669 shutdown 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 msgr2=0x5598a49a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0854024040 tx=0x7f0854007030 comp rx=0 tx=0).stop 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 shutdown_connections 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x5598a49cbf80 0x5598a49cc390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5598a4601270 0x5598a49c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5598a45e0590 0x5598a49c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 --2- 172.21.15.55:0/3824627470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5598a45d83b0 0x5598a49a01b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f0842000640 0 client.6669 ms_handle_reset on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 >> 172.21.15.55:0/3824627470 conn(0x5598a45f53c0 msgr2=0x5598a49c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 shutdown_connections 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 1 -- 172.21.15.55:0/3824627470 wait complete. 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.718+0000 7f086ee0e740 20 client.6669 trim_cache size 0 max 16384 2024-05-10T03:33:42.722 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6540, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.7524153619999998, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 39, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 78, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 78, 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.457846848000001 2024-05-10T03:33:42.723 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:33:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 msgr2=0x7f084406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 0x7f084406d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0854001eb0 tx=0x7f0854009040 comp rx=0 tx=0).stop 2024-05-10T03:33:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 msgr2=0x7f0860142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0860159870 tx=0x7f085800f4a0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 shutdown_connections 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0860157e40 0x7f0860142900 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f084406ac00 0x7f084406d110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0860157a30 0x7f08601423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 --2- 172.21.15.55:0/946536992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0848003f20 0x7f08600cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 >> 172.21.15.55:0/946536992 conn(0x7f08600cd2f0 msgr2=0x7f08600b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 shutdown_connections 2024-05-10T03:33:42.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:42.719+0000 7f0867e00640 1 -- 172.21.15.55:0/946536992 wait complete. 2024-05-10T03:33:42.748 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:33:43.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.026+0000 7f1a23e00640 1 Processor -- start 2024-05-10T03:33:43.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.027+0000 7f1a23e00640 1 -- start start 2024-05-10T03:33:43.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a2415e190 0x7f1a240ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1a240b1f60 0x7f1a240b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1a2406cbf0 0x7f1a2406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2415d740 con 0x7f1a240b1f60 2024-05-10T03:33:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2406d540 con 0x7f1a2406cbf0 2024-05-10T03:33:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a23e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b2a20 con 0x7f1a2415e190 2024-05-10T03:33:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a1a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1a240b1f60 0x7f1a240b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a1a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1a240b1f60 0x7f1a240b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57230/0 (socket says 172.21.15.55:57230) 2024-05-10T03:33:43.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.029+0000 7f1a1a000640 1 -- 172.21.15.55:0/111865818 learned_addr learned my addr 172.21.15.55:0/111865818 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a23400640 1 --2- 172.21.15.55:0/111865818 >> v2:172.21.15.162:3300/0 conn(0x7f1a2406cbf0 0x7f1a2406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a22a00640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a2415e190 0x7f1a240ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a1a000640 1 -- 172.21.15.55:0/111865818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a2415e190 msgr2=0x7f1a240ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a1a000640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a2415e190 0x7f1a240ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a1a000640 1 -- 172.21.15.55:0/111865818 >> v2:172.21.15.162:3300/0 conn(0x7f1a2406cbf0 msgr2=0x7f1a2406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:43.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a1a000640 1 --2- 172.21.15.55:0/111865818 >> v2:172.21.15.162:3300/0 conn(0x7f1a2406cbf0 0x7f1a2406d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.030+0000 7f1a1a000640 1 -- 172.21.15.55:0/111865818 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240b2d00 con 0x7f1a240b1f60 2024-05-10T03:33:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.031+0000 7f1a1a000640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a240b1f60 0x7f1a240b2370 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1a2406e350 tx=0x7f1a0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32165f9bed8db24 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.031+0000 7f1a22000640 1 -- 172.21.15.55:0/111865818 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c02ae90 con 0x7f1a240b1f60 2024-05-10T03:33:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.031+0000 7f1a22000640 1 -- 172.21.15.55:0/111865818 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c032440 con 0x7f1a240b1f60 2024-05-10T03:33:43.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.031+0000 7f1a22000640 1 -- 172.21.15.55:0/111865818 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0327d0 con 0x7f1a240b1f60 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 -- 172.21.15.55:0/111865818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a240b1f60 msgr2=0x7f1a240b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a240b1f60 0x7f1a240b2370 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1a2406e350 tx=0x7f1a0c02a140 comp rx=0 tx=0).stop 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 -- 172.21.15.55:0/111865818 shutdown_connections 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 --2- 172.21.15.55:0/111865818 >> v2:172.21.15.162:3300/0 conn(0x7f1a2406cbf0 0x7f1a2406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a240b1f60 0x7f1a240b2370 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 --2- 172.21.15.55:0/111865818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a2415e190 0x7f1a240ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 -- 172.21.15.55:0/111865818 >> 172.21.15.55:0/111865818 conn(0x7f1a2406a110 msgr2=0x7f1a2406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 -- 172.21.15.55:0/111865818 shutdown_connections 2024-05-10T03:33:43.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.032+0000 7f1a23e00640 1 -- 172.21.15.55:0/111865818 wait complete. 2024-05-10T03:33:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.035+0000 7f1a23e00640 1 Processor -- start 2024-05-10T03:33:43.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.035+0000 7f1a23e00640 1 -- start start 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a2406cbf0 0x7f1a2413f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a240b1f60 0x7f1a2413c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2413cac0 con 0x7f1a2415e190 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a241e3be0 con 0x7f1a2406cbf0 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24158c20 con 0x7f1a240b1f60 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57236/0 (socket says 172.21.15.55:57236) 2024-05-10T03:33:43.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 -- 172.21.15.55:0/1912846207 learned_addr learned my addr 172.21.15.55:0/1912846207 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a22a00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a2406cbf0 0x7f1a2413f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 -- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a240b1f60 msgr2=0x7f1a2413c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a240b1f60 0x7f1a2413c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 -- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a2406cbf0 msgr2=0x7f1a2413f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a2406cbf0 0x7f1a2413f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.037+0000 7f1a23400640 1 -- 172.21.15.55:0/1912846207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0094d0 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a23400640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00cbe0 tx=0x7f1a1c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00ced0 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c00ab70 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011410 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a241e9140 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.038+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a24158e70 con 0x7f1a2415e190 2024-05-10T03:33:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.039+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a24066ce0 con 0x7f1a2415e190 2024-05-10T03:33:43.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.041+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1a1c011570 con 0x7f1a2415e190 2024-05-10T03:33:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.042+0000 7f1a1b400640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 0x7f1a0006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.042+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a1c0893b0 con 0x7f1a2415e190 2024-05-10T03:33:43.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.043+0000 7f1a22a00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 0x7f1a0006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:43.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.044+0000 7f1a22a00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 0x7f1a0006d140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b7c0 tx=0x7f1a14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:43.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.047+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1a1c054320 con 0x7f1a2415e190 2024-05-10T03:33:43.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.284+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1a2404e290 con 0x7f1a2415e190 2024-05-10T03:33:43.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.285+0000 7f1a1b400640 1 -- 172.21.15.55:0/1912846207 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a1c01f060 con 0x7f1a2415e190 2024-05-10T03:33:43.288 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 msgr2=0x7f1a0006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 0x7f1a0006d140 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b7c0 tx=0x7f1a14008040 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 msgr2=0x7f1a2413ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00cbe0 tx=0x7f1a1c00a4c0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 shutdown_connections 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1a2415e190 0x7f1a2413ed10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a240b1f60 0x7f1a2413c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1a0006ac30 0x7f1a0006d140 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 --2- 172.21.15.55:0/1912846207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a2406cbf0 0x7f1a2413f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 >> 172.21.15.55:0/1912846207 conn(0x7f1a2406a110 msgr2=0x7f1a24151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:43.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.288+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 shutdown_connections 2024-05-10T03:33:43.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:43.289+0000 7f1a23e00640 1 -- 172.21.15.55:0/1912846207 wait complete. 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:33:43.321 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:43.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.322 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.372 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42228sec preferred_lft 42228sec 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:43.373 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:33:43.374 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:33:43.375 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:43.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.375 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:33:43.446 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:33:43.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.450 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:33:43.495 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:33:43.496 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:33:43.496 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:33:43.496 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:33:43.496 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:33:43.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.496 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:33:43.496 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:33:43.497 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:33:43.497 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:33:43.497 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:33:43.497 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:33:43.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.690 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:33:43.744 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:33:43.744 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:33:43.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:43.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.744 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:33:43.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:43.836 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:43.893 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:33:43.894 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:43.894 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:43.946 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:33:43.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:43.947 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:44.154 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:44.154 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:44.154 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:44.182 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:44.182 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:44.182 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:44.182 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:44.182 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:33:44.224 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:33:44.227 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:33:44.227 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:33:44.227 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:33:44.227 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:33:44.227 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:33:44.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:33:44.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:44.232 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:44.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:44.321 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:33:44.403 INFO:teuthology.orchestra.run.smithi055.stdout:client6678 2024-05-10T03:33:44.403 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:44.403 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:44.462 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:33:44.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.747+0000 7f0630a00640 1 Processor -- start 2024-05-10T03:33:44.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.747+0000 7f0630a00640 1 -- start start 2024-05-10T03:33:44.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.749+0000 7f0630a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c144120 0x7f062c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0630a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f062c0b3190 0x7f062c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0630a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c06d350 con 0x7f062c144710 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0630a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c144530 con 0x7f062c0b3190 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0630a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c06dd20 con 0x7f062c144120 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f062a800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f062c144710 0x7f062c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48722/0 (socket says 172.21.15.55:48722) 2024-05-10T03:33:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f062a800640 1 -- 172.21.15.55:0/743769505 learned_addr learned my addr 172.21.15.55:0/743769505 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f062b200640 1 --2- 172.21.15.55:0/743769505 >> v2:172.21.15.162:3300/0 conn(0x7f062c0b3190 0x7f062c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298522170 0 0) 0x7f062c06d350 con 0x7f062c144710 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0621e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c144120 0x7f062c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.750+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0610003520 con 0x7f062c144710 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819680563 0 0) 0x7f0610003520 con 0x7f062c144710 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c144120 msgr2=0x7f062c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:44.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c144120 0x7f062c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 >> v2:172.21.15.162:3300/0 conn(0x7f062c0b3190 msgr2=0x7f062c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 --2- 172.21.15.55:0/743769505 >> v2:172.21.15.162:3300/0 conn(0x7f062c0b3190 0x7f062c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c06c740 con 0x7f062c144710 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f061c001670 con 0x7f062c144710 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 >> v1:172.21.15.55:6789/0 conn(0x7f062c144710 legacy=0x7f062c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06100039e0 0x7f0610003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06100047c0 0x7f0610004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0610005230 0x7f061000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c144530 con 0x7f0610005230 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c06dd20 con 0x7f06100047c0 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061000c1a0 con 0x7f06100039e0 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.751+0000 7f0621e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c144120 0x7f062c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0610005230 0x7f061000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f0621e00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06100047c0 0x7f0610004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062a800640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06100039e0 0x7f0610003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 -- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06100039e0 msgr2=0x7f0610003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:44.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06100039e0 0x7f0610003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 -- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06100047c0 msgr2=0x7f0610004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:44.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06100047c0 0x7f0610004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.752+0000 7f062b200640 1 -- 172.21.15.55:0/743769505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610003520 con 0x7f0610005230 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.753+0000 7f062b200640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0610005230 0x7f061000b980 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0624009fc0 tx=0x7f06100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee7c611095a96ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.753+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624031670 con 0x7f0610005230 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.753+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062400f040 con 0x7f0610005230 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.753+0000 7f0629e00640 1 -- 172.21.15.55:0/743769505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624031a10 con 0x7f0610005230 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.754+0000 7f0630a00640 1 -- 172.21.15.55:0/743769505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0610005230 msgr2=0x7f061000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.754+0000 7f0630a00640 1 --2- 172.21.15.55:0/743769505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0610005230 0x7f061000b980 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0624009fc0 tx=0x7f06100057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.754+0000 7f062b200640 1 -- 172.21.15.55:0/743769505 reap_dead start 2024-05-10T03:33:44.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.754+0000 7f0630a00640 1 -- 172.21.15.55:0/743769505 shutdown_connections 2024-05-10T03:33:44.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.754+0000 7f0630a00640 1 -- 172.21.15.55:0/743769505 >> 172.21.15.55:0/743769505 conn(0x7f062c06a110 msgr2=0x7f062c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:44.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.755+0000 7f0630a00640 1 -- 172.21.15.55:0/743769505 shutdown_connections 2024-05-10T03:33:44.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.755+0000 7f0630a00640 1 -- 172.21.15.55:0/743769505 wait complete. 2024-05-10T03:33:44.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.757+0000 7f0630a00640 1 Processor -- start 2024-05-10T03:33:44.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.757+0000 7f0630a00640 1 -- start start 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 0x7f062c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 0x7f062c139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c139b50 con 0x7f062c15a910 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c15ca00 con 0x7f0610005230 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0630a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c15d860 con 0x7f062c138d50 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0621e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0621e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57262/0 (socket says 172.21.15.55:57262) 2024-05-10T03:33:44.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.759+0000 7f0621e00640 1 -- 172.21.15.55:0/3669534280 learned_addr learned my addr 172.21.15.55:0/3669534280 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:44.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f062b200640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 0x7f062c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f062a800640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 0x7f062c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 -- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 msgr2=0x7f062c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:44.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 0x7f062c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 -- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 msgr2=0x7f062c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 0x7f062c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 -- 172.21.15.55:0/3669534280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c1396a0 con 0x7f062c15a910 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f062b200640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 0x7f062c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f062a800640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 0x7f062c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0621e00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0614010870 tx=0x7f0614010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061400c930 con 0x7f062c15a910 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061400ca90 con 0x7f062c15a910 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061400ce20 con 0x7f062c15a910 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.760+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c0711a0 con 0x7f062c15a910 2024-05-10T03:33:44.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.761+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c15cf30 con 0x7f062c15a910 2024-05-10T03:33:44.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.762+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c066ce0 con 0x7f062c15a910 2024-05-10T03:33:44.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.763+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0614015470 con 0x7f062c15a910 2024-05-10T03:33:44.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.764+0000 7f0623400640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 0x7f060806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:44.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.764+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0614089ef0 con 0x7f062c15a910 2024-05-10T03:33:44.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.765+0000 7f062a800640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 0x7f060806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:44.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.766+0000 7f062a800640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 0x7f060806d140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f061c001200 tx=0x7f061c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:44.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:44.769+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0614054f00 con 0x7f062c15a910 2024-05-10T03:33:45.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.006+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f062c06c360 con 0x7f062c15a910 2024-05-10T03:33:45.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.007+0000 7f0623400640 1 -- 172.21.15.55:0/3669534280 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0614058520 con 0x7f062c15a910 2024-05-10T03:33:45.010 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:33:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 msgr2=0x7f060806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 0x7f060806d140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f061c001200 tx=0x7f061c009040 comp rx=0 tx=0).stop 2024-05-10T03:33:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 msgr2=0x7f062c138810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0614010870 tx=0x7f0614010d30 comp rx=0 tx=0).stop 2024-05-10T03:33:45.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 shutdown_connections 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f062c138d50 0x7f062c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f062c15a910 0x7f062c138810 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f060806ac30 0x7f060806d140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 --2- 172.21.15.55:0/3669534280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0610005230 0x7f062c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 >> 172.21.15.55:0/3669534280 conn(0x7f062c06a110 msgr2=0x7f062c0cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 shutdown_connections 2024-05-10T03:33:45.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:45.010+0000 7f0630a00640 1 -- 172.21.15.55:0/3669534280 wait complete. 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:33:45.041 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:45.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.042 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:33:45.101 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:33:45.101 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:45.102 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:33:45.103 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.104 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:33:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:33:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:33:45.104 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:33:45.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.105 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:33:45.185 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:33:45.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.189 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:33:45.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.240 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:33:45.240 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:33:45.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:33:45.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:33:45.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:33:45.241 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:33:45.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.386 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:33:45.434 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:33:45.434 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:33:45.435 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:45.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.435 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:33:45.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:45.519 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:45.566 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:33:45.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:33:45.567 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:45.615 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:33:45.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:45.813 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:45.813 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:45.814 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:45.839 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:45.840 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:45.840 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:45.840 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:45.840 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:33:45.874 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:33:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:33:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:33:45.877 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:33:45.878 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:33:45.878 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:33:45.881 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:33:45.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.881 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:45.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:45.959 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:33:46.015 INFO:teuthology.orchestra.run.smithi162.stdout:client5369 2024-05-10T03:33:46.015 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:46.015 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:46.063 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-05-10T03:33:46.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.358+0000 7f537be00640 1 Processor -- start 2024-05-10T03:33:46.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.359+0000 7f537be00640 1 -- start start 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.361+0000 7f537be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f537c06b830 0x7f537c161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.361+0000 7f537be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 0x7f537c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.361+0000 7f537be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cd660 con 0x7f537c0ce890 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.361+0000 7f537be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0b1ef0 con 0x7f537c06b830 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.361+0000 7f537be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c0cd920 con 0x7f537c071de0 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f537c06b830 0x7f537c161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f537c06b830 0x7f537c161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57268/0 (socket says 172.21.15.55:57268) 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 -- 172.21.15.55:0/1529505051 learned_addr learned my addr 172.21.15.55:0/1529505051 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537b400640 1 --2- 172.21.15.55:0/1529505051 >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 0x7f537c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 -- 172.21.15.55:0/1529505051 >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 msgr2=0x7f537c0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:46.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 --2- 172.21.15.55:0/1529505051 >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 0x7f537c0b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 -- 172.21.15.55:0/1529505051 >> v1:172.21.15.162:6789/0 conn(0x7f537c0ce890 legacy=0x7f537c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:46.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537a000640 1 -- 172.21.15.55:0/1529505051 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0cdc00 con 0x7f537c06b830 2024-05-10T03:33:46.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.362+0000 7f537b400640 1 --2- 172.21.15.55:0/1529505051 >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 0x7f537c0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:46.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.363+0000 7f537a000640 1 --2- 172.21.15.55:0/1529505051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c06b830 0x7f537c161ae0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f537c162020 tx=0x7f536402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a081a5f33d680312 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:46.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f5379600640 1 -- 172.21.15.55:0/1529505051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364032440 con 0x7f537c06b830 2024-05-10T03:33:46.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f5379600640 1 -- 172.21.15.55:0/1529505051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53640325a0 con 0x7f537c06b830 2024-05-10T03:33:46.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f5379600640 1 -- 172.21.15.55:0/1529505051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364032930 con 0x7f537c06b830 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 -- 172.21.15.55:0/1529505051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c06b830 msgr2=0x7f537c161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 --2- 172.21.15.55:0/1529505051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c06b830 0x7f537c161ae0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f537c162020 tx=0x7f536402a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 -- 172.21.15.55:0/1529505051 shutdown_connections 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 --2- 172.21.15.55:0/1529505051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c06b830 0x7f537c161ae0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 --2- 172.21.15.55:0/1529505051 >> v2:172.21.15.162:3300/0 conn(0x7f537c071de0 0x7f537c0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.364+0000 7f537be00640 1 -- 172.21.15.55:0/1529505051 >> 172.21.15.55:0/1529505051 conn(0x7f537c06c750 msgr2=0x7f537c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.365+0000 7f537be00640 1 -- 172.21.15.55:0/1529505051 shutdown_connections 2024-05-10T03:33:46.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.365+0000 7f537be00640 1 -- 172.21.15.55:0/1529505051 wait complete. 2024-05-10T03:33:46.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.367+0000 7f537be00640 1 Processor -- start 2024-05-10T03:33:46.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.367+0000 7f537be00640 1 -- start start 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 0x7f537c162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537c0ce890 0x7f537c1487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c164c10 con 0x7f537c14a650 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c162c80 con 0x7f537c071de0 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c149360 con 0x7f537c0ce890 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57276/0 (socket says 172.21.15.55:57276) 2024-05-10T03:33:46.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537b400640 1 -- 172.21.15.55:0/2208308544 learned_addr learned my addr 172.21.15.55:0/2208308544 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:46.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537aa00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 0x7f537c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537a000640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537c0ce890 0x7f537c1487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.369+0000 7f537b400640 1 -- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537c0ce890 msgr2=0x7f537c1487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537b400640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537c0ce890 0x7f537c1487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537b400640 1 -- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 msgr2=0x7f537c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537b400640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 0x7f537c162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537b400640 1 -- 172.21.15.55:0/2208308544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53640094d0 con 0x7f537c14a650 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537aa00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 0x7f537c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537b400640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f537400c5f0 tx=0x7f537400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374011800 con 0x7f537c14a650 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374011980 con 0x7f537c14a650 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374011d30 con 0x7f537c14a650 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.370+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c0a8760 con 0x7f537c14a650 2024-05-10T03:33:46.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.371+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1495b0 con 0x7f537c14a650 2024-05-10T03:33:46.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.372+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c04e290 con 0x7f537c14a650 2024-05-10T03:33:46.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.373+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5374022350 con 0x7f537c14a650 2024-05-10T03:33:46.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.374+0000 7f5372a00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 0x7f535806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:46.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.374+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5374089560 con 0x7f537c14a650 2024-05-10T03:33:46.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.375+0000 7f537aa00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 0x7f535806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:46.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.376+0000 7f537aa00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 0x7f535806d140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f536c007eb0 tx=0x7f536c0073c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:46.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.378+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5374054570 con 0x7f537c14a650 2024-05-10T03:33:46.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.585+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) v1 -- 0x7f537c0b0750 con 0x7f537c14a650 2024-05-10T03:33:46.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.596+0000 7f5372a00640 1 -- 172.21.15.55:0/2208308544 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v19)=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5374057b90 con 0x7f537c14a650 2024-05-10T03:33:46.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 msgr2=0x7f535806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:46.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 0x7f535806d140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f536c007eb0 tx=0x7f536c0073c0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 msgr2=0x7f537c14aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:46.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f537400c5f0 tx=0x7f537400cab0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 shutdown_connections 2024-05-10T03:33:46.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f537c14a650 0x7f537c14aa60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f537c0ce890 0x7f537c1487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f535806ac30 0x7f535806d140 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 --2- 172.21.15.55:0/2208308544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f537c071de0 0x7f537c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 >> 172.21.15.55:0/2208308544 conn(0x7f537c06c750 msgr2=0x7f537c0b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 shutdown_connections 2024-05-10T03:33:46.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:46.599+0000 7f537be00640 1 -- 172.21.15.55:0/2208308544 wait complete. 2024-05-10T03:33:46.635 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:46.635 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:46.687 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:46.687 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:46.788 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:47.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.074+0000 7f5838600640 1 Processor -- start 2024-05-10T03:33:47.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.074+0000 7f5838600640 1 -- start start 2024-05-10T03:33:47.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.077+0000 7f5838600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f583006b6e0 0x7f58300ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.077+0000 7f5838600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58300b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.077+0000 7f5838600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58300715e0 con 0x7f58301659c0 2024-05-10T03:33:47.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.077+0000 7f5838600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58300711c0 con 0x7f583006b6e0 2024-05-10T03:33:47.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.077+0000 7f5838600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583006baf0 con 0x7f58300b2700 2024-05-10T03:33:47.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.078+0000 7f5835c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f583006b6e0 0x7f58300ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.078+0000 7f5836600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58300b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.078+0000 7f5835c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f583006b6e0 0x7f58300ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:57032/0 (socket says 172.21.15.55:57032) 2024-05-10T03:33:47.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.078+0000 7f5835c00640 1 -- 172.21.15.55:0/991438715 learned_addr learned my addr 172.21.15.55:0/991438715 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:47.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.079+0000 7f5835c00640 1 -- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 msgr2=0x7f58300b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:47.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.079+0000 7f5835c00640 1 --2- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58300b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.079+0000 7f5835c00640 1 -- 172.21.15.55:0/991438715 >> v1:172.21.15.162:6789/0 conn(0x7f58301659c0 legacy=0x7f5830165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.079+0000 7f5835c00640 1 -- 172.21.15.55:0/991438715 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58301667f0 con 0x7f583006b6e0 2024-05-10T03:33:47.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.080+0000 7f5835c00640 1 --2- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f58300ce3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5824009870 tx=0x7f582402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3052224fff4d8ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.081+0000 7f582fe00640 1 -- 172.21.15.55:0/991438715 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824032440 con 0x7f583006b6e0 2024-05-10T03:33:47.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.081+0000 7f582fe00640 1 -- 172.21.15.55:0/991438715 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240325a0 con 0x7f583006b6e0 2024-05-10T03:33:47.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.081+0000 7f582fe00640 1 -- 172.21.15.55:0/991438715 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824032930 con 0x7f583006b6e0 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 -- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 msgr2=0x7f58300ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 --2- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f58300ce3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5824009870 tx=0x7f582402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 -- 172.21.15.55:0/991438715 shutdown_connections 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 --2- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58300b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 --2- 172.21.15.55:0/991438715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f58300ce3b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 -- 172.21.15.55:0/991438715 >> 172.21.15.55:0/991438715 conn(0x7f58300cbb90 msgr2=0x7f58300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:47.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 -- 172.21.15.55:0/991438715 shutdown_connections 2024-05-10T03:33:47.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.082+0000 7f5838600640 1 -- 172.21.15.55:0/991438715 wait complete. 2024-05-10T03:33:47.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.085+0000 7f5838600640 1 Processor -- start 2024-05-10T03:33:47.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.085+0000 7f5838600640 1 -- start start 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f583013dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58301f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583013e890 con 0x7f58301659c0 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58301582f0 con 0x7f583006b6e0 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.086+0000 7f5838600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5830154ff0 con 0x7f58300b2700 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57286/0 (socket says 172.21.15.55:57286) 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 -- 172.21.15.55:0/800036827 learned_addr learned my addr 172.21.15.55:0/800036827 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5835c00640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f583013dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5835200640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58301f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 -- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 msgr2=0x7f58301f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58301f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 -- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 msgr2=0x7f583013dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f583013dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.087+0000 7f5836600640 1 -- 172.21.15.55:0/800036827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240094d0 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f5836600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f582800efc0 tx=0x7f582800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828009b20 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582800f040 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58280146c0 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58301552d0 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.088+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5830159260 con 0x7f58301659c0 2024-05-10T03:33:47.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.089+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5830159420 con 0x7f58301659c0 2024-05-10T03:33:47.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.090+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5828014820 con 0x7f58301659c0 2024-05-10T03:33:47.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.091+0000 7f582d600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 0x7f581006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.092+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f582808a310 con 0x7f58301659c0 2024-05-10T03:33:47.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.092+0000 7f5835c00640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 0x7f581006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.093+0000 7f5835c00640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 0x7f581006d110 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5824034000 tx=0x7f582402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.097+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5828055310 con 0x7f58301659c0 2024-05-10T03:33:47.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.297+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5830066f60 con 0x7f58301659c0 2024-05-10T03:33:47.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.301+0000 7f582d600640 1 -- 172.21.15.55:0/800036827 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f5828009e60 con 0x7f58301659c0 2024-05-10T03:33:47.305 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:47.305 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 msgr2=0x7f581006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 0x7f581006d110 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5824034000 tx=0x7f582402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:47.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 msgr2=0x7f5830159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f582800efc0 tx=0x7f582800c490 comp rx=0 tx=0).stop 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 shutdown_connections 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58301659c0 0x7f5830159780 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58300b2700 0x7f58301f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f581006ac00 0x7f581006d110 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 --2- 172.21.15.55:0/800036827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f583006b6e0 0x7f583013dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 >> 172.21.15.55:0/800036827 conn(0x7f58300cbb90 msgr2=0x7f58300b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 shutdown_connections 2024-05-10T03:33:47.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.304+0000 7f5838600640 1 -- 172.21.15.55:0/800036827 wait complete. 2024-05-10T03:33:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:33:33.167391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:33:33.995610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:47.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:47.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.624+0000 7f9400600640 1 Processor -- start 2024-05-10T03:33:47.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.624+0000 7f9400600640 1 -- start start 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f9400600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 0x7f93f8161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f9400600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93f8161b90 0x7f93f80b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f9400600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f81624d0 con 0x7f93f80b8af0 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f9400600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f80b8f00 con 0x7f93f8161b90 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f9400600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80cdab0 con 0x7f93f80d2c60 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f93fdc00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f93f80b8af0 0x7f93f80b6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48736/0 (socket says 172.21.15.55:48736) 2024-05-10T03:33:47.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.626+0000 7f93fdc00640 1 -- 172.21.15.55:0/1717842118 learned_addr learned my addr 172.21.15.55:0/1717842118 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:47.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102569665 0 0) 0x7f93f81624d0 con 0x7f93f80b8af0 2024-05-10T03:33:47.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93dc003520 con 0x7f93f80b8af0 2024-05-10T03:33:47.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93fd200640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 0x7f93f8161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93fe600640 1 --2- 172.21.15.55:0/1717842118 >> v2:172.21.15.162:3300/0 conn(0x7f93f8161b90 0x7f93f80b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739840072 0 0) 0x7f93dc003520 con 0x7f93f80b8af0 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 msgr2=0x7f93f8161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 0x7f93f8161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 >> v2:172.21.15.162:3300/0 conn(0x7f93f8161b90 msgr2=0x7f93f80b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 --2- 172.21.15.55:0/1717842118 >> v2:172.21.15.162:3300/0 conn(0x7f93f8161b90 0x7f93f80b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80b2d10 con 0x7f93f80b8af0 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f93e8001670 con 0x7f93f80b8af0 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93fe600640 1 --2- 172.21.15.55:0/1717842118 >> v2:172.21.15.162:3300/0 conn(0x7f93f8161b90 0x7f93f80b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.627+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 >> v1:172.21.15.55:6789/0 conn(0x7f93f80b8af0 legacy=0x7f93f80b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93dc003970 0x7f93dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93dc0047c0 0x7f93dc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 0x7f93dc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80b8f00 con 0x7f93dc0047c0 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80cdab0 con 0x7f93dc003970 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc00bae0 con 0x7f93f80d2c60 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93dc0047c0 0x7f93dc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 -- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 msgr2=0x7f93dc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93f80d2c60 0x7f93dc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fdc00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93dc003970 0x7f93dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 -- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93dc003970 msgr2=0x7f93dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93dc003970 0x7f93dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fd200640 1 -- 172.21.15.55:0/1717842118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc003520 con 0x7f93dc0047c0 2024-05-10T03:33:47.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.628+0000 7f93fdc00640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93dc003970 0x7f93dc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:47.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.629+0000 7f93fd200640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93dc0047c0 0x7f93dc004bd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ba00 tx=0x7f93dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c216713d16724a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e402ade0 con 0x7f93dc0047c0 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e402e070 con 0x7f93dc0047c0 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f93f7e00640 1 -- 172.21.15.55:0/1717842118 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4034620 con 0x7f93dc0047c0 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f9400600640 1 -- 172.21.15.55:0/1717842118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93dc0047c0 msgr2=0x7f93dc004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f9400600640 1 --2- 172.21.15.55:0/1717842118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93dc0047c0 0x7f93dc004bd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ba00 tx=0x7f93dc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.630+0000 7f93fe600640 1 -- 172.21.15.55:0/1717842118 reap_dead start 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.631+0000 7f9400600640 1 -- 172.21.15.55:0/1717842118 shutdown_connections 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.631+0000 7f9400600640 1 -- 172.21.15.55:0/1717842118 >> 172.21.15.55:0/1717842118 conn(0x7f93f80cfe20 msgr2=0x7f93f80d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.631+0000 7f9400600640 1 -- 172.21.15.55:0/1717842118 shutdown_connections 2024-05-10T03:33:47.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.631+0000 7f9400600640 1 -- 172.21.15.55:0/1717842118 wait complete. 2024-05-10T03:33:47.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.633+0000 7f9400600640 1 Processor -- start 2024-05-10T03:33:47.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.633+0000 7f9400600640 1 -- start start 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 0x7f93f80c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f8150e80 0x7f93f81f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80ccaf0 con 0x7f93f814da00 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80cb550 con 0x7f93f8150e80 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f9400600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81f70e0 con 0x7f93dc0047c0 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f93fe600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f93fe600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57296/0 (socket says 172.21.15.55:57296) 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.635+0000 7f93fe600640 1 -- 172.21.15.55:0/3777830399 learned_addr learned my addr 172.21.15.55:0/3777830399 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fdc00640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 0x7f93f80c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 -- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 msgr2=0x7f93f80c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 0x7f93f80c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 -- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f8150e80 msgr2=0x7f93f81f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fd200640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f8150e80 0x7f93f81f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f8150e80 0x7f93f81f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 -- 172.21.15.55:0/3777830399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f81f7ab0 con 0x7f93f814da00 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fdc00640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 0x7f93f80c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:47.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93fe600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f93f808baa0 tx=0x7f93f000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.636+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000dc90 con 0x7f93f814da00 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.637+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000f040 con 0x7f93f814da00 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.637+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0003990 con 0x7f93f814da00 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.637+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f80cb7a0 con 0x7f93f814da00 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.637+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f81f6d70 con 0x7f93f814da00 2024-05-10T03:33:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.638+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f80cbcc0 con 0x7f93f814da00 2024-05-10T03:33:47.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.639+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f93f0003af0 con 0x7f93f814da00 2024-05-10T03:33:47.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.640+0000 7f93f5600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 0x7f93d806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:47.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.640+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f93f0089300 con 0x7f93f814da00 2024-05-10T03:33:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.641+0000 7f93fdc00640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 0x7f93d806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:47.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.642+0000 7f93fdc00640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 0x7f93d806d110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f93e80017e0 tx=0x7f93e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:47.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.645+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f93f80cbcc0 con 0x7f93f814da00 2024-05-10T03:33:47.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:33:47.771 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:33:47.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.851+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93f81f6bd0 con 0x7f93f814da00 2024-05-10T03:33:47.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.852+0000 7f93f5600640 1 -- 172.21.15.55:0/3777830399 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93f81f6bd0 con 0x7f93f814da00 2024-05-10T03:33:47.854 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:47.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.854+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 msgr2=0x7f93d806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.854+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 0x7f93d806d110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f93e80017e0 tx=0x7f93e8009040 comp rx=0 tx=0).stop 2024-05-10T03:33:47.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.854+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 msgr2=0x7f93f80cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:47.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.854+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f93f808baa0 tx=0x7f93f000d4a0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 shutdown_connections 2024-05-10T03:33:47.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93f814da00 0x7f93f80cbee0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93d806ac00 0x7f93d806d110 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93f8150e80 0x7f93f81f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 --2- 172.21.15.55:0/3777830399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93dc0047c0 0x7f93f80c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:47.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 >> 172.21.15.55:0/3777830399 conn(0x7f93f80cfe20 msgr2=0x7f93f815d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:47.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 shutdown_connections 2024-05-10T03:33:47.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:47.855+0000 7f9400600640 1 -- 172.21.15.55:0/3777830399 wait complete. 2024-05-10T03:33:47.883 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:48.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.210+0000 7f8d31800640 1 Processor -- start 2024-05-10T03:33:48.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.211+0000 7f8d31800640 1 -- start start 2024-05-10T03:33:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d2c0719f0 0x7f8d2c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b840 0x7f8d2c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 0x7f8d2c070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c15fdf0 con 0x7f8d2c0719f0 2024-05-10T03:33:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c15f1e0 con 0x7f8d2c071e00 2024-05-10T03:33:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.213+0000 7f8d31800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15f4c0 con 0x7f8d2c06b840 2024-05-10T03:33:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d2c0719f0 0x7f8d2c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d2c0719f0 0x7f8d2c06b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57298/0 (socket says 172.21.15.55:57298) 2024-05-10T03:33:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 -- 172.21.15.55:0/384232002 learned_addr learned my addr 172.21.15.55:0/384232002 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 -- 172.21.15.55:0/384232002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b840 msgr2=0x7f8d2c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 --2- 172.21.15.55:0/384232002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b840 0x7f8d2c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d30e00640 1 --2- 172.21.15.55:0/384232002 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 0x7f8d2c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 -- 172.21.15.55:0/384232002 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 msgr2=0x7f8d2c070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 --2- 172.21.15.55:0/384232002 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 0x7f8d2c070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d2be00640 1 -- 172.21.15.55:0/384232002 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c15f7a0 con 0x7f8d2c0719f0 2024-05-10T03:33:48.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.214+0000 7f8d30e00640 1 --2- 172.21.15.55:0/384232002 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 0x7f8d2c070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:48.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.215+0000 7f8d2be00640 1 --2- 172.21.15.55:0/384232002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c0719f0 0x7f8d2c06b300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c071940 tx=0x7f8d1402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d4c109e24aa43c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.216+0000 7f8d2aa00640 1 -- 172.21.15.55:0/384232002 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1402ae90 con 0x7f8d2c0719f0 2024-05-10T03:33:48.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.216+0000 7f8d2aa00640 1 -- 172.21.15.55:0/384232002 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14032440 con 0x7f8d2c0719f0 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.216+0000 7f8d2aa00640 1 -- 172.21.15.55:0/384232002 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140327d0 con 0x7f8d2c0719f0 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.216+0000 7f8d31800640 1 -- 172.21.15.55:0/384232002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c0719f0 msgr2=0x7f8d2c06b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.216+0000 7f8d31800640 1 --2- 172.21.15.55:0/384232002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c0719f0 0x7f8d2c06b300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c071940 tx=0x7f8d1402a140 comp rx=0 tx=0).stop 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 -- 172.21.15.55:0/384232002 shutdown_connections 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 --2- 172.21.15.55:0/384232002 >> v2:172.21.15.162:3300/0 conn(0x7f8d2c071e00 0x7f8d2c070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 --2- 172.21.15.55:0/384232002 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c06b840 0x7f8d2c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 --2- 172.21.15.55:0/384232002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c0719f0 0x7f8d2c06b300 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 -- 172.21.15.55:0/384232002 >> 172.21.15.55:0/384232002 conn(0x7f8d2c06c750 msgr2=0x7f8d2c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 -- 172.21.15.55:0/384232002 shutdown_connections 2024-05-10T03:33:48.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.217+0000 7f8d31800640 1 -- 172.21.15.55:0/384232002 wait complete. 2024-05-10T03:33:48.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.219+0000 7f8d31800640 1 Processor -- start 2024-05-10T03:33:48.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.219+0000 7f8d31800640 1 -- start start 2024-05-10T03:33:48.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c06b840 0x7f8d2c13b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c0719f0 0x7f8d2c13caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1f44f0 con 0x7f8d2c071e00 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c13be50 con 0x7f8d2c06b840 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d31800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c4390 con 0x7f8d2c0719f0 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d30e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d30e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57302/0 (socket says 172.21.15.55:57302) 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d30e00640 1 -- 172.21.15.55:0/2389718585 learned_addr learned my addr 172.21.15.55:0/2389718585 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d2be00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c06b840 0x7f8d2c13b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.221+0000 7f8d2b400640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c0719f0 0x7f8d2c13caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 -- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c0719f0 msgr2=0x7f8d2c13caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c0719f0 0x7f8d2c13caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 -- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c06b840 msgr2=0x7f8d2c13b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c06b840 0x7f8d2c13b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 -- 172.21.15.55:0/2389718585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140094d0 con 0x7f8d2c071e00 2024-05-10T03:33:48.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d30e00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c5a0 tx=0x7f8d2400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004220 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d240043a0 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24003850 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.222+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0c4670 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.223+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c0c4c30 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.224+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c04e290 con 0x7f8d2c071e00 2024-05-10T03:33:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.225+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8d240039b0 con 0x7f8d2c071e00 2024-05-10T03:33:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.226+0000 7f8d23e00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 0x7f8d0806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.226+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8d24089200 con 0x7f8d2c071e00 2024-05-10T03:33:48.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.227+0000 7f8d2be00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 0x7f8d0806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.228+0000 7f8d2be00640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 0x7f8d0806d110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.231+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8d24054210 con 0x7f8d2c071e00 2024-05-10T03:33:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.427+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d2c15ebf0 con 0x7f8d2c071e00 2024-05-10T03:33:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.428+0000 7f8d23e00640 1 -- 172.21.15.55:0/2389718585 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d24057830 con 0x7f8d2c071e00 2024-05-10T03:33:48.431 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:48.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 msgr2=0x7f8d0806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 0x7f8d0806d110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).stop 2024-05-10T03:33:48.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 msgr2=0x7f8d2c0ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c5a0 tx=0x7f8d2400ca60 comp rx=0 tx=0).stop 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 shutdown_connections 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d2c071e00 0x7f8d2c0ad410 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d2c0719f0 0x7f8d2c13caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8d0806ac00 0x7f8d0806d110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 --2- 172.21.15.55:0/2389718585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d2c06b840 0x7f8d2c13b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 >> 172.21.15.55:0/2389718585 conn(0x7f8d2c06c750 msgr2=0x7f8d2c140210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.431+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 shutdown_connections 2024-05-10T03:33:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.432+0000 7f8d31800640 1 -- 172.21.15.55:0/2389718585 wait complete. 2024-05-10T03:33:48.461 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.740+0000 7f6daa200640 1 Processor -- start 2024-05-10T03:33:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.741+0000 7f6daa200640 1 -- start start 2024-05-10T03:33:48.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.742+0000 7f6daa200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da406b830 0x7f6da4143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.742+0000 7f6daa200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40cdbe0 con 0x7f6da41435d0 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.742+0000 7f6daa200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da415e8b0 con 0x7f6da40b2040 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.742+0000 7f6daa200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da40ce140 con 0x7f6da406b830 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da9800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da406b830 0x7f6da4143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da3e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6da41435d0 0x7f6da406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48746/0 (socket says 172.21.15.55:48746) 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da9800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da406b830 0x7f6da4143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57316/0 (socket says 172.21.15.55:57316) 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da3e00640 1 -- 172.21.15.55:0/1587037556 learned_addr learned my addr 172.21.15.55:0/1587037556 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da9800640 1 -- 172.21.15.55:0/1587037556 >> v1:172.21.15.162:6789/0 conn(0x7f6da40b2040 legacy=0x7f6da40ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:48.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da9800640 1 -- 172.21.15.55:0/1587037556 >> v1:172.21.15.55:6789/0 conn(0x7f6da41435d0 legacy=0x7f6da406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da9800640 1 -- 172.21.15.55:0/1587037556 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da415ede0 con 0x7f6da406b830 2024-05-10T03:33:48.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.743+0000 7f6da3400640 1 -- 172.21.15.55:0/1587037556 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666202162 0 0) 0x7f6da40cdbe0 con 0x7f6da41435d0 2024-05-10T03:33:48.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.745+0000 7f6da9800640 1 --2- 172.21.15.55:0/1587037556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da406b830 0x7f6da4143020 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009dc0 tx=0x7f6d9c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c43a04d26a11f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.745+0000 7f6da3400640 1 -- 172.21.15.55:0/1587037556 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c007740 con 0x7f6da406b830 2024-05-10T03:33:48.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.745+0000 7f6da3400640 1 -- 172.21.15.55:0/1587037556 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c02d050 con 0x7f6da406b830 2024-05-10T03:33:48.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.745+0000 7f6da3400640 1 -- 172.21.15.55:0/1587037556 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c007ac0 con 0x7f6da406b830 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 -- 172.21.15.55:0/1587037556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da406b830 msgr2=0x7f6da4143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 --2- 172.21.15.55:0/1587037556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da406b830 0x7f6da4143020 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009dc0 tx=0x7f6d9c02a640 comp rx=0 tx=0).stop 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 -- 172.21.15.55:0/1587037556 shutdown_connections 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 --2- 172.21.15.55:0/1587037556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da406b830 0x7f6da4143020 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 -- 172.21.15.55:0/1587037556 >> 172.21.15.55:0/1587037556 conn(0x7f6da406c730 msgr2=0x7f6da40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 -- 172.21.15.55:0/1587037556 shutdown_connections 2024-05-10T03:33:48.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.746+0000 7f6daa200640 1 -- 172.21.15.55:0/1587037556 wait complete. 2024-05-10T03:33:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.748+0000 7f6daa200640 1 Processor -- start 2024-05-10T03:33:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.748+0000 7f6daa200640 1 -- start start 2024-05-10T03:33:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b2040 0x7f6da4136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da41435d0 0x7f6da40c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4137440 con 0x7f6da40afd00 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4135590 con 0x7f6da41435d0 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6daa200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da415cf20 con 0x7f6da40b2040 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6da9800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6da9800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57318/0 (socket says 172.21.15.55:57318) 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.750+0000 7f6da9800640 1 -- 172.21.15.55:0/1176307437 learned_addr learned my addr 172.21.15.55:0/1176307437 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da3e00640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da41435d0 0x7f6da40c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da8e00640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b2040 0x7f6da4136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 -- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b2040 msgr2=0x7f6da4136f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b2040 0x7f6da4136f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 -- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da41435d0 msgr2=0x7f6da40c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da41435d0 0x7f6da40c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 -- 172.21.15.55:0/1176307437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c009a70 con 0x7f6da40afd00 2024-05-10T03:33:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.751+0000 7f6da9800640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009ee0 tx=0x7f6d9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.752+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031890 con 0x7f6da40afd00 2024-05-10T03:33:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.752+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0319f0 con 0x7f6da40afd00 2024-05-10T03:33:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.752+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031d60 con 0x7f6da40afd00 2024-05-10T03:33:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.752+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da40b0650 con 0x7f6da40afd00 2024-05-10T03:33:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.752+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da415c5f0 con 0x7f6da40afd00 2024-05-10T03:33:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.753+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da40b1680 con 0x7f6da40afd00 2024-05-10T03:33:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.754+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d9c03c420 con 0x7f6da40afd00 2024-05-10T03:33:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.755+0000 7f6da0c00640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.755+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6d9c0ab330 con 0x7f6da40afd00 2024-05-10T03:33:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.756+0000 7f6da8e00640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:48.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.757+0000 7f6da8e00640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6d98006fd0 tx=0x7f6d98008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:48.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.760+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6d9c0762a0 con 0x7f6da40afd00 2024-05-10T03:33:48.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.958+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6da415d0b0 con 0x7f6da40afd00 2024-05-10T03:33:48.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.959+0000 7f6da0c00640 1 -- 172.21.15.55:0/1176307437 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d9c0798c0 con 0x7f6da40afd00 2024-05-10T03:33:48.962 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.962+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 msgr2=0x7f6d8406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.962+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6d98006fd0 tx=0x7f6d98008040 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.962+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 msgr2=0x7f6da40b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.962+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009ee0 tx=0x7f6d9c004a40 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 shutdown_connections 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40afd00 0x7f6da40b0110 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da41435d0 0x7f6da40c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 --2- 172.21.15.55:0/1176307437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b2040 0x7f6da4136f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 >> 172.21.15.55:0/1176307437 conn(0x7f6da406c730 msgr2=0x7f6da4071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 shutdown_connections 2024-05-10T03:33:48.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:48.963+0000 7f6daa200640 1 -- 172.21.15.55:0/1176307437 wait complete. 2024-05-10T03:33:48.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:33:49.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.276+0000 7fb307e00640 1 Processor -- start 2024-05-10T03:33:49.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.276+0000 7fb307e00640 1 -- start start 2024-05-10T03:33:49.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.278+0000 7fb307e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb30813e390 0x7fb30815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.278+0000 7fb307e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb30806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.278+0000 7fb307e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30813eae0 con 0x7fb30806b660 2024-05-10T03:33:49.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.278+0000 7fb307e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30813e7a0 con 0x7fb30813e390 2024-05-10T03:33:49.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.278+0000 7fb307e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30806ba70 con 0x7fb3081414c0 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.279+0000 7fb307400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb30806b660 0x7fb30815efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48756/0 (socket says 172.21.15.55:48756) 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.279+0000 7fb307400640 1 -- 172.21.15.55:0/726581914 learned_addr learned my addr 172.21.15.55:0/726581914 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.279+0000 7fb306a00640 1 --2- 172.21.15.55:0/726581914 >> v2:172.21.15.162:3300/0 conn(0x7fb30813e390 0x7fb30815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.279+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb30806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.279+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598561624 0 0) 0x7fb30813eae0 con 0x7fb30806b660 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec003520 con 0x7fb30806b660 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192860937 0 0) 0x7fb2ec003520 con 0x7fb30806b660 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 msgr2=0x7fb30806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb30806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 >> v2:172.21.15.162:3300/0 conn(0x7fb30813e390 msgr2=0x7fb30815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:49.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 --2- 172.21.15.55:0/726581914 >> v2:172.21.15.162:3300/0 conn(0x7fb30813e390 0x7fb30815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30815c020 con 0x7fb30806b660 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fb3000019f0 con 0x7fb30806b660 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 >> v1:172.21.15.55:6789/0 conn(0x7fb30806b660 legacy=0x7fb30815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb2ec003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2ec003ef0 0x7fb2ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec004d00 0x7fb2ec005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30813e7a0 con 0x7fb2ec003ef0 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30806ba70 con 0x7fb2ec004d00 2024-05-10T03:33:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec00b9d0 con 0x7fb3081414c0 2024-05-10T03:33:49.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.280+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2ec003ef0 0x7fb2ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb307400640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec004d00 0x7fb2ec005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb306a00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb2ec003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb2ffe00640 1 -- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 msgr2=0x7fb2ec003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb2ec003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb2ffe00640 1 -- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec004d00 msgr2=0x7fb2ec005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec004d00 0x7fb2ec005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb2ffe00640 1 -- 172.21.15.55:0/726581914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec003520 con 0x7fb2ec003ef0 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.281+0000 7fb306a00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3081414c0 0x7fb2ec003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.282+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2ec003ef0 0x7fb2ec0047c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb30806fe70 tx=0x7fb2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c730d30591621d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.282+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0005990 con 0x7fb2ec003ef0 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.282+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f002d050 con 0x7fb2ec003ef0 2024-05-10T03:33:49.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.282+0000 7fb306000640 1 -- 172.21.15.55:0/726581914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0005d30 con 0x7fb2ec003ef0 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 -- 172.21.15.55:0/726581914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2ec003ef0 msgr2=0x7fb2ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 --2- 172.21.15.55:0/726581914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2ec003ef0 0x7fb2ec0047c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb30806fe70 tx=0x7fb2ec0057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307400640 1 -- 172.21.15.55:0/726581914 reap_dead start 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 -- 172.21.15.55:0/726581914 shutdown_connections 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 -- 172.21.15.55:0/726581914 >> 172.21.15.55:0/726581914 conn(0x7fb30806c730 msgr2=0x7fb3080b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 -- 172.21.15.55:0/726581914 shutdown_connections 2024-05-10T03:33:49.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.284+0000 7fb307e00640 1 -- 172.21.15.55:0/726581914 wait complete. 2024-05-10T03:33:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.286+0000 7fb307e00640 1 Processor -- start 2024-05-10T03:33:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.287+0000 7fb307e00640 1 -- start start 2024-05-10T03:33:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec003ef0 0x7fb308156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb308155ae0 0x7fb308151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081556c0 con 0x7fb3080b0180 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308159c00 con 0x7fb2ec003ef0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.288+0000 7fb307e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081547f0 con 0x7fb308155ae0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57332/0 (socket says 172.21.15.55:57332) 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 -- 172.21.15.55:0/1541556713 learned_addr learned my addr 172.21.15.55:0/1541556713 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb306a00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec003ef0 0x7fb308156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb2ffe00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb308155ae0 0x7fb308151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 -- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb308155ae0 msgr2=0x7fb308151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb308155ae0 0x7fb308151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 -- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec003ef0 msgr2=0x7fb308156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec003ef0 0x7fb308156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 -- 172.21.15.55:0/1541556713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080b0ad0 con 0x7fb3080b0180 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.289+0000 7fb307400640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb300002230 tx=0x7fb30000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.290+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000ec10 con 0x7fb3080b0180 2024-05-10T03:33:49.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.290+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000ed70 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.290+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000eee0 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.290+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30813aad0 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.290+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb308154d20 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.291+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308155800 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.292+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb300013660 con 0x7fb3080b0180 2024-05-10T03:33:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.293+0000 7fb2fea00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 0x7fb2e406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:49.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.293+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb300089930 con 0x7fb3080b0180 2024-05-10T03:33:49.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.294+0000 7fb306a00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 0x7fb2e406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:49.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.295+0000 7fb306a00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 0x7fb2e406d110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3080b4350 tx=0x7fb2f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.298+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb308155800 con 0x7fb3080b0180 2024-05-10T03:33:49.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.510+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb308155800 con 0x7fb3080b0180 2024-05-10T03:33:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.511+0000 7fb2fea00640 1 -- 172.21.15.55:0/1541556713 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb308155800 con 0x7fb3080b0180 2024-05-10T03:33:49.514 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 msgr2=0x7fb2e406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 0x7fb2e406d110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3080b4350 tx=0x7fb2f8008040 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 msgr2=0x7fb3080b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb300002230 tx=0x7fb30000e530 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 shutdown_connections 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3080b0180 0x7fb3080b0590 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb308155ae0 0x7fb308151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2e406ac00 0x7fb2e406d110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 --2- 172.21.15.55:0/1541556713 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ec003ef0 0x7fb308156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 >> 172.21.15.55:0/1541556713 conn(0x7fb30806c730 msgr2=0x7fb308140920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:49.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 shutdown_connections 2024-05-10T03:33:49.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:49.514+0000 7fb307e00640 1 -- 172.21.15.55:0/1541556713 wait complete. 2024-05-10T03:33:49.544 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:33:49.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:49.544 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:33:49.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:33:49.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:49.645 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:33:49.701 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:49.701 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:49.756 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:49.756 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:49.803 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:50.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.091+0000 7fc914a00640 1 Processor -- start 2024-05-10T03:33:50.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.091+0000 7fc914a00640 1 -- start start 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc914a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc9101612f0 0x7fc910161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc914a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc910161c40 con 0x7fc9101549e0 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc914a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9100b39f0 con 0x7fc910071100 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc914a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9100b3cd0 con 0x7fc9101612f0 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc90e800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc9101549e0 0x7fc9100b2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48762/0 (socket says 172.21.15.55:48762) 2024-05-10T03:33:50.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.094+0000 7fc90e800640 1 -- 172.21.15.55:0/1544593714 learned_addr learned my addr 172.21.15.55:0/1544593714 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:50.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.095+0000 7fc90de00640 1 --2- 172.21.15.55:0/1544593714 >> v2:172.21.15.162:3300/0 conn(0x7fc9101612f0 0x7fc910161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.095+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992450197 0 0) 0x7fc910161c40 con 0x7fc9101549e0 2024-05-10T03:33:50.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.095+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f4003520 con 0x7fc9101549e0 2024-05-10T03:33:50.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.096+0000 7fc90de00640 1 -- 172.21.15.55:0/1544593714 >> v1:172.21.15.162:6789/0 conn(0x7fc910071100 legacy=0x7fc910071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.096+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3981984076 0 0) 0x7fc8f4003520 con 0x7fc9101549e0 2024-05-10T03:33:50.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.096+0000 7fc90de00640 1 -- 172.21.15.55:0/1544593714 >> v1:172.21.15.55:6789/0 conn(0x7fc9101549e0 legacy=0x7fc9100b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.096+0000 7fc90de00640 1 -- 172.21.15.55:0/1544593714 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100b3fb0 con 0x7fc9101612f0 2024-05-10T03:33:50.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.097+0000 7fc90de00640 1 --2- 172.21.15.55:0/1544593714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9101612f0 0x7fc910161700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097d0 tx=0x7fc8fc02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f575515c002f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.098+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900001670 con 0x7fc9101612f0 2024-05-10T03:33:50.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.098+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc02abf0 con 0x7fc9101612f0 2024-05-10T03:33:50.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.098+0000 7fc90d400640 1 -- 172.21.15.55:0/1544593714 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc032440 con 0x7fc9101612f0 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.098+0000 7fc914a00640 1 -- 172.21.15.55:0/1544593714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9101612f0 msgr2=0x7fc910161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.098+0000 7fc914a00640 1 --2- 172.21.15.55:0/1544593714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9101612f0 0x7fc910161700 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097d0 tx=0x7fc8fc02a160 comp rx=0 tx=0).stop 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.099+0000 7fc914a00640 1 -- 172.21.15.55:0/1544593714 shutdown_connections 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.099+0000 7fc914a00640 1 --2- 172.21.15.55:0/1544593714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc9101612f0 0x7fc910161700 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.099+0000 7fc914a00640 1 -- 172.21.15.55:0/1544593714 >> 172.21.15.55:0/1544593714 conn(0x7fc91006a110 msgr2=0x7fc91006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.099+0000 7fc914a00640 1 -- 172.21.15.55:0/1544593714 shutdown_connections 2024-05-10T03:33:50.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.099+0000 7fc914a00640 1 -- 172.21.15.55:0/1544593714 wait complete. 2024-05-10T03:33:50.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.101+0000 7fc914a00640 1 Processor -- start 2024-05-10T03:33:50.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.101+0000 7fc914a00640 1 -- start start 2024-05-10T03:33:50.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc910071100 0x7fc910143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9101612f0 0x7fc9100cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100ccf80 con 0x7fc9101549e0 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9101573e0 con 0x7fc910071100 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc914a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100cbed0 con 0x7fc9101612f0 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc90de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc90de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57342/0 (socket says 172.21.15.55:57342) 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc90de00640 1 -- 172.21.15.55:0/3916608863 learned_addr learned my addr 172.21.15.55:0/3916608863 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.103+0000 7fc90f200640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9101612f0 0x7fc9100cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90e800640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc910071100 0x7fc910143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 -- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9101612f0 msgr2=0x7fc9100cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9101612f0 0x7fc9100cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 -- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc910071100 msgr2=0x7fc910143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc910071100 0x7fc910143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 -- 172.21.15.55:0/3916608863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc009400 con 0x7fc9101549e0 2024-05-10T03:33:50.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc90de00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009720 tx=0x7fc8fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc03c8e0 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc03ca60 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc910143ff0 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.104+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc03ce50 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.105+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9101f7340 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.106+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91004e290 con 0x7fc9101549e0 2024-05-10T03:33:50.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.107+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc8fc039050 con 0x7fc9101549e0 2024-05-10T03:33:50.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.108+0000 7fc906a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 0x7fc8f006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.108+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc8fc0a9e00 con 0x7fc9101549e0 2024-05-10T03:33:50.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.109+0000 7fc90e800640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 0x7fc8f006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.110+0000 7fc90e800640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 0x7fc8f006d110 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9000014a0 tx=0x7fc900009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.113+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc8fc074e00 con 0x7fc9101549e0 2024-05-10T03:33:50.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.313+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc91006a800 con 0x7fc9101549e0 2024-05-10T03:33:50.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.315+0000 7fc906a00640 1 -- 172.21.15.55:0/3916608863 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7fc8fc04c090 con 0x7fc9101549e0 2024-05-10T03:33:50.322 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:50.322 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 msgr2=0x7fc8f006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 0x7fc8f006d110 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9000014a0 tx=0x7fc900009040 comp rx=0 tx=0).stop 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 msgr2=0x7fc9100cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009720 tx=0x7fc8fc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 shutdown_connections 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc9101612f0 0x7fc9100cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9101549e0 0x7fc9100cc500 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc8f006ac00 0x7fc8f006d110 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 --2- 172.21.15.55:0/3916608863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc910071100 0x7fc910143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.318+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 >> 172.21.15.55:0/3916608863 conn(0x7fc91006a110 msgr2=0x7fc910071510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:50.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.319+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 shutdown_connections 2024-05-10T03:33:50.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.319+0000 7fc914a00640 1 -- 172.21.15.55:0/3916608863 wait complete. 2024-05-10T03:33:50.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:33:33.167391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:33:33.995610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:50.349 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:50.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.637+0000 7f1fd3e00640 1 Processor -- start 2024-05-10T03:33:50.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.638+0000 7f1fd3e00640 1 -- start start 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.640+0000 7f1fd3e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1fd40719f0 0x7f1fd406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.640+0000 7f1fd3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1fd406b840 0x7f1fd40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.640+0000 7f1fd3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd415fdf0 con 0x7f1fd4071e00 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.640+0000 7f1fd3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd415f1e0 con 0x7f1fd406b840 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.640+0000 7f1fd3e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd415f4c0 con 0x7f1fd40719f0 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1fd406b840 0x7f1fd40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1fd406b840 0x7f1fd40700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57344/0 (socket says 172.21.15.55:57344) 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 -- 172.21.15.55:0/3693746618 learned_addr learned my addr 172.21.15.55:0/3693746618 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2a00640 1 --2- 172.21.15.55:0/3693746618 >> v2:172.21.15.162:3300/0 conn(0x7f1fd40719f0 0x7f1fd406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 -- 172.21.15.55:0/3693746618 >> v2:172.21.15.162:3300/0 conn(0x7f1fd40719f0 msgr2=0x7f1fd406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 --2- 172.21.15.55:0/3693746618 >> v2:172.21.15.162:3300/0 conn(0x7f1fd40719f0 0x7f1fd406b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 -- 172.21.15.55:0/3693746618 >> v1:172.21.15.162:6789/0 conn(0x7f1fd4071e00 legacy=0x7f1fd4070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:50.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.641+0000 7f1fd2000640 1 -- 172.21.15.55:0/3693746618 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd415f7a0 con 0x7f1fd406b840 2024-05-10T03:33:50.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.642+0000 7f1fd2000640 1 --2- 172.21.15.55:0/3693746618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd406b840 0x7f1fd40700e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40b41f0 tx=0x7f1fc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af27e13e09cc96c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd1600640 1 -- 172.21.15.55:0/3693746618 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0032440 con 0x7f1fd406b840 2024-05-10T03:33:50.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd1600640 1 -- 172.21.15.55:0/3693746618 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc00325a0 con 0x7f1fd406b840 2024-05-10T03:33:50.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd1600640 1 -- 172.21.15.55:0/3693746618 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0032930 con 0x7f1fd406b840 2024-05-10T03:33:50.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 -- 172.21.15.55:0/3693746618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd406b840 msgr2=0x7f1fd40700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/3693746618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd406b840 0x7f1fd40700e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40b41f0 tx=0x7f1fc002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 -- 172.21.15.55:0/3693746618 shutdown_connections 2024-05-10T03:33:50.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/3693746618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd406b840 0x7f1fd40700e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/3693746618 >> v2:172.21.15.162:3300/0 conn(0x7f1fd40719f0 0x7f1fd406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.643+0000 7f1fd3e00640 1 -- 172.21.15.55:0/3693746618 >> 172.21.15.55:0/3693746618 conn(0x7f1fd406c750 msgr2=0x7f1fd40b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:50.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.644+0000 7f1fd3e00640 1 -- 172.21.15.55:0/3693746618 shutdown_connections 2024-05-10T03:33:50.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.644+0000 7f1fd3e00640 1 -- 172.21.15.55:0/3693746618 wait complete. 2024-05-10T03:33:50.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.646+0000 7f1fd3e00640 1 Processor -- start 2024-05-10T03:33:50.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.646+0000 7f1fd3e00640 1 -- start start 2024-05-10T03:33:50.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd406b840 0x7f1fd413d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 0x7f1fd413b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41f4830 con 0x7f1fd4071e00 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd413df10 con 0x7f1fd406b840 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd413ed70 con 0x7f1fd40719f0 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57346/0 (socket says 172.21.15.55:57346) 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 -- 172.21.15.55:0/2386857346 learned_addr learned my addr 172.21.15.55:0/2386857346 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd2a00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd406b840 0x7f1fd413d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd2000640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 0x7f1fd413b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 -- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 msgr2=0x7f1fd413b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 0x7f1fd413b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 -- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd406b840 msgr2=0x7f1fd413d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd406b840 0x7f1fd413d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.648+0000 7f1fd3400640 1 -- 172.21.15.55:0/2386857346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00094d0 con 0x7f1fd4071e00 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fd2000640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 0x7f1fd413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:50.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fd3400640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b9e0 tx=0x7f1fcc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00cb20 con 0x7f1fd4071e00 2024-05-10T03:33:50.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00cc80 con 0x7f1fd4071e00 2024-05-10T03:33:50.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc014680 con 0x7f1fd4071e00 2024-05-10T03:33:50.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd413e160 con 0x7f1fd4071e00 2024-05-10T03:33:50.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.649+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd413e720 con 0x7f1fd4071e00 2024-05-10T03:33:50.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.650+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd413eee0 con 0x7f1fd4071e00 2024-05-10T03:33:50.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.651+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1fcc01d3b0 con 0x7f1fd4071e00 2024-05-10T03:33:50.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.652+0000 7f1fcaa00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 0x7f1fb406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:50.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.653+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1fcc089430 con 0x7f1fd4071e00 2024-05-10T03:33:50.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.653+0000 7f1fd2a00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 0x7f1fb406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:50.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.654+0000 7f1fd2a00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 0x7f1fb406d110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006fd0 tx=0x7f1fc4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:50.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.659+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1fcc054430 con 0x7f1fd4071e00 2024-05-10T03:33:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.860+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fd404e290 con 0x7f1fd4071e00 2024-05-10T03:33:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.861+0000 7f1fcaa00640 1 -- 172.21.15.55:0/2386857346 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fcc057a50 con 0x7f1fd4071e00 2024-05-10T03:33:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.863+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 msgr2=0x7f1fb406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.863+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 0x7f1fb406d110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4006fd0 tx=0x7f1fc4008040 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.863+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 msgr2=0x7f1fd40b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.863+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b9e0 tx=0x7f1fcc00bea0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 shutdown_connections 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fd4071e00 0x7f1fd40b2ea0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fd40719f0 0x7f1fd413b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1fb406ac00 0x7f1fb406d110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 --2- 172.21.15.55:0/2386857346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd406b840 0x7f1fd413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 >> 172.21.15.55:0/2386857346 conn(0x7f1fd406c750 msgr2=0x7f1fd415e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:50.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 shutdown_connections 2024-05-10T03:33:50.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:50.864+0000 7f1fd3e00640 1 -- 172.21.15.55:0/2386857346 wait complete. 2024-05-10T03:33:50.893 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:51.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.183+0000 7f0f41400640 1 Processor -- start 2024-05-10T03:33:51.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.183+0000 7f0f41400640 1 -- start start 2024-05-10T03:33:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.185+0000 7f0f41400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f3c0719f0 0x7f0f3c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.185+0000 7f0f41400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 0x7f0f3c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.185+0000 7f0f41400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3c15fdf0 con 0x7f0f3c071e00 2024-05-10T03:33:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.185+0000 7f0f41400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3c15f1e0 con 0x7f0f3c0719f0 2024-05-10T03:33:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.185+0000 7f0f41400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c15f4c0 con 0x7f0f3c06b840 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f40a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0f3c071e00 0x7f0f3c070620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48766/0 (socket says 172.21.15.55:48766) 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f40a00640 1 -- 172.21.15.55:0/644867858 learned_addr learned my addr 172.21.15.55:0/644867858 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f3be00640 1 --2- 172.21.15.55:0/644867858 >> v2:172.21.15.162:3300/0 conn(0x7f0f3c0719f0 0x7f0f3c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f33e00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 0x7f0f3c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640787751 0 0) 0x7f0f3c15fdf0 con 0x7f0f3c071e00 2024-05-10T03:33:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.186+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f20003520 con 0x7f0f3c071e00 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433475510 0 0) 0x7f0f20003520 con 0x7f0f3c071e00 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 msgr2=0x7f0f3c0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 0x7f0f3c0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 >> v2:172.21.15.162:3300/0 conn(0x7f0f3c0719f0 msgr2=0x7f0f3c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 --2- 172.21.15.55:0/644867858 >> v2:172.21.15.162:3300/0 conn(0x7f0f3c0719f0 0x7f0f3c06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c15f7a0 con 0x7f0f3c071e00 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0f340019f0 con 0x7f0f3c071e00 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 >> v1:172.21.15.55:6789/0 conn(0x7f0f3c071e00 legacy=0x7f0f3c070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200039e0 0x7f0f20003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 0x7f0f200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f20004d00 0x7f0f20005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c15f1e0 con 0x7f0f200039e0 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c15f4c0 con 0x7f0f20004d00 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2000bab0 con 0x7f0f3c06b840 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.187+0000 7f0f3be00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200039e0 0x7f0f20003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f40a00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f20004d00 0x7f0f20005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f3be00640 1 -- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 msgr2=0x7f0f200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f3be00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c06b840 0x7f0f200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f3be00640 1 -- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f20004d00 msgr2=0x7f0f20005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f3be00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f20004d00 0x7f0f20005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f3be00640 1 -- 172.21.15.55:0/644867858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20003520 con 0x7f0f200039e0 2024-05-10T03:33:51.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.188+0000 7f0f40a00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f20004d00 0x7f0f20005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.189+0000 7f0f3be00640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200039e0 0x7f0f20003e50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009ac0 tx=0x7f0f200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd84d6e7c261a6a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.189+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c031400 con 0x7f0f200039e0 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.189+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c02d050 con 0x7f0f200039e0 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.189+0000 7f0f3b400640 1 -- 172.21.15.55:0/644867858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c031780 con 0x7f0f200039e0 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.190+0000 7f0f41400640 1 -- 172.21.15.55:0/644867858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200039e0 msgr2=0x7f0f20003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.190+0000 7f0f41400640 1 --2- 172.21.15.55:0/644867858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f200039e0 0x7f0f20003e50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009ac0 tx=0x7f0f200057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.190+0000 7f0f40a00640 1 -- 172.21.15.55:0/644867858 reap_dead start 2024-05-10T03:33:51.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.190+0000 7f0f41400640 1 -- 172.21.15.55:0/644867858 shutdown_connections 2024-05-10T03:33:51.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.190+0000 7f0f41400640 1 -- 172.21.15.55:0/644867858 >> 172.21.15.55:0/644867858 conn(0x7f0f3c06c750 msgr2=0x7f0f3c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:51.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.191+0000 7f0f41400640 1 -- 172.21.15.55:0/644867858 shutdown_connections 2024-05-10T03:33:51.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.191+0000 7f0f41400640 1 -- 172.21.15.55:0/644867858 wait complete. 2024-05-10T03:33:51.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.193+0000 7f0f41400640 1 Processor -- start 2024-05-10T03:33:51.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.193+0000 7f0f41400640 1 -- start start 2024-05-10T03:33:51.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f200039e0 0x7f0f3c13eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c13f820 0x7f0f3c0a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c0c3df0 con 0x7f0f3c1f2130 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c0a93f0 con 0x7f0f200039e0 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f41400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1f0e40 con 0x7f0f3c13f820 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57358/0 (socket says 172.21.15.55:57358) 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 -- 172.21.15.55:0/131549947 learned_addr learned my addr 172.21.15.55:0/131549947 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 -- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c13f820 msgr2=0x7f0f3c0a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c13f820 0x7f0f3c0a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 -- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f200039e0 msgr2=0x7f0f3c13eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f200039e0 0x7f0f3c13eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.195+0000 7f0f33e00640 1 -- 172.21.15.55:0/131549947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c13f090 con 0x7f0f3c1f2130 2024-05-10T03:33:51.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f33e00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400e780 tx=0x7f0f2400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400ee90 con 0x7f0f3c1f2130 2024-05-10T03:33:51.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2400c940 con 0x7f0f3c1f2130 2024-05-10T03:33:51.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400ccf0 con 0x7f0f3c1f2130 2024-05-10T03:33:51.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c1f1ca0 con 0x7f0f3c1f2130 2024-05-10T03:33:51.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.196+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c0a8f50 con 0x7f0f3c1f2130 2024-05-10T03:33:51.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.197+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c04e290 con 0x7f0f3c1f2130 2024-05-10T03:33:51.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.198+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0f24013470 con 0x7f0f3c1f2130 2024-05-10T03:33:51.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.199+0000 7f0f38c00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 0x7f0f1806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.199+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0f24089640 con 0x7f0f3c1f2130 2024-05-10T03:33:51.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.200+0000 7f0f3be00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 0x7f0f1806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.201+0000 7f0f3be00640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 0x7f0f1806d140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c033000 tx=0x7f0f2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.204+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0f24054640 con 0x7f0f3c1f2130 2024-05-10T03:33:51.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.393+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f3c071480 con 0x7f0f3c1f2130 2024-05-10T03:33:51.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.394+0000 7f0f38c00640 1 -- 172.21.15.55:0/131549947 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f24057c60 con 0x7f0f3c1f2130 2024-05-10T03:33:51.397 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:51.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 msgr2=0x7f0f1806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 0x7f0f1806d140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c033000 tx=0x7f0f2c02e000 comp rx=0 tx=0).stop 2024-05-10T03:33:51.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 msgr2=0x7f0f3c0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400e780 tx=0x7f0f2400ec40 comp rx=0 tx=0).stop 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 shutdown_connections 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f3c13f820 0x7f0f3c0a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f3c1f2130 0x7f0f3c0a84d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0f1806ac30 0x7f0f1806d140 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 --2- 172.21.15.55:0/131549947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f200039e0 0x7f0f3c13eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 >> 172.21.15.55:0/131549947 conn(0x7f0f3c06c750 msgr2=0x7f0f3c0705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 shutdown_connections 2024-05-10T03:33:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.397+0000 7f0f41400640 1 -- 172.21.15.55:0/131549947 wait complete. 2024-05-10T03:33:51.426 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.710+0000 7f0cb9800640 1 Processor -- start 2024-05-10T03:33:51.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.711+0000 7f0cb9800640 1 -- start start 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb9800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0cb415b9d0 0x7f0cb406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb9800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 0x7f0cb413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb9800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb415bde0 con 0x7f0cb413e390 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb9800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb413e7a0 con 0x7f0cb415b9d0 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb9800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb413eae0 con 0x7f0cb406b660 2024-05-10T03:33:51.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb3e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0cb413e390 0x7f0cb41414c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48772/0 (socket says 172.21.15.55:48772) 2024-05-10T03:33:51.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.713+0000 7f0cb3e00640 1 -- 172.21.15.55:0/1176900538 learned_addr learned my addr 172.21.15.55:0/1176900538 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.714+0000 7f0cab400640 1 --2- 172.21.15.55:0/1176900538 >> v2:172.21.15.162:3300/0 conn(0x7f0cb415b9d0 0x7f0cb406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.714+0000 7f0cb8e00640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 0x7f0cb413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.714+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843772783 0 0) 0x7f0ca4001110 con 0x7f0cb413e390 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.714+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c98003520 con 0x7f0cb413e390 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635036788 0 0) 0x7f0c98003520 con 0x7f0cb413e390 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 msgr2=0x7f0cb413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 0x7f0cb413ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 >> v2:172.21.15.162:3300/0 conn(0x7f0cb415b9d0 msgr2=0x7f0cb406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 --2- 172.21.15.55:0/1176900538 >> v2:172.21.15.162:3300/0 conn(0x7f0cb415b9d0 0x7f0cb406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb406ba70 con 0x7f0cb413e390 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0ca4001800 con 0x7f0cb413e390 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 >> v1:172.21.15.55:6789/0 conn(0x7f0cb413e390 legacy=0x7f0cb41414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c98003970 0x7f0c98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c980047c0 0x7f0c98004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 0x7f0c98005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb413e7a0 con 0x7f0c980047c0 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb413eae0 con 0x7f0c98003970 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9800ba30 con 0x7f0cb406b660 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cab400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c980047c0 0x7f0c98004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.715+0000 7f0cb3e00640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c98003970 0x7f0c98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cab400640 1 -- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 msgr2=0x7f0c98005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cab400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cb406b660 0x7f0c98005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cab400640 1 -- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c98003970 msgr2=0x7f0c98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cab400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c98003970 0x7f0c98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cab400640 1 -- 172.21.15.55:0/1176900538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98003520 con 0x7f0c980047c0 2024-05-10T03:33:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.716+0000 7f0cb3e00640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c98003970 0x7f0c98003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:51.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.717+0000 7f0cab400640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c980047c0 0x7f0c98004bd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0cb406e290 tx=0x7f0c980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cd9d9ab118a56e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.717+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c005990 con 0x7f0c980047c0 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.717+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c02d050 con 0x7f0c980047c0 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.717+0000 7f0cb3400640 1 -- 172.21.15.55:0/1176900538 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c005cf0 con 0x7f0c980047c0 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 -- 172.21.15.55:0/1176900538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c980047c0 msgr2=0x7f0c98004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 --2- 172.21.15.55:0/1176900538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c980047c0 0x7f0c98004bd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0cb406e290 tx=0x7f0c980057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb8e00640 1 -- 172.21.15.55:0/1176900538 reap_dead start 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 -- 172.21.15.55:0/1176900538 shutdown_connections 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 -- 172.21.15.55:0/1176900538 >> 172.21.15.55:0/1176900538 conn(0x7f0cb406c730 msgr2=0x7f0cb40b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 -- 172.21.15.55:0/1176900538 shutdown_connections 2024-05-10T03:33:51.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.718+0000 7f0cb9800640 1 -- 172.21.15.55:0/1176900538 wait complete. 2024-05-10T03:33:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.720+0000 7f0cb9800640 1 Processor -- start 2024-05-10T03:33:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.721+0000 7f0cb9800640 1 -- start start 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c980047c0 0x7f0cb41e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cb41e4cb0 0x7f0cb415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41f4820 con 0x7f0cb413a5e0 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41e51b0 con 0x7f0cb41e4cb0 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.722+0000 7f0cb9800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41e6010 con 0x7f0c980047c0 2024-05-10T03:33:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57370/0 (socket says 172.21.15.55:57370) 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 -- 172.21.15.55:0/3545376808 learned_addr learned my addr 172.21.15.55:0/3545376808 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb3e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c980047c0 0x7f0cb41e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cab400640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cb41e4cb0 0x7f0cb415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 -- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c980047c0 msgr2=0x7f0cb41e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c980047c0 0x7f0cb41e64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 -- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cb41e4cb0 msgr2=0x7f0cb415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cb41e4cb0 0x7f0cb415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.723+0000 7f0cb8e00640 1 -- 172.21.15.55:0/3545376808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb41e5400 con 0x7f0cb413a5e0 2024-05-10T03:33:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb8e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0cb406fcc0 tx=0x7f0cac00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac010040 con 0x7f0cb413a5e0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00bae0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00be70 con 0x7f0cb413a5e0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb41e56e0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.724+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb41e5ca0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.725+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb40680c0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.726+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0cac01e410 con 0x7f0cb413a5e0 2024-05-10T03:33:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.727+0000 7f0cb0c00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 0x7f0c9006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.727+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0cac089840 con 0x7f0cb413a5e0 2024-05-10T03:33:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.728+0000 7f0cb3e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 0x7f0c9006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:51.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.729+0000 7f0cb3e00640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 0x7f0c9006d110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4001930 tx=0x7f0ca4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:51.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.732+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0cac0547b0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.930+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0cb41e6180 con 0x7f0cb413a5e0 2024-05-10T03:33:51.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.931+0000 7f0cb0c00640 1 -- 172.21.15.55:0/3545376808 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0cac057dd0 con 0x7f0cb413a5e0 2024-05-10T03:33:51.934 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 msgr2=0x7f0c9006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 0x7f0c9006d110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4001930 tx=0x7f0ca4009040 comp rx=0 tx=0).stop 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 msgr2=0x7f0cb40ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0cb406fcc0 tx=0x7f0cac00b480 comp rx=0 tx=0).stop 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 shutdown_connections 2024-05-10T03:33:51.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cb413a5e0 0x7f0cb40ac8e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0c9006ac00 0x7f0c9006d110 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cb41e4cb0 0x7f0cb415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 --2- 172.21.15.55:0/3545376808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c980047c0 0x7f0cb41e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.934+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 >> 172.21.15.55:0/3545376808 conn(0x7f0cb406c730 msgr2=0x7f0cb4071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.935+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 shutdown_connections 2024-05-10T03:33:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:51.935+0000 7f0cb9800640 1 -- 172.21.15.55:0/3545376808 wait complete. 2024-05-10T03:33:51.966 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:33:52.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.245+0000 7f2f7fe00640 1 Processor -- start 2024-05-10T03:33:52.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.245+0000 7f2f7fe00640 1 -- start start 2024-05-10T03:33:52.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8006b830 0x7f2f80164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800cec90 con 0x7f2f8015adc0 2024-05-10T03:33:52.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800b4330 con 0x7f2f800ce880 2024-05-10T03:33:52.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015a0d0 con 0x7f2f8006b830 2024-05-10T03:33:52.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2f8015adc0 0x7f2f8015e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:48780/0 (socket says 172.21.15.55:48780) 2024-05-10T03:33:52.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.248+0000 7f2f7ea00640 1 -- 172.21.15.55:0/2004687399 learned_addr learned my addr 172.21.15.55:0/2004687399 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.249+0000 7f2f7f400640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8006b830 0x7f2f80164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.249+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276188603 0 0) 0x7f2f800cec90 con 0x7f2f8015adc0 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.249+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f64003520 con 0x7f2f8015adc0 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.249+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2526902866 0 0) 0x7f2f64003520 con 0x7f2f8015adc0 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8006b830 msgr2=0x7f2f80164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:52.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8006b830 0x7f2f80164840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 >> v1:172.21.15.162:6789/0 conn(0x7f2f800ce880 legacy=0x7f2f8006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8015a2e0 con 0x7f2f8015adc0 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2f70001670 con 0x7f2f8015adc0 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 >> v1:172.21.15.55:6789/0 conn(0x7f2f8015adc0 legacy=0x7f2f8015e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f640039a0 0x7f2f64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f640047c0 0x7f2f64004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f64005110 0x7f2f6400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8015a0d0 con 0x7f2f64005110 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6400e100 con 0x7f2f640039a0 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6400e2c0 con 0x7f2f640047c0 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7f400640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f64005110 0x7f2f6400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f77e00640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f640047c0 0x7f2f64004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7ea00640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f640039a0 0x7f2f64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7f400640 1 -- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f640047c0 msgr2=0x7f2f64004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7f400640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f640047c0 0x7f2f64004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7f400640 1 -- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f640039a0 msgr2=0x7f2f64003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.250+0000 7f2f7f400640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f640039a0 0x7f2f64003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.251+0000 7f2f7f400640 1 -- 172.21.15.55:0/2004687399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64003520 con 0x7f2f64005110 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.251+0000 7f2f77e00640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f640047c0 0x7f2f64004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.251+0000 7f2f7ea00640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f640039a0 0x7f2f64003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7f400640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f64005110 0x7f2f6400db90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800a250 tx=0x7f2f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c8ac566632bf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7802c040 con 0x7f2f64005110 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78032610 con 0x7f2f64005110 2024-05-10T03:33:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7e000640 1 -- 172.21.15.55:0/2004687399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f780329a0 con 0x7f2f64005110 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7fe00640 1 -- 172.21.15.55:0/2004687399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f64005110 msgr2=0x7f2f6400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.252+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/2004687399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f64005110 0x7f2f6400db90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800a250 tx=0x7f2f640057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.253+0000 7f2f7f400640 1 -- 172.21.15.55:0/2004687399 reap_dead start 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.253+0000 7f2f7fe00640 1 -- 172.21.15.55:0/2004687399 shutdown_connections 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.253+0000 7f2f7fe00640 1 -- 172.21.15.55:0/2004687399 >> 172.21.15.55:0/2004687399 conn(0x7f2f8006c730 msgr2=0x7f2f800b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.253+0000 7f2f7fe00640 1 -- 172.21.15.55:0/2004687399 shutdown_connections 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.253+0000 7f2f7fe00640 1 -- 172.21.15.55:0/2004687399 wait complete. 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.255+0000 7f2f7fe00640 1 Processor -- start 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.255+0000 7f2f7fe00640 1 -- start start 2024-05-10T03:33:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f64005110 0x7f2f8014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 0x7f2f801ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80159480 con 0x7f2f801520b0 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8014d610 con 0x7f2f801524c0 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f7fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801efb80 con 0x7f2f64005110 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f77e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f77e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57392/0 (socket says 172.21.15.55:57392) 2024-05-10T03:33:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.257+0000 7f2f77e00640 1 -- 172.21.15.55:0/1048157143 learned_addr learned my addr 172.21.15.55:0/1048157143 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f7ea00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f64005110 0x7f2f8014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f7f400640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 0x7f2f801ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 -- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f64005110 msgr2=0x7f2f8014da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f64005110 0x7f2f8014da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 -- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 msgr2=0x7f2f801ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 0x7f2f801ef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 -- 172.21.15.55:0/1048157143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8014c740 con 0x7f2f801520b0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f7f400640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 0x7f2f801ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77e00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d7d0 tx=0x7f2f6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68017070 con 0x7f2f801520b0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.258+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68005070 con 0x7f2f801520b0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.259+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8014c520 con 0x7f2f801520b0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.259+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68005400 con 0x7f2f801520b0 2024-05-10T03:33:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.259+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8014cc70 con 0x7f2f801520b0 2024-05-10T03:33:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.260+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8014d880 con 0x7f2f801520b0 2024-05-10T03:33:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.261+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f68006820 con 0x7f2f801520b0 2024-05-10T03:33:52.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.262+0000 7f2f77400640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 0x7f2f5c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.262+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2f68088d90 con 0x7f2f801520b0 2024-05-10T03:33:52.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.263+0000 7f2f7ea00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 0x7f2f5c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.264+0000 7f2f7ea00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 0x7f2f5c06d110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2f70001200 tx=0x7f2f70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.267+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f68053da0 con 0x7f2f801520b0 2024-05-10T03:33:52.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.482+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2f8004e290 con 0x7f2f801520b0 2024-05-10T03:33:52.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.483+0000 7f2f77400640 1 -- 172.21.15.55:0/1048157143 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2f680573c0 con 0x7f2f801520b0 2024-05-10T03:33:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 msgr2=0x7f2f5c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 0x7f2f5c06d110 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2f70001200 tx=0x7f2f70009040 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 msgr2=0x7f2f801eedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d7d0 tx=0x7f2f6800dc90 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 shutdown_connections 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f5c06ac00 0x7f2f5c06d110 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f801524c0 0x7f2f801ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f801520b0 0x7f2f801eedf0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 --2- 172.21.15.55:0/1048157143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f64005110 0x7f2f8014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 >> 172.21.15.55:0/1048157143 conn(0x7f2f8006c730 msgr2=0x7f2f800b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:52.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 shutdown_connections 2024-05-10T03:33:52.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.486+0000 7f2f7fe00640 1 -- 172.21.15.55:0/1048157143 wait complete. 2024-05-10T03:33:52.515 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:33:52.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:52.515 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:33:52.616 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:33:52.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:33:52.616 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:33:52.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:52.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.957+0000 7fc17b200640 1 Processor -- start 2024-05-10T03:33:52.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.957+0000 7fc17b200640 1 -- start start 2024-05-10T03:33:52.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.959+0000 7fc17b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc17406a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.959+0000 7fc17b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc174071cf0 0x7fc1740b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.959+0000 7fc17b200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740cd710 con 0x7fc17415ab30 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc17b200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1740cdfb0 con 0x7fc174071cf0 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc17b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17406b4d0 con 0x7fc1740cdba0 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc179200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc174071cf0 0x7fc1740b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc173400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc17415ab30 0x7fc1740715e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35280/0 (socket says 172.21.15.55:35280) 2024-05-10T03:33:52.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc173400640 1 -- 172.21.15.55:0/884206628 learned_addr learned my addr 172.21.15.55:0/884206628 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc173e00640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc17406a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc179200640 1 -- 172.21.15.55:0/884206628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 msgr2=0x7fc17406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.960+0000 7fc179200640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc17406a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.961+0000 7fc179200640 1 -- 172.21.15.55:0/884206628 >> v1:172.21.15.55:6789/0 conn(0x7fc17415ab30 legacy=0x7fc1740715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.961+0000 7fc179200640 1 -- 172.21.15.55:0/884206628 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17406b1f0 con 0x7fc174071cf0 2024-05-10T03:33:52.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.961+0000 7fc172a00640 1 -- 172.21.15.55:0/884206628 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118600973 0 0) 0x7fc1740cd710 con 0x7fc17415ab30 2024-05-10T03:33:52.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.962+0000 7fc179200640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc174071cf0 0x7fc1740b2fa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0097a0 tx=0x7fc16c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d8c1da69db3c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.962+0000 7fc172a00640 1 -- 172.21.15.55:0/884206628 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c02adb0 con 0x7fc174071cf0 2024-05-10T03:33:52.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.962+0000 7fc172a00640 1 -- 172.21.15.55:0/884206628 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c032440 con 0x7fc174071cf0 2024-05-10T03:33:52.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.962+0000 7fc172a00640 1 -- 172.21.15.55:0/884206628 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0327d0 con 0x7fc174071cf0 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 -- 172.21.15.55:0/884206628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc174071cf0 msgr2=0x7fc1740b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc174071cf0 0x7fc1740b2fa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0097a0 tx=0x7fc16c02a060 comp rx=0 tx=0).stop 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 -- 172.21.15.55:0/884206628 shutdown_connections 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc174071cf0 0x7fc1740b2fa0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 --2- 172.21.15.55:0/884206628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc17406a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 -- 172.21.15.55:0/884206628 >> 172.21.15.55:0/884206628 conn(0x7fc1740cbb90 msgr2=0x7fc1740cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 -- 172.21.15.55:0/884206628 shutdown_connections 2024-05-10T03:33:52.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.963+0000 7fc17b200640 1 -- 172.21.15.55:0/884206628 wait complete. 2024-05-10T03:33:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.966+0000 7fc17b200640 1 Processor -- start 2024-05-10T03:33:52.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.966+0000 7fc17b200640 1 -- start start 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc174151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17415ab30 0x7fc17413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17414c300 con 0x7fc17415dec0 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc174151920 con 0x7fc17415ab30 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc17b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc174152780 con 0x7fc1740cdba0 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc179200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc179200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34472/0 (socket says 172.21.15.55:34472) 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc179200640 1 -- 172.21.15.55:0/555977671 learned_addr learned my addr 172.21.15.55:0/555977671 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc173e00640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc174151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.968+0000 7fc173400640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17415ab30 0x7fc17413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 -- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 msgr2=0x7fc174151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc174151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 -- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17415ab30 msgr2=0x7fc17413ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17415ab30 0x7fc17413ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 -- 172.21.15.55:0/555977671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009400 con 0x7fc17415dec0 2024-05-10T03:33:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc179200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098c0 tx=0x7fc16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c03d850 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c03d9b0 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c03dd40 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.969+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc174152c10 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.970+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc174151e50 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.971+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc174066f60 con 0x7fc17415dec0 2024-05-10T03:33:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.972+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc16c039040 con 0x7fc17415dec0 2024-05-10T03:33:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.973+0000 7fc16be00640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 0x7fc15406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.973+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc16c0a9f30 con 0x7fc17415dec0 2024-05-10T03:33:52.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.974+0000 7fc173e00640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 0x7fc15406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.975+0000 7fc173e00640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 0x7fc15406d140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc1640099c0 tx=0x7fc164003990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:52.978+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc16c074ea0 con 0x7fc17415dec0 2024-05-10T03:33:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.219+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc174067220 con 0x7fc17415dec0 2024-05-10T03:33:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.220+0000 7fc16be00640 1 -- 172.21.15.55:0/555977671 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fc16c0784c0 con 0x7fc17415dec0 2024-05-10T03:33:53.223 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:53.223 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 msgr2=0x7fc15406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 0x7fc15406d140 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc1640099c0 tx=0x7fc164003990 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 msgr2=0x7fc174158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0098c0 tx=0x7fc16c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 shutdown_connections 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc17415dec0 0x7fc174158e10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc15406ac30 0x7fc15406d140 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17415ab30 0x7fc17413ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 --2- 172.21.15.55:0/555977671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1740cdba0 0x7fc174151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.222+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 >> 172.21.15.55:0/555977671 conn(0x7fc1740cbb90 msgr2=0x7fc17406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.223+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 shutdown_connections 2024-05-10T03:33:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.223+0000 7fc17b200640 1 -- 172.21.15.55:0/555977671 wait complete. 2024-05-10T03:33:53.252 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-05-10T03:33:53.555 INFO:teuthology.orchestra.run.smithi162.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":65,"msgr_send_messages":59,"msgr_recv_bytes":115203,"msgr_send_bytes":16682,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.021480190,"msgr_running_send_time":0.002118311,"msgr_running_recv_time":0.017127577,"msgr_running_fast_dispatch_time":0.002063605,"msgr_send_messages_queue_lat":{"avgcount":59,"sum":0.142629737,"avgtime":0.002417453},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":112112,"msgr_send_encrypted_bytes":11344},"AsyncMessenger::Worker-1":{"msgr_recv_messages":40,"msgr_send_messages":43,"msgr_recv_bytes":10675,"msgr_send_bytes":127156,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.017515946,"msgr_running_send_time":0.004883011,"msgr_running_recv_time":0.012195489,"msgr_running_fast_dispatch_time":0.000532649,"msgr_send_messages_queue_lat":{"avgcount":41,"sum":0.037528484,"avgtime":0.000915328},"msgr_handle_ack_lat":{"avgcount":27,"sum":0.000019240,"avgtime":0.000000712},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":92976},"AsyncMessenger::Worker-2":{"msgr_recv_messages":38,"msgr_send_messages":33,"msgr_recv_bytes":10533,"msgr_send_bytes":21919,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.015557611,"msgr_running_send_time":0.002647713,"msgr_running_recv_time":0.009054739,"msgr_running_fast_dispatch_time":0.003429997,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.090288718,"avgtime":0.002821522},"msgr_handle_ack_lat":{"avgcount":4,"sum":0.000002910,"avgtime":0.000000727},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":33,"sum":0.004554117,"avgtime":0.000138003}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000022626,"avgtime":0.000011313}},"mds":{"request":26,"reply":26,"reply_latency":{"avgcount":26,"sum":0.011737876,"avgtime":0.000451456},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":51,"traverse_hit":51,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196981,"rdpos":0,"jlat":{"avgcount":5,"sum":0.022098572,"avgtime":0.004419714},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":107520,"heap":207224},"mds_server":{"dispatch_client_request":26,"dispatch_server_request":0,"handle_client_request":26,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":25,"sum":0.011073256,"avgtime":0.000442930},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000664620,"avgtime":0.000664620},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":24,"average_load":12,"avg_session_uptime":4,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":219565,"buffer_anon_items":87,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52240,"mds_co_items":213,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":13887,"op_resend":30,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.145976410,"avgtime":0.026044918},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":78,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":140,"get_sum":120836,"get_or_fail_fail":0,"get_or_fail_success":140,"take":0,"take_sum":0,"put":140,"put_sum":120836,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":13887,"put":44,"put_sum":13887,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555c25036340":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2677,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2677,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-10T03:33:53.556 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 115203, "msgr_recv_encrypted_bytes": 112112, "msgr_recv_messages": 65, "msgr_running_fast_dispatch_time": 0.002063605, "msgr_running_recv_time": 0.017127577, "msgr_running_send_time": 0.002118311, "msgr_running_total_time": 0.02148019, "msgr_send_bytes": 16682, "msgr_send_encrypted_bytes": 11344, "msgr_send_messages": 59, "msgr_send_messages_queue_lat": { "avgcount": 59, "avgtime": 0.002417453, "sum": 0.142629737 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 27, "avgtime": 7.12e-07, "sum": 1.924e-05 }, "msgr_recv_bytes": 10675, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 40, "msgr_running_fast_dispatch_time": 0.000532649, "msgr_running_recv_time": 0.012195489, "msgr_running_send_time": 0.004883011, "msgr_running_total_time": 0.017515946, "msgr_send_bytes": 127156, "msgr_send_encrypted_bytes": 92976, "msgr_send_messages": 43, "msgr_send_messages_queue_lat": { "avgcount": 41, "avgtime": 0.000915328, "sum": 0.037528484 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 4, "avgtime": 7.27e-07, "sum": 2.91e-06 }, "msgr_recv_bytes": 10533, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 38, "msgr_running_fast_dispatch_time": 0.003429997, "msgr_running_recv_time": 0.009054739, "msgr_running_send_time": 0.002647713, "msgr_running_total_time": 0.015557611, "msgr_send_bytes": 21919, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.002821522, "sum": 0.090288718 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 33, "avgtime": 0.000138003, "sum": 0.004554117 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.1313e-05, "sum": 2.2626e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 26, "reply_latency": { "avgcount": 26, "avgtime": 0.000451456, "sum": 0.011737876 }, "request": 26, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 51, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 51, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.004419714, "sum": 0.022098572 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4196981 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207224, "ino": 13, "ino+": 13, "ino-": 0, "rss": 107520 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 26, "dispatch_server_request": 0, "handle_client_request": 26, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 25, "avgtime": 0.00044293, "sum": 0.011073256 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.00066462, "sum": 0.00066462 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 12, "avg_session_uptime": 4, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 24 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 219565, "buffer_anon_items": 87, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52240, "mds_co_items": 213, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 78, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.026044918, "sum": 1.14597641 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 30, "op_rmw": 0, "op_send": 44, "op_send_bytes": 13887, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 140, "get_or_fail_fail": 0, "get_or_fail_success": 140, "get_started": 0, "get_sum": 120836, "max": 104857600, "put": 140, "put_sum": 120836, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 13887, "take": 44, "take_sum": 13887, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555c25036340": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2677, "max": 3758096384, "put": 5, "put_sum": 2677, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-10T03:33:53.557 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 session ls 2024-05-10T03:33:53.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.850+0000 7ff83fe00640 1 Processor -- start 2024-05-10T03:33:53.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.850+0000 7ff83fe00640 1 -- start start 2024-05-10T03:33:53.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8400715c0 0x7ff84016c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84006d8a0 con 0x7ff8400b2c10 2024-05-10T03:33:53.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400b3020 con 0x7ff840165600 2024-05-10T03:33:53.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401663d0 con 0x7ff8400715c0 2024-05-10T03:33:53.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff8400b2c10 0x7ff84006c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35284/0 (socket says 172.21.15.55:35284) 2024-05-10T03:33:53.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83ea00640 1 -- 172.21.15.55:0/1255049607 learned_addr learned my addr 172.21.15.55:0/1255049607 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:53.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.853+0000 7ff83f400640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8400715c0 0x7ff84016c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772564985 0 0) 0x7ff84006d8a0 con 0x7ff8400b2c10 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff824003520 con 0x7ff8400b2c10 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294902610 0 0) 0x7ff8400b3020 con 0x7ff840165600 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff84006d8a0 con 0x7ff840165600 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487997463 0 0) 0x7ff824003520 con 0x7ff8400b2c10 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8400715c0 msgr2=0x7ff84016c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8400715c0 0x7ff84016c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 >> v1:172.21.15.162:6789/0 conn(0x7ff840165600 legacy=0x7ff840165a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84006cc90 con 0x7ff8400b2c10 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7ff830001670 con 0x7ff8400b2c10 2024-05-10T03:33:53.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.854+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 >> v1:172.21.15.55:6789/0 conn(0x7ff8400b2c10 legacy=0x7ff84006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff824003b30 0x7ff824003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240047c0 0x7ff824004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8240051d0 0x7ff82400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401663d0 con 0x7ff824003b30 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82400c100 con 0x7ff8240047c0 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82400c2c0 con 0x7ff8240051d0 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83f400640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8400715c0 0x7ff84016c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:33:53.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff824003b30 0x7ff824003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83e000640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240047c0 0x7ff824004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.855+0000 7ff83f400640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8240051d0 0x7ff82400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.856+0000 7ff83ea00640 1 -- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8240051d0 msgr2=0x7ff82400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.856+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8240051d0 0x7ff82400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.856+0000 7ff83ea00640 1 -- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240047c0 msgr2=0x7ff824004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.856+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8240047c0 0x7ff824004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.856+0000 7ff83ea00640 1 -- 172.21.15.55:0/1255049607 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824003520 con 0x7ff824003b30 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.857+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff824003b30 0x7ff824003f80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff840163140 tx=0x7ff8240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9043933247ddc39f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.857+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830005ad0 con 0x7ff824003b30 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.857+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83002d070 con 0x7ff824003b30 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.857+0000 7ff83d600640 1 -- 172.21.15.55:0/1255049607 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830005e90 con 0x7ff824003b30 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 -- 172.21.15.55:0/1255049607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff824003b30 msgr2=0x7ff824003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 --2- 172.21.15.55:0/1255049607 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff824003b30 0x7ff824003f80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff840163140 tx=0x7ff8240057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83f400640 1 -- 172.21.15.55:0/1255049607 reap_dead start 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 -- 172.21.15.55:0/1255049607 shutdown_connections 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 -- 172.21.15.55:0/1255049607 >> 172.21.15.55:0/1255049607 conn(0x7ff8400b1080 msgr2=0x7ff8400b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 -- 172.21.15.55:0/1255049607 shutdown_connections 2024-05-10T03:33:53.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.858+0000 7ff83fe00640 1 -- 172.21.15.55:0/1255049607 wait complete. 2024-05-10T03:33:53.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.860+0000 7ff83fe00640 1 Processor -- start 2024-05-10T03:33:53.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.861+0000 7ff83fe00640 1 -- start start 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 0x7ff8401ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8401f7840 0x7ff8401f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401ed0d0 con 0x7ff8401554a0 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401600d0 con 0x7ff8401ec780 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8401603b0 con 0x7ff8401f7840 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34494/0 (socket says 172.21.15.55:34494) 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.862+0000 7ff83ea00640 1 -- 172.21.15.55:0/240371301 learned_addr learned my addr 172.21.15.55:0/240371301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83e000640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 0x7ff8401ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83f400640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8401f7840 0x7ff8401f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 -- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8401f7840 msgr2=0x7ff8401f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8401f7840 0x7ff8401f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 -- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 msgr2=0x7ff8401ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 0x7ff8401ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 -- 172.21.15.55:0/240371301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff840160690 con 0x7ff8401554a0 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83e000640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 0x7ff8401ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:53.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff83ea00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff8300018b0 tx=0x7ff830001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.863+0000 7ff837400640 1 -- 172.21.15.55:0/240371301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830004cb0 con 0x7ff8401554a0 2024-05-10T03:33:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.864+0000 7ff837400640 1 -- 172.21.15.55:0/240371301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff830004e10 con 0x7ff8401554a0 2024-05-10T03:33:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.864+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff840160970 con 0x7ff8401554a0 2024-05-10T03:33:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.864+0000 7ff837400640 1 -- 172.21.15.55:0/240371301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830007290 con 0x7ff8401554a0 2024-05-10T03:33:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.864+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8401486c0 con 0x7ff8401554a0 2024-05-10T03:33:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.866+0000 7ff837400640 1 -- 172.21.15.55:0/240371301 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff83002f030 con 0x7ff8401554a0 2024-05-10T03:33:53.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.867+0000 7ff837400640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 0x7ff82006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.867+0000 7ff837400640 1 -- 172.21.15.55:0/240371301 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff8300a8d60 con 0x7ff8401554a0 2024-05-10T03:33:53.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.868+0000 7ff83e000640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 0x7ff82006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.868+0000 7ff84cd8c740 1 Processor -- start 2024-05-10T03:33:53.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.869+0000 7ff84cd8c740 1 -- start start 2024-05-10T03:33:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.869+0000 7ff83e000640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 0x7ff82006d110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff84015bbc0 tx=0x7ff82c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433af73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433a61c80 con 0x5614337304d0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433a5f410 con 0x5614337386b0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561433a5ad00 con 0x561433759390 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.870+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34502/0 (socket says 172.21.15.55:34502) 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 -- 172.21.15.55:0/2807576815 learned_addr learned my addr 172.21.15.55:0/2807576815 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83ea00640 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83f400640 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433af73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 -- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 msgr2=0x561433b22dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b22dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 -- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 msgr2=0x561433af73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433af73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 -- 172.21.15.55:0/2807576815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561433a0cc80 con 0x5614337304d0 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.871+0000 7ff83e000640 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007a80 tx=0x7ff82c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cd6097e7ceca99 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.872+0000 7ff82b400640 1 -- 172.21.15.55:0/2807576815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0149d0 con 0x5614337304d0 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.872+0000 7ff82b400640 1 -- 172.21.15.55:0/2807576815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00d050 con 0x5614337304d0 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.872+0000 7ff82b400640 1 -- 172.21.15.55:0/2807576815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c014d30 con 0x5614337304d0 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2807576815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 msgr2=0x561433af6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007a80 tx=0x7ff82c012000 comp rx=0 tx=0).stop 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2807576815 shutdown_connections 2024-05-10T03:33:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433af73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433af6ea0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2807576815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2807576815 >> 172.21.15.55:0/2807576815 conn(0x56143374d4e0 msgr2=0x561433b3dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2807576815 shutdown_connections 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.873+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2807576815 wait complete. 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.874+0000 7ff84cd8c740 1 -- start start 2024-05-10T03:33:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.874+0000 7ff84cd8c740 1 Processor -- start 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433b1af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614339fc1d0 con 0x5614337304d0 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614339fbe70 con 0x5614337386b0 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561433757f90 con 0x561433759390 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff84cd8c740 10 client.0 _ll_register_callbacks cb 0x561433a10790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:53.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff83ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff83ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34510/0 (socket says 172.21.15.55:34510) 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.876+0000 7ff83ea00640 1 -- 172.21.15.55:0/4000673797 learned_addr learned my addr 172.21.15.55:0/4000673797 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83f400640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 -- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 msgr2=0x561433b1ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b1ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83e000640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433b1af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 -- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 msgr2=0x561433b1af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433b1af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 -- 172.21.15.55:0/4000673797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84006d8a0 con 0x5614337304d0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff83ea00640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff8300a7140 tx=0x7ff830007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7ecdfe16139495 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff81fe00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83007fea0 con 0x5614337304d0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff83007fa10 con 0x5614337304d0 2024-05-10T03:33:53.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.877+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561433a84db0 con 0x5614337304d0 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff83007fbd0 con 0x5614337304d0 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 10 client.6771 fetch_fsmap learned FSMap version 39 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x561433a248c0 con 0x5614337304d0 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561433a84db0 con 0x5614337304d0 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff8300073f0 con 0x5614337304d0 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 10 client.6771 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 10 client.6771 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 20 client.6771 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433717f60 0x561433b1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 4 client.6771 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:33:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.878+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 --> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561433a4df30 con 0x561433717f60 2024-05-10T03:33:53.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.879+0000 7ff83e000640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433717f60 0x561433b1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.880+0000 7ff83e000640 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433717f60 0x561433b1f140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70111bca262c2196 server_cookie=89067700eac64163 in_seq=0 out_seq=0 2024-05-10T03:33:53.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.880+0000 7ff81fe00640 10 client.6771 ms_handle_connect on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff81fe00640 1 -- 172.21.15.55:0/4000673797 <== mds.0 v2:172.21.15.162:6835/3533670538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561433a4df30 con 0x561433717f60 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff81fe00640 10 client.6771 handle_command_reply: tid=0 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 client.6771 shutdown 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 msgr2=0x561433b252a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff8300a7140 tx=0x7ff830007dc0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 shutdown_connections 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433717f60 0x561433b1f140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561433759390 0x561433b1ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614337386b0 0x561433b1af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/4000673797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337304d0 0x561433b252a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff81fe00640 0 client.6771 ms_handle_reset on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 >> 172.21.15.55:0/4000673797 conn(0x56143374d4e0 msgr2=0x561433af1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 shutdown_connections 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.882+0000 7ff84cd8c740 1 -- 172.21.15.55:0/4000673797 wait complete. 2024-05-10T03:33:53.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.883+0000 7ff84cd8c740 20 client.6771 trim_cache size 0 max 16384 2024-05-10T03:33:53.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.901+0000 7ff84cd8c740 1 Processor -- start 2024-05-10T03:33:53.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.901+0000 7ff84cd8c740 1 -- start start 2024-05-10T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433af8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433b4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433b3d370 con 0x5614337386b0 2024-05-10T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433b24c30 con 0x561433759390 2024-05-10T03:33:53.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.902+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561433b3d060 con 0x5614337304d0 2024-05-10T03:33:53.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34518/0 (socket says 172.21.15.55:34518) 2024-05-10T03:33:53.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 -- 172.21.15.55:0/2341132344 learned_addr learned my addr 172.21.15.55:0/2341132344 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 -- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 msgr2=0x561433af8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433af8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 -- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 msgr2=0x561433b4be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433b4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 -- 172.21.15.55:0/2341132344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614339fb580 con 0x5614337386b0 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.903+0000 7ff83e000640 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff82c001eb0 tx=0x7ff82c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f64b6915cc476c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff82be00640 1 -- 172.21.15.55:0/2341132344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c014e70 con 0x5614337386b0 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff82be00640 1 -- 172.21.15.55:0/2341132344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c001430 con 0x5614337386b0 2024-05-10T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff82be00640 1 -- 172.21.15.55:0/2341132344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c02b650 con 0x5614337386b0 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2341132344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 msgr2=0x561433af9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff82c001eb0 tx=0x7ff82c014000 comp rx=0 tx=0).stop 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2341132344 shutdown_connections 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433b4be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433af9140 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/2341132344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433af8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.904+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2341132344 >> 172.21.15.55:0/2341132344 conn(0x56143374d4e0 msgr2=0x561433b37580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.905+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2341132344 shutdown_connections 2024-05-10T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.905+0000 7ff84cd8c740 1 -- 172.21.15.55:0/2341132344 wait complete. 2024-05-10T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.906+0000 7ff84cd8c740 1 -- start start 2024-05-10T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.906+0000 7ff84cd8c740 1 Processor -- start 2024-05-10T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.907+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433b1ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.907+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.907+0000 7ff84cd8c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433baa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433af7350 con 0x5614337386b0 2024-05-10T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561433b3c8e0 con 0x561433759390 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff84cd8c740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561433b59080 con 0x5614337304d0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff84cd8c740 10 client.0 _ll_register_callbacks cb 0x561433b23cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34524/0 (socket says 172.21.15.55:34524) 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 -- 172.21.15.55:0/1766342675 learned_addr learned my addr 172.21.15.55:0/1766342675 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83f400640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433baa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433b1ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 -- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 msgr2=0x561433b1ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433b1ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 -- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 msgr2=0x561433baa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433baa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.908+0000 7ff83e000640 1 -- 172.21.15.55:0/1766342675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561433a0cc80 con 0x5614337386b0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff83e000640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff82c020000 tx=0x7ff82c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820853d5d950cd23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff81fe00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c02b650 con 0x5614337386b0 2024-05-10T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff82c01c040 con 0x5614337386b0 2024-05-10T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56143374e4e0 con 0x5614337386b0 2024-05-10T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82c02b7b0 con 0x5614337386b0 2024-05-10T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff84cd8c740 10 client.6777 fetch_fsmap learned FSMap version 39 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.909+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x561433a20a90 con 0x5614337386b0 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56143374e4e0 con 0x5614337386b0 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff82c012070 con 0x5614337386b0 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 10 client.6777 fetch_fsmap finished waiting for FSMap version 39 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 10 client.6777 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 20 client.6777 populate_metadata read hostname 'smithi055' 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433b8f260 0x561433b22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 4 client.6777 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 --> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] -- command(tid 0: {"prefix": "session ls"}) v1 -- 0x56143375f7e0 con 0x561433b8f260 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.910+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433b8f260 0x561433b22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff83ea00640 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433b8f260 0x561433b22dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1807178f131b56a2 server_cookie=6a32e258bc8a6c42 in_seq=0 out_seq=0 2024-05-10T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff81fe00640 10 client.6777 ms_handle_connect on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:53.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff81fe00640 1 -- 172.21.15.55:0/1766342675 <== mds.0 v2:172.21.15.162:6835/3533670538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56143375f7e0 con 0x561433b8f260 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff81fe00640 10 client.6777 handle_command_reply: tid=0 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 client.6777 shutdown 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 msgr2=0x561433baa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff82c020000 tx=0x7ff82c0012d0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 shutdown_connections 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:6835/3533670538,v1:172.21.15.162:6837/3533670538] conn(0x561433b8f260 0x561433b22dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561433759390 0x561433baa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5614337386b0 0x561433baa0b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 --2- 172.21.15.55:0/1766342675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5614337304d0 0x561433b1ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff81fe00640 0 client.6777 ms_handle_reset on v2:172.21.15.162:6835/3533670538 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.912+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 >> 172.21.15.55:0/1766342675 conn(0x56143374d4e0 msgr2=0x561433af9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 shutdown_connections 2024-05-10T03:33:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff84cd8c740 1 -- 172.21.15.55:0/1766342675 wait complete. 2024-05-10T03:33:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff84cd8c740 20 client.6777 trim_cache size 0 max 16384 2024-05-10T03:33:53.918 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 msgr2=0x7ff82006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 0x7ff82006d110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff84015bbc0 tx=0x7ff82c009040 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 msgr2=0x7ff840148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.913+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff8300018b0 tx=0x7ff830001980 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 shutdown_connections 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8401f7840 0x7ff8401f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff82006ac00 0x7ff82006d110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8401ec780 0x7ff8401ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 --2- 172.21.15.55:0/240371301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff8401554a0 0x7ff840148180 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 >> 172.21.15.55:0/240371301 conn(0x7ff8400b1080 msgr2=0x7ff840164420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 shutdown_connections 2024-05-10T03:33:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:53.914+0000 7ff83fe00640 1 -- 172.21.15.55:0/240371301 wait complete. 2024-05-10T03:33:53.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-05-10T03:33:54.246 INFO:teuthology.orchestra.run.smithi162.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":65,"msgr_send_messages":59,"msgr_recv_bytes":115203,"msgr_send_bytes":16682,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.021953902,"msgr_running_send_time":0.002118311,"msgr_running_recv_time":0.017127577,"msgr_running_fast_dispatch_time":0.002063605,"msgr_send_messages_queue_lat":{"avgcount":59,"sum":0.142629737,"avgtime":0.002417453},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":112112,"msgr_send_encrypted_bytes":11344},"AsyncMessenger::Worker-1":{"msgr_recv_messages":42,"msgr_send_messages":45,"msgr_recv_bytes":10965,"msgr_send_bytes":146428,"msgr_created_connections":10,"msgr_active_connections":3,"msgr_running_total_time":0.020410042,"msgr_running_send_time":0.005157145,"msgr_running_recv_time":0.014544136,"msgr_running_fast_dispatch_time":0.000532649,"msgr_send_messages_queue_lat":{"avgcount":43,"sum":0.037675951,"avgtime":0.000876184},"msgr_handle_ack_lat":{"avgcount":29,"sum":0.000019853,"avgtime":0.000000684},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":92976},"AsyncMessenger::Worker-2":{"msgr_recv_messages":38,"msgr_send_messages":33,"msgr_recv_bytes":10533,"msgr_send_bytes":21919,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.015557611,"msgr_running_send_time":0.002647713,"msgr_running_recv_time":0.009054739,"msgr_running_fast_dispatch_time":0.003429997,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.090288718,"avgtime":0.002821522},"msgr_handle_ack_lat":{"avgcount":4,"sum":0.000002910,"avgtime":0.000000727},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":33,"sum":0.004554117,"avgtime":0.000138003}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000022626,"avgtime":0.000011313}},"mds":{"request":26,"reply":26,"reply_latency":{"avgcount":26,"sum":0.011737876,"avgtime":0.000451456},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":2,"subtrees":2,"traverse":51,"traverse_hit":51,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196981,"rdpos":0,"jlat":{"avgcount":5,"sum":0.022098572,"avgtime":0.004419714},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":107648,"heap":207224},"mds_server":{"dispatch_client_request":26,"dispatch_server_request":0,"handle_client_request":26,"handle_client_session":6,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":25,"sum":0.011073256,"avgtime":0.000442930},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000664620,"avgtime":0.000664620},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":24,"average_load":12,"avg_session_uptime":4,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":219565,"buffer_anon_items":87,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5424,"osdmap_items":76,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52240,"mds_co_items":213,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":13887,"op_resend":30,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.145976410,"avgtime":0.026044918},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":78,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":142,"get_sum":120946,"get_or_fail_fail":0,"get_or_fail_success":142,"take":0,"take_sum":0,"put":142,"put_sum":120946,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":13887,"put":44,"put_sum":13887,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x555c25036340":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2677,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2677,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-05-10T03:33:54.247 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 115203, "msgr_recv_encrypted_bytes": 112112, "msgr_recv_messages": 65, "msgr_running_fast_dispatch_time": 0.002063605, "msgr_running_recv_time": 0.017127577, "msgr_running_send_time": 0.002118311, "msgr_running_total_time": 0.021953902, "msgr_send_bytes": 16682, "msgr_send_encrypted_bytes": 11344, "msgr_send_messages": 59, "msgr_send_messages_queue_lat": { "avgcount": 59, "avgtime": 0.002417453, "sum": 0.142629737 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 10, "msgr_handle_ack_lat": { "avgcount": 29, "avgtime": 6.84e-07, "sum": 1.9853e-05 }, "msgr_recv_bytes": 10965, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 42, "msgr_running_fast_dispatch_time": 0.000532649, "msgr_running_recv_time": 0.014544136, "msgr_running_send_time": 0.005157145, "msgr_running_total_time": 0.020410042, "msgr_send_bytes": 146428, "msgr_send_encrypted_bytes": 92976, "msgr_send_messages": 45, "msgr_send_messages_queue_lat": { "avgcount": 43, "avgtime": 0.000876184, "sum": 0.037675951 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 4, "avgtime": 7.27e-07, "sum": 2.91e-06 }, "msgr_recv_bytes": 10533, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 38, "msgr_running_fast_dispatch_time": 0.003429997, "msgr_running_recv_time": 0.009054739, "msgr_running_send_time": 0.002647713, "msgr_running_total_time": 0.015557611, "msgr_send_bytes": 21919, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.002821522, "sum": 0.090288718 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 33, "avgtime": 0.000138003, "sum": 0.004554117 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.1313e-05, "sum": 2.2626e-05 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 26, "reply_latency": { "avgcount": 26, "avgtime": 0.000451456, "sum": 0.011737876 }, "request": 26, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 51, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 51, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.004419714, "sum": 0.022098572 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4196981 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 207224, "ino": 13, "ino+": 13, "ino-": 0, "rss": 107648 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 26, "dispatch_server_request": 0, "handle_client_request": 26, "handle_client_session": 6, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 25, "avgtime": 0.00044293, "sum": 0.011073256 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.00066462, "sum": 0.00066462 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 12, "avg_session_uptime": 4, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 24 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 219565, "buffer_anon_items": 87, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52240, "mds_co_items": 213, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5424, "osdmap_items": 76, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 78, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.026044918, "sum": 1.14597641 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 30, "op_rmw": 0, "op_send": 44, "op_send_bytes": 13887, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 142, "get_or_fail_fail": 0, "get_or_fail_success": 142, "get_started": 0, "get_sum": 120946, "max": 104857600, "put": 142, "put_sum": 120946, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 13887, "take": 44, "take_sum": 13887, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x555c25036340": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2677, "max": 3758096384, "put": 5, "put_sum": 2677, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-05-10T03:33:54.247 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-10T03:33:54.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:33:54.248 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:54.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:33:54.476 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:33:54.476 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:54.476 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:54.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:33:54.538 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:54.538 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:54.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:33:54.595 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:33:54.595 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:54.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:33:54.655 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:33:54.655 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:33:54.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:33:54.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:55.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.005+0000 7f8afe800640 1 Processor -- start 2024-05-10T03:33:55.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.005+0000 7f8afe800640 1 -- start start 2024-05-10T03:33:55.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8afe800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af815fe50 0x7f8af8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8afe800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 0x7f8af806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8afe800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af806ad20 con 0x7f8af80b2140 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8afe800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af8141f50 con 0x7f8af815fe50 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8afe800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af806a110 con 0x7f8af80b2550 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8af7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af815fe50 0x7f8af8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8af7400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af815fe50 0x7f8af8141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34534/0 (socket says 172.21.15.55:34534) 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.008+0000 7f8af7400640 1 -- 172.21.15.55:0/2673466231 learned_addr learned my addr 172.21.15.55:0/2673466231 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7e00640 1 --2- 172.21.15.55:0/2673466231 >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 0x7f8af806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7400640 1 -- 172.21.15.55:0/2673466231 >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 msgr2=0x7f8af806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7400640 1 --2- 172.21.15.55:0/2673466231 >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 0x7f8af806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7400640 1 -- 172.21.15.55:0/2673466231 >> v1:172.21.15.162:6789/0 conn(0x7f8af80b2140 legacy=0x7f8af8142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7400640 1 -- 172.21.15.55:0/2673466231 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af806a3f0 con 0x7f8af815fe50 2024-05-10T03:33:55.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.009+0000 7f8af7e00640 1 --2- 172.21.15.55:0/2673466231 >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 0x7f8af806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.010+0000 7f8af7400640 1 --2- 172.21.15.55:0/2673466231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815fe50 0x7f8af8141a10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8af806d130 tx=0x7f8ae802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6746d3dc98e4c56b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.010+0000 7f8af6000640 1 -- 172.21.15.55:0/2673466231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8032440 con 0x7f8af815fe50 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.010+0000 7f8af6000640 1 -- 172.21.15.55:0/2673466231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae80325a0 con 0x7f8af815fe50 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.010+0000 7f8af6000640 1 -- 172.21.15.55:0/2673466231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8032930 con 0x7f8af815fe50 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 -- 172.21.15.55:0/2673466231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815fe50 msgr2=0x7f8af8141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 --2- 172.21.15.55:0/2673466231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815fe50 0x7f8af8141a10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8af806d130 tx=0x7f8ae802a2c0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 -- 172.21.15.55:0/2673466231 shutdown_connections 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 --2- 172.21.15.55:0/2673466231 >> v2:172.21.15.162:3300/0 conn(0x7f8af80b2550 0x7f8af806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 --2- 172.21.15.55:0/2673466231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815fe50 0x7f8af8141a10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 -- 172.21.15.55:0/2673466231 >> 172.21.15.55:0/2673466231 conn(0x7f8af80cbb90 msgr2=0x7f8af80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 -- 172.21.15.55:0/2673466231 shutdown_connections 2024-05-10T03:33:55.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.011+0000 7f8afe800640 1 -- 172.21.15.55:0/2673466231 wait complete. 2024-05-10T03:33:55.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.014+0000 7f8afe800640 1 Processor -- start 2024-05-10T03:33:55.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.014+0000 7f8afe800640 1 -- start start 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 0x7f8af81f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af815fe50 0x7f8af81f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80addd0 con 0x7f8af80b2140 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8070ab0 con 0x7f8af80b2550 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.015+0000 7f8afe800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8070d00 con 0x7f8af815fe50 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34548/0 (socket says 172.21.15.55:34548) 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 -- 172.21.15.55:0/1020534867 learned_addr learned my addr 172.21.15.55:0/1020534867 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af6a00640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 0x7f8af81f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 -- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af815fe50 msgr2=0x7f8af81f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7e00640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af815fe50 0x7f8af81f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af815fe50 0x7f8af81f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 -- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 msgr2=0x7f8af81f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 0x7f8af81f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af7400640 1 -- 172.21.15.55:0/1020534867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80094d0 con 0x7f8af80b2140 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.016+0000 7f8af6a00640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 0x7f8af81f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8af7400640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000c00 tx=0x7f8ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8032440 con 0x7f8af80b2140 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8030b80 con 0x7f8af80b2140 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af8070fe0 con 0x7f8af80b2140 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae803d420 con 0x7f8af80b2140 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.017+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af80715a0 con 0x7f8af80b2140 2024-05-10T03:33:55.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.018+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80aec20 con 0x7f8af80b2140 2024-05-10T03:33:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.019+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8ae8039040 con 0x7f8af80b2140 2024-05-10T03:33:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.021+0000 7f8aef400640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 0x7f8ad806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.021+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8ae80aa150 con 0x7f8af80b2140 2024-05-10T03:33:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.022+0000 7f8af6a00640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 0x7f8ad806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.023+0000 7f8af6a00640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 0x7f8ad806d160 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8af806b750 tx=0x7f8ae4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.026+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8ae8075240 con 0x7f8af80b2140 2024-05-10T03:33:55.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.262+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8af8071ac0 con 0x7f8af80b2140 2024-05-10T03:33:55.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.263+0000 7f8aef400640 1 -- 172.21.15.55:0/1020534867 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f8af8071ac0 con 0x7f8af80b2140 2024-05-10T03:33:55.266 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:55.266 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:55.267 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:55.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.265+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 msgr2=0x7f8ad806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.265+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 0x7f8ad806d160 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8af806b750 tx=0x7f8ae4008040 comp rx=0 tx=0).stop 2024-05-10T03:33:55.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.265+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 msgr2=0x7f8af8070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.265+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000c00 tx=0x7f8ae80046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 shutdown_connections 2024-05-10T03:33:55.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8af815fe50 0x7f8af81f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8ad806ac50 0x7f8ad806d160 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8af80b2550 0x7f8af81f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 --2- 172.21.15.55:0/1020534867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af80b2140 0x7f8af8070540 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 >> 172.21.15.55:0/1020534867 conn(0x7f8af80cbb90 msgr2=0x7f8af815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 shutdown_connections 2024-05-10T03:33:55.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.266+0000 7f8afe800640 1 -- 172.21.15.55:0/1020534867 wait complete. 2024-05-10T03:33:55.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:55.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.573+0000 7f5c2ca00640 1 Processor -- start 2024-05-10T03:33:55.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.573+0000 7f5c2ca00640 1 -- start start 2024-05-10T03:33:55.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.575+0000 7f5c2ca00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5c2806d300 0x7f5c280b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.575+0000 7f5c2ca00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2815d740 con 0x7f5c2815e190 2024-05-10T03:33:55.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.575+0000 7f5c2ca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280b2a20 con 0x7f5c280b1f60 2024-05-10T03:33:55.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.575+0000 7f5c2ca00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280b2d00 con 0x7f5c2806d300 2024-05-10T03:33:55.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c27400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c2815e190 0x7f5c280ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35292/0 (socket says 172.21.15.55:35292) 2024-05-10T03:33:55.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c27400640 1 -- 172.21.15.55:0/1106716123 learned_addr learned my addr 172.21.15.55:0/1106716123 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815599059 0 0) 0x7f5c2815d740 con 0x7f5c2815e190 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c27e00640 1 --2- 172.21.15.55:0/1106716123 >> v2:172.21.15.162:3300/0 conn(0x7f5c2806d300 0x7f5c280b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c003440 con 0x7f5c2815e190 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1446777492 0 0) 0x7f5c0c003440 con 0x7f5c2815e190 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 >> v2:172.21.15.162:3300/0 conn(0x7f5c2806d300 msgr2=0x7f5c280b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 --2- 172.21.15.55:0/1106716123 >> v2:172.21.15.162:3300/0 conn(0x7f5c2806d300 0x7f5c280b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.576+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 >> v1:172.21.15.162:6789/0 conn(0x7f5c280b1f60 legacy=0x7f5c2806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2815d990 con 0x7f5c2815e190 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f5c180019f0 con 0x7f5c2815e190 2024-05-10T03:33:55.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 >> v1:172.21.15.55:6789/0 conn(0x7f5c2815e190 legacy=0x7f5c280ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 0x7f5c0c003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0c008950 0x7f5c0c00ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c00b6d0 0x7f5c0c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b2d00 con 0x7f5c0c003ad0 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c00e4c0 con 0x7f5c0c00b6d0 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c00e680 con 0x7f5c0c008950 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c27400640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 0x7f5c0c003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c27e00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c00b6d0 0x7f5c0c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c27400640 1 -- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0c008950 msgr2=0x7f5c0c00ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.577+0000 7f5c1ea00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0c008950 0x7f5c0c00ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27400640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0c008950 0x7f5c0c00ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27400640 1 -- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c00b6d0 msgr2=0x7f5c0c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27400640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c00b6d0 0x7f5c0c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27400640 1 -- 172.21.15.55:0/1106716123 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c003440 con 0x7f5c0c003ad0 2024-05-10T03:33:55.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27e00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c00b6d0 0x7f5c0c00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:55.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c27e00640 1 -- 172.21.15.55:0/1106716123 reap_dead start 2024-05-10T03:33:55.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.578+0000 7f5c1ea00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0c008950 0x7f5c0c00ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.579+0000 7f5c27400640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 0x7f5c0c003f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5c18002230 tx=0x7f5c0c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78308899942a444 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.579+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1802ba20 con 0x7f5c0c003ad0 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.579+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1802bba0 con 0x7f5c0c003ad0 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.579+0000 7f5c26a00640 1 -- 172.21.15.55:0/1106716123 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18034470 con 0x7f5c0c003ad0 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 -- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 msgr2=0x7f5c0c003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 0x7f5c0c003f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5c18002230 tx=0x7f5c0c0036f0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 -- 172.21.15.55:0/1106716123 shutdown_connections 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/1106716123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c0c003ad0 0x7f5c0c003f20 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 -- 172.21.15.55:0/1106716123 >> 172.21.15.55:0/1106716123 conn(0x7f5c2806a110 msgr2=0x7f5c2806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 -- 172.21.15.55:0/1106716123 shutdown_connections 2024-05-10T03:33:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.580+0000 7f5c2ca00640 1 -- 172.21.15.55:0/1106716123 wait complete. 2024-05-10T03:33:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.582+0000 7f5c2ca00640 1 Processor -- start 2024-05-10T03:33:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.583+0000 7f5c2ca00640 1 -- start start 2024-05-10T03:33:55.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c003ad0 0x7f5c281520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c28139d70 0x7f5c2813a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2813a6c0 con 0x7f5c28139550 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2813ab00 con 0x7f5c0c003ad0 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c2ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281549a0 con 0x7f5c28139d70 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c1ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c1ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34566/0 (socket says 172.21.15.55:34566) 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.584+0000 7f5c1ea00640 1 -- 172.21.15.55:0/352662119 learned_addr learned my addr 172.21.15.55:0/352662119 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c27e00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c28139d70 0x7f5c2813a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c27400640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c003ad0 0x7f5c281520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 -- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c28139d70 msgr2=0x7f5c2813a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c28139d70 0x7f5c2813a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 -- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c003ad0 msgr2=0x7f5c281520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c003ad0 0x7f5c281520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 -- 172.21.15.55:0/352662119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28151c20 con 0x7f5c28139550 2024-05-10T03:33:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1ea00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d430 tx=0x7f5c1000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10007020 con 0x7f5c28139550 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.585+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100071a0 con 0x7f5c28139550 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.586+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10007550 con 0x7f5c28139550 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.586+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2814d180 con 0x7f5c28139550 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.586+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c281515d0 con 0x7f5c28139550 2024-05-10T03:33:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.587+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28151db0 con 0x7f5c28139550 2024-05-10T03:33:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.588+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c100067d0 con 0x7f5c28139550 2024-05-10T03:33:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.589+0000 7f5c1fe00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 0x7f5c0406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.589+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5c100892a0 con 0x7f5c28139550 2024-05-10T03:33:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.590+0000 7f5c27400640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 0x7f5c0406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.591+0000 7f5c27400640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 0x7f5c0406d110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c18033000 tx=0x7f5c18009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:55.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.594+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c10054340 con 0x7f5c28139550 2024-05-10T03:33:55.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.797+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c28066ce0 con 0x7f5c28139550 2024-05-10T03:33:55.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.800+0000 7f5c1fe00640 1 -- 172.21.15.55:0/352662119 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f5c10057960 con 0x7f5c28139550 2024-05-10T03:33:55.803 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:55.803 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:55.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 msgr2=0x7f5c0406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 0x7f5c0406d110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c18033000 tx=0x7f5c18009030 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 msgr2=0x7f5c2814df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000d430 tx=0x7f5c1000d8f0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 shutdown_connections 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c28139d70 0x7f5c2813a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c28139550 0x7f5c2814df90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5c0406ac00 0x7f5c0406d110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 --2- 172.21.15.55:0/352662119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0c003ad0 0x7f5c281520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.802+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 >> 172.21.15.55:0/352662119 conn(0x7f5c2806a110 msgr2=0x7f5c2806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.803+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 shutdown_connections 2024-05-10T03:33:55.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:55.803+0000 7f5c2ca00640 1 -- 172.21.15.55:0/352662119 wait complete. 2024-05-10T03:33:55.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:33:33.167391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:33:33.995610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:33:55.835 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:33:56.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.112+0000 7fbb61200640 1 Processor -- start 2024-05-10T03:33:56.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.112+0000 7fbb61200640 1 -- start start 2024-05-10T03:33:56.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb61200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb5c1546f0 0x7fbb5c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb61200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c072270 con 0x7fbb5c161e70 2024-05-10T03:33:56.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb61200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c06ba70 con 0x7fbb5c06b660 2024-05-10T03:33:56.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb61200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c161a50 con 0x7fbb5c1546f0 2024-05-10T03:33:56.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb5b400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbb5c161e70 0x7fbb5c0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35294/0 (socket says 172.21.15.55:35294) 2024-05-10T03:33:56.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.114+0000 7fbb5b400640 1 -- 172.21.15.55:0/3104707559 learned_addr learned my addr 172.21.15.55:0/3104707559 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:56.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/3104707559 >> v2:172.21.15.162:3300/0 conn(0x7fbb5c1546f0 0x7fbb5c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749903378 0 0) 0x7fbb5c072270 con 0x7fbb5c161e70 2024-05-10T03:33:56.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40003520 con 0x7fbb5c161e70 2024-05-10T03:33:56.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496618028 0 0) 0x7fbb40003520 con 0x7fbb5c161e70 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 >> v2:172.21.15.162:3300/0 conn(0x7fbb5c1546f0 msgr2=0x7fbb5c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 --2- 172.21.15.55:0/3104707559 >> v2:172.21.15.162:3300/0 conn(0x7fbb5c1546f0 0x7fbb5c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 >> v1:172.21.15.162:6789/0 conn(0x7fbb5c06b660 legacy=0x7fbb5c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c071f90 con 0x7fbb5c161e70 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbb4c001670 con 0x7fbb5c161e70 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.115+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 >> v1:172.21.15.55:6789/0 conn(0x7fbb5c161e70 legacy=0x7fbb5c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb40003a10 0x7fbb40003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 0x7fbb40004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb40005310 0x7fbb4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c161a50 con 0x7fbb400047c0 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4000c240 con 0x7fbb40005310 2024-05-10T03:33:56.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4000c400 con 0x7fbb40003a10 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 0x7fbb40004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5b400640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb40003a10 0x7fbb40003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 -- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb40003a10 msgr2=0x7fbb40003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb40003a10 0x7fbb40003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 -- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb40005310 msgr2=0x7fbb4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb40005310 0x7fbb4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5aa00640 1 -- 172.21.15.55:0/3104707559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40003520 con 0x7fbb400047c0 2024-05-10T03:33:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.116+0000 7fbb5be00640 1 -- 172.21.15.55:0/3104707559 reap_dead start 2024-05-10T03:33:56.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.117+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 0x7fbb40004bd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b2160 tx=0x7fbb400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71088da90cf82a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440315f0 con 0x7fbb400047c0 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4400f040 con 0x7fbb400047c0 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb5a000640 1 -- 172.21.15.55:0/3104707559 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44031950 con 0x7fbb400047c0 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb61200640 1 -- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 msgr2=0x7fbb40004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb61200640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 0x7fbb40004bd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b2160 tx=0x7fbb400057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb61200640 1 -- 172.21.15.55:0/3104707559 shutdown_connections 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb61200640 1 --2- 172.21.15.55:0/3104707559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb400047c0 0x7fbb40004bd0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.118+0000 7fbb61200640 1 -- 172.21.15.55:0/3104707559 >> 172.21.15.55:0/3104707559 conn(0x7fbb5c06c750 msgr2=0x7fbb5c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.119+0000 7fbb61200640 1 -- 172.21.15.55:0/3104707559 shutdown_connections 2024-05-10T03:33:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.119+0000 7fbb61200640 1 -- 172.21.15.55:0/3104707559 wait complete. 2024-05-10T03:33:56.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.121+0000 7fbb61200640 1 Processor -- start 2024-05-10T03:33:56.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.121+0000 7fbb61200640 1 -- start start 2024-05-10T03:33:56.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400047c0 0x7fbb5c13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb5c13a5e0 0x7fbb5c1e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c13b650 con 0x7fbb5c1e9600 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c1e9a10 con 0x7fbb5c13a5e0 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb61200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c1e8020 con 0x7fbb400047c0 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34586/0 (socket says 172.21.15.55:34586) 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb5be00640 1 -- 172.21.15.55:0/4092928182 learned_addr learned my addr 172.21.15.55:0/4092928182 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb5b400640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400047c0 0x7fbb5c13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.123+0000 7fbb5aa00640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb5c13a5e0 0x7fbb5c1e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 -- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400047c0 msgr2=0x7fbb5c13ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400047c0 0x7fbb5c13ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 -- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb5c13a5e0 msgr2=0x7fbb5c1e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb5c13a5e0 0x7fbb5c1e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 -- 172.21.15.55:0/4092928182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c1e8300 con 0x7fbb5c1e9600 2024-05-10T03:33:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb5be00640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b0b30 tx=0x7fbb5400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540074e0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54007660 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54005a00 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.124+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5c1e85e0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.125+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c1e8ba0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.126+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5c1e8d30 con 0x7fbb5c1e9600 2024-05-10T03:33:56.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.127+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbb54005b60 con 0x7fbb5c1e9600 2024-05-10T03:33:56.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.128+0000 7fbb53400640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 0x7fbb3806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.128+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb540887d0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.129+0000 7fbb5b400640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 0x7fbb3806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.130+0000 7fbb5b400640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 0x7fbb3806d110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0017a0 tx=0x7fbb4c001470 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.132+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbb540537d0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.327+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb5c071890 con 0x7fbb5c1e9600 2024-05-10T03:33:56.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.328+0000 7fbb53400640 1 -- 172.21.15.55:0/4092928182 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb5400cea0 con 0x7fbb5c1e9600 2024-05-10T03:33:56.331 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.330+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 msgr2=0x7fbb3806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.330+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 0x7fbb3806d110 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0017a0 tx=0x7fbb4c001470 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.330+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 msgr2=0x7fbb5c155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.330+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b0b30 tx=0x7fbb5400c820 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 shutdown_connections 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb5c1e9600 0x7fbb5c155cd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbb3806ac00 0x7fbb3806d110 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb5c13a5e0 0x7fbb5c1e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 --2- 172.21.15.55:0/4092928182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb400047c0 0x7fbb5c13ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 >> 172.21.15.55:0/4092928182 conn(0x7fbb5c06c750 msgr2=0x7fbb5c160620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 shutdown_connections 2024-05-10T03:33:56.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.331+0000 7fbb61200640 1 -- 172.21.15.55:0/4092928182 wait complete. 2024-05-10T03:33:56.362 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:33:56.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.636+0000 7fb2e2e00640 1 Processor -- start 2024-05-10T03:33:56.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.636+0000 7fb2e2e00640 1 -- start start 2024-05-10T03:33:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.638+0000 7fb2e2e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 0x7fb2dc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.638+0000 7fb2e2e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.638+0000 7fb2e2e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc0b2a30 con 0x7fb2dc06b120 2024-05-10T03:33:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e2e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc15a900 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e2e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc15eec0 con 0x7fb2dc15e920 2024-05-10T03:33:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 0x7fb2dc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e2400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 0x7fb2dc0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:45568/0 (socket says 172.21.15.55:45568) 2024-05-10T03:33:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e2400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:54546/0 (socket says 172.21.15.55:54546) 2024-05-10T03:33:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.639+0000 7fb2e2400640 1 -- 172.21.15.55:0/1122516403 learned_addr learned my addr 172.21.15.55:0/1122516403 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.640+0000 7fb2e2400640 1 -- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 msgr2=0x7fb2dc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.640+0000 7fb2e2400640 1 --2- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 0x7fb2dc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.640+0000 7fb2e2400640 1 -- 172.21.15.55:0/1122516403 >> v1:172.21.15.162:6789/0 conn(0x7fb2dc06b120 legacy=0x7fb2dc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.640+0000 7fb2e2400640 1 -- 172.21.15.55:0/1122516403 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc15b8b0 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.642+0000 7fb2e2400640 1 --2- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009e50 tx=0x7fb2d402c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219edcf182fc1818 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.642+0000 7fb2dbe00640 1 -- 172.21.15.55:0/1122516403 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4032610 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.642+0000 7fb2dbe00640 1 -- 172.21.15.55:0/1122516403 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d402e040 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.642+0000 7fb2dbe00640 1 -- 172.21.15.55:0/1122516403 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4032990 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 -- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc0cdb70 msgr2=0x7fb2dc0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009e50 tx=0x7fb2d402c920 comp rx=0 tx=0).stop 2024-05-10T03:33:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 -- 172.21.15.55:0/1122516403 shutdown_connections 2024-05-10T03:33:56.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc0cdb70 0x7fb2dc0cdf80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/1122516403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc15e920 0x7fb2dc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.643+0000 7fb2e2e00640 1 -- 172.21.15.55:0/1122516403 >> 172.21.15.55:0/1122516403 conn(0x7fb2dc06c750 msgr2=0x7fb2dc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.644+0000 7fb2e2e00640 1 -- 172.21.15.55:0/1122516403 shutdown_connections 2024-05-10T03:33:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.644+0000 7fb2e2e00640 1 -- 172.21.15.55:0/1122516403 wait complete. 2024-05-10T03:33:56.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.646+0000 7fb2e2e00640 1 Processor -- start 2024-05-10T03:33:56.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.646+0000 7fb2e2e00640 1 -- start start 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc0cdb70 0x7fb2dc1f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc15e920 0x7fb2dc1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1f4dc0 con 0x7fb2dc06b120 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1f4aa0 con 0x7fb2dc15e920 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e2e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1f4fd0 con 0x7fb2dc0cdb70 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34600/0 (socket says 172.21.15.55:34600) 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 -- 172.21.15.55:0/3821555077 learned_addr learned my addr 172.21.15.55:0/3821555077 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 -- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc0cdb70 msgr2=0x7fb2dc1f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc0cdb70 0x7fb2dc1f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 -- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc15e920 msgr2=0x7fb2dc1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc15e920 0x7fb2dc1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.648+0000 7fb2e1a00640 1 -- 172.21.15.55:0/3821555077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4009b30 con 0x7fb2dc06b120 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2e1a00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e870 tx=0x7fb2d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00094f0 con 0x7fb2dc06b120 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000c940 con 0x7fb2dc06b120 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000ccf0 con 0x7fb2dc06b120 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc1f52b0 con 0x7fb2dc06b120 2024-05-10T03:33:56.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.649+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc1f5870 con 0x7fb2dc06b120 2024-05-10T03:33:56.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.650+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc04e290 con 0x7fb2dc06b120 2024-05-10T03:33:56.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.651+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb2d0013470 con 0x7fb2dc06b120 2024-05-10T03:33:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.652+0000 7fb2d9600640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 0x7fb2bc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.652+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2d00896b0 con 0x7fb2dc06b120 2024-05-10T03:33:56.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.653+0000 7fb2e1000640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 0x7fb2bc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:56.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.655+0000 7fb2e1000640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 0x7fb2bc06d110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000f60 tx=0x7fb2c80079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:56.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.663+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb2d0054620 con 0x7fb2dc06b120 2024-05-10T03:33:56.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.866+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2dc13ed50 con 0x7fb2dc06b120 2024-05-10T03:33:56.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.867+0000 7fb2d9600640 1 -- 172.21.15.55:0/3821555077 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2d0057c40 con 0x7fb2dc06b120 2024-05-10T03:33:56.870 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.869+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 msgr2=0x7fb2bc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.869+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 0x7fb2bc06d110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000f60 tx=0x7fb2c80079b0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.869+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 msgr2=0x7fb2dc159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.869+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e870 tx=0x7fb2d000ed30 comp rx=0 tx=0).stop 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 shutdown_connections 2024-05-10T03:33:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb2bc06ac00 0x7fb2bc06d110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2dc15e920 0x7fb2dc1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2dc0cdb70 0x7fb2dc1f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 --2- 172.21.15.55:0/3821555077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2dc06b120 0x7fb2dc159ba0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 >> 172.21.15.55:0/3821555077 conn(0x7fb2dc06c750 msgr2=0x7fb2dc0cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 shutdown_connections 2024-05-10T03:33:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:56.870+0000 7fb2e2e00640 1 -- 172.21.15.55:0/3821555077 wait complete. 2024-05-10T03:33:56.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:33:57.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.171+0000 7f9387200640 1 Processor -- start 2024-05-10T03:33:57.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.171+0000 7f9387200640 1 -- start start 2024-05-10T03:33:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.173+0000 7f9387200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93881612f0 0x7f938815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.173+0000 7f9387200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938806b520 0x7f93880ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.173+0000 7f9387200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93880cea20 con 0x7f93880ce610 2024-05-10T03:33:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.173+0000 7f9387200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938815dc80 con 0x7f93881612f0 2024-05-10T03:33:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.173+0000 7f9387200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93880cfa00 con 0x7f938806b520 2024-05-10T03:33:57.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.174+0000 7f9385e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93881612f0 0x7f938815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.174+0000 7f9385400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938806b520 0x7f93880ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.174+0000 7f9385e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f93881612f0 0x7f938815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:54560/0 (socket says 172.21.15.55:54560) 2024-05-10T03:33:57.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.174+0000 7f9385e00640 1 -- 172.21.15.55:0/3584079901 learned_addr learned my addr 172.21.15.55:0/3584079901 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.175+0000 7f9385e00640 1 -- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938806b520 msgr2=0x7f93880ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.175+0000 7f9385e00640 1 --2- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938806b520 0x7f93880ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.175+0000 7f9385e00640 1 -- 172.21.15.55:0/3584079901 >> v1:172.21.15.162:6789/0 conn(0x7f93880ce610 legacy=0x7f93880b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.175+0000 7f9385e00640 1 -- 172.21.15.55:0/3584079901 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938806a480 con 0x7f93881612f0 2024-05-10T03:33:57.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.176+0000 7f9385e00640 1 --2- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93881612f0 0x7f938815d740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93780098a0 tx=0x7f937802a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ea42b23dd9afde server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.177+0000 7f9384a00640 1 -- 172.21.15.55:0/3584079901 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378032440 con 0x7f93881612f0 2024-05-10T03:33:57.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.177+0000 7f9384a00640 1 -- 172.21.15.55:0/3584079901 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93780325a0 con 0x7f93881612f0 2024-05-10T03:33:57.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.177+0000 7f9384a00640 1 -- 172.21.15.55:0/3584079901 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378032930 con 0x7f93881612f0 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 -- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93881612f0 msgr2=0x7f938815d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 --2- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93881612f0 0x7f938815d740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93780098a0 tx=0x7f937802a290 comp rx=0 tx=0).stop 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 -- 172.21.15.55:0/3584079901 shutdown_connections 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 --2- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938806b520 0x7f93880ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 --2- 172.21.15.55:0/3584079901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93881612f0 0x7f938815d740 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 -- 172.21.15.55:0/3584079901 >> 172.21.15.55:0/3584079901 conn(0x7f93880d0830 msgr2=0x7f93880d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 -- 172.21.15.55:0/3584079901 shutdown_connections 2024-05-10T03:33:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.178+0000 7f9387200640 1 -- 172.21.15.55:0/3584079901 wait complete. 2024-05-10T03:33:57.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.181+0000 7f9387200640 1 Processor -- start 2024-05-10T03:33:57.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.181+0000 7f9387200640 1 -- start start 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 0x7f9388160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93881612f0 0x7f93881f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388150020 con 0x7f938806b520 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880cd600 con 0x7f93880ce610 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.183+0000 7f9387200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93881f4450 con 0x7f93881612f0 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34604/0 (socket says 172.21.15.55:34604) 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 -- 172.21.15.55:0/216964198 learned_addr learned my addr 172.21.15.55:0/216964198 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:57.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385400640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 0x7f9388160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9386800640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93881612f0 0x7f93881f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 -- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93881612f0 msgr2=0x7f93881f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93881612f0 0x7f93881f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 -- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 msgr2=0x7f9388160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 0x7f9388160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385e00640 1 -- 172.21.15.55:0/216964198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780094d0 con 0x7f938806b520 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.184+0000 7f9385400640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 0x7f9388160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f9385e00640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f937802a880 tx=0x7f93780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937803d8f0 con 0x7f938806b520 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937803da70 con 0x7f938806b520 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937803de20 con 0x7f938806b520 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9388150190 con 0x7f938806b520 2024-05-10T03:33:57.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.185+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881506c0 con 0x7f938806b520 2024-05-10T03:33:57.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.186+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938804e290 con 0x7f938806b520 2024-05-10T03:33:57.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.187+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9378039040 con 0x7f938806b520 2024-05-10T03:33:57.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.188+0000 7f937e000640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 0x7f936406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.188+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f93780a9fc0 con 0x7f938806b520 2024-05-10T03:33:57.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.189+0000 7f9385400640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 0x7f936406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.190+0000 7f9385400640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 0x7f936406d110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93881608d0 tx=0x7f937000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.194+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9378074fc0 con 0x7f938806b520 2024-05-10T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.391+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93880d1690 con 0x7f938806b520 2024-05-10T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.391+0000 7f937e000640 1 -- 172.21.15.55:0/216964198 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93780785e0 con 0x7f938806b520 2024-05-10T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 msgr2=0x7f936406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 0x7f936406d110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93881608d0 tx=0x7f937000b040 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 msgr2=0x7f93881455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f937802a880 tx=0x7f93780046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 shutdown_connections 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93881612f0 0x7f93881f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f936406ac00 0x7f936406d110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880ce610 0x7f9388160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 --2- 172.21.15.55:0/216964198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938806b520 0x7f93881455a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 >> 172.21.15.55:0/216964198 conn(0x7f93880d0830 msgr2=0x7f938815b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 shutdown_connections 2024-05-10T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.395+0000 7f9387200640 1 -- 172.21.15.55:0/216964198 wait complete. 2024-05-10T03:33:57.427 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:57.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.703+0000 7f5751400640 1 Processor -- start 2024-05-10T03:33:57.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.703+0000 7f5751400640 1 -- start start 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5751400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f574c1545d0 0x7f574c0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5751400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0b2890 con 0x7f574c0b4960 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5751400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0ceb20 con 0x7f574c06b830 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5751400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c15b280 con 0x7f574c1545d0 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f574a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f574c1545d0 0x7f574c0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5741e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f574c0b4960 0x7f574c06b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35300/0 (socket says 172.21.15.55:35300) 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.706+0000 7f5741e00640 1 -- 172.21.15.55:0/4041184101 learned_addr learned my addr 172.21.15.55:0/4041184101 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.707+0000 7f574a800640 1 -- 172.21.15.55:0/4041184101 >> v1:172.21.15.162:6789/0 conn(0x7f574c06b830 legacy=0x7f574c06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.707+0000 7f574a800640 1 -- 172.21.15.55:0/4041184101 >> v1:172.21.15.55:6789/0 conn(0x7f574c0b4960 legacy=0x7f574c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.707+0000 7f574a800640 1 -- 172.21.15.55:0/4041184101 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c15a600 con 0x7f574c1545d0 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.707+0000 7f5749e00640 1 -- 172.21.15.55:0/4041184101 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061757106 0 0) 0x7f574c0b2890 con 0x7f574c0b4960 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.708+0000 7f574a800640 1 --2- 172.21.15.55:0/4041184101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c0b4420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f573c0097a0 tx=0x7f573c02b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c67303ce390497 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.708+0000 7f5749e00640 1 -- 172.21.15.55:0/4041184101 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c033440 con 0x7f574c1545d0 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.708+0000 7f5749e00640 1 -- 172.21.15.55:0/4041184101 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c0335a0 con 0x7f574c1545d0 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.708+0000 7f5749e00640 1 -- 172.21.15.55:0/4041184101 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c033930 con 0x7f574c1545d0 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 -- 172.21.15.55:0/4041184101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 msgr2=0x7f574c0b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 --2- 172.21.15.55:0/4041184101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c0b4420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f573c0097a0 tx=0x7f573c02b2f0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 -- 172.21.15.55:0/4041184101 shutdown_connections 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 --2- 172.21.15.55:0/4041184101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c0b4420 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 -- 172.21.15.55:0/4041184101 >> 172.21.15.55:0/4041184101 conn(0x7f574c070900 msgr2=0x7f574c070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:57.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 -- 172.21.15.55:0/4041184101 shutdown_connections 2024-05-10T03:33:57.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.709+0000 7f5751400640 1 -- 172.21.15.55:0/4041184101 wait complete. 2024-05-10T03:33:57.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.712+0000 7f5751400640 1 Processor -- start 2024-05-10T03:33:57.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.712+0000 7f5751400640 1 -- start start 2024-05-10T03:33:57.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 0x7f574c1e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 0x7f574c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c1583b0 con 0x7f574c1545d0 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0ad5b0 con 0x7f574c0b4960 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5751400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0ad800 con 0x7f574c157c40 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34626/0 (socket says 172.21.15.55:34626) 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 -- 172.21.15.55:0/851076653 learned_addr learned my addr 172.21.15.55:0/851076653 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f574a800640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 0x7f574c1e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 -- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 msgr2=0x7f574c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f574b200640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 0x7f574c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 0x7f574c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 -- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 msgr2=0x7f574c1e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 0x7f574c1e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f5741e00640 1 -- 172.21.15.55:0/851076653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c009400 con 0x7f574c1545d0 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f574a800640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 0x7f574c1e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.714+0000 7f574b200640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 0x7f574c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:33:57.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5741e00640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f573400c430 tx=0x7f573400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57340074e0 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734007660 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734004a20 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c0adae0 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.715+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1562b0 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.717+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c067220 con 0x7f574c1545d0 2024-05-10T03:33:57.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.717+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5734004b80 con 0x7f574c1545d0 2024-05-10T03:33:57.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.718+0000 7f5743400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 0x7f572806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:57.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.719+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f57340893f0 con 0x7f574c1545d0 2024-05-10T03:33:57.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.719+0000 7f574a800640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 0x7f572806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:57.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.720+0000 7f574a800640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 0x7f572806d140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f573c035000 tx=0x7f573c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:57.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.723+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5734054360 con 0x7f574c1545d0 2024-05-10T03:33:57.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.962+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f574c06e090 con 0x7f574c1545d0 2024-05-10T03:33:57.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.964+0000 7f5743400640 1 -- 172.21.15.55:0/851076653 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f5734057980 con 0x7f574c1545d0 2024-05-10T03:33:57.967 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:57.967 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:57.967 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:57.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.966+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 msgr2=0x7f572806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.966+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 0x7f572806d140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f573c035000 tx=0x7f573c02f000 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.966+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 msgr2=0x7f574c1e3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.966+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f573400c430 tx=0x7f573400c8f0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 shutdown_connections 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f574c157c40 0x7f574c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f574c1545d0 0x7f574c1e3c90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f572806ac30 0x7f572806d140 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 --2- 172.21.15.55:0/851076653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c0b4960 0x7f574c1e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 >> 172.21.15.55:0/851076653 conn(0x7f574c070900 msgr2=0x7f574c06b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 shutdown_connections 2024-05-10T03:33:57.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:57.967+0000 7f5751400640 1 -- 172.21.15.55:0/851076653 wait complete. 2024-05-10T03:33:58.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:58.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.278+0000 7f7b13800640 1 Processor -- start 2024-05-10T03:33:58.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.278+0000 7f7b13800640 1 -- start start 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b13800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7b0c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b13800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b0c0b2140 0x7f7b0c142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b13800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c06ad20 con 0x7f7b0c0b2550 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b13800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c141f50 con 0x7f7b0c0b2140 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b13800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06a110 con 0x7f7b0c15fe50 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b11800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7b0c0b2550 0x7f7b0c06b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35316/0 (socket says 172.21.15.55:35316) 2024-05-10T03:33:58.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.281+0000 7f7b11800640 1 -- 172.21.15.55:0/1673930744 learned_addr learned my addr 172.21.15.55:0/1673930744 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:58.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.282+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> v2:172.21.15.162:3300/0 conn(0x7f7b0c0b2140 0x7f7b0c142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.282+0000 7f7b10e00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7b0c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.282+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320578340 0 0) 0x7f7b0c06ad20 con 0x7f7b0c0b2550 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.282+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af4003520 con 0x7f7b0c0b2550 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.282+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038186438 0 0) 0x7f7af4003520 con 0x7f7b0c0b2550 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 msgr2=0x7f7b0c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7b0c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 >> v2:172.21.15.162:3300/0 conn(0x7f7b0c0b2140 msgr2=0x7f7b0c142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 --2- 172.21.15.55:0/1673930744 >> v2:172.21.15.162:3300/0 conn(0x7f7b0c0b2140 0x7f7b0c142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c06a3f0 con 0x7f7b0c0b2550 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7b040019f0 con 0x7f7b0c0b2550 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 >> v1:172.21.15.55:6789/0 conn(0x7f7b0c0b2550 legacy=0x7f7b0c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af4003970 0x7f7af4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af40047c0 0x7f7af4004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7af4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c141f50 con 0x7f7af40047c0 2024-05-10T03:33:58.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06a110 con 0x7f7af4003970 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af400ba00 con 0x7f7b0c15fe50 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> v2:172.21.15.162:3300/0 conn(0x7f7b0c0b2140 0x7f7b0c142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.283+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af40047c0 0x7f7af4004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b10e00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af4003970 0x7f7af4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b11800640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7af4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b0be00640 1 -- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 msgr2=0x7f7af4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c15fe50 0x7f7af4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b0be00640 1 -- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af4003970 msgr2=0x7f7af4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af4003970 0x7f7af4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.284+0000 7f7b0be00640 1 -- 172.21.15.55:0/1673930744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4003520 con 0x7f7af40047c0 2024-05-10T03:33:58.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.285+0000 7f7b0be00640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af40047c0 0x7f7af4004bd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009c70 tx=0x7f7af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda163eb66cc3adf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.285+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004020 con 0x7f7af40047c0 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.285+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0002d050 con 0x7f7af40047c0 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.285+0000 7f7b0b400640 1 -- 172.21.15.55:0/1673930744 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004380 con 0x7f7af40047c0 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 -- 172.21.15.55:0/1673930744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af40047c0 msgr2=0x7f7af4004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 --2- 172.21.15.55:0/1673930744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af40047c0 0x7f7af4004bd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009c70 tx=0x7f7af40057f0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b11800640 1 -- 172.21.15.55:0/1673930744 reap_dead start 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 -- 172.21.15.55:0/1673930744 shutdown_connections 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 -- 172.21.15.55:0/1673930744 >> 172.21.15.55:0/1673930744 conn(0x7f7b0c0cbb90 msgr2=0x7f7b0c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 -- 172.21.15.55:0/1673930744 shutdown_connections 2024-05-10T03:33:58.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.286+0000 7f7b13800640 1 -- 172.21.15.55:0/1673930744 wait complete. 2024-05-10T03:33:58.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.288+0000 7f7b13800640 1 Processor -- start 2024-05-10T03:33:58.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.289+0000 7f7b13800640 1 -- start start 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af40047c0 0x7f7b0c0a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c0adf70 0x7f7b0c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0aabb0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1f30b0 con 0x7f7af40047c0 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.290+0000 7f7b13800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c0a9ce0 con 0x7f7b0c0adf70 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34642/0 (socket says 172.21.15.55:34642) 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 -- 172.21.15.55:0/4000878919 learned_addr learned my addr 172.21.15.55:0/4000878919 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b10e00640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af40047c0 0x7f7b0c0a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b0be00640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c0adf70 0x7f7b0c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 -- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c0adf70 msgr2=0x7f7b0c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c0adf70 0x7f7b0c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 -- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af40047c0 msgr2=0x7f7b0c0a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af40047c0 0x7f7b0c0a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 -- 172.21.15.55:0/4000878919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c15ae90 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.291+0000 7f7b11800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7b04002230 tx=0x7f7b0400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.292+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04007a80 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.292+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0400ee20 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.292+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04013660 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.292+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c13e030 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.292+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c0aa210 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.293+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c0aacf0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.294+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7b040137c0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.295+0000 7f7b08c00640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 0x7f7af006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.295+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7b04089990 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.296+0000 7f7b10e00640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 0x7f7af006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.297+0000 7f7b10e00640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 0x7f7af006d110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009160 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.300+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7b0c0aacf0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.503+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b0c0aacf0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.506+0000 7f7b08c00640 1 -- 172.21.15.55:0/4000878919 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f7b0c0aacf0 con 0x7f7b0c13d6e0 2024-05-10T03:33:58.509 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:58.509 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.508+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 msgr2=0x7f7af006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.508+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 0x7f7af006d110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009160 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.508+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 msgr2=0x7f7b0c13daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.508+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7b04002230 tx=0x7f7b0400e530 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 shutdown_connections 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0c13d6e0 0x7f7b0c13daf0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b0c0adf70 0x7f7b0c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7af006ac00 0x7f7af006d110 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 --2- 172.21.15.55:0/4000878919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7af40047c0 0x7f7b0c0a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 >> 172.21.15.55:0/4000878919 conn(0x7f7b0c0cbb90 msgr2=0x7f7b0c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 shutdown_connections 2024-05-10T03:33:58.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.509+0000 7f7b13800640 1 -- 172.21.15.55:0/4000878919 wait complete. 2024-05-10T03:33:58.537 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:33:58.537 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:58.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.813+0000 7f469d600640 1 Processor -- start 2024-05-10T03:33:58.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.813+0000 7f469d600640 1 -- start start 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f469d600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46980cdba0 0x7f469806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f469d600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980cd710 con 0x7f4698071cf0 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f469d600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980cdfb0 con 0x7f469815ab30 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f469d600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469806b4d0 con 0x7f46980cdba0 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f4696a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46980cdba0 0x7f469806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f4696a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46980cdba0 0x7f469806a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34656/0 (socket says 172.21.15.55:34656) 2024-05-10T03:33:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f4696a00640 1 -- 172.21.15.55:0/1578418847 learned_addr learned my addr 172.21.15.55:0/1578418847 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.815+0000 7f4697400640 1 --1- 172.21.15.55:0/1578418847 >> v1:172.21.15.55:6789/0 conn(0x7f4698071cf0 0x7f46980b2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35332/0 (socket says 172.21.15.55:35332) 2024-05-10T03:33:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.816+0000 7f4696a00640 1 -- 172.21.15.55:0/1578418847 >> v1:172.21.15.162:6789/0 conn(0x7f469815ab30 legacy=0x7f46980715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.816+0000 7f4696a00640 1 -- 172.21.15.55:0/1578418847 >> v1:172.21.15.55:6789/0 conn(0x7f4698071cf0 legacy=0x7f46980b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.816+0000 7f4696a00640 1 -- 172.21.15.55:0/1578418847 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469806b1f0 con 0x7f46980cdba0 2024-05-10T03:33:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.817+0000 7f4696a00640 1 --2- 172.21.15.55:0/1578418847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f469806a9a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84ba0fe59413d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.817+0000 7f4695600640 1 -- 172.21.15.55:0/1578418847 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468802ae90 con 0x7f46980cdba0 2024-05-10T03:33:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.817+0000 7f4695600640 1 -- 172.21.15.55:0/1578418847 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688032440 con 0x7f46980cdba0 2024-05-10T03:33:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.817+0000 7f4695600640 1 -- 172.21.15.55:0/1578418847 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880327d0 con 0x7f46980cdba0 2024-05-10T03:33:58.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 -- 172.21.15.55:0/1578418847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 msgr2=0x7f469806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 --2- 172.21.15.55:0/1578418847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f469806a9a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468802a180 comp rx=0 tx=0).stop 2024-05-10T03:33:58.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 -- 172.21.15.55:0/1578418847 shutdown_connections 2024-05-10T03:33:58.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 --2- 172.21.15.55:0/1578418847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f469806a9a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 -- 172.21.15.55:0/1578418847 >> 172.21.15.55:0/1578418847 conn(0x7f46980cbb90 msgr2=0x7f46980cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:58.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 -- 172.21.15.55:0/1578418847 shutdown_connections 2024-05-10T03:33:58.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.818+0000 7f469d600640 1 -- 172.21.15.55:0/1578418847 wait complete. 2024-05-10T03:33:58.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.821+0000 7f469d600640 1 Processor -- start 2024-05-10T03:33:58.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.821+0000 7f469d600640 1 -- start start 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469815ab30 0x7f46981e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46981e4b20 0x7f469813b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981e50f0 con 0x7f46980cdba0 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981e5340 con 0x7f469815ab30 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f469d600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4698139a00 con 0x7f46981e4b20 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34662/0 (socket says 172.21.15.55:34662) 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 -- 172.21.15.55:0/1911463673 learned_addr learned my addr 172.21.15.55:0/1911463673 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 -- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46981e4b20 msgr2=0x7f469813b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46981e4b20 0x7f469813b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 -- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469815ab30 msgr2=0x7f46981e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469815ab30 0x7f46981e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:58.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.823+0000 7f4696a00640 1 -- 172.21.15.55:0/1911463673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880094d0 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f4696a00640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f46880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688030b80 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688030ce0 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468803b640 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4698139ce0 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.824+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469813a2a0 con 0x7f46980cdba0 2024-05-10T03:33:58.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.825+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698066f60 con 0x7f46980cdba0 2024-05-10T03:33:58.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.827+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4688039040 con 0x7f46980cdba0 2024-05-10T03:33:58.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.827+0000 7f468ea00640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 0x7f467806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:58.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.828+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46880a9e70 con 0x7f46980cdba0 2024-05-10T03:33:58.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.828+0000 7f4696000640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 0x7f467806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:58.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.830+0000 7f4696000640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 0x7f467806d140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:58.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:58.832+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4688074e70 con 0x7f46980cdba0 2024-05-10T03:33:59.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.069+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4698067220 con 0x7f46980cdba0 2024-05-10T03:33:59.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.070+0000 7f468ea00640 1 -- 172.21.15.55:0/1911463673 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f4688037090 con 0x7f46980cdba0 2024-05-10T03:33:59.073 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:59.073 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:59.073 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.072+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 msgr2=0x7f467806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.072+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 0x7f467806d140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4684006fd0 tx=0x7f4684006bc0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.072+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 msgr2=0x7f4698139030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.072+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f46880046c0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 shutdown_connections 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46981e4b20 0x7f469813b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac30 0x7f467806d140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469815ab30 0x7f46981e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 --2- 172.21.15.55:0/1911463673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980cdba0 0x7f4698139030 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 >> 172.21.15.55:0/1911463673 conn(0x7f46980cbb90 msgr2=0x7f4698070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:59.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 shutdown_connections 2024-05-10T03:33:59.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.073+0000 7f469d600640 1 -- 172.21.15.55:0/1911463673 wait complete. 2024-05-10T03:33:59.101 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:33:59.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.382+0000 7f21aae00640 1 Processor -- start 2024-05-10T03:33:59.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.382+0000 7f21aae00640 1 -- start start 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aae00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21a40ce710 0x7f21a40b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aae00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21a415d890 0x7f21a40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aae00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a40b31a0 con 0x7f21a40b1dc0 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aae00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40b6c20 con 0x7f21a415d890 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aae00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40b2870 con 0x7f21a40ce710 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aa400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21a415d890 0x7f21a40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aa400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21a415d890 0x7f21a40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34664/0 (socket says 172.21.15.55:34664) 2024-05-10T03:33:59.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21aa400640 1 -- 172.21.15.55:0/3012494850 learned_addr learned my addr 172.21.15.55:0/3012494850 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:59.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.384+0000 7f21a9000640 1 --2- 172.21.15.55:0/3012494850 >> v2:172.21.15.162:3300/0 conn(0x7f21a40ce710 0x7f21a40b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.385+0000 7f21aa400640 1 -- 172.21.15.55:0/3012494850 >> v2:172.21.15.162:3300/0 conn(0x7f21a40ce710 msgr2=0x7f21a40b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:59.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.385+0000 7f21aa400640 1 --2- 172.21.15.55:0/3012494850 >> v2:172.21.15.162:3300/0 conn(0x7f21a40ce710 0x7f21a40b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.385+0000 7f21aa400640 1 -- 172.21.15.55:0/3012494850 >> v1:172.21.15.162:6789/0 conn(0x7f21a40b1dc0 legacy=0x7f21a40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:33:59.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.385+0000 7f21aa400640 1 -- 172.21.15.55:0/3012494850 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40b2b50 con 0x7f21a415d890 2024-05-10T03:33:59.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.386+0000 7f21aa400640 1 --2- 172.21.15.55:0/3012494850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40b3630 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f21a40d4090 tx=0x7f219c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c41e0c6c111e1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.386+0000 7f21a3e00640 1 -- 172.21.15.55:0/3012494850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c032440 con 0x7f21a415d890 2024-05-10T03:33:59.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.386+0000 7f21a3e00640 1 -- 172.21.15.55:0/3012494850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0325a0 con 0x7f21a415d890 2024-05-10T03:33:59.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.386+0000 7f21a3e00640 1 -- 172.21.15.55:0/3012494850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c032930 con 0x7f21a415d890 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 -- 172.21.15.55:0/3012494850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 msgr2=0x7f21a40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 --2- 172.21.15.55:0/3012494850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40b3630 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f21a40d4090 tx=0x7f219c02a1a0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 -- 172.21.15.55:0/3012494850 shutdown_connections 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 --2- 172.21.15.55:0/3012494850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40b3630 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 --2- 172.21.15.55:0/3012494850 >> v2:172.21.15.162:3300/0 conn(0x7f21a40ce710 0x7f21a40b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 -- 172.21.15.55:0/3012494850 >> 172.21.15.55:0/3012494850 conn(0x7f21a40d3030 msgr2=0x7f21a40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 -- 172.21.15.55:0/3012494850 shutdown_connections 2024-05-10T03:33:59.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.387+0000 7f21aae00640 1 -- 172.21.15.55:0/3012494850 wait complete. 2024-05-10T03:33:59.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.390+0000 7f21aae00640 1 Processor -- start 2024-05-10T03:33:59.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.390+0000 7f21aae00640 1 -- start start 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21a40b1dc0 0x7f21a40c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21a40c96b0 0x7f21a40caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40cbbf0 con 0x7f21a415d890 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40c81c0 con 0x7f21a40c96b0 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.391+0000 7f21aae00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40c6be0 con 0x7f21a40b1dc0 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34672/0 (socket says 172.21.15.55:34672) 2024-05-10T03:33:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 -- 172.21.15.55:0/454184975 learned_addr learned my addr 172.21.15.55:0/454184975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9a00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21a40b1dc0 0x7f21a40c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21aa400640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21a40c96b0 0x7f21a40caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 -- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21a40b1dc0 msgr2=0x7f21a40c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21a40b1dc0 0x7f21a40c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 -- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21a40c96b0 msgr2=0x7f21a40caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21a40c96b0 0x7f21a40caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.392+0000 7f21a9000640 1 -- 172.21.15.55:0/454184975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0094d0 con 0x7f21a415d890 2024-05-10T03:33:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21a9000640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2198007e30 tx=0x7f219800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198014620 con 0x7f21a415d890 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800ee90 con 0x7f21a415d890 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980149c0 con 0x7f21a415d890 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a40c6ec0 con 0x7f21a415d890 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.393+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a40c7480 con 0x7f21a415d890 2024-05-10T03:33:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.394+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a40b5950 con 0x7f21a415d890 2024-05-10T03:33:59.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.396+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2198014b20 con 0x7f21a415d890 2024-05-10T03:33:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.397+0000 7f21a1600640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 0x7f218806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.397+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2198089df0 con 0x7f21a415d890 2024-05-10T03:33:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.397+0000 7f21a9a00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 0x7f218806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.399+0000 7f21a9a00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 0x7f218806d110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.402+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2198054e90 con 0x7f21a415d890 2024-05-10T03:33:59.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.639+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a404e290 con 0x7f21a415d890 2024-05-10T03:33:59.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.640+0000 7f21a1600640 1 -- 172.21.15.55:0/454184975 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f21980584b0 con 0x7f21a415d890 2024-05-10T03:33:59.643 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:33:59.643 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:33:34.958997+0000","modified":"2024-05-10T03:33:38.424470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6837/3533670538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":3533670538},{"type":"v1","addr":"172.21.15.162:6837","nonce":3533670538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-05-10T03:33:59.643 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 39 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.642+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 msgr2=0x7f218806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.642+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 0x7f218806d110 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2194006fd0 tx=0x7f2194008040 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 msgr2=0x7f21a40c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2198007e30 tx=0x7f219800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 shutdown_connections 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f218806ac00 0x7f218806d110 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21a40c96b0 0x7f21a40caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21a415d890 0x7f21a40c9170 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 --2- 172.21.15.55:0/454184975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21a40b1dc0 0x7f21a40c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 >> 172.21.15.55:0/454184975 conn(0x7f21a40d3030 msgr2=0x7f21a40d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:59.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 shutdown_connections 2024-05-10T03:33:59.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.643+0000 7f21aae00640 1 -- 172.21.15.55:0/454184975 wait complete. 2024-05-10T03:33:59.672 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:33:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.955+0000 7f24d6200640 1 Processor -- start 2024-05-10T03:33:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.955+0000 7f24d6200640 1 -- start start 2024-05-10T03:33:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.957+0000 7f24d6200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24d0141c70 0x7f24d00d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24d6200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d00d0e80 con 0x7f24d006f210 2024-05-10T03:33:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24d6200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d00b5ec0 con 0x7f24d0144aa0 2024-05-10T03:33:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24d6200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d0145340 con 0x7f24d0141c70 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24cfe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f24d006f210 0x7f24d00cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35348/0 (socket says 172.21.15.55:35348) 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24cfe00640 1 -- 172.21.15.55:0/3434860704 learned_addr learned my addr 172.21.15.55:0/3434860704 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.958+0000 7f24cf400640 1 --2- 172.21.15.55:0/3434860704 >> v2:172.21.15.162:3300/0 conn(0x7f24d0141c70 0x7f24d00d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799708393 0 0) 0x7f24c8001110 con 0x7f24d006f210 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b4003520 con 0x7f24d006f210 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24cf400640 1 -- 172.21.15.55:0/3434860704 >> v1:172.21.15.162:6789/0 conn(0x7f24d0144aa0 legacy=0x7f24d006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24cf400640 1 -- 172.21.15.55:0/3434860704 >> v1:172.21.15.55:6789/0 conn(0x7f24d006f210 legacy=0x7f24d00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658755247 0 0) 0x7f24b4003520 con 0x7f24d006f210 2024-05-10T03:33:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.959+0000 7f24cf400640 1 -- 172.21.15.55:0/3434860704 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00b52f0 con 0x7f24d0141c70 2024-05-10T03:33:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.961+0000 7f24cf400640 1 --2- 172.21.15.55:0/3434860704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d00d07a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24c00097a0 tx=0x7f24c002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6597da02127786a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.961+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0032440 con 0x7f24d0141c70 2024-05-10T03:33:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.961+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c00325a0 con 0x7f24d0141c70 2024-05-10T03:33:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.961+0000 7f24ce000640 1 -- 172.21.15.55:0/3434860704 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0032930 con 0x7f24d0141c70 2024-05-10T03:33:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 -- 172.21.15.55:0/3434860704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 msgr2=0x7f24d00d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 --2- 172.21.15.55:0/3434860704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d00d07a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24c00097a0 tx=0x7f24c002a230 comp rx=0 tx=0).stop 2024-05-10T03:33:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 -- 172.21.15.55:0/3434860704 shutdown_connections 2024-05-10T03:33:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 --2- 172.21.15.55:0/3434860704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d00d07a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 -- 172.21.15.55:0/3434860704 >> 172.21.15.55:0/3434860704 conn(0x7f24d00b2c10 msgr2=0x7f24d00b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:33:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.962+0000 7f24d6200640 1 -- 172.21.15.55:0/3434860704 shutdown_connections 2024-05-10T03:33:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.963+0000 7f24d6200640 1 -- 172.21.15.55:0/3434860704 wait complete. 2024-05-10T03:33:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.965+0000 7f24d6200640 1 Processor -- start 2024-05-10T03:33:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.965+0000 7f24d6200640 1 -- start start 2024-05-10T03:33:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d01e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d0144aa0 0x7f24d01312b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0135410 con 0x7f24d01317f0 2024-05-10T03:33:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01e6ba0 con 0x7f24d0141c70 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24d6200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01e6df0 con 0x7f24d0144aa0 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34684/0 (socket says 172.21.15.55:34684) 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24cfe00640 1 -- 172.21.15.55:0/3450788956 learned_addr learned my addr 172.21.15.55:0/3450788956 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.967+0000 7f24cf400640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d01e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cea00640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d0144aa0 0x7f24d01312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 -- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d0144aa0 msgr2=0x7f24d01312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d0144aa0 0x7f24d01312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 -- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 msgr2=0x7f24d01e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d01e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:33:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 -- 172.21.15.55:0/3450788956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0009400 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24cfe00640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c410 tx=0x7f24c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8005ce0 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8005e40 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8007310 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.968+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01e70d0 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.969+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01e7690 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.970+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d01e7bb0 con 0x7f24d01317f0 2024-05-10T03:33:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.971+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f24c8007530 con 0x7f24d01317f0 2024-05-10T03:33:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.972+0000 7f24c7400640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 0x7f24b006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:33:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.972+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f24c8088a00 con 0x7f24d01317f0 2024-05-10T03:33:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.972+0000 7f24cf400640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 0x7f24b006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:33:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.974+0000 7f24cf400640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 0x7f24b006d110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f24c0034000 tx=0x7f24c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:33:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:33:59.977+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f24d01e7bb0 con 0x7f24d01317f0 2024-05-10T03:34:00.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.178+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24d00d0e80 con 0x7f24d01317f0 2024-05-10T03:34:00.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.180+0000 7f24c7400640 1 -- 172.21.15.55:0/3450788956 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16309 (secure 0 0 0) 0x7f24d00d0e80 con 0x7f24d01317f0 2024-05-10T03:34:00.183 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:00.184 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:33:35.964486+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:33:33.167391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-10T03:33:33.995610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.182+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 msgr2=0x7f24b006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.182+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 0x7f24b006d110 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f24c0034000 tx=0x7f24c002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.182+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 msgr2=0x7f24d01f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.182+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c410 tx=0x7f24c800c8d0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 shutdown_connections 2024-05-10T03:34:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24d01317f0 0x7f24d01f2960 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d0144aa0 0x7f24d01312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f24b006ac00 0x7f24b006d110 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 --2- 172.21.15.55:0/3450788956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d0141c70 0x7f24d01e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 >> 172.21.15.55:0/3450788956 conn(0x7f24d00b2c10 msgr2=0x7f24d0141140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 shutdown_connections 2024-05-10T03:34:00.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.183+0000 7f24d6200640 1 -- 172.21.15.55:0/3450788956 wait complete. 2024-05-10T03:34:00.212 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:34:00.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.494+0000 7fcb00600640 1 Processor -- start 2024-05-10T03:34:00.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.494+0000 7fcb00600640 1 -- start start 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcaf815fe50 0x7fcaf8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2140 0x7fcaf8142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 0x7fcaf8142980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf806ad20 con 0x7fcaf815fe50 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf8141f50 con 0x7fcaf80b2550 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcb00600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf806a110 con 0x7fcaf80b2140 2024-05-10T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcafdc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcaf815fe50 0x7fcaf8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcafdc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcaf815fe50 0x7fcaf8141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34698/0 (socket says 172.21.15.55:34698) 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.497+0000 7fcafdc00640 1 -- 172.21.15.55:0/1080517386 learned_addr learned my addr 172.21.15.55:0/1080517386 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafe600640 1 --2- 172.21.15.55:0/1080517386 >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 0x7fcaf8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafdc00640 1 -- 172.21.15.55:0/1080517386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2140 msgr2=0x7fcaf8142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafdc00640 1 --2- 172.21.15.55:0/1080517386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2140 0x7fcaf8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafdc00640 1 -- 172.21.15.55:0/1080517386 >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 msgr2=0x7fcaf8142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafdc00640 1 --2- 172.21.15.55:0/1080517386 >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 0x7fcaf8142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafdc00640 1 -- 172.21.15.55:0/1080517386 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf806a3f0 con 0x7fcaf815fe50 2024-05-10T03:34:00.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.498+0000 7fcafe600640 1 --2- 172.21.15.55:0/1080517386 >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 0x7fcaf8142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:00.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.499+0000 7fcafdc00640 1 --2- 172.21.15.55:0/1080517386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf815fe50 0x7fcaf8141a10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80ccd70 tx=0x7fcaec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd3ef05fe95951e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:00.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.500+0000 7fcaf7e00640 1 -- 172.21.15.55:0/1080517386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032440 con 0x7fcaf815fe50 2024-05-10T03:34:00.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.500+0000 7fcaf7e00640 1 -- 172.21.15.55:0/1080517386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec0325a0 con 0x7fcaf815fe50 2024-05-10T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.500+0000 7fcaf7e00640 1 -- 172.21.15.55:0/1080517386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032930 con 0x7fcaf815fe50 2024-05-10T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.500+0000 7fcb00600640 1 -- 172.21.15.55:0/1080517386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf815fe50 msgr2=0x7fcaf8141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.500+0000 7fcb00600640 1 --2- 172.21.15.55:0/1080517386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf815fe50 0x7fcaf8141a10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80ccd70 tx=0x7fcaec02a270 comp rx=0 tx=0).stop 2024-05-10T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 -- 172.21.15.55:0/1080517386 shutdown_connections 2024-05-10T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 --2- 172.21.15.55:0/1080517386 >> v2:172.21.15.162:3300/0 conn(0x7fcaf80b2550 0x7fcaf8142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 --2- 172.21.15.55:0/1080517386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2140 0x7fcaf8142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 --2- 172.21.15.55:0/1080517386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf815fe50 0x7fcaf8141a10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 -- 172.21.15.55:0/1080517386 >> 172.21.15.55:0/1080517386 conn(0x7fcaf80cbb90 msgr2=0x7fcaf80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 -- 172.21.15.55:0/1080517386 shutdown_connections 2024-05-10T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.501+0000 7fcb00600640 1 -- 172.21.15.55:0/1080517386 wait complete. 2024-05-10T03:34:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.503+0000 7fcb00600640 1 Processor -- start 2024-05-10T03:34:00.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.503+0000 7fcb00600640 1 -- start start 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 0x7fcaf815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 0x7fcaf815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81f4db0 con 0x7fcaf80b2140 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf814af80 con 0x7fcaf815fe50 2024-05-10T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcb00600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf814aab0 con 0x7fcaf80b2550 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcafdc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcafdc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34710/0 (socket says 172.21.15.55:34710) 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.505+0000 7fcafdc00640 1 -- 172.21.15.55:0/2832636417 learned_addr learned my addr 172.21.15.55:0/2832636417 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafe600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 0x7fcaf815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafd200640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 0x7fcaf815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 -- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 msgr2=0x7fcaf815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 0x7fcaf815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 -- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 msgr2=0x7fcaf815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 0x7fcaf815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 -- 172.21.15.55:0/2832636417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0094d0 con 0x7fcaf80b2140 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafd200640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 0x7fcaf815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafe600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 0x7fcaf815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.506+0000 7fcafdc00640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:00.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.507+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032440 con 0x7fcaf80b2140 2024-05-10T03:34:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.507+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec030b80 con 0x7fcaf80b2140 2024-05-10T03:34:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.507+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec03d420 con 0x7fcaf80b2140 2024-05-10T03:34:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.507+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf81f5b40 con 0x7fcaf80b2140 2024-05-10T03:34:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.507+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf81f4fc0 con 0x7fcaf80b2140 2024-05-10T03:34:00.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.508+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf80aec00 con 0x7fcaf80b2140 2024-05-10T03:34:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.509+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcaec039040 con 0x7fcaf80b2140 2024-05-10T03:34:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.510+0000 7fcaf5600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 0x7fcad806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.510+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcaec0aa280 con 0x7fcaf80b2140 2024-05-10T03:34:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.511+0000 7fcafd200640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 0x7fcad806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:00.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.512+0000 7fcafd200640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 0x7fcad806d110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8141360 tx=0x7fcae4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:00.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.515+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcaec075280 con 0x7fcaf80b2140 2024-05-10T03:34:00.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.751+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcaf8066f60 con 0x7fcaf80b2140 2024-05-10T03:34:00.755 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:34:00.752+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:34:00.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.770+0000 7fcaf5600640 1 -- 172.21.15.55:0/2832636417 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcaec0788a0 con 0x7fcaf80b2140 2024-05-10T03:34:00.774 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:34:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 msgr2=0x7fcad806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 0x7fcad806d110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8141360 tx=0x7fcae4008040 comp rx=0 tx=0).stop 2024-05-10T03:34:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 msgr2=0x7fcaf814d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 shutdown_connections 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcad806ac00 0x7fcad806d110 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcaf815fe50 0x7fcaf815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcaf80b2550 0x7fcaf815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 --2- 172.21.15.55:0/2832636417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcaf80b2140 0x7fcaf814d510 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 >> 172.21.15.55:0/2832636417 conn(0x7fcaf80cbb90 msgr2=0x7fcaf80af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:00.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 shutdown_connections 2024-05-10T03:34:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:00.773+0000 7fcb00600640 1 -- 172.21.15.55:0/2832636417 wait complete. 2024-05-10T03:34:00.790 INFO:tasks.ceph.mds.d.smithi162.stderr: -2218> 2024-05-10T03:33:39.328+0000 7feedf000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:34:00.805 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:34:00.962 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:34:01.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.092+0000 7f68d8a00640 1 Processor -- start 2024-05-10T03:34:01.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.092+0000 7f68d8a00640 1 -- start start 2024-05-10T03:34:01.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.094+0000 7f68d8a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68d406a4b0 0x7f68d4158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.094+0000 7f68d8a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40708b0 con 0x7f68d40b3860 2024-05-10T03:34:01.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d8a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d415b940 con 0x7f68d4071db0 2024-05-10T03:34:01.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d8a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40b2490 con 0x7f68d406a4b0 2024-05-10T03:34:01.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d2800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f68d40b3860 0x7f68d40b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35358/0 (socket says 172.21.15.55:35358) 2024-05-10T03:34:01.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d2800640 1 -- 172.21.15.55:0/6707175 learned_addr learned my addr 172.21.15.55:0/6707175 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d1400640 1 --2- 172.21.15.55:0/6707175 >> v2:172.21.15.162:3300/0 conn(0x7f68d406a4b0 0x7f68d4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212233101 0 0) 0x7f68d40708b0 con 0x7f68d40b3860 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.095+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68b8003520 con 0x7f68d40b3860 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362011177 0 0) 0x7f68b8003520 con 0x7f68d40b3860 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 >> v2:172.21.15.162:3300/0 conn(0x7f68d406a4b0 msgr2=0x7f68d4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 --2- 172.21.15.55:0/6707175 >> v2:172.21.15.162:3300/0 conn(0x7f68d406a4b0 0x7f68d4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 >> v1:172.21.15.162:6789/0 conn(0x7f68d4071db0 legacy=0x7f68d40ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4070260 con 0x7f68d40b3860 2024-05-10T03:34:01.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f68cc001670 con 0x7f68d40b3860 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 >> v1:172.21.15.55:6789/0 conn(0x7f68d40b3860 legacy=0x7f68d40b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 0x7f68b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b80047c0 0x7f68b8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8005310 0x7f68b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b800c210 con 0x7f68b8003970 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b800c380 con 0x7f68b8005310 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b800c540 con 0x7f68b80047c0 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.096+0000 7f68d1e00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 0x7f68b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d1e00640 1 -- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b80047c0 msgr2=0x7f68b8004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d1e00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b80047c0 0x7f68b8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d1e00640 1 -- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8005310 msgr2=0x7f68b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d1e00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8005310 0x7f68b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d1e00640 1 -- 172.21.15.55:0/6707175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8003520 con 0x7f68b8003970 2024-05-10T03:34:01.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.097+0000 7f68d2800640 1 -- 172.21.15.55:0/6707175 reap_dead start 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.098+0000 7f68d1e00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 0x7f68b8003de0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f68d406e510 tx=0x7f68b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec194f2fb2d9f68b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.098+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00052d0 con 0x7f68b8003970 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.098+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c0005430 con 0x7f68b8003970 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.098+0000 7f68d0a00640 1 -- 172.21.15.55:0/6707175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c002ab70 con 0x7f68b8003970 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 -- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 msgr2=0x7f68b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 0x7f68b8003de0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f68d406e510 tx=0x7f68b80057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 -- 172.21.15.55:0/6707175 shutdown_connections 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 --2- 172.21.15.55:0/6707175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68b8003970 0x7f68b8003de0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 -- 172.21.15.55:0/6707175 >> 172.21.15.55:0/6707175 conn(0x7f68d40cbb90 msgr2=0x7f68d40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:01.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 -- 172.21.15.55:0/6707175 shutdown_connections 2024-05-10T03:34:01.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.099+0000 7f68d8a00640 1 -- 172.21.15.55:0/6707175 wait complete. 2024-05-10T03:34:01.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.102+0000 7f68d8a00640 1 Processor -- start 2024-05-10T03:34:01.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.102+0000 7f68d8a00640 1 -- start start 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8003970 0x7f68d41398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d4141e60 0x7f68d415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41382f0 con 0x7f68d40ae390 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40ad890 con 0x7f68b8003970 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.103+0000 7f68d8a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40acc80 con 0x7f68d4141e60 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34730/0 (socket says 172.21.15.55:34730) 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 -- 172.21.15.55:0/64543686 learned_addr learned my addr 172.21.15.55:0/64543686 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 -- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d4141e60 msgr2=0x7f68d415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d4141e60 0x7f68d415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 -- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8003970 msgr2=0x7f68d41398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8003970 0x7f68d41398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.104+0000 7f68d1400640 1 -- 172.21.15.55:0/64543686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d40acf60 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68d1400640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f68c400ef70 tx=0x7f68c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4009e20 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400f040 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d40ad240 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40146c0 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.105+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d4139de0 con 0x7f68d40ae390 2024-05-10T03:34:01.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.106+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d4066f60 con 0x7f68d40ae390 2024-05-10T03:34:01.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.107+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f68c4014820 con 0x7f68d40ae390 2024-05-10T03:34:01.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.108+0000 7f68ca000640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 0x7f68b406d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:01.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.108+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f68c408a2a0 con 0x7f68d40ae390 2024-05-10T03:34:01.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.109+0000 7f68d1e00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 0x7f68b406d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:01.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.111+0000 7f68d1e00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 0x7f68b406d0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f68c0032000 tx=0x7f68c002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:01.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.113+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f68c4055240 con 0x7f68d40ae390 2024-05-10T03:34:01.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.355+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f68d4067220 con 0x7f68d40ae390 2024-05-10T03:34:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.782+0000 7f68ca000640 1 -- 172.21.15.55:0/64543686 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f68c4058860 con 0x7f68d40ae390 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 msgr2=0x7f68b406d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 0x7f68b406d0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f68c0032000 tx=0x7f68c002c000 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 msgr2=0x7f68d415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f68c400ef70 tx=0x7f68c400c490 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 shutdown_connections 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d4141e60 0x7f68d415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d40ae390 0x7f68d415bcf0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f68b406ab90 0x7f68b406d0a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 --2- 172.21.15.55:0/64543686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8003970 0x7f68d41398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 >> 172.21.15.55:0/64543686 conn(0x7f68d40cbb90 msgr2=0x7f68d40661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 shutdown_connections 2024-05-10T03:34:01.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:01.786+0000 7f68d8a00640 1 -- 172.21.15.55:0/64543686 wait complete. 2024-05-10T03:34:01.816 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:34:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.099+0000 7fcf25c00640 1 Processor -- start 2024-05-10T03:34:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.099+0000 7fcf25c00640 1 -- start start 2024-05-10T03:34:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf25c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf200cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf25c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf200ad730 con 0x7fcf2006b7c0 2024-05-10T03:34:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf25c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2006bbd0 con 0x7fcf2015b820 2024-05-10T03:34:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf25c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200cd380 con 0x7fcf200714d0 2024-05-10T03:34:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf1ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcf2006b7c0 0x7fcf2006a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35366/0 (socket says 172.21.15.55:35366) 2024-05-10T03:34:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.102+0000 7fcf1ea00640 1 -- 172.21.15.55:0/3249242225 learned_addr learned my addr 172.21.15.55:0/3249242225 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1f400640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf200cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356925299 0 0) 0x7fcf10001490 con 0x7fcf2006b7c0 2024-05-10T03:34:02.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf04003520 con 0x7fcf2006b7c0 2024-05-10T03:34:02.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367079895 0 0) 0x7fcf04003520 con 0x7fcf2006b7c0 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 msgr2=0x7fcf200cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf200cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 >> v1:172.21.15.162:6789/0 conn(0x7fcf2015b820 legacy=0x7fcf20070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2015bc30 con 0x7fcf2006b7c0 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fcf10001b80 con 0x7fcf2006b7c0 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 >> v1:172.21.15.55:6789/0 conn(0x7fcf2006b7c0 legacy=0x7fcf2006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf040039a0 0x7fcf04003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.103+0000 7fcf1d600640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1d600640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf04004d00 0x7fcf04005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200cd380 con 0x7fcf04004d00 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0400fec0 con 0x7fcf040039a0 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04010080 con 0x7fcf200714d0 2024-05-10T03:34:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1f400640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf04004d00 0x7fcf04005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1ea00640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf040039a0 0x7fcf04003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.104+0000 7fcf1e000640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.105+0000 7fcf1f400640 1 -- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 msgr2=0x7fcf040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.105+0000 7fcf1f400640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf200714d0 0x7fcf040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.105+0000 7fcf1f400640 1 -- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf040039a0 msgr2=0x7fcf04003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.105+0000 7fcf1f400640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf040039a0 0x7fcf04003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.105+0000 7fcf1f400640 1 -- 172.21.15.55:0/3249242225 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04003520 con 0x7fcf04004d00 2024-05-10T03:34:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.106+0000 7fcf1f400640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf04004d00 0x7fcf04005110 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fcf20067040 tx=0x7fcf040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa96f655afb85f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.106+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18031400 con 0x7fcf04004d00 2024-05-10T03:34:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.106+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1802d050 con 0x7fcf04004d00 2024-05-10T03:34:02.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.106+0000 7fcf1d600640 1 -- 172.21.15.55:0/3249242225 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180317a0 con 0x7fcf04004d00 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.107+0000 7fcf25c00640 1 -- 172.21.15.55:0/3249242225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf04004d00 msgr2=0x7fcf04005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.107+0000 7fcf25c00640 1 --2- 172.21.15.55:0/3249242225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf04004d00 0x7fcf04005110 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fcf20067040 tx=0x7fcf040057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.107+0000 7fcf1f400640 1 -- 172.21.15.55:0/3249242225 reap_dead start 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.107+0000 7fcf25c00640 1 -- 172.21.15.55:0/3249242225 shutdown_connections 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.107+0000 7fcf25c00640 1 -- 172.21.15.55:0/3249242225 >> 172.21.15.55:0/3249242225 conn(0x7fcf200b0cf0 msgr2=0x7fcf200b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.108+0000 7fcf25c00640 1 -- 172.21.15.55:0/3249242225 shutdown_connections 2024-05-10T03:34:02.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.108+0000 7fcf25c00640 1 -- 172.21.15.55:0/3249242225 wait complete. 2024-05-10T03:34:02.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.110+0000 7fcf25c00640 1 Processor -- start 2024-05-10T03:34:02.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.110+0000 7fcf25c00640 1 -- start start 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf04004d00 0x7fcf2015e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf20154a70 0x7fcf2015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2015dd60 con 0x7fcf2015ea70 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf20155040 con 0x7fcf04004d00 2024-05-10T03:34:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf25c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2015c590 con 0x7fcf20154a70 2024-05-10T03:34:02.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf1e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf1e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34754/0 (socket says 172.21.15.55:34754) 2024-05-10T03:34:02.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.112+0000 7fcf1e000640 1 -- 172.21.15.55:0/2670541484 learned_addr learned my addr 172.21.15.55:0/2670541484 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:02.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1ea00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf04004d00 0x7fcf2015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1f400640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf20154a70 0x7fcf2015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 -- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf20154a70 msgr2=0x7fcf2015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf20154a70 0x7fcf2015ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 -- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf04004d00 msgr2=0x7fcf2015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf04004d00 0x7fcf2015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 -- 172.21.15.55:0/2670541484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf20155570 con 0x7fcf2015ea70 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf1e000640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e550 tx=0x7fcf0800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.113+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf080052e0 con 0x7fcf2015ea70 2024-05-10T03:34:02.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.114+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf08005440 con 0x7fcf2015ea70 2024-05-10T03:34:02.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.114+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08003810 con 0x7fcf2015ea70 2024-05-10T03:34:02.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.114+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf20155290 con 0x7fcf2015ea70 2024-05-10T03:34:02.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.114+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2015c220 con 0x7fcf2015ea70 2024-05-10T03:34:02.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.115+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf200b1a40 con 0x7fcf2015ea70 2024-05-10T03:34:02.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.116+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcf08003970 con 0x7fcf2015ea70 2024-05-10T03:34:02.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.117+0000 7fcf16a00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 0x7fcefc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:02.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.117+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fcf08089140 con 0x7fcf2015ea70 2024-05-10T03:34:02.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.118+0000 7fcf1ea00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 0x7fcefc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.119+0000 7fcf1ea00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 0x7fcefc06d110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcf100021a0 tx=0x7fcf10009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:02.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.124+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcf080540e0 con 0x7fcf2015ea70 2024-05-10T03:34:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.322+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf2006bbd0 con 0x7fcf2015ea70 2024-05-10T03:34:02.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.787+0000 7fcf16a00640 1 -- 172.21.15.55:0/2670541484 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcf2006bbd0 con 0x7fcf2015ea70 2024-05-10T03:34:02.791 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 msgr2=0x7fcefc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 0x7fcefc06d110 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcf100021a0 tx=0x7fcf10009040 comp rx=0 tx=0).stop 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 msgr2=0x7fcf20154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e550 tx=0x7fcf0800ea10 comp rx=0 tx=0).stop 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 shutdown_connections 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf20154a70 0x7fcf2015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf2015ea70 0x7fcf20154530 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcefc06ac00 0x7fcefc06d110 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 --2- 172.21.15.55:0/2670541484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf04004d00 0x7fcf2015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:02.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 >> 172.21.15.55:0/2670541484 conn(0x7fcf200b0cf0 msgr2=0x7fcf200718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:02.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 shutdown_connections 2024-05-10T03:34:02.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:02.792+0000 7fcf25c00640 1 -- 172.21.15.55:0/2670541484 wait complete. 2024-05-10T03:34:02.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:34:03.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.102+0000 7f40e3e00640 1 Processor -- start 2024-05-10T03:34:03.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.102+0000 7f40e3e00640 1 -- start start 2024-05-10T03:34:03.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40dc06b7c0 0x7f40dc15b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e3e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40dc0ad730 con 0x7f40dc0cce40 2024-05-10T03:34:03.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40dc06bbd0 con 0x7f40dc06b7c0 2024-05-10T03:34:03.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc071710 con 0x7f40dc070dc0 2024-05-10T03:34:03.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e1400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40dc06b7c0 0x7f40dc15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e1400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40dc06b7c0 0x7f40dc15b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39584/0 (socket says 172.21.15.55:39584) 2024-05-10T03:34:03.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e1400640 1 -- 172.21.15.55:0/1142620662 learned_addr learned my addr 172.21.15.55:0/1142620662 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:03.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.105+0000 7f40e1e00640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.106+0000 7f40e1400640 1 -- 172.21.15.55:0/1142620662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 msgr2=0x7f40dc0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.106+0000 7f40e1400640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc0711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.106+0000 7f40e1400640 1 -- 172.21.15.55:0/1142620662 >> v1:172.21.15.162:6789/0 conn(0x7f40dc0cce40 legacy=0x7f40dc06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.106+0000 7f40e1400640 1 -- 172.21.15.55:0/1142620662 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0ad170 con 0x7f40dc06b7c0 2024-05-10T03:34:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.106+0000 7f40e1e00640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:03.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.107+0000 7f40e1400640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc06b7c0 0x7f40dc15b820 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009870 tx=0x7f40d002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff95442f4346ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:03.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.107+0000 7f40dbe00640 1 -- 172.21.15.55:0/1142620662 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0032440 con 0x7f40dc06b7c0 2024-05-10T03:34:03.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.107+0000 7f40dbe00640 1 -- 172.21.15.55:0/1142620662 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d00325a0 con 0x7f40dc06b7c0 2024-05-10T03:34:03.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.107+0000 7f40dbe00640 1 -- 172.21.15.55:0/1142620662 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0032930 con 0x7f40dc06b7c0 2024-05-10T03:34:03.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 -- 172.21.15.55:0/1142620662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc06b7c0 msgr2=0x7f40dc15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc06b7c0 0x7f40dc15b820 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009870 tx=0x7f40d002a200 comp rx=0 tx=0).stop 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 -- 172.21.15.55:0/1142620662 shutdown_connections 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 --2- 172.21.15.55:0/1142620662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc06b7c0 0x7f40dc15b820 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 -- 172.21.15.55:0/1142620662 >> 172.21.15.55:0/1142620662 conn(0x7f40dc0b0cf0 msgr2=0x7f40dc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 -- 172.21.15.55:0/1142620662 shutdown_connections 2024-05-10T03:34:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.108+0000 7f40e3e00640 1 -- 172.21.15.55:0/1142620662 wait complete. 2024-05-10T03:34:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.111+0000 7f40e3e00640 1 Processor -- start 2024-05-10T03:34:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.111+0000 7f40e3e00640 1 -- start start 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40dc06b7c0 0x7f40dc0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc09af40 con 0x7f40dc0cce40 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0c8920 con 0x7f40dc06b7c0 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.112+0000 7f40e3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc09c9b0 con 0x7f40dc070dc0 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39598/0 (socket says 172.21.15.55:39598) 2024-05-10T03:34:03.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 -- 172.21.15.55:0/3654172859 learned_addr learned my addr 172.21.15.55:0/3654172859 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e0a00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1400640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40dc06b7c0 0x7f40dc0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 -- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 msgr2=0x7f40dc14b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc14b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 -- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40dc06b7c0 msgr2=0x7f40dc0c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40dc06b7c0 0x7f40dc0c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.113+0000 7f40e1e00640 1 -- 172.21.15.55:0/3654172859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00094d0 con 0x7f40dc0cce40 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40e1e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f40d400cd40 tx=0x7f40d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4003b90 con 0x7f40dc0cce40 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4003d10 con 0x7f40dc0cce40 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004260 con 0x7f40dc0cce40 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40dc0c90c0 con 0x7f40dc0cce40 2024-05-10T03:34:03.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.114+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc09b1e0 con 0x7f40dc0cce40 2024-05-10T03:34:03.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.115+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40dc0b1a40 con 0x7f40dc0cce40 2024-05-10T03:34:03.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.117+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f40d400b040 con 0x7f40dc0cce40 2024-05-10T03:34:03.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.117+0000 7f40d9600640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 0x7f40bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:03.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.118+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5677+0+0 (secure 0 0 0) 0x7f40d4089af0 con 0x7f40dc0cce40 2024-05-10T03:34:03.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.118+0000 7f40e1400640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 0x7f40bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:03.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.119+0000 7f40e1400640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 0x7f40bc06d140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40d0034000 tx=0x7f40d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:03.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.122+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f40d4054bc0 con 0x7f40dc0cce40 2024-05-10T03:34:03.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.324+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40dc09b370 con 0x7f40dc0cce40 2024-05-10T03:34:03.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.812+0000 7f40d9600640 1 -- 172.21.15.55:0/3654172859 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f40d401d070 con 0x7f40dc0cce40 2024-05-10T03:34:03.815 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:34:03.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.816+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 msgr2=0x7f40bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:03.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.816+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 0x7f40bc06d140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40d0034000 tx=0x7f40d002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:03.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.816+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 msgr2=0x7f40dc1f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.816+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f40d400cd40 tx=0x7f40d400a650 comp rx=0 tx=0).stop 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 shutdown_connections 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40dc0cce40 0x7f40dc1f6e50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40dc070dc0 0x7f40dc14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f40bc06ac30 0x7f40bc06d140 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 --2- 172.21.15.55:0/3654172859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40dc06b7c0 0x7f40dc0c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 >> 172.21.15.55:0/3654172859 conn(0x7f40dc0b0cf0 msgr2=0x7f40dc0abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 shutdown_connections 2024-05-10T03:34:03.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:03.817+0000 7f40e3e00640 1 -- 172.21.15.55:0/3654172859 wait complete. 2024-05-10T03:34:03.852 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:34:04.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.123+0000 7f9683a00640 1 Processor -- start 2024-05-10T03:34:04.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.123+0000 7f9683a00640 1 -- start start 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9683a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f967c0cc9a0 0x7f967c0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9683a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f967c163320 0x7f967c165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9683a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c0c7060 con 0x7f967c0b0c40 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9683a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c0cd3b0 con 0x7f967c163320 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9683a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c0c7300 con 0x7f967c0cc9a0 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9681a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f967c163320 0x7f967c165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9681a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f967c163320 0x7f967c165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39614/0 (socket says 172.21.15.55:39614) 2024-05-10T03:34:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.126+0000 7f9681a00640 1 -- 172.21.15.55:0/1217882281 learned_addr learned my addr 172.21.15.55:0/1217882281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.127+0000 7f967be00640 1 --2- 172.21.15.55:0/1217882281 >> v2:172.21.15.162:3300/0 conn(0x7f967c0cc9a0 0x7f967c0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.127+0000 7f9681a00640 1 -- 172.21.15.55:0/1217882281 >> v2:172.21.15.162:3300/0 conn(0x7f967c0cc9a0 msgr2=0x7f967c0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.127+0000 7f9681a00640 1 --2- 172.21.15.55:0/1217882281 >> v2:172.21.15.162:3300/0 conn(0x7f967c0cc9a0 0x7f967c0b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.127+0000 7f9681a00640 1 -- 172.21.15.55:0/1217882281 >> v1:172.21.15.162:6789/0 conn(0x7f967c0b0c40 legacy=0x7f967c0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.127+0000 7f9681a00640 1 -- 172.21.15.55:0/1217882281 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0c75e0 con 0x7f967c163320 2024-05-10T03:34:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.129+0000 7f9681a00640 1 --2- 172.21.15.55:0/1217882281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c163320 0x7f967c165740 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f967c073310 tx=0x7f967402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e372e0b9883e487 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.129+0000 7f967b400640 1 -- 172.21.15.55:0/1217882281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674032440 con 0x7f967c163320 2024-05-10T03:34:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.129+0000 7f967b400640 1 -- 172.21.15.55:0/1217882281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96740325a0 con 0x7f967c163320 2024-05-10T03:34:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.129+0000 7f967b400640 1 -- 172.21.15.55:0/1217882281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674032930 con 0x7f967c163320 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 -- 172.21.15.55:0/1217882281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c163320 msgr2=0x7f967c165740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 --2- 172.21.15.55:0/1217882281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c163320 0x7f967c165740 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f967c073310 tx=0x7f967402a230 comp rx=0 tx=0).stop 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 -- 172.21.15.55:0/1217882281 shutdown_connections 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 --2- 172.21.15.55:0/1217882281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c163320 0x7f967c165740 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 --2- 172.21.15.55:0/1217882281 >> v2:172.21.15.162:3300/0 conn(0x7f967c0cc9a0 0x7f967c0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 -- 172.21.15.55:0/1217882281 >> 172.21.15.55:0/1217882281 conn(0x7f967c0cdbe0 msgr2=0x7f967c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 -- 172.21.15.55:0/1217882281 shutdown_connections 2024-05-10T03:34:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.130+0000 7f9683a00640 1 -- 172.21.15.55:0/1217882281 wait complete. 2024-05-10T03:34:04.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.133+0000 7f9683a00640 1 Processor -- start 2024-05-10T03:34:04.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.133+0000 7f9683a00640 1 -- start start 2024-05-10T03:34:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967c0cc9a0 0x7f967c14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967c163320 0x7f967c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c15c5b0 con 0x7f967c0b0c40 2024-05-10T03:34:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0b5510 con 0x7f967c163320 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.134+0000 7f9683a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c14e5c0 con 0x7f967c0cc9a0 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39618/0 (socket says 172.21.15.55:39618) 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 -- 172.21.15.55:0/3762718808 learned_addr learned my addr 172.21.15.55:0/3762718808 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 -- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967c0cc9a0 msgr2=0x7f967c14e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f967be00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967c0cc9a0 0x7f967c14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967c163320 0x7f967c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967c0cc9a0 0x7f967c14e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 -- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967c163320 msgr2=0x7f967c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967c163320 0x7f967c15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.135+0000 7f9681000640 1 -- 172.21.15.55:0/3762718808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96740094d0 con 0x7f967c0b0c40 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9681000640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f966c00b9c0 tx=0x7f966c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c00cba0 con 0x7f967c0b0c40 2024-05-10T03:34:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c00cd00 con 0x7f967c0b0c40 2024-05-10T03:34:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c150e00 con 0x7f967c0b0c40 2024-05-10T03:34:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c003bb0 con 0x7f967c0b0c40 2024-05-10T03:34:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.136+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c1513c0 con 0x7f967c0b0c40 2024-05-10T03:34:04.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.137+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c067930 con 0x7f967c0b0c40 2024-05-10T03:34:04.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.138+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f966c014440 con 0x7f967c0b0c40 2024-05-10T03:34:04.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.140+0000 7f9678c00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 0x7f965806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.140+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f966c088fe0 con 0x7f967c0b0c40 2024-05-10T03:34:04.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.140+0000 7f967be00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 0x7f965806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.142+0000 7f967be00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 0x7f965806d110 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.145+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f966c054320 con 0x7f967c0b0c40 2024-05-10T03:34:04.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.359+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f967c1561e0 con 0x7f967c0b0c40 2024-05-10T03:34:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.367+0000 7f9678c00640 1 -- 172.21.15.55:0/3762718808 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f966c057940 con 0x7f967c0b0c40 2024-05-10T03:34:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 msgr2=0x7f965806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 0x7f965806d110 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 msgr2=0x7f967c0b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f966c00b9c0 tx=0x7f966c00be80 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 shutdown_connections 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f965806ac00 0x7f965806d110 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967c163320 0x7f967c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967c0cc9a0 0x7f967c14e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 --2- 172.21.15.55:0/3762718808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f967c0b0c40 0x7f967c0b4f70 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 >> 172.21.15.55:0/3762718808 conn(0x7f967c0cdbe0 msgr2=0x7f967c15abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 shutdown_connections 2024-05-10T03:34:04.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.370+0000 7f9683a00640 1 -- 172.21.15.55:0/3762718808 wait complete. 2024-05-10T03:34:04.400 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-05-10T03:34:04.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.668+0000 7f826fc00640 1 Processor -- start 2024-05-10T03:34:04.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.668+0000 7f826fc00640 1 -- start start 2024-05-10T03:34:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.670+0000 7f826fc00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8268071db0 0x7f8268158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.670+0000 7f826fc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f826806a4b0 0x7f82680ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826fc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680708b0 con 0x7f82680b3860 2024-05-10T03:34:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826fc00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82680b2490 con 0x7f826806a4b0 2024-05-10T03:34:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826fc00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8268070050 con 0x7f8268071db0 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f825fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f826806a4b0 0x7f82680ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826dc00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f82680b3860 0x7f82680b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42454/0 (socket says 172.21.15.55:42454) 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826dc00640 1 -- 172.21.15.55:0/2860536468 learned_addr learned my addr 172.21.15.55:0/2860536468 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.671+0000 7f826d200640 1 --2- 172.21.15.55:0/2860536468 >> v2:172.21.15.162:3300/0 conn(0x7f8268071db0 0x7f8268158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.672+0000 7f825fe00640 1 -- 172.21.15.55:0/2860536468 >> v2:172.21.15.162:3300/0 conn(0x7f8268071db0 msgr2=0x7f8268158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:04.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.672+0000 7f825fe00640 1 --2- 172.21.15.55:0/2860536468 >> v2:172.21.15.162:3300/0 conn(0x7f8268071db0 0x7f8268158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.672+0000 7f825fe00640 1 -- 172.21.15.55:0/2860536468 >> v1:172.21.15.55:6789/0 conn(0x7f82680b3860 legacy=0x7f82680b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.672+0000 7f825fe00640 1 -- 172.21.15.55:0/2860536468 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826815b940 con 0x7f826806a4b0 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.673+0000 7f825fe00640 1 --2- 172.21.15.55:0/2860536468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82680ccf90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb80 tx=0x7f825002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3f748dc2ffddde server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.673+0000 7f8267e00640 1 -- 172.21.15.55:0/2860536468 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825002add0 con 0x7f826806a4b0 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.673+0000 7f8267e00640 1 -- 172.21.15.55:0/2860536468 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250032440 con 0x7f826806a4b0 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.673+0000 7f8267e00640 1 -- 172.21.15.55:0/2860536468 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500327d0 con 0x7f826806a4b0 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 -- 172.21.15.55:0/2860536468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 msgr2=0x7f82680ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 --2- 172.21.15.55:0/2860536468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82680ccf90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f82680ceb80 tx=0x7f825002a080 comp rx=0 tx=0).stop 2024-05-10T03:34:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 -- 172.21.15.55:0/2860536468 shutdown_connections 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 --2- 172.21.15.55:0/2860536468 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82680ccf90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 --2- 172.21.15.55:0/2860536468 >> v2:172.21.15.162:3300/0 conn(0x7f8268071db0 0x7f8268158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 -- 172.21.15.55:0/2860536468 >> 172.21.15.55:0/2860536468 conn(0x7f82680cbb90 msgr2=0x7f82680cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 -- 172.21.15.55:0/2860536468 shutdown_connections 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.674+0000 7f826fc00640 1 -- 172.21.15.55:0/2860536468 wait complete. 2024-05-10T03:34:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.676+0000 7f826fc00640 1 Processor -- start 2024-05-10T03:34:04.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.677+0000 7f826fc00640 1 -- start start 2024-05-10T03:34:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 0x7f82681e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82680b3860 0x7f82680b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680acfa0 con 0x7f826806a4b0 2024-05-10T03:34:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82681e4be0 con 0x7f82680b3860 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826fc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8268157770 con 0x7f8268071db0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.678+0000 7f826d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39642/0 (socket says 172.21.15.55:39642) 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 -- 172.21.15.55:0/996641160 learned_addr learned my addr 172.21.15.55:0/996641160 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826dc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82680b3860 0x7f82680b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f825fe00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 0x7f82681e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 -- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 msgr2=0x7f82681e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 0x7f82681e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 -- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82680b3860 msgr2=0x7f82680b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82680b3860 0x7f82680b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f826d200640 1 -- 172.21.15.55:0/996641160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009400 con 0x7f826806a4b0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.679+0000 7f825fe00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 0x7f82681e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f826d200640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f825800d5d0 tx=0x7f825800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258004240 con 0x7f826806a4b0 2024-05-10T03:34:04.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82580043a0 con 0x7f826806a4b0 2024-05-10T03:34:04.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82681e33c0 con 0x7f826806a4b0 2024-05-10T03:34:04.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258003830 con 0x7f826806a4b0 2024-05-10T03:34:04.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.680+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82681e7a60 con 0x7f826806a4b0 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.681+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82680ad210 con 0x7f826806a4b0 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.682+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8258003990 con 0x7f826806a4b0 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.683+0000 7f8265600640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 0x7f824406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.683+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f82580893b0 con 0x7f826806a4b0 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.684+0000 7f825fe00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 0x7f824406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:04.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.685+0000 7f825fe00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 0x7f824406d110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8250034000 tx=0x7f825002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:04.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.688+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f82580546b0 con 0x7f826806a4b0 2024-05-10T03:34:04.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.888+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) v1 -- 0x7f8268066f60 con 0x7f826806a4b0 2024-05-10T03:34:04.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.900+0000 7f8265600640 1 -- 172.21.15.55:0/996641160 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v21) v1 ==== 106+0+0 (secure 0 0 0) 0x7f8258057cd0 con 0x7f826806a4b0 2024-05-10T03:34:04.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 msgr2=0x7f824406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 0x7f824406d110 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8250034000 tx=0x7f825002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:04.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 msgr2=0x7f82681e2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:04.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f825800d5d0 tx=0x7f825800da90 comp rx=0 tx=0).stop 2024-05-10T03:34:04.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 shutdown_connections 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f824406ac00 0x7f824406d110 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82680b3860 0x7f82680b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8268071db0 0x7f82681e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 --2- 172.21.15.55:0/996641160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f826806a4b0 0x7f82681e2de0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 >> 172.21.15.55:0/996641160 conn(0x7f82680cbb90 msgr2=0x7f82680ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 shutdown_connections 2024-05-10T03:34:04.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:04.903+0000 7f826fc00640 1 -- 172.21.15.55:0/996641160 wait complete. 2024-05-10T03:34:04.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-05-10T03:34:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.209+0000 7fd455800640 1 Processor -- start 2024-05-10T03:34:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.210+0000 7fd455800640 1 -- start start 2024-05-10T03:34:05.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.212+0000 7fd455800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.212+0000 7fd455800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd450161a00 0x7fd450071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.212+0000 7fd455800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4500b37a0 con 0x7fd4500b3390 2024-05-10T03:34:05.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.212+0000 7fd455800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500b39f0 con 0x7fd450161a00 2024-05-10T03:34:05.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.212+0000 7fd455800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b3cd0 con 0x7fd4501549e0 2024-05-10T03:34:05.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.213+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.213+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:53422/0 (socket says 172.21.15.55:53422) 2024-05-10T03:34:05.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.213+0000 7fd44fe00640 1 -- 172.21.15.55:0/3661970911 learned_addr learned my addr 172.21.15.55:0/3661970911 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:05.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.213+0000 7fd454e00640 1 --2- 172.21.15.55:0/3661970911 >> v2:172.21.15.162:3300/0 conn(0x7fd450161a00 0x7fd450071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.214+0000 7fd44fe00640 1 -- 172.21.15.55:0/3661970911 >> v2:172.21.15.162:3300/0 conn(0x7fd450161a00 msgr2=0x7fd450071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.214+0000 7fd44fe00640 1 --2- 172.21.15.55:0/3661970911 >> v2:172.21.15.162:3300/0 conn(0x7fd450161a00 0x7fd450071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.214+0000 7fd44fe00640 1 -- 172.21.15.55:0/3661970911 >> v1:172.21.15.162:6789/0 conn(0x7fd4500b3390 legacy=0x7fd4501612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.214+0000 7fd44fe00640 1 -- 172.21.15.55:0/3661970911 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500b3fb0 con 0x7fd4501549e0 2024-05-10T03:34:05.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.215+0000 7fd44fe00640 1 --2- 172.21.15.55:0/3661970911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fefb789c6390b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:05.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd44f400640 1 -- 172.21.15.55:0/3661970911 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440032440 con 0x7fd4501549e0 2024-05-10T03:34:05.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd44f400640 1 -- 172.21.15.55:0/3661970911 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400325a0 con 0x7fd4501549e0 2024-05-10T03:34:05.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd44f400640 1 -- 172.21.15.55:0/3661970911 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440032930 con 0x7fd4501549e0 2024-05-10T03:34:05.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 -- 172.21.15.55:0/3661970911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 msgr2=0x7fd4500b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 --2- 172.21.15.55:0/3661970911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 -- 172.21.15.55:0/3661970911 shutdown_connections 2024-05-10T03:34:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 --2- 172.21.15.55:0/3661970911 >> v2:172.21.15.162:3300/0 conn(0x7fd450161a00 0x7fd450071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 --2- 172.21.15.55:0/3661970911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4501549e0 0x7fd4500b2e50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.216+0000 7fd455800640 1 -- 172.21.15.55:0/3661970911 >> 172.21.15.55:0/3661970911 conn(0x7fd45006a110 msgr2=0x7fd45006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.217+0000 7fd455800640 1 -- 172.21.15.55:0/3661970911 shutdown_connections 2024-05-10T03:34:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.217+0000 7fd455800640 1 -- 172.21.15.55:0/3661970911 wait complete. 2024-05-10T03:34:05.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.219+0000 7fd455800640 1 Processor -- start 2024-05-10T03:34:05.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.219+0000 7fd455800640 1 -- start start 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4500b3390 0x7fd4500ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4501549e0 0x7fd450146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f44b0 con 0x7fd450152e70 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500cd460 con 0x7fd4501549e0 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.220+0000 7fd455800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd450146f40 con 0x7fd4500b3390 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39650/0 (socket says 172.21.15.55:39650) 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 -- 172.21.15.55:0/2115935387 learned_addr learned my addr 172.21.15.55:0/2115935387 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd44fe00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4500b3390 0x7fd4500ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd447400640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4501549e0 0x7fd450146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 -- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4500b3390 msgr2=0x7fd4500ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4500b3390 0x7fd4500ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 -- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4501549e0 msgr2=0x7fd450146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4501549e0 0x7fd450146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.221+0000 7fd454e00640 1 -- 172.21.15.55:0/2115935387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400094d0 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd454e00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd448007e60 tx=0x7fd44800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448014420 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480145a0 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448014950 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4500cd6b0 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.222+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4500cdc70 con 0x7fd450152e70 2024-05-10T03:34:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.223+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45004e290 con 0x7fd450152e70 2024-05-10T03:34:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.224+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd448014ab0 con 0x7fd450152e70 2024-05-10T03:34:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.225+0000 7fd44cc00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 0x7fd42c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.225+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd448089bd0 con 0x7fd450152e70 2024-05-10T03:34:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.226+0000 7fd44fe00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 0x7fd42c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.227+0000 7fd44fe00640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 0x7fd42c06d110 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.230+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd448054e40 con 0x7fd450152e70 2024-05-10T03:34:05.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.497+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) v1 -- 0x7fd450071590 con 0x7fd450152e70 2024-05-10T03:34:05.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.819+0000 7fd44cc00640 1 -- 172.21.15.55:0/2115935387 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v277) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd448058460 con 0x7fd450152e70 2024-05-10T03:34:05.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.822+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 msgr2=0x7fd42c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.822+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 0x7fd42c06d110 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.822+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 msgr2=0x7fd4500c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.822+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd448007e60 tx=0x7fd44800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 shutdown_connections 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd450152e70 0x7fd4500c4750 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd42c06ac00 0x7fd42c06d110 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4501549e0 0x7fd450146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 --2- 172.21.15.55:0/2115935387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4500b3390 0x7fd4500ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 >> 172.21.15.55:0/2115935387 conn(0x7fd45006a110 msgr2=0x7fd450153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 shutdown_connections 2024-05-10T03:34:05.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:05.823+0000 7fd455800640 1 -- 172.21.15.55:0/2115935387 wait complete. 2024-05-10T03:34:05.853 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-05-10T03:34:05.853 INFO:tasks.cephfs_test_runner:Starting test: test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-05-10T03:34:05.853 INFO:tasks.cephfs_test_runner:test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:34:05.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:06.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.138+0000 7fc015200640 1 Processor -- start 2024-05-10T03:34:06.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.138+0000 7fc015200640 1 -- start start 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc015200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fc0100b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc015200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc010161a00 0x7fc010071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc015200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0100b37a0 con 0x7fc0100b3390 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc015200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100b39f0 con 0x7fc010161a00 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc015200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b3cd0 con 0x7fc0101549e0 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc00ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc0100b3390 0x7fc0101612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42458/0 (socket says 172.21.15.55:42458) 2024-05-10T03:34:06.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.141+0000 7fc00ea00640 1 -- 172.21.15.55:0/2782609534 learned_addr learned my addr 172.21.15.55:0/2782609534 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:06.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00fe00640 1 --2- 172.21.15.55:0/2782609534 >> v2:172.21.15.162:3300/0 conn(0x7fc010161a00 0x7fc010071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00f400640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fc0100b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138661546 0 0) 0x7fc0100b37a0 con 0x7fc0100b3390 2024-05-10T03:34:06.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff4003520 con 0x7fc0100b3390 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587559009 0 0) 0x7fbff4003520 con 0x7fc0100b3390 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 msgr2=0x7fc0100b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fc0100b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 >> v2:172.21.15.162:3300/0 conn(0x7fc010161a00 msgr2=0x7fc010071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 --2- 172.21.15.55:0/2782609534 >> v2:172.21.15.162:3300/0 conn(0x7fc010161a00 0x7fc010071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100b3fb0 con 0x7fc0100b3390 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fbffc001670 con 0x7fc0100b3390 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 >> v1:172.21.15.55:6789/0 conn(0x7fc0100b3390 legacy=0x7fc0101612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff40039e0 0x7fbff4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fbff40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.142+0000 7fc00e000640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff4004d00 0x7fbff4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b39f0 con 0x7fbff40039e0 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100b3cd0 con 0x7fbff4004d00 2024-05-10T03:34:06.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff400bab0 con 0x7fc0101549e0 2024-05-10T03:34:06.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff40039e0 0x7fbff4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00ea00640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fbff40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00fe00640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff4004d00 0x7fbff4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 -- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 msgr2=0x7fbff40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fbff40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 -- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff4004d00 msgr2=0x7fbff4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff4004d00 0x7fbff4005110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00f400640 1 -- 172.21.15.55:0/2782609534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4003520 con 0x7fbff40039e0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.143+0000 7fc00ea00640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0101549e0 0x7fbff40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.145+0000 7fc00f400640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff40039e0 0x7fbff4003e50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc000009a90 tx=0x7fbff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3449b268d39a0649 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.145+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000031400 con 0x7fbff40039e0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.145+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00002d050 con 0x7fbff40039e0 2024-05-10T03:34:06.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.145+0000 7fc00e000640 1 -- 172.21.15.55:0/2782609534 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000317a0 con 0x7fbff40039e0 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 -- 172.21.15.55:0/2782609534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff40039e0 msgr2=0x7fbff4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 --2- 172.21.15.55:0/2782609534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff40039e0 0x7fbff4003e50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc000009a90 tx=0x7fbff40057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc00fe00640 1 -- 172.21.15.55:0/2782609534 reap_dead start 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 -- 172.21.15.55:0/2782609534 shutdown_connections 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 -- 172.21.15.55:0/2782609534 >> 172.21.15.55:0/2782609534 conn(0x7fc01006a110 msgr2=0x7fc01006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:06.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 -- 172.21.15.55:0/2782609534 shutdown_connections 2024-05-10T03:34:06.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.146+0000 7fc015200640 1 -- 172.21.15.55:0/2782609534 wait complete. 2024-05-10T03:34:06.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.148+0000 7fc015200640 1 Processor -- start 2024-05-10T03:34:06.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.149+0000 7fc015200640 1 -- start start 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff40039e0 0x7fc0100ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc010152ef0 0x7fc0100cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0101f4230 con 0x7fc0101529a0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100cba40 con 0x7fc010152ef0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc015200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100cd270 con 0x7fbff40039e0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc00ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc00ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39676/0 (socket says 172.21.15.55:39676) 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.150+0000 7fc00ea00640 1 -- 172.21.15.55:0/1066382195 learned_addr learned my addr 172.21.15.55:0/1066382195 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00f400640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff40039e0 0x7fc0100ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00fe00640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc010152ef0 0x7fc0100cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 -- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff40039e0 msgr2=0x7fc0100ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff40039e0 0x7fc0100ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 -- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc010152ef0 msgr2=0x7fc0100cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc010152ef0 0x7fc0100cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 -- 172.21.15.55:0/1066382195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100cbc90 con 0x7fc0101529a0 2024-05-10T03:34:06.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.151+0000 7fc00ea00640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e780 tx=0x7fbffc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.152+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00ee30 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.152+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc00c870 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.152+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00cc00 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.152+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0100cbf70 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.152+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0100cc530 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.153+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01004e290 con 0x7fc0101529a0 2024-05-10T03:34:06.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.154+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbffc01d460 con 0x7fc0101529a0 2024-05-10T03:34:06.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.155+0000 7fc007400640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 0x7fbff006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.155+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fbffc089110 con 0x7fc0101529a0 2024-05-10T03:34:06.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.156+0000 7fc00f400640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 0x7fbff006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.157+0000 7fc00f400640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 0x7fbff006d110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc000033000 tx=0x7fc000008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.159+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbffc054410 con 0x7fc0101529a0 2024-05-10T03:34:06.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.358+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc01006dc80 con 0x7fc0101529a0 2024-05-10T03:34:06.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.360+0000 7fc007400640 1 -- 172.21.15.55:0/1066382195 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+12726 (secure 0 0 0) 0x7fbffc057a30 con 0x7fc0101529a0 2024-05-10T03:34:06.363 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:06.363 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:03.781803+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/3533670538":"2024-05-11T03:34:00.752706+0000","172.21.15.162:6835/3533670538":"2024-05-11T03:34:00.752706+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.363+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 msgr2=0x7fbff006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.363+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 0x7fbff006d110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc000033000 tx=0x7fc000008030 comp rx=0 tx=0).stop 2024-05-10T03:34:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.363+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 msgr2=0x7fc0101520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.363+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00e780 tx=0x7fbffc00ec40 comp rx=0 tx=0).stop 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 shutdown_connections 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbff006ac00 0x7fbff006d110 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc010152ef0 0x7fc0100cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0101529a0 0x7fc0101520e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 --2- 172.21.15.55:0/1066382195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff40039e0 0x7fc0100ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 >> 172.21.15.55:0/1066382195 conn(0x7fc01006a110 msgr2=0x7fc0101537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 shutdown_connections 2024-05-10T03:34:06.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.364+0000 7fc015200640 1 -- 172.21.15.55:0/1066382195 wait complete. 2024-05-10T03:34:06.393 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:06.393 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:06.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.673+0000 7fcb69000640 1 Processor -- start 2024-05-10T03:34:06.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.673+0000 7fcb69000640 1 -- start start 2024-05-10T03:34:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb69000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcb640719f0 0x7fcb6406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb69000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb64071e00 0x7fcb64141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb69000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6415fdf0 con 0x7fcb6406b840 2024-05-10T03:34:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb69000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6415f1e0 con 0x7fcb64071e00 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb69000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6415f4c0 con 0x7fcb640719f0 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb63e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb64071e00 0x7fcb64141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb63e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb64071e00 0x7fcb64141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39678/0 (socket says 172.21.15.55:39678) 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.676+0000 7fcb63e00640 1 -- 172.21.15.55:0/1759950089 learned_addr learned my addr 172.21.15.55:0/1759950089 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.677+0000 7fcb63e00640 1 -- 172.21.15.55:0/1759950089 >> v2:172.21.15.162:3300/0 conn(0x7fcb640719f0 msgr2=0x7fcb6406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.677+0000 7fcb63e00640 1 --2- 172.21.15.55:0/1759950089 >> v2:172.21.15.162:3300/0 conn(0x7fcb640719f0 0x7fcb6406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.677+0000 7fcb63e00640 1 -- 172.21.15.55:0/1759950089 >> v1:172.21.15.162:6789/0 conn(0x7fcb6406b840 legacy=0x7fcb640700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.677+0000 7fcb63e00640 1 -- 172.21.15.55:0/1759950089 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415f7a0 con 0x7fcb64071e00 2024-05-10T03:34:06.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.678+0000 7fcb63e00640 1 --2- 172.21.15.55:0/1759950089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64141600 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcb640b41f0 tx=0x7fcb5c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9f8566bd65d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.678+0000 7fcb62000640 1 -- 172.21.15.55:0/1759950089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c031600 con 0x7fcb64071e00 2024-05-10T03:34:06.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.678+0000 7fcb62000640 1 -- 172.21.15.55:0/1759950089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c02d050 con 0x7fcb64071e00 2024-05-10T03:34:06.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.678+0000 7fcb62000640 1 -- 172.21.15.55:0/1759950089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c031980 con 0x7fcb64071e00 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 -- 172.21.15.55:0/1759950089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 msgr2=0x7fcb64141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 --2- 172.21.15.55:0/1759950089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64141600 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcb640b41f0 tx=0x7fcb5c02a660 comp rx=0 tx=0).stop 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 -- 172.21.15.55:0/1759950089 shutdown_connections 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 --2- 172.21.15.55:0/1759950089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64141600 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 --2- 172.21.15.55:0/1759950089 >> v2:172.21.15.162:3300/0 conn(0x7fcb640719f0 0x7fcb6406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 -- 172.21.15.55:0/1759950089 >> 172.21.15.55:0/1759950089 conn(0x7fcb6406c750 msgr2=0x7fcb640b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 -- 172.21.15.55:0/1759950089 shutdown_connections 2024-05-10T03:34:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.679+0000 7fcb69000640 1 -- 172.21.15.55:0/1759950089 wait complete. 2024-05-10T03:34:06.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.681+0000 7fcb69000640 1 Processor -- start 2024-05-10T03:34:06.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.682+0000 7fcb69000640 1 -- start start 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6406b840 0x7fcb640cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 0x7fcb6412c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64130a90 con 0x7fcb64071e00 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6412cf30 con 0x7fcb6406b840 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.683+0000 7fcb69000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6412dd90 con 0x7fcb640719f0 2024-05-10T03:34:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39684/0 (socket says 172.21.15.55:39684) 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 -- 172.21.15.55:0/3247833818 learned_addr learned my addr 172.21.15.55:0/3247833818 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb62a00640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 0x7fcb6412c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63400640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6406b840 0x7fcb640cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 -- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 msgr2=0x7fcb6412c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 0x7fcb6412c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 -- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6406b840 msgr2=0x7fcb640cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6406b840 0x7fcb640cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb63e00640 1 -- 172.21.15.55:0/3247833818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009b30 con 0x7fcb64071e00 2024-05-10T03:34:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.684+0000 7fcb62a00640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 0x7fcb6412c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb63e00640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c034040 tx=0x7fcb5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c03b820 con 0x7fcb64071e00 2024-05-10T03:34:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c03b9a0 con 0x7fcb64071e00 2024-05-10T03:34:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6412e220 con 0x7fcb64071e00 2024-05-10T03:34:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c03bd50 con 0x7fcb64071e00 2024-05-10T03:34:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.685+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6412d460 con 0x7fcb64071e00 2024-05-10T03:34:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.687+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6412df40 con 0x7fcb64071e00 2024-05-10T03:34:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.687+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb5c038050 con 0x7fcb64071e00 2024-05-10T03:34:06.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.689+0000 7fcb5b400640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 0x7fcb4006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:06.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.689+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fcb5c02ae40 con 0x7fcb64071e00 2024-05-10T03:34:06.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.690+0000 7fcb63400640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 0x7fcb4006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.691+0000 7fcb63400640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 0x7fcb4006d140 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b810 tx=0x7fcb54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.694+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcb6412df40 con 0x7fcb64071e00 2024-05-10T03:34:06.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.893+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb6412df00 con 0x7fcb64071e00 2024-05-10T03:34:06.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.894+0000 7fcb5b400640 1 -- 172.21.15.55:0/3247833818 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb5c04b020 con 0x7fcb64071e00 2024-05-10T03:34:06.897 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.897+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 msgr2=0x7fcb4006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.897+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 0x7fcb4006d140 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b810 tx=0x7fcb54008040 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.897+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 msgr2=0x7fcb64143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.897+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c034040 tx=0x7fcb5c004a40 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 shutdown_connections 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb64071e00 0x7fcb64143200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb640719f0 0x7fcb6412c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcb4006ac30 0x7fcb4006d140 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 --2- 172.21.15.55:0/3247833818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6406b840 0x7fcb640cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 >> 172.21.15.55:0/3247833818 conn(0x7fcb6406c750 msgr2=0x7fcb64142300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 shutdown_connections 2024-05-10T03:34:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:06.898+0000 7fcb69000640 1 -- 172.21.15.55:0/3247833818 wait complete. 2024-05-10T03:34:06.928 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-05-10T03:34:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.205+0000 7f919d200640 1 Processor -- start 2024-05-10T03:34:07.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.205+0000 7f919d200640 1 -- start start 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.207+0000 7f919d200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f9198144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f919d200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91980b3190 0x7f91980cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f919d200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919806d350 con 0x7f919806d7e0 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f919d200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919806c740 con 0x7f91980b3190 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f919d200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919806ca20 con 0x7f9198144710 2024-05-10T03:34:07.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f9197e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91980b3190 0x7f91980cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f9197e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91980b3190 0x7f91980cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39694/0 (socket says 172.21.15.55:39694) 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f9197e00640 1 -- 172.21.15.55:0/627147099 learned_addr learned my addr 172.21.15.55:0/627147099 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f9197400640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f9198144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.208+0000 7f9197e00640 1 -- 172.21.15.55:0/627147099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 msgr2=0x7f9198144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.209+0000 7f9197e00640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f9198144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.209+0000 7f9197e00640 1 -- 172.21.15.55:0/627147099 >> v1:172.21.15.162:6789/0 conn(0x7f919806d7e0 legacy=0x7f91980b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.209+0000 7f9197e00640 1 -- 172.21.15.55:0/627147099 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919806cd00 con 0x7f91980b3190 2024-05-10T03:34:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.210+0000 7f9197e00640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91980b3190 0x7f91980cd9c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9190009e50 tx=0x7f919002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33eaf97bbf974a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.210+0000 7f9196a00640 1 -- 172.21.15.55:0/627147099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190031600 con 0x7f91980b3190 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.210+0000 7f9196a00640 1 -- 172.21.15.55:0/627147099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919002d050 con 0x7f91980b3190 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.210+0000 7f9196a00640 1 -- 172.21.15.55:0/627147099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190031980 con 0x7f91980b3190 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 -- 172.21.15.55:0/627147099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91980b3190 msgr2=0x7f91980cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91980b3190 0x7f91980cd9c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9190009e50 tx=0x7f919002a5b0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 -- 172.21.15.55:0/627147099 shutdown_connections 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91980b3190 0x7f91980cd9c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 --2- 172.21.15.55:0/627147099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f9198144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 -- 172.21.15.55:0/627147099 >> 172.21.15.55:0/627147099 conn(0x7f919806a110 msgr2=0x7f919806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 -- 172.21.15.55:0/627147099 shutdown_connections 2024-05-10T03:34:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.211+0000 7f919d200640 1 -- 172.21.15.55:0/627147099 wait complete. 2024-05-10T03:34:07.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.214+0000 7f919d200640 1 Processor -- start 2024-05-10T03:34:07.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.214+0000 7f919d200640 1 -- start start 2024-05-10T03:34:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91980b3190 0x7f919815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f919815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91980a89b0 con 0x7f919806d7e0 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9198151050 con 0x7f91980b3190 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.215+0000 7f919d200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91980a8c00 con 0x7f9198144710 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39700/0 (socket says 172.21.15.55:39700) 2024-05-10T03:34:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 -- 172.21.15.55:0/2630884137 learned_addr learned my addr 172.21.15.55:0/2630884137 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 -- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 msgr2=0x7f919815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f919815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f918ea00640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91980b3190 0x7f919815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 -- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91980b3190 msgr2=0x7f919815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91980b3190 0x7f919815cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.216+0000 7f9197400640 1 -- 172.21.15.55:0/2630884137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9190009b30 con 0x7f919806d7e0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f9197400640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f918800d5d0 tx=0x7f918800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188004240 con 0x7f919806d7e0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880043a0 con 0x7f919806d7e0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919814b290 con 0x7f919806d7e0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188003830 con 0x7f919806d7e0 2024-05-10T03:34:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.217+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919814b850 con 0x7f919806d7e0 2024-05-10T03:34:07.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.218+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9198066ce0 con 0x7f919806d7e0 2024-05-10T03:34:07.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.220+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9188003990 con 0x7f919806d7e0 2024-05-10T03:34:07.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.220+0000 7f918fe00640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 0x7f917406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:07.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.221+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9188089510 con 0x7f919806d7e0 2024-05-10T03:34:07.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.221+0000 7f918ea00640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 0x7f917406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:07.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.223+0000 7f918ea00640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 0x7f917406d110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f91980cb5a0 tx=0x7f9180008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:07.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.225+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9188054780 con 0x7f919806d7e0 2024-05-10T03:34:07.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.490+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) v1 -- 0x7f919804e290 con 0x7f919806d7e0 2024-05-10T03:34:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.832+0000 7f918fe00640 1 -- 172.21.15.55:0/2630884137 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v279) v1 ==== 148+0+0 (secure 0 0 0) 0x7f9188057da0 con 0x7f919806d7e0 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 msgr2=0x7f917406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 0x7f917406d110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f91980cb5a0 tx=0x7f9180008040 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 msgr2=0x7f9198150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f918800d5d0 tx=0x7f918800da90 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 shutdown_connections 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9198144710 0x7f919815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f917406ac00 0x7f917406d110 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91980b3190 0x7f919815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 --2- 172.21.15.55:0/2630884137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f919806d7e0 0x7f9198150b10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 >> 172.21.15.55:0/2630884137 conn(0x7f919806a110 msgr2=0x7f91980ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 shutdown_connections 2024-05-10T03:34:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:07.836+0000 7f919d200640 1 -- 172.21.15.55:0/2630884137 wait complete. 2024-05-10T03:34:07.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:34:08.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.155+0000 7fe396000640 1 Processor -- start 2024-05-10T03:34:08.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.155+0000 7fe396000640 1 -- start start 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe396000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe390161e70 0x7fe3901546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe396000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 0x7fe39006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe396000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe390072270 con 0x7fe39006b660 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe396000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe390161a50 con 0x7fe390161e70 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe396000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe390154c30 con 0x7fe3900ce660 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe394c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe390161e70 0x7fe3901546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe394c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe390161e70 0x7fe3901546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39710/0 (socket says 172.21.15.55:39710) 2024-05-10T03:34:08.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.158+0000 7fe394c00640 1 -- 172.21.15.55:0/2279296983 learned_addr learned my addr 172.21.15.55:0/2279296983 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:08.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe38fe00640 1 --2- 172.21.15.55:0/2279296983 >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 0x7fe39006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe394c00640 1 -- 172.21.15.55:0/2279296983 >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 msgr2=0x7fe39006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:08.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe394c00640 1 --2- 172.21.15.55:0/2279296983 >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 0x7fe39006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe394c00640 1 -- 172.21.15.55:0/2279296983 >> v1:172.21.15.162:6789/0 conn(0x7fe39006b660 legacy=0x7fe3900ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:08.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe394c00640 1 -- 172.21.15.55:0/2279296983 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39006ba70 con 0x7fe390161e70 2024-05-10T03:34:08.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.159+0000 7fe38fe00640 1 --2- 172.21.15.55:0/2279296983 >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 0x7fe39006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:08.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.160+0000 7fe394c00640 1 --2- 172.21.15.55:0/2279296983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe3901546f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe390158300 tx=0x7fe38402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31258b4a6b4c68c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:08.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.161+0000 7fe38f400640 1 -- 172.21.15.55:0/2279296983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384032440 con 0x7fe390161e70 2024-05-10T03:34:08.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.161+0000 7fe38f400640 1 -- 172.21.15.55:0/2279296983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3840325a0 con 0x7fe390161e70 2024-05-10T03:34:08.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.161+0000 7fe38f400640 1 -- 172.21.15.55:0/2279296983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384032930 con 0x7fe390161e70 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 -- 172.21.15.55:0/2279296983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 msgr2=0x7fe3901546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 --2- 172.21.15.55:0/2279296983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe3901546f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe390158300 tx=0x7fe38402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 -- 172.21.15.55:0/2279296983 shutdown_connections 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 --2- 172.21.15.55:0/2279296983 >> v2:172.21.15.162:3300/0 conn(0x7fe3900ce660 0x7fe39006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 --2- 172.21.15.55:0/2279296983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe3901546f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 -- 172.21.15.55:0/2279296983 >> 172.21.15.55:0/2279296983 conn(0x7fe39006c750 msgr2=0x7fe3900b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 -- 172.21.15.55:0/2279296983 shutdown_connections 2024-05-10T03:34:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.162+0000 7fe396000640 1 -- 172.21.15.55:0/2279296983 wait complete. 2024-05-10T03:34:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.164+0000 7fe396000640 1 Processor -- start 2024-05-10T03:34:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.164+0000 7fe396000640 1 -- start start 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 0x7fe390160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3900ce660 0x7fe390160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39015d030 con 0x7fe390161e70 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39015c160 con 0x7fe39006b660 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe396000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39015a140 con 0x7fe3900ce660 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe395600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe395600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39724/0 (socket says 172.21.15.55:39724) 2024-05-10T03:34:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.166+0000 7fe395600640 1 -- 172.21.15.55:0/1897951077 learned_addr learned my addr 172.21.15.55:0/1897951077 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe38fe00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3900ce660 0x7fe390160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe394c00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 0x7fe390160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 -- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3900ce660 msgr2=0x7fe390160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3900ce660 0x7fe390160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 -- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 msgr2=0x7fe390160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 0x7fe390160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 -- 172.21.15.55:0/1897951077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840094d0 con 0x7fe390161e70 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe394c00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 0x7fe390160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe395600640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ba10 tx=0x7fe38800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800cb40 con 0x7fe390161e70 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38800cca0 con 0x7fe390161e70 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388014680 con 0x7fe390161e70 2024-05-10T03:34:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39015c3b0 con 0x7fe390161e70 2024-05-10T03:34:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.167+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39015c970 con 0x7fe390161e70 2024-05-10T03:34:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.169+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39004e290 con 0x7fe390161e70 2024-05-10T03:34:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.170+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe38801d3b0 con 0x7fe390161e70 2024-05-10T03:34:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.171+0000 7fe38cc00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 0x7fe37006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.171+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe388088fd0 con 0x7fe390161e70 2024-05-10T03:34:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.171+0000 7fe394c00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 0x7fe37006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.173+0000 7fe394c00640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 0x7fe37006d140 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe384034000 tx=0x7fe38402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:08.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.176+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3880542d0 con 0x7fe390161e70 2024-05-10T03:34:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.378+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe390070400 con 0x7fe390161e70 2024-05-10T03:34:08.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.385+0000 7fe38cc00640 1 -- 172.21.15.55:0/1897951077 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3880578f0 con 0x7fe390161e70 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 msgr2=0x7fe37006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 0x7fe37006d140 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe384034000 tx=0x7fe38402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 msgr2=0x7fe390155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ba10 tx=0x7fe38800bed0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 shutdown_connections 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe390161e70 0x7fe390155e00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3900ce660 0x7fe390160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe37006ac30 0x7fe37006d140 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 --2- 172.21.15.55:0/1897951077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe39006b660 0x7fe390160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 >> 172.21.15.55:0/1897951077 conn(0x7fe39006c750 msgr2=0x7fe39006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:08.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 shutdown_connections 2024-05-10T03:34:08.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:08.388+0000 7fe396000640 1 -- 172.21.15.55:0/1897951077 wait complete. 2024-05-10T03:34:08.416 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:34:08.416 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:08.528 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:08.645 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:34:08.645 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:08.645 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:08.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:08.705 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:08.705 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:08.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:08.760 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:08.760 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:08.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:08.821 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:08.821 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:08.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:08.878 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:34:08.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:09.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.169+0000 7f991b200640 1 Processor -- start 2024-05-10T03:34:09.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.169+0000 7f991b200640 1 -- start start 2024-05-10T03:34:09.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.171+0000 7f991b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c06c9e0 0x7f991c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.171+0000 7f991b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f991c161280 0x7f991c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.171+0000 7f991b200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06c550 con 0x7f991c15b2c0 2024-05-10T03:34:09.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.171+0000 7f991b200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c15b6d0 con 0x7f991c161280 2024-05-10T03:34:09.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.171+0000 7f991b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c161bd0 con 0x7f991c06c9e0 2024-05-10T03:34:09.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f9919e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f991c15b2c0 0x7f991c0b4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42462/0 (socket says 172.21.15.55:42462) 2024-05-10T03:34:09.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f991a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f991c161280 0x7f991c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f9919e00640 1 -- 172.21.15.55:0/3850617529 learned_addr learned my addr 172.21.15.55:0/3850617529 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:09.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f9919400640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c06c9e0 0x7f991c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f991a800640 1 -- 172.21.15.55:0/3850617529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c06c9e0 msgr2=0x7f991c06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:09.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f991a800640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c06c9e0 0x7f991c06cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f991a800640 1 -- 172.21.15.55:0/3850617529 >> v1:172.21.15.55:6789/0 conn(0x7f991c15b2c0 legacy=0x7f991c0b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f991a800640 1 -- 172.21.15.55:0/3850617529 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c06bf20 con 0x7f991c161280 2024-05-10T03:34:09.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.172+0000 7f9918a00640 1 -- 172.21.15.55:0/3850617529 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901624773 0 0) 0x7f991c06c550 con 0x7f991c15b2c0 2024-05-10T03:34:09.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.173+0000 7f991a800640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c161280 0x7f991c161690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d90 tx=0x7f991402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8b18861a0078 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.174+0000 7f9918a00640 1 -- 172.21.15.55:0/3850617529 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914031600 con 0x7f991c161280 2024-05-10T03:34:09.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.174+0000 7f9918a00640 1 -- 172.21.15.55:0/3850617529 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991402d050 con 0x7f991c161280 2024-05-10T03:34:09.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.174+0000 7f9918a00640 1 -- 172.21.15.55:0/3850617529 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914031980 con 0x7f991c161280 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.174+0000 7f991b200640 1 -- 172.21.15.55:0/3850617529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c161280 msgr2=0x7f991c161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.174+0000 7f991b200640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c161280 0x7f991c161690 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d90 tx=0x7f991402a5d0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 -- 172.21.15.55:0/3850617529 shutdown_connections 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c06c9e0 0x7f991c06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 --2- 172.21.15.55:0/3850617529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c161280 0x7f991c161690 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 -- 172.21.15.55:0/3850617529 >> 172.21.15.55:0/3850617529 conn(0x7f991c0b2d60 msgr2=0x7f991c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 -- 172.21.15.55:0/3850617529 shutdown_connections 2024-05-10T03:34:09.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.175+0000 7f991b200640 1 -- 172.21.15.55:0/3850617529 wait complete. 2024-05-10T03:34:09.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.177+0000 7f991b200640 1 Processor -- start 2024-05-10T03:34:09.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.177+0000 7f991b200640 1 -- start start 2024-05-10T03:34:09.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c15b2c0 0x7f991c147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f991c161280 0x7f991c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c157ed0 con 0x7f991c06c9e0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c155160 con 0x7f991c161280 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f991b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c148cc0 con 0x7f991c15b2c0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f9919e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f9919e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39734/0 (socket says 172.21.15.55:39734) 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.179+0000 7f9919e00640 1 -- 172.21.15.55:0/822640288 learned_addr learned my addr 172.21.15.55:0/822640288 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919400640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c15b2c0 0x7f991c147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 -- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c15b2c0 msgr2=0x7f991c147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c15b2c0 0x7f991c147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 -- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f991c161280 msgr2=0x7f991c0cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f991c161280 0x7f991c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 -- 172.21.15.55:0/822640288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9914009a70 con 0x7f991c06c9e0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9919e00640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c430 tx=0x7f990c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0074e0 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c007660 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004a20 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.180+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c157000 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.181+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c157530 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.182+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c158140 con 0x7f991c06c9e0 2024-05-10T03:34:09.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.183+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f990c004b80 con 0x7f991c06c9e0 2024-05-10T03:34:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.184+0000 7f9912000640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 0x7f98fc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.184+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f990c088fc0 con 0x7f991c06c9e0 2024-05-10T03:34:09.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.185+0000 7f9919400640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 0x7f98fc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.186+0000 7f9919400640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 0x7f98fc06d110 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9908006fd0 tx=0x7f9908008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.189+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f990c054350 con 0x7f991c06c9e0 2024-05-10T03:34:09.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.430+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991c04e290 con 0x7f991c06c9e0 2024-05-10T03:34:09.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.431+0000 7f9912000640 1 -- 172.21.15.55:0/822640288 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f990c057970 con 0x7f991c06c9e0 2024-05-10T03:34:09.435 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:09.435 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1949269718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1949269718},{"type":"v1","addr":"172.21.15.55:6834","nonce":1949269718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6836/368065200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":368065200},{"type":"v1","addr":"172.21.15.162:6836","nonce":368065200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/1951187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6835","nonce":1951187831},{"type":"v1","addr":"172.21.15.162:6837","nonce":1951187831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6525,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/643527999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":643527999},{"type":"v1","addr":"172.21.15.55:6835","nonce":643527999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2024-05-10T03:34:09.435 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 41 2024-05-10T03:34:09.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 msgr2=0x7f98fc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 0x7f98fc06d110 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9908006fd0 tx=0x7f9908008040 comp rx=0 tx=0).stop 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 msgr2=0x7f991c148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f990c00c430 tx=0x7f990c00c8f0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 shutdown_connections 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f98fc06ac00 0x7f98fc06d110 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f991c161280 0x7f991c0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f991c15b2c0 0x7f991c147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 --2- 172.21.15.55:0/822640288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991c06c9e0 0x7f991c148240 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 >> 172.21.15.55:0/822640288 conn(0x7f991c0b2d60 msgr2=0x7f991c06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 shutdown_connections 2024-05-10T03:34:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.434+0000 7f991b200640 1 -- 172.21.15.55:0/822640288 wait complete. 2024-05-10T03:34:09.559 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:34:09.559 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:34:09.559 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:34:09.559 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:09.560 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:34:09.560 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:34:09.560 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:34:09.560 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:09.560 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:34:09.560 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:34:09.560 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:34:09.561 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:09.561 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:34:09.561 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:34:09.561 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:34:09.561 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:09.587 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:34:09.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:34:09.600 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:34:09.600 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:34:09.624 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:34:09.624 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:34:09.625 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:34:09.625 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:34:09.648 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:34:09.648 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:09.654 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:34:09.655 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:09.672 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:34:09.672 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:09.678 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:34:09.678 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:09.679 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:34:09.875 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:34:09.879 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:34:09.906 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:34:09.908 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:34:09.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.938+0000 7ff68d800640 1 Processor -- start 2024-05-10T03:34:09.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.938+0000 7ff68d800640 1 -- start start 2024-05-10T03:34:09.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.940+0000 7ff68d800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6880b47f0 0x7ff6880b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.940+0000 7ff68d800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880b2e10 con 0x7ff6880d27f0 2024-05-10T03:34:09.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.940+0000 7ff68d800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880b3c50 con 0x7ff68815d7e0 2024-05-10T03:34:09.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.940+0000 7ff68d800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6880b3040 con 0x7ff6880b47f0 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff687400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6880b47f0 0x7ff6880b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff686a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff6880d27f0 0x7ff6880b6860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42492/0 (socket says 172.21.15.55:42492) 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff686a00640 1 -- 172.21.15.55:0/989736624 learned_addr learned my addr 172.21.15.55:0/989736624 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff687400640 1 -- 172.21.15.55:0/989736624 >> v1:172.21.15.162:6789/0 conn(0x7ff68815d7e0 legacy=0x7ff6880b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff686000640 1 -- 172.21.15.55:0/989736624 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104537449 0 0) 0x7ff6880b2e10 con 0x7ff6880d27f0 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff687400640 1 -- 172.21.15.55:0/989736624 >> v1:172.21.15.55:6789/0 conn(0x7ff6880d27f0 legacy=0x7ff6880b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.941+0000 7ff687400640 1 -- 172.21.15.55:0/989736624 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880b3320 con 0x7ff6880b47f0 2024-05-10T03:34:09.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.942+0000 7ff687400640 1 --2- 172.21.15.55:0/989736624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6880b47f0 0x7ff6880b28d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097a0 tx=0x7ff68002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57b2652fa385456 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.943+0000 7ff686000640 1 -- 172.21.15.55:0/989736624 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680032440 con 0x7ff6880b47f0 2024-05-10T03:34:09.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.943+0000 7ff686000640 1 -- 172.21.15.55:0/989736624 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6800325a0 con 0x7ff6880b47f0 2024-05-10T03:34:09.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.943+0000 7ff686000640 1 -- 172.21.15.55:0/989736624 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680032930 con 0x7ff6880b47f0 2024-05-10T03:34:09.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.943+0000 7ff68d800640 1 -- 172.21.15.55:0/989736624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6880b47f0 msgr2=0x7ff6880b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.943+0000 7ff68d800640 1 --2- 172.21.15.55:0/989736624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6880b47f0 0x7ff6880b28d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097a0 tx=0x7ff68002a1a0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.944+0000 7ff68d800640 1 -- 172.21.15.55:0/989736624 shutdown_connections 2024-05-10T03:34:09.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.944+0000 7ff68d800640 1 --2- 172.21.15.55:0/989736624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6880b47f0 0x7ff6880b28d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.944+0000 7ff68d800640 1 -- 172.21.15.55:0/989736624 >> 172.21.15.55:0/989736624 conn(0x7ff6880d0880 msgr2=0x7ff6880d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:09.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.944+0000 7ff68d800640 1 -- 172.21.15.55:0/989736624 shutdown_connections 2024-05-10T03:34:09.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.944+0000 7ff68d800640 1 -- 172.21.15.55:0/989736624 wait complete. 2024-05-10T03:34:09.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.947+0000 7ff68d800640 1 Processor -- start 2024-05-10T03:34:09.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.947+0000 7ff68d800640 1 -- start start 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6880d27f0 0x7ff6880c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff68815d7e0 0x7ff6880c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881e83d0 con 0x7ff6880b47f0 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881e9b30 con 0x7ff68815d7e0 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff68d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880c74b0 con 0x7ff6880d27f0 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff686a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff686a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39780/0 (socket says 172.21.15.55:39780) 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff686a00640 1 -- 172.21.15.55:0/2859728017 learned_addr learned my addr 172.21.15.55:0/2859728017 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:09.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff687400640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff68815d7e0 0x7ff6880c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.949+0000 7ff67e000640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6880d27f0 0x7ff6880c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 -- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6880d27f0 msgr2=0x7ff6880c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6880d27f0 0x7ff6880c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 -- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff68815d7e0 msgr2=0x7ff6880c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff68815d7e0 0x7ff6880c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 -- 172.21.15.55:0/2859728017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680009400 con 0x7ff6880b47f0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff686a00640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c7f0 tx=0x7ff67800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800a640 con 0x7ff6880b47f0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800a7c0 con 0x7ff6880b47f0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.950+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880c7700 con 0x7ff6880b47f0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.951+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800ab70 con 0x7ff6880b47f0 2024-05-10T03:34:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.951+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880c7cc0 con 0x7ff6880b47f0 2024-05-10T03:34:09.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.952+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688067220 con 0x7ff6880b47f0 2024-05-10T03:34:09.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.953+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff67800acd0 con 0x7ff6880b47f0 2024-05-10T03:34:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.954+0000 7ff67f400640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 0x7ff66406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.954+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff678088980 con 0x7ff6880b47f0 2024-05-10T03:34:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.955+0000 7ff67e000640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 0x7ff66406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:09.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.956+0000 7ff67e000640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 0x7ff66406d110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff6881f7fb0 tx=0x7ff670009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:09.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:09.959+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff678053d30 con 0x7ff6880b47f0 2024-05-10T03:34:10.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.174+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6880c63a0 con 0x7ff6880b47f0 2024-05-10T03:34:10.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.934+0000 7ff67f400640 1 -- 172.21.15.55:0/2859728017 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff678018020 con 0x7ff6880b47f0 2024-05-10T03:34:10.938 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.937+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 msgr2=0x7ff66406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.937+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 0x7ff66406d110 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff6881f7fb0 tx=0x7ff670009040 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 msgr2=0x7ff6880c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c7f0 tx=0x7ff67800ccb0 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 shutdown_connections 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff66406ac00 0x7ff66406d110 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff68815d7e0 0x7ff6880c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6880d27f0 0x7ff6880c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 --2- 172.21.15.55:0/2859728017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6880b47f0 0x7ff6880c6dd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:10.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 >> 172.21.15.55:0/2859728017 conn(0x7ff6880d0880 msgr2=0x7ff68815e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:10.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 shutdown_connections 2024-05-10T03:34:10.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:10.938+0000 7ff68d800640 1 -- 172.21.15.55:0/2859728017 wait complete. 2024-05-10T03:34:10.973 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:34:11.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.250+0000 7f20d7200640 1 Processor -- start 2024-05-10T03:34:11.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.250+0000 7f20d7200640 1 -- start start 2024-05-10T03:34:11.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.252+0000 7f20d7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20d815d970 0x7f20d80b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.252+0000 7f20d7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d80ce3a0 0x7f20d80b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.252+0000 7f20d7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80b2aa0 con 0x7f20d80b3d30 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.252+0000 7f20d7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80b6190 con 0x7f20d815d970 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.252+0000 7f20d7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80b6c30 con 0x7f20d80ce3a0 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20d815d970 0x7f20d80b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20d815d970 0x7f20d80b66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39790/0 (socket says 172.21.15.55:39790) 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 -- 172.21.15.55:0/2169587086 learned_addr learned my addr 172.21.15.55:0/2169587086 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:11.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5400640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d80ce3a0 0x7f20d80b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 -- 172.21.15.55:0/2169587086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d80ce3a0 msgr2=0x7f20d80b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d80ce3a0 0x7f20d80b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 -- 172.21.15.55:0/2169587086 >> v1:172.21.15.55:6789/0 conn(0x7f20d80b3d30 legacy=0x7f20d8161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.253+0000 7f20d5e00640 1 -- 172.21.15.55:0/2169587086 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80b2d10 con 0x7f20d815d970 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.254+0000 7f20d5e00640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d815d970 0x7f20d80b66f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009870 tx=0x7f20c802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5fabd2089575a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.255+0000 7f20d4a00640 1 -- 172.21.15.55:0/2169587086 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8032440 con 0x7f20d815d970 2024-05-10T03:34:11.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.255+0000 7f20d4a00640 1 -- 172.21.15.55:0/2169587086 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80325a0 con 0x7f20d815d970 2024-05-10T03:34:11.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.255+0000 7f20d4a00640 1 -- 172.21.15.55:0/2169587086 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8032930 con 0x7f20d815d970 2024-05-10T03:34:11.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.255+0000 7f20d7200640 1 -- 172.21.15.55:0/2169587086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d815d970 msgr2=0x7f20d80b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.255+0000 7f20d7200640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d815d970 0x7f20d80b66f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009870 tx=0x7f20c802a250 comp rx=0 tx=0).stop 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 -- 172.21.15.55:0/2169587086 shutdown_connections 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d80ce3a0 0x7f20d80b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 --2- 172.21.15.55:0/2169587086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d815d970 0x7f20d80b66f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 -- 172.21.15.55:0/2169587086 >> 172.21.15.55:0/2169587086 conn(0x7f20d80d0370 msgr2=0x7f20d80d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 -- 172.21.15.55:0/2169587086 shutdown_connections 2024-05-10T03:34:11.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.256+0000 7f20d7200640 1 -- 172.21.15.55:0/2169587086 wait complete. 2024-05-10T03:34:11.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.258+0000 7f20d7200640 1 Processor -- start 2024-05-10T03:34:11.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.258+0000 7f20d7200640 1 -- start start 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d80b3d30 0x7f20d8149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d815d970 0x7f20d81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d814a310 con 0x7f20d80ce3a0 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d814a770 con 0x7f20d80b3d30 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d815f1c0 con 0x7f20d815d970 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39800/0 (socket says 172.21.15.55:39800) 2024-05-10T03:34:11.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.260+0000 7f20d5400640 1 -- 172.21.15.55:0/1317655885 learned_addr learned my addr 172.21.15.55:0/1317655885 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:11.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 -- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d815d970 msgr2=0x7f20d81604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d815d970 0x7f20d81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 -- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d80b3d30 msgr2=0x7f20d8149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5e00640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d80b3d30 0x7f20d8149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d80b3d30 0x7f20d8149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 -- 172.21.15.55:0/1317655885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80094d0 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d5400640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f20c000c820 tx=0x7f20c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0004020 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0004180 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d8156c60 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.261+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0007830 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.262+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d815f6f0 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.263+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d804e290 con 0x7f20d80ce3a0 2024-05-10T03:34:11.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.264+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f20c0007990 con 0x7f20d80ce3a0 2024-05-10T03:34:11.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.265+0000 7f20ce000640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 0x7f20b406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.265+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f20c0088f50 con 0x7f20d80ce3a0 2024-05-10T03:34:11.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.266+0000 7f20d5e00640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 0x7f20b406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.267+0000 7f20d5e00640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 0x7f20b406d110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f20c8034000 tx=0x7f20c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.270+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f20c0054210 con 0x7f20d80ce3a0 2024-05-10T03:34:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.546+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f20d80b1f00 con 0x7f20d80ce3a0 2024-05-10T03:34:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.547+0000 7f20ce000640 1 -- 172.21.15.55:0/1317655885 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f20c001e070 con 0x7f20d80ce3a0 2024-05-10T03:34:11.551 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:11.551 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:34:11.551 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:34:11.552 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.553 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.554 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.555 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.556 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.557 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.558 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.559 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:11.560 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.561 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:11.562 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:11.563 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:34:11.564 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:34:11.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 msgr2=0x7f20b406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 0x7f20b406d110 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f20c8034000 tx=0x7f20c802e000 comp rx=0 tx=0).stop 2024-05-10T03:34:11.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 msgr2=0x7f20d815e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f20c000c820 tx=0x7f20c000cce0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 shutdown_connections 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20d815d970 0x7f20d81604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20d80ce3a0 0x7f20d815e830 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f20b406ac00 0x7f20b406d110 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 --2- 172.21.15.55:0/1317655885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20d80b3d30 0x7f20d8149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 >> 172.21.15.55:0/1317655885 conn(0x7f20d80d0370 msgr2=0x7f20d80d2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 shutdown_connections 2024-05-10T03:34:11.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.550+0000 7f20d7200640 1 -- 172.21.15.55:0/1317655885 wait complete. 2024-05-10T03:34:11.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.860+0000 7f2ca2600640 1 Processor -- start 2024-05-10T03:34:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.860+0000 7f2ca2600640 1 -- start start 2024-05-10T03:34:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.862+0000 7f2ca2600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.862+0000 7f2ca2600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.862+0000 7f2ca2600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0cd710 con 0x7f2c9c15af40 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.862+0000 7f2ca2600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0cdfb0 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2ca2600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c06b4d0 con 0x7f2c9c15ab30 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39812/0 (socket says 172.21.15.55:39812) 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 -- 172.21.15.55:0/850395140 learned_addr learned my addr 172.21.15.55:0/850395140 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c92a00640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 -- 172.21.15.55:0/850395140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 msgr2=0x7f2c9c0715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c0715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 -- 172.21.15.55:0/850395140 >> v1:172.21.15.162:6789/0 conn(0x7f2c9c15af40 legacy=0x7f2c9c0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.863+0000 7f2c9b400640 1 -- 172.21.15.55:0/850395140 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c06b1f0 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.865+0000 7f2c9b400640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009870 tx=0x7f2c8c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa45fc47085eb57 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.865+0000 7f2c9aa00640 1 -- 172.21.15.55:0/850395140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c02ae10 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.865+0000 7f2c9aa00640 1 -- 172.21.15.55:0/850395140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c032440 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.865+0000 7f2c9aa00640 1 -- 172.21.15.55:0/850395140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0327d0 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 -- 172.21.15.55:0/850395140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c0cdba0 msgr2=0x7f2c9c06a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009870 tx=0x7f2c8c02a100 comp rx=0 tx=0).stop 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 -- 172.21.15.55:0/850395140 shutdown_connections 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 --2- 172.21.15.55:0/850395140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c06a9a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 -- 172.21.15.55:0/850395140 >> 172.21.15.55:0/850395140 conn(0x7f2c9c0cbb90 msgr2=0x7f2c9c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 -- 172.21.15.55:0/850395140 shutdown_connections 2024-05-10T03:34:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.866+0000 7f2ca2600640 1 -- 172.21.15.55:0/850395140 wait complete. 2024-05-10T03:34:11.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.868+0000 7f2ca2600640 1 Processor -- start 2024-05-10T03:34:11.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.869+0000 7f2ca2600640 1 -- start start 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c1534f0 con 0x7f2c9c15af40 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0afb30 con 0x7f2c9c0cdba0 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.870+0000 7f2ca2600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c151cb0 con 0x7f2c9c15ab30 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39814/0 (socket says 172.21.15.55:39814) 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 -- 172.21.15.55:0/1996990577 learned_addr learned my addr 172.21.15.55:0/1996990577 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c92a00640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9b400640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 -- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 msgr2=0x7f2c9c143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 -- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 msgr2=0x7f2c9c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9be00640 1 -- 172.21.15.55:0/1996990577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0094d0 con 0x7f2c9c15af40 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.871+0000 7f2c9b400640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.872+0000 7f2c9be00640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400be60 tx=0x7f2c9400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.876+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94014440 con 0x7f2c9c15af40 2024-05-10T03:34:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.876+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c0afd80 con 0x7f2c9c15af40 2024-05-10T03:34:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.876+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c940145a0 con 0x7f2c9c15af40 2024-05-10T03:34:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.876+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94014950 con 0x7f2c9c15af40 2024-05-10T03:34:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.876+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c151660 con 0x7f2c9c15af40 2024-05-10T03:34:11.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.877+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c066f60 con 0x7f2c9c15af40 2024-05-10T03:34:11.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.878+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2c9401c360 con 0x7f2c9c15af40 2024-05-10T03:34:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.879+0000 7f2c93e00640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 0x7f2c7806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.879+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2c94089540 con 0x7f2c9c15af40 2024-05-10T03:34:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.880+0000 7f2c9b400640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 0x7f2c7806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.881+0000 7f2c9b400640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 0x7f2c7806d110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c034000 tx=0x7f2c8c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:11.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:11.884+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2c94054780 con 0x7f2c9c15af40 2024-05-10T03:34:12.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.084+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c9c067220 con 0x7f2c9c15af40 2024-05-10T03:34:12.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.086+0000 7f2c93e00640 1 -- 172.21.15.55:0/1996990577 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+12595 (secure 0 0 0) 0x7f2c94057da0 con 0x7f2c9c15af40 2024-05-10T03:34:12.090 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:12.090 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":86,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:10.926686+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 msgr2=0x7f2c7806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 0x7f2c7806d110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c034000 tx=0x7f2c8c02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 msgr2=0x7f2c9c14e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400be60 tx=0x7f2c9400bf50 comp rx=0 tx=0).stop 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 shutdown_connections 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c9c15af40 0x7f2c9c14e060 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c9c15ab30 0x7f2c9c143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c7806ac00 0x7f2c7806d110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 --2- 172.21.15.55:0/1996990577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c9c0cdba0 0x7f2c9c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 >> 172.21.15.55:0/1996990577 conn(0x7f2c9c0cbb90 msgr2=0x7f2c9c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:12.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 shutdown_connections 2024-05-10T03:34:12.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.089+0000 7f2ca2600640 1 -- 172.21.15.55:0/1996990577 wait complete. 2024-05-10T03:34:12.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:12.119 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.399+0000 7f9cf0400640 1 Processor -- start 2024-05-10T03:34:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.399+0000 7f9cf0400640 1 -- start start 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cf0400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ce8070050 0x7f9ce81658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cf0400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8161b90 0x7f9ce80cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cf0400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce80cee70 con 0x7f9ce806b120 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cf0400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80b3590 con 0x7f9ce8070050 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cf0400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8166830 con 0x7f9ce8161b90 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cee400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9ce806b120 0x7f9ce806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42508/0 (socket says 172.21.15.55:42508) 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9cee400640 1 -- 172.21.15.55:0/2247103833 learned_addr learned my addr 172.21.15.55:0/2247103833 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.402+0000 7f9ceda00640 1 --2- 172.21.15.55:0/2247103833 >> v2:172.21.15.55:3300/0 conn(0x7f9ce8070050 0x7f9ce81658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ced000640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8161b90 0x7f9ce80cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ce7e00640 1 -- 172.21.15.55:0/2247103833 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348994772 0 0) 0x7f9ce80cee70 con 0x7f9ce806b120 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ce7e00640 1 -- 172.21.15.55:0/2247103833 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd0003520 con 0x7f9ce806b120 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ceda00640 1 -- 172.21.15.55:0/2247103833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8161b90 msgr2=0x7f9ce80cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ceda00640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8161b90 0x7f9ce80cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ceda00640 1 -- 172.21.15.55:0/2247103833 >> v1:172.21.15.55:6789/0 conn(0x7f9ce806b120 legacy=0x7f9ce806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.403+0000 7f9ceda00640 1 -- 172.21.15.55:0/2247103833 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8165f90 con 0x7f9ce8070050 2024-05-10T03:34:12.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.405+0000 7f9ceda00640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce81658d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0097a0 tx=0x7f9cdc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2d3f52b67d3d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.406+0000 7f9ce7e00640 1 -- 172.21.15.55:0/2247103833 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc032440 con 0x7f9ce8070050 2024-05-10T03:34:12.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.406+0000 7f9ce7e00640 1 -- 172.21.15.55:0/2247103833 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0325a0 con 0x7f9ce8070050 2024-05-10T03:34:12.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.406+0000 7f9ce7e00640 1 -- 172.21.15.55:0/2247103833 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc032930 con 0x7f9ce8070050 2024-05-10T03:34:12.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.406+0000 7f9cf0400640 1 -- 172.21.15.55:0/2247103833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 msgr2=0x7f9ce81658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.406+0000 7f9cf0400640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce81658d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0097a0 tx=0x7f9cdc02a180 comp rx=0 tx=0).stop 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 -- 172.21.15.55:0/2247103833 shutdown_connections 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8161b90 0x7f9ce80cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 --2- 172.21.15.55:0/2247103833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce81658d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 -- 172.21.15.55:0/2247103833 >> 172.21.15.55:0/2247103833 conn(0x7f9ce806c120 msgr2=0x7f9ce806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 -- 172.21.15.55:0/2247103833 shutdown_connections 2024-05-10T03:34:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.407+0000 7f9cf0400640 1 -- 172.21.15.55:0/2247103833 wait complete. 2024-05-10T03:34:12.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.409+0000 7f9cf0400640 1 Processor -- start 2024-05-10T03:34:12.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.410+0000 7f9cf0400640 1 -- start start 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce806b120 0x7f9ce80a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 0x7f9ce81646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce814c710 con 0x7f9ce8070050 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce814d210 con 0x7f9ce8161b90 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.411+0000 7f9cf0400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80a8ab0 con 0x7f9ce806b120 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39828/0 (socket says 172.21.15.55:39828) 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 -- 172.21.15.55:0/919124292 learned_addr learned my addr 172.21.15.55:0/919124292 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ceda00640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce806b120 0x7f9ce80a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9cee400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 0x7f9ce81646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 -- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce806b120 msgr2=0x7f9ce80a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce806b120 0x7f9ce80a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 -- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 msgr2=0x7f9ce81646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 0x7f9ce81646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9ced000640 1 -- 172.21.15.55:0/919124292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009400 con 0x7f9ce8070050 2024-05-10T03:34:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.412+0000 7f9cee400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 0x7f9ce81646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9ced000640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400d5c0 tx=0x7f9cd400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd40125f0 con 0x7f9ce8070050 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd4012750 con 0x7f9ce8070050 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012b00 con 0x7f9ce8070050 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce8164230 con 0x7f9ce8070050 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.413+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce8163900 con 0x7f9ce8070050 2024-05-10T03:34:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.414+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce8066f60 con 0x7f9ce8070050 2024-05-10T03:34:12.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.415+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9cd4012c60 con 0x7f9ce8070050 2024-05-10T03:34:12.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.416+0000 7f9ce5600640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 0x7f9cc806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.416+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9cd40896a0 con 0x7f9ce8070050 2024-05-10T03:34:12.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.417+0000 7f9ceda00640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 0x7f9cc806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.418+0000 7f9ceda00640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 0x7f9cc806d110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc034000 tx=0x7f9cdc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.421+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9cd40549f0 con 0x7f9ce8070050 2024-05-10T03:34:12.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.621+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ce8067220 con 0x7f9ce8070050 2024-05-10T03:34:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.622+0000 7f9ce5600640 1 -- 172.21.15.55:0/919124292 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cd4058010 con 0x7f9ce8070050 2024-05-10T03:34:12.626 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 msgr2=0x7f9cc806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 0x7f9cc806d110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc034000 tx=0x7f9cdc02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 msgr2=0x7f9ce814c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400d5c0 tx=0x7f9cd400da80 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 shutdown_connections 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9cc806ac00 0x7f9cc806d110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8161b90 0x7f9ce81646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8070050 0x7f9ce814c1d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 --2- 172.21.15.55:0/919124292 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce806b120 0x7f9ce80a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 >> 172.21.15.55:0/919124292 conn(0x7f9ce806c120 msgr2=0x7f9ce80b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 shutdown_connections 2024-05-10T03:34:12.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.625+0000 7f9cf0400640 1 -- 172.21.15.55:0/919124292 wait complete. 2024-05-10T03:34:12.659 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:34:12.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.932+0000 7fd68dc00640 1 Processor -- start 2024-05-10T03:34:12.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.932+0000 7fd68dc00640 1 -- start start 2024-05-10T03:34:12.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.934+0000 7fd68dc00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6880d1ba0 0x7fd6880b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.934+0000 7fd68dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6880ce680 0x7fd68815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.934+0000 7fd68dc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6880b7b50 con 0x7fd68815d700 2024-05-10T03:34:12.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.934+0000 7fd68dc00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68815e050 con 0x7fd6880d1ba0 2024-05-10T03:34:12.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.934+0000 7fd68dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880b8ff0 con 0x7fd6880ce680 2024-05-10T03:34:12.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.935+0000 7fd68d200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6880ce680 0x7fd68815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.935+0000 7fd687e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6880d1ba0 0x7fd6880b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.935+0000 7fd687e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6880d1ba0 0x7fd6880b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46496/0 (socket says 172.21.15.55:46496) 2024-05-10T03:34:12.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.935+0000 7fd687e00640 1 -- 172.21.15.55:0/130641811 learned_addr learned my addr 172.21.15.55:0/130641811 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:12.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.935+0000 7fd687400640 1 --1- 172.21.15.55:0/130641811 >> v1:172.21.15.162:6789/0 conn(0x7fd68815d700 0x7fd6880cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:53238/0 (socket says 172.21.15.55:53238) 2024-05-10T03:34:12.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.936+0000 7fd687e00640 1 -- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6880ce680 msgr2=0x7fd68815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:12.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.936+0000 7fd687e00640 1 --2- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6880ce680 0x7fd68815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.936+0000 7fd687e00640 1 -- 172.21.15.55:0/130641811 >> v1:172.21.15.162:6789/0 conn(0x7fd68815d700 legacy=0x7fd6880cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.936+0000 7fd687e00640 1 -- 172.21.15.55:0/130641811 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6880b6f40 con 0x7fd6880d1ba0 2024-05-10T03:34:12.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.938+0000 7fd687e00640 1 --2- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6880d1ba0 0x7fd6880b65f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd678009870 tx=0x7fd67802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2a080110408b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.938+0000 7fd686a00640 1 -- 172.21.15.55:0/130641811 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678032440 con 0x7fd6880d1ba0 2024-05-10T03:34:12.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.938+0000 7fd686a00640 1 -- 172.21.15.55:0/130641811 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6780325a0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.938+0000 7fd686a00640 1 -- 172.21.15.55:0/130641811 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678032930 con 0x7fd6880d1ba0 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 -- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6880d1ba0 msgr2=0x7fd6880b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 --2- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6880d1ba0 0x7fd6880b65f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd678009870 tx=0x7fd67802a250 comp rx=0 tx=0).stop 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 -- 172.21.15.55:0/130641811 shutdown_connections 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 --2- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6880ce680 0x7fd68815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 --2- 172.21.15.55:0/130641811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6880d1ba0 0x7fd6880b65f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 -- 172.21.15.55:0/130641811 >> 172.21.15.55:0/130641811 conn(0x7fd6880d0620 msgr2=0x7fd6880d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:12.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 -- 172.21.15.55:0/130641811 shutdown_connections 2024-05-10T03:34:12.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.939+0000 7fd68dc00640 1 -- 172.21.15.55:0/130641811 wait complete. 2024-05-10T03:34:12.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.943+0000 7fd68dc00640 1 Processor -- start 2024-05-10T03:34:12.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.943+0000 7fd68dc00640 1 -- start start 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd68815d700 0x7fd68815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd688160010 0x7fd688160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68815f8b0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6881f8720 con 0x7fd688160010 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd68dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd688160a40 con 0x7fd68815d700 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.945+0000 7fd687e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58124/0 (socket says 172.21.15.55:58124) 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 -- 172.21.15.55:0/1868157122 learned_addr learned my addr 172.21.15.55:0/1868157122 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd68d200640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd688160010 0x7fd688160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687400640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd68815d700 0x7fd68815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 -- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd68815d700 msgr2=0x7fd68815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd68815d700 0x7fd68815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 -- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd688160010 msgr2=0x7fd688160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd688160010 0x7fd688160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.946+0000 7fd687e00640 1 -- 172.21.15.55:0/1868157122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780094d0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd687e00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd678009990 tx=0x7fd6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678032440 con 0x7fd6880d1ba0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd678030980 con 0x7fd6880d1ba0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678030d10 con 0x7fd6880d1ba0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68815e2d0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.947+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68815e890 con 0x7fd6880d1ba0 2024-05-10T03:34:12.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.948+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6880d14a0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.950+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd678039040 con 0x7fd6880d1ba0 2024-05-10T03:34:12.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.951+0000 7fd67fe00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 0x7fd66806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:12.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.951+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd6780a9bd0 con 0x7fd6880d1ba0 2024-05-10T03:34:12.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.952+0000 7fd687400640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 0x7fd66806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:12.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.953+0000 7fd687400640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 0x7fd66806d140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd6880cdff0 tx=0x7fd674007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:12.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:12.960+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd678074e90 con 0x7fd6880d1ba0 2024-05-10T03:34:13.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.159+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6880cda60 con 0x7fd6880d1ba0 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.160+0000 7fd67fe00640 1 -- 172.21.15.55:0/1868157122 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd678037090 con 0x7fd6880d1ba0 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:34:13.164 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:13.165 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:34:13.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 msgr2=0x7fd66806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 0x7fd66806d140 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd6880cdff0 tx=0x7fd674007ce0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 msgr2=0x7fd688154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd678009990 tx=0x7fd6780046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 shutdown_connections 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66806ac30 0x7fd66806d140 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd688160010 0x7fd688160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd68815d700 0x7fd68815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 --2- 172.21.15.55:0/1868157122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6880d1ba0 0x7fd688154550 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 >> 172.21.15.55:0/1868157122 conn(0x7fd6880d0620 msgr2=0x7fd6880ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 shutdown_connections 2024-05-10T03:34:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.163+0000 7fd68dc00640 1 -- 172.21.15.55:0/1868157122 wait complete. 2024-05-10T03:34:13.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:34:13.193 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:34:13.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.474+0000 7f1cd3e00640 1 Processor -- start 2024-05-10T03:34:13.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.474+0000 7f1cd3e00640 1 -- start start 2024-05-10T03:34:13.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.476+0000 7f1cd3e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1cd4071cb0 0x7f1cd415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.476+0000 7f1cd3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1cd40ce840 0x7f1cd406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.476+0000 7f1cd3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd406ba70 con 0x7f1cd406b660 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.476+0000 7f1cd3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd4070f80 con 0x7f1cd40ce840 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.477+0000 7f1cd3e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd4072220 con 0x7f1cd4071cb0 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.477+0000 7f1cd2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1cd40ce840 0x7f1cd406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.477+0000 7f1cd3400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1cd406b660 0x7f1cd4161400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52752/0 (socket says 172.21.15.55:52752) 2024-05-10T03:34:13.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.477+0000 7f1cd3400640 1 -- 172.21.15.55:0/1418558513 learned_addr learned my addr 172.21.15.55:0/1418558513 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.477+0000 7f1cd2a00640 1 --2- 172.21.15.55:0/1418558513 >> v2:172.21.15.162:3300/0 conn(0x7f1cd4071cb0 0x7f1cd415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.478+0000 7f1cd2000640 1 -- 172.21.15.55:0/1418558513 >> v2:172.21.15.162:3300/0 conn(0x7f1cd4071cb0 msgr2=0x7f1cd415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.478+0000 7f1cd2000640 1 --2- 172.21.15.55:0/1418558513 >> v2:172.21.15.162:3300/0 conn(0x7f1cd4071cb0 0x7f1cd415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.478+0000 7f1cd2000640 1 -- 172.21.15.55:0/1418558513 >> v1:172.21.15.55:6789/0 conn(0x7f1cd406b660 legacy=0x7f1cd4161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.478+0000 7f1cd2000640 1 -- 172.21.15.55:0/1418558513 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd40ac980 con 0x7f1cd40ce840 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.478+0000 7f1cd1600640 1 -- 172.21.15.55:0/1418558513 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775573942 0 0) 0x7f1cd406ba70 con 0x7f1cd406b660 2024-05-10T03:34:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.479+0000 7f1cd2000640 1 --2- 172.21.15.55:0/1418558513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd40ce840 0x7f1cd406b120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1cd406bce0 tx=0x7f1cbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68567213e55ee90d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:13.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.479+0000 7f1cd1600640 1 -- 172.21.15.55:0/1418558513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc032440 con 0x7f1cd40ce840 2024-05-10T03:34:13.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.479+0000 7f1cd1600640 1 -- 172.21.15.55:0/1418558513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc0325a0 con 0x7f1cd40ce840 2024-05-10T03:34:13.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.479+0000 7f1cd1600640 1 -- 172.21.15.55:0/1418558513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc032930 con 0x7f1cd40ce840 2024-05-10T03:34:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 -- 172.21.15.55:0/1418558513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd40ce840 msgr2=0x7f1cd406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/1418558513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd40ce840 0x7f1cd406b120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1cd406bce0 tx=0x7f1cbc02a1a0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 -- 172.21.15.55:0/1418558513 shutdown_connections 2024-05-10T03:34:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/1418558513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd40ce840 0x7f1cd406b120 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/1418558513 >> v2:172.21.15.162:3300/0 conn(0x7f1cd4071cb0 0x7f1cd415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 -- 172.21.15.55:0/1418558513 >> 172.21.15.55:0/1418558513 conn(0x7f1cd406c070 msgr2=0x7f1cd406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 -- 172.21.15.55:0/1418558513 shutdown_connections 2024-05-10T03:34:13.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.480+0000 7f1cd3e00640 1 -- 172.21.15.55:0/1418558513 wait complete. 2024-05-10T03:34:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.483+0000 7f1cd3e00640 1 Processor -- start 2024-05-10T03:34:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.483+0000 7f1cd3e00640 1 -- start start 2024-05-10T03:34:13.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd4071cb0 0x7f1cd41f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 0x7f1cd40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd4148140 con 0x7f1cd406b660 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c4240 con 0x7f1cd40ce840 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.484+0000 7f1cd3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd41f6690 con 0x7f1cd4071cb0 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58134/0 (socket says 172.21.15.55:58134) 2024-05-10T03:34:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 -- 172.21.15.55:0/3724252682 learned_addr learned my addr 172.21.15.55:0/3724252682 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd3400640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 0x7f1cd40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 -- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd4071cb0 msgr2=0x7f1cd41f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2000640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd4071cb0 0x7f1cd41f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd4071cb0 0x7f1cd41f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 -- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 msgr2=0x7f1cd40aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 0x7f1cd40aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd2a00640 1 -- 172.21.15.55:0/3724252682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc009400 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.485+0000 7f1cd3400640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 0x7f1cd40aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1cd2a00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4007e60 tx=0x7f1cc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4003b90 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc400ee90 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4004030 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd41f6970 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.486+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd41f6f30 con 0x7f1cd406b660 2024-05-10T03:34:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.487+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd4067220 con 0x7f1cd406b660 2024-05-10T03:34:13.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.488+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1cc4004190 con 0x7f1cd406b660 2024-05-10T03:34:13.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.489+0000 7f1ccb400640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 0x7f1cb006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:13.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.489+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1cc4089560 con 0x7f1cd406b660 2024-05-10T03:34:13.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.490+0000 7f1cd2000640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 0x7f1cb006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:13.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.491+0000 7f1cd2000640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 0x7f1cb006d140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc034000 tx=0x7f1cbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:13.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.494+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1cc4054960 con 0x7f1cd406b660 2024-05-10T03:34:13.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.695+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1cd40ae100 con 0x7f1cd406b660 2024-05-10T03:34:13.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.911+0000 7f1ccb400640 1 -- 172.21.15.55:0/3724252682 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1cc4057f80 con 0x7f1cd406b660 2024-05-10T03:34:13.916 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:34:13.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 msgr2=0x7f1cb006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 0x7f1cb006d140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc034000 tx=0x7f1cbc02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:13.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 msgr2=0x7f1cd4149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:13.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4007e60 tx=0x7f1cc400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 shutdown_connections 2024-05-10T03:34:13.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1cb006ac30 0x7f1cb006d140 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd40ce840 0x7f1cd40aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd4071cb0 0x7f1cd41f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 --2- 172.21.15.55:0/3724252682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1cd406b660 0x7f1cd4149ae0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.920+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 >> 172.21.15.55:0/3724252682 conn(0x7f1cd406c070 msgr2=0x7f1cd406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.921+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 shutdown_connections 2024-05-10T03:34:13.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:13.921+0000 7f1cd3e00640 1 -- 172.21.15.55:0/3724252682 wait complete. 2024-05-10T03:34:13.958 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:34:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.237+0000 7f5da7200640 1 Processor -- start 2024-05-10T03:34:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.237+0000 7f5da7200640 1 -- start start 2024-05-10T03:34:14.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da7200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5da015fe80 0x7f5da015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5da0141360 0x7f5da0141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da7200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da015cb30 con 0x7f5da00cc230 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00b4540 con 0x7f5da0141360 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da7200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da015c1d0 con 0x7f5da015fe80 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.240+0000 7f5da6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5da0141360 0x7f5da0141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5da0141360 0x7f5da0141770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58138/0 (socket says 172.21.15.55:58138) 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 -- 172.21.15.55:0/776104489 learned_addr learned my addr 172.21.15.55:0/776104489 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:14.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da5e00640 1 --2- 172.21.15.55:0/776104489 >> v2:172.21.15.162:3300/0 conn(0x7f5da015fe80 0x7f5da015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 -- 172.21.15.55:0/776104489 >> v2:172.21.15.162:3300/0 conn(0x7f5da015fe80 msgr2=0x7f5da015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:14.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 --2- 172.21.15.55:0/776104489 >> v2:172.21.15.162:3300/0 conn(0x7f5da015fe80 0x7f5da015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 -- 172.21.15.55:0/776104489 >> v1:172.21.15.162:6789/0 conn(0x7f5da00cc230 legacy=0x7f5da00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.241+0000 7f5da6800640 1 -- 172.21.15.55:0/776104489 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00b3f80 con 0x7f5da0141360 2024-05-10T03:34:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.242+0000 7f5da6800640 1 --2- 172.21.15.55:0/776104489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da0141770 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5da006f740 tx=0x7f5d9c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dad3020d3ebf69 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.243+0000 7f5da4a00640 1 -- 172.21.15.55:0/776104489 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c031600 con 0x7f5da0141360 2024-05-10T03:34:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.243+0000 7f5da4a00640 1 -- 172.21.15.55:0/776104489 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c02d050 con 0x7f5da0141360 2024-05-10T03:34:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.243+0000 7f5da4a00640 1 -- 172.21.15.55:0/776104489 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c031980 con 0x7f5da0141360 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.243+0000 7f5da7200640 1 -- 172.21.15.55:0/776104489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 msgr2=0x7f5da0141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.243+0000 7f5da7200640 1 --2- 172.21.15.55:0/776104489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da0141770 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5da006f740 tx=0x7f5d9c02a730 comp rx=0 tx=0).stop 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 -- 172.21.15.55:0/776104489 shutdown_connections 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 --2- 172.21.15.55:0/776104489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da0141770 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 --2- 172.21.15.55:0/776104489 >> v2:172.21.15.162:3300/0 conn(0x7f5da015fe80 0x7f5da015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 -- 172.21.15.55:0/776104489 >> 172.21.15.55:0/776104489 conn(0x7f5da00b1810 msgr2=0x7f5da006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 -- 172.21.15.55:0/776104489 shutdown_connections 2024-05-10T03:34:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.244+0000 7f5da7200640 1 -- 172.21.15.55:0/776104489 wait complete. 2024-05-10T03:34:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.246+0000 7f5da7200640 1 Processor -- start 2024-05-10T03:34:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.246+0000 7f5da7200640 1 -- start start 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5da00cc230 0x7f5da0159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 0x7f5da013fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015a070 con 0x7f5da0141360 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00ab720 con 0x7f5da015fe80 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015a280 con 0x7f5da00cc230 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58144/0 (socket says 172.21.15.55:58144) 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.248+0000 7f5da5400640 1 -- 172.21.15.55:0/1035471103 learned_addr learned my addr 172.21.15.55:0/1035471103 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5e00640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5da00cc230 0x7f5da0159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da6800640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 0x7f5da013fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 -- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5da00cc230 msgr2=0x7f5da0159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5da00cc230 0x7f5da0159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 -- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 msgr2=0x7f5da013fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 0x7f5da013fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 -- 172.21.15.55:0/1035471103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c009b30 con 0x7f5da0141360 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da6800640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 0x7f5da013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5da5400640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3c0 tx=0x7f5d9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.249+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900052d0 con 0x7f5da0141360 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.250+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90005450 con 0x7f5da0141360 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.250+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90003830 con 0x7f5da0141360 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.250+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da015a4d0 con 0x7f5da0141360 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.250+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00a8850 con 0x7f5da0141360 2024-05-10T03:34:14.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.251+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da006d170 con 0x7f5da0141360 2024-05-10T03:34:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.253+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d90003990 con 0x7f5da0141360 2024-05-10T03:34:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.254+0000 7f5d9a000640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 0x7f5d8406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.254+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f5d90088ad0 con 0x7f5da0141360 2024-05-10T03:34:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.255+0000 7f5da5e00640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 0x7f5d8406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.256+0000 7f5da5e00640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 0x7f5d8406d110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:14.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.258+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d90053ca0 con 0x7f5da0141360 2024-05-10T03:34:14.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.458+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5da0067430 con 0x7f5da0141360 2024-05-10T03:34:14.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.915+0000 7f5d9a000640 1 -- 172.21.15.55:0/1035471103 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5d900572c0 con 0x7f5da0141360 2024-05-10T03:34:14.920 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:34:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.923+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 msgr2=0x7f5d8406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.923+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 0x7f5d8406d110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006fd0 tx=0x7f5d94008040 comp rx=0 tx=0).stop 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.923+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 msgr2=0x7f5da013e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.923+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3c0 tx=0x7f5d9000c880 comp rx=0 tx=0).stop 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.927+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 shutdown_connections 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.927+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d8406ac00 0x7f5d8406d110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.927+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5da015fe80 0x7f5da013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.927+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5da0141360 0x7f5da013e260 secure :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3c0 tx=0x7f5d9000c880 comp rx=0 tx=0).stop 2024-05-10T03:34:14.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.927+0000 7f5da7200640 1 --2- 172.21.15.55:0/1035471103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5da00cc230 0x7f5da0159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:14.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.928+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 >> 172.21.15.55:0/1035471103 conn(0x7f5da00b1810 msgr2=0x7f5da006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.930+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 shutdown_connections 2024-05-10T03:34:14.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:14.930+0000 7f5da7200640 1 -- 172.21.15.55:0/1035471103 wait complete. 2024-05-10T03:34:14.975 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:34:15.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.225+0000 7fea7ee00640 1 Processor -- start 2024-05-10T03:34:15.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.225+0000 7fea7ee00640 1 -- start start 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.227+0000 7fea7ee00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea780b2190 0x7fea7815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.227+0000 7fea7ee00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea7815e030 0x7fea7806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.227+0000 7fea7ee00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7815ef50 con 0x7fea7806ff50 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.227+0000 7fea7ee00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780b0a00 con 0x7fea7815e030 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.227+0000 7fea7ee00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7815e440 con 0x7fea780b2190 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea7815e030 0x7fea7806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea7815e030 0x7fea7806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58146/0 (socket says 172.21.15.55:58146) 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 -- 172.21.15.55:0/2075195411 learned_addr learned my addr 172.21.15.55:0/2075195411 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77e00640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea780b2190 0x7fea7815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 -- 172.21.15.55:0/2075195411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea780b2190 msgr2=0x7fea7815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea780b2190 0x7fea7815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 -- 172.21.15.55:0/2075195411 >> v1:172.21.15.162:6789/0 conn(0x7fea7806ff50 legacy=0x7fea78161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.228+0000 7fea77400640 1 -- 172.21.15.55:0/2075195411 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780b61a0 con 0x7fea7815e030 2024-05-10T03:34:15.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.229+0000 7fea77400640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea7815e030 0x7fea7806fa10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fea64009e50 tx=0x7fea6402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea37c08a3bae718 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.230+0000 7fea76a00640 1 -- 172.21.15.55:0/2075195411 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6402aea0 con 0x7fea7815e030 2024-05-10T03:34:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.230+0000 7fea76a00640 1 -- 172.21.15.55:0/2075195411 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6402d050 con 0x7fea7815e030 2024-05-10T03:34:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.230+0000 7fea76a00640 1 -- 172.21.15.55:0/2075195411 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64031820 con 0x7fea7815e030 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 -- 172.21.15.55:0/2075195411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea7815e030 msgr2=0x7fea7806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea7815e030 0x7fea7806fa10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fea64009e50 tx=0x7fea6402a6c0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 -- 172.21.15.55:0/2075195411 shutdown_connections 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea7815e030 0x7fea7806fa10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 --2- 172.21.15.55:0/2075195411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea780b2190 0x7fea7815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 -- 172.21.15.55:0/2075195411 >> 172.21.15.55:0/2075195411 conn(0x7fea780b36e0 msgr2=0x7fea780d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 -- 172.21.15.55:0/2075195411 shutdown_connections 2024-05-10T03:34:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.231+0000 7fea7ee00640 1 -- 172.21.15.55:0/2075195411 wait complete. 2024-05-10T03:34:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.234+0000 7fea7ee00640 1 Processor -- start 2024-05-10T03:34:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.234+0000 7fea7ee00640 1 -- start start 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea7806ff50 0x7fea780afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea7815e030 0x7fea7815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780cd460 con 0x7fea780b2190 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea781f7530 con 0x7fea7815e030 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea7ee00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780cb610 con 0x7fea7806ff50 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea77400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea77400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58158/0 (socket says 172.21.15.55:58158) 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.236+0000 7fea77400640 1 -- 172.21.15.55:0/3366899789 learned_addr learned my addr 172.21.15.55:0/3366899789 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77e00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea7806ff50 0x7fea780afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea7ce00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea7815e030 0x7fea7815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 -- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea7806ff50 msgr2=0x7fea780afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea7806ff50 0x7fea780afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 -- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea7815e030 msgr2=0x7fea7815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea7815e030 0x7fea7815c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 -- 172.21.15.55:0/3366899789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009b30 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.237+0000 7fea77400640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea64009e50 tx=0x7fea64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.238+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6403b820 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.238+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6403b980 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.238+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6403bd30 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.238+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea781f7740 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.238+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea781f7d00 con 0x7fea780b2190 2024-05-10T03:34:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.239+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea780d1980 con 0x7fea780b2190 2024-05-10T03:34:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.240+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fea64038050 con 0x7fea780b2190 2024-05-10T03:34:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.241+0000 7fea6fe00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 0x7fea5806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.241+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fea640a9da0 con 0x7fea780b2190 2024-05-10T03:34:15.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.242+0000 7fea77e00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 0x7fea5806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.243+0000 7fea77e00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 0x7fea5806d160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fea68006fd0 tx=0x7fea68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.247+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fea64074e80 con 0x7fea780b2190 2024-05-10T03:34:15.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.484+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fea78067220 con 0x7fea780b2190 2024-05-10T03:34:15.502 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 13 and data pool 14 2024-05-10T03:34:15.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.498+0000 7fea6fe00640 1 -- 172.21.15.55:0/3366899789 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7fea6404b0c0 con 0x7fea780b2190 2024-05-10T03:34:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.502+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 msgr2=0x7fea5806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.502+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 0x7fea5806d160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fea68006fd0 tx=0x7fea68008040 comp rx=0 tx=0).stop 2024-05-10T03:34:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.502+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 msgr2=0x7fea780ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.502+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea64009e50 tx=0x7fea64004a40 comp rx=0 tx=0).stop 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 shutdown_connections 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fea5806ac50 0x7fea5806d160 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea7815e030 0x7fea7815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea780b2190 0x7fea780ccac0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 --2- 172.21.15.55:0/3366899789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea7806ff50 0x7fea780afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 >> 172.21.15.55:0/3366899789 conn(0x7fea780b36e0 msgr2=0x7fea7806d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 shutdown_connections 2024-05-10T03:34:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.512+0000 7fea7ee00640 1 -- 172.21.15.55:0/3366899789 wait complete. 2024-05-10T03:34:15.560 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.796+0000 7f2f90800640 1 Processor -- start 2024-05-10T03:34:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.796+0000 7f2f90800640 1 -- start start 2024-05-10T03:34:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f90800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2f8815fe50 0x7f2f88141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f90800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8806ad20 con 0x7f2f880b2550 2024-05-10T03:34:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f90800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f88141f50 con 0x7f2f880b2140 2024-05-10T03:34:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f90800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8806a110 con 0x7f2f8815fe50 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f8e800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2f880b2550 0x7f2f8806b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52762/0 (socket says 172.21.15.55:52762) 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.799+0000 7f2f8e800640 1 -- 172.21.15.55:0/1108323117 learned_addr learned my addr 172.21.15.55:0/1108323117 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8de00640 1 --2- 172.21.15.55:0/1108323117 >> v2:172.21.15.162:3300/0 conn(0x7f2f8815fe50 0x7f2f88141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986969254 0 0) 0x7f2f8806ad20 con 0x7f2f880b2550 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f70003520 con 0x7f2f880b2550 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346475739 0 0) 0x7f2f70003520 con 0x7f2f880b2550 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 >> v2:172.21.15.162:3300/0 conn(0x7f2f8815fe50 msgr2=0x7f2f88141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 --2- 172.21.15.55:0/1108323117 >> v2:172.21.15.162:3300/0 conn(0x7f2f8815fe50 0x7f2f88141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 >> v1:172.21.15.162:6789/0 conn(0x7f2f880b2140 legacy=0x7f2f88142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8806a3f0 con 0x7f2f880b2550 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2f840019f0 con 0x7f2f880b2550 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 >> v1:172.21.15.55:6789/0 conn(0x7f2f880b2550 legacy=0x7f2f8806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f70003970 0x7f2f70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f700047c0 0x7f2f70004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806a110 con 0x7f2f70005160 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7000c0e0 con 0x7f2f70003970 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.800+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7000c2a0 con 0x7f2f700047c0 2024-05-10T03:34:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8de00640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f70003970 0x7f2f70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8d400640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f700047c0 0x7f2f70004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 -- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f700047c0 msgr2=0x7f2f70004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f700047c0 0x7f2f70004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 -- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f70003970 msgr2=0x7f2f70003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f70003970 0x7f2f70003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.801+0000 7f2f8e800640 1 -- 172.21.15.55:0/1108323117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70003520 con 0x7f2f70005160 2024-05-10T03:34:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.802+0000 7f2f8e800640 1 -- 172.21.15.55:0/1108323117 reap_dead start 2024-05-10T03:34:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.802+0000 7f2f8e800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f7000b980 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400a810 tx=0x7f2f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10122854efe379b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.802+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8402ba10 con 0x7f2f70005160 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.802+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8402bb70 con 0x7f2f70005160 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.802+0000 7f2f8ca00640 1 -- 172.21.15.55:0/1108323117 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84034470 con 0x7f2f70005160 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 -- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 msgr2=0x7f2f7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f7000b980 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400a810 tx=0x7f2f700057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 -- 172.21.15.55:0/1108323117 shutdown_connections 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 --2- 172.21.15.55:0/1108323117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f7000b980 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 -- 172.21.15.55:0/1108323117 >> 172.21.15.55:0/1108323117 conn(0x7f2f880cbb90 msgr2=0x7f2f880cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:15.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 -- 172.21.15.55:0/1108323117 shutdown_connections 2024-05-10T03:34:15.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.803+0000 7f2f90800640 1 -- 172.21.15.55:0/1108323117 wait complete. 2024-05-10T03:34:15.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.806+0000 7f2f90800640 1 Processor -- start 2024-05-10T03:34:15.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.806+0000 7f2f90800640 1 -- start start 2024-05-10T03:34:15.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8813e970 0x7f2f8814c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8813ed80 0x7f2f8815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8814c980 con 0x7f2f70005160 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8815be00 con 0x7f2f8813e970 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f90800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f881f4200 con 0x7f2f8813ed80 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.807+0000 7f2f8de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58170/0 (socket says 172.21.15.55:58170) 2024-05-10T03:34:15.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 -- 172.21.15.55:0/823251986 learned_addr learned my addr 172.21.15.55:0/823251986 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:15.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8d400640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8813e970 0x7f2f8814c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 -- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8813ed80 msgr2=0x7f2f8815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8813ed80 0x7f2f8815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 -- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8813e970 msgr2=0x7f2f8814c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8813e970 0x7f2f8814c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 -- 172.21.15.55:0/823251986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8815c050 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.808+0000 7f2f8de00640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c011540 tx=0x7f2f7c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.809+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c015930 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.809+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c015a90 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.809+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8815c330 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.809+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c015e60 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.809+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8815c8f0 con 0x7f2f70005160 2024-05-10T03:34:15.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.810+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f88066f60 con 0x7f2f70005160 2024-05-10T03:34:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.811+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2f7c00f460 con 0x7f2f70005160 2024-05-10T03:34:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.812+0000 7f2f82000640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 0x7f2f6c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.813+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2f7c089df0 con 0x7f2f70005160 2024-05-10T03:34:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.813+0000 7f2f8d400640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 0x7f2f6c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.815+0000 7f2f8d400640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 0x7f2f6c06d110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2f78007fd0 tx=0x7f2f78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:15.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:15.817+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2f7c054d60 con 0x7f2f70005160 2024-05-10T03:34:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.015+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f88067220 con 0x7f2f70005160 2024-05-10T03:34:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.018+0000 7f2f82000640 1 -- 172.21.15.55:0/823251986 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16262 (secure 0 0 0) 0x7f2f7c058380 con 0x7f2f70005160 2024-05-10T03:34:16.022 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:16.022 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:15.486985+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:16.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.020+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 msgr2=0x7f2f6c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.020+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 0x7f2f6c06d110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2f78007fd0 tx=0x7f2f78009040 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.020+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 msgr2=0x7f2f881f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.020+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c011540 tx=0x7f2f7c011a00 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 shutdown_connections 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f8813ed80 0x7f2f8815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2f6c06ac00 0x7f2f6c06d110 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8813e970 0x7f2f8814c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 --2- 172.21.15.55:0/823251986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f70005160 0x7f2f881f3cc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 >> 172.21.15.55:0/823251986 conn(0x7f2f880cbb90 msgr2=0x7f2f88140cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 shutdown_connections 2024-05-10T03:34:16.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.021+0000 7f2f90800640 1 -- 172.21.15.55:0/823251986 wait complete. 2024-05-10T03:34:16.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:16.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.317+0000 7fd259200640 1 Processor -- start 2024-05-10T03:34:16.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.318+0000 7fd259200640 1 -- start start 2024-05-10T03:34:16.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.319+0000 7fd259200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2540b2170 0x7fd25415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.319+0000 7fd259200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd25406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.319+0000 7fd259200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2540d25a0 con 0x7fd2540b6680 2024-05-10T03:34:16.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.319+0000 7fd259200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2540d2b20 con 0x7fd2540b2170 2024-05-10T03:34:16.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.319+0000 7fd259200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25415db20 con 0x7fd2541618b0 2024-05-10T03:34:16.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd253400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd2540b6680 0x7fd2541611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52764/0 (socket says 172.21.15.55:52764) 2024-05-10T03:34:16.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd253400640 1 -- 172.21.15.55:0/448421961 learned_addr learned my addr 172.21.15.55:0/448421961 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037826794 0 0) 0x7fd2540d25a0 con 0x7fd2540b6680 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd238003520 con 0x7fd2540b6680 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd252a00640 1 --2- 172.21.15.55:0/448421961 >> v2:172.21.15.162:3300/0 conn(0x7fd2540b2170 0x7fd25415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd253e00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd25406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.320+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129191176 0 0) 0x7fd238003520 con 0x7fd2540b6680 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 msgr2=0x7fd25406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd25406aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 >> v2:172.21.15.162:3300/0 conn(0x7fd2540b2170 msgr2=0x7fd25415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 --2- 172.21.15.55:0/448421961 >> v2:172.21.15.162:3300/0 conn(0x7fd2540b2170 0x7fd25415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540b28d0 con 0x7fd2540b6680 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fd244001670 con 0x7fd2540b6680 2024-05-10T03:34:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 >> v1:172.21.15.55:6789/0 conn(0x7fd2540b6680 legacy=0x7fd2541611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd238003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd238004030 0x7fd2380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd238004d00 0x7fd238005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540d2b20 con 0x7fd238004d00 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25415db20 con 0x7fd238004030 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23800bbc0 con 0x7fd2541618b0 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd253e00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd238004d00 0x7fd238005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd252a00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd238004030 0x7fd2380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.321+0000 7fd253400640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd238003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.322+0000 7fd253e00640 1 -- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 msgr2=0x7fd238003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.322+0000 7fd253e00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2541618b0 0x7fd238003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.322+0000 7fd253e00640 1 -- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd238004030 msgr2=0x7fd2380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.322+0000 7fd253e00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd238004030 0x7fd2380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.322+0000 7fd253e00640 1 -- 172.21.15.55:0/448421961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238003520 con 0x7fd238004d00 2024-05-10T03:34:16.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.323+0000 7fd253e00640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd238004d00 0x7fd238005150 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd24c007330 tx=0x7fd2380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e3ec529dd22bcc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.323+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c02f660 con 0x7fd238004d00 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.323+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c02f7c0 con 0x7fd238004d00 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.323+0000 7fd252000640 1 -- 172.21.15.55:0/448421961 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c02fb50 con 0x7fd238004d00 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 -- 172.21.15.55:0/448421961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd238004d00 msgr2=0x7fd238005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 --2- 172.21.15.55:0/448421961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd238004d00 0x7fd238005150 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd24c007330 tx=0x7fd2380057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd253e00640 1 -- 172.21.15.55:0/448421961 reap_dead start 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 -- 172.21.15.55:0/448421961 shutdown_connections 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 -- 172.21.15.55:0/448421961 >> 172.21.15.55:0/448421961 conn(0x7fd2540b5730 msgr2=0x7fd2540d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 -- 172.21.15.55:0/448421961 shutdown_connections 2024-05-10T03:34:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.324+0000 7fd259200640 1 -- 172.21.15.55:0/448421961 wait complete. 2024-05-10T03:34:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.326+0000 7fd259200640 1 Processor -- start 2024-05-10T03:34:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.326+0000 7fd259200640 1 -- start start 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd254159d10 0x7fd2541595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 0x7fd2541562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254159110 con 0x7fd2541f3b00 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254159ae0 con 0x7fd254159d10 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd259200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540ccc90 con 0x7fd2540cc6d0 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58184/0 (socket says 172.21.15.55:58184) 2024-05-10T03:34:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253e00640 1 -- 172.21.15.55:0/1593989310 learned_addr learned my addr 172.21.15.55:0/1593989310 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd252a00640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 0x7fd2541562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253400640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd254159d10 0x7fd2541595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253e00640 1 -- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 msgr2=0x7fd2541562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.328+0000 7fd253e00640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 0x7fd2541562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd253e00640 1 -- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd254159d10 msgr2=0x7fd2541595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd253e00640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd254159d10 0x7fd2541595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd253e00640 1 -- 172.21.15.55:0/1593989310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2541587e0 con 0x7fd2541f3b00 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd252a00640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 0x7fd2541562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd253e00640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd24c02b040 tx=0x7fd24c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c03e070 con 0x7fd2541f3b00 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c003990 con 0x7fd2541f3b00 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c003d40 con 0x7fd2541f3b00 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd254158ac0 con 0x7fd2541f3b00 2024-05-10T03:34:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.329+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd254156810 con 0x7fd2541f3b00 2024-05-10T03:34:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.330+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254067740 con 0x7fd2541f3b00 2024-05-10T03:34:16.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.331+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd24c038660 con 0x7fd2541f3b00 2024-05-10T03:34:16.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.332+0000 7fd24b400640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 0x7fd23406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.332+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd24c0a9d60 con 0x7fd2541f3b00 2024-05-10T03:34:16.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.333+0000 7fd253400640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 0x7fd23406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.334+0000 7fd253400640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 0x7fd23406d140 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd25406b1d0 tx=0x7fd244001db0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.337+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd24c074d60 con 0x7fd2541f3b00 2024-05-10T03:34:16.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.533+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2541569d0 con 0x7fd2541f3b00 2024-05-10T03:34:16.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.535+0000 7fd24b400640 1 -- 172.21.15.55:0/1593989310 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16249 (secure 0 0 0) 0x7fd2541569d0 con 0x7fd2541f3b00 2024-05-10T03:34:16.539 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:16.539 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":90,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:16.492872+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 msgr2=0x7fd23406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 0x7fd23406d140 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd25406b1d0 tx=0x7fd244001db0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 msgr2=0x7fd2541f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd24c02b040 tx=0x7fd24c004ab0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 shutdown_connections 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2541f3b00 0x7fd2541f3f10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2540cc6d0 0x7fd2541562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd23406ac30 0x7fd23406d140 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 --2- 172.21.15.55:0/1593989310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd254159d10 0x7fd2541595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 >> 172.21.15.55:0/1593989310 conn(0x7fd2540b5730 msgr2=0x7fd2540cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 shutdown_connections 2024-05-10T03:34:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.538+0000 7fd259200640 1 -- 172.21.15.55:0/1593989310 wait complete. 2024-05-10T03:34:16.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:34:16.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.841+0000 7efcbae00640 1 Processor -- start 2024-05-10T03:34:16.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.841+0000 7efcbae00640 1 -- start start 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcbae00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efcb40ce260 0x7efcb415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcbae00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 0x7efcb406a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcbae00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb40ceb00 con 0x7efcb40b2110 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcbae00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb415dcd0 con 0x7efcb40ce260 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcbae00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40d2150 con 0x7efcb40b1bc0 2024-05-10T03:34:16.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcb9000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efcb40ce260 0x7efcb415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcb9a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efcb40b2110 0x7efcb40cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52770/0 (socket says 172.21.15.55:52770) 2024-05-10T03:34:16.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.843+0000 7efcb9a00640 1 -- 172.21.15.55:0/2738414349 learned_addr learned my addr 172.21.15.55:0/2738414349 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:16.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcba400640 1 --2- 172.21.15.55:0/2738414349 >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 0x7efcb406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcb9000640 1 -- 172.21.15.55:0/2738414349 >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 msgr2=0x7efcb406a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcb9000640 1 --2- 172.21.15.55:0/2738414349 >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 0x7efcb406a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcb9000640 1 -- 172.21.15.55:0/2738414349 >> v1:172.21.15.55:6789/0 conn(0x7efcb40b2110 legacy=0x7efcb40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcb9000640 1 -- 172.21.15.55:0/2738414349 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40b2910 con 0x7efcb40ce260 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.844+0000 7efcba400640 1 --2- 172.21.15.55:0/2738414349 >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 0x7efcb406a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:16.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.845+0000 7efcb9000640 1 --2- 172.21.15.55:0/2738414349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415d790 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efcb406c9d0 tx=0x7efca002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc17d390903c231 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.845+0000 7efcb3e00640 1 -- 172.21.15.55:0/2738414349 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca002ae60 con 0x7efcb40ce260 2024-05-10T03:34:16.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.845+0000 7efcb3e00640 1 -- 172.21.15.55:0/2738414349 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0032440 con 0x7efcb40ce260 2024-05-10T03:34:16.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.845+0000 7efcb3e00640 1 -- 172.21.15.55:0/2738414349 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00327d0 con 0x7efcb40ce260 2024-05-10T03:34:16.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 -- 172.21.15.55:0/2738414349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 msgr2=0x7efcb415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 --2- 172.21.15.55:0/2738414349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415d790 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efcb406c9d0 tx=0x7efca002a110 comp rx=0 tx=0).stop 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 -- 172.21.15.55:0/2738414349 shutdown_connections 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 --2- 172.21.15.55:0/2738414349 >> v2:172.21.15.162:3300/0 conn(0x7efcb40b1bc0 0x7efcb406a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 --2- 172.21.15.55:0/2738414349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415d790 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 -- 172.21.15.55:0/2738414349 >> 172.21.15.55:0/2738414349 conn(0x7efcb40d30d0 msgr2=0x7efcb40d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 -- 172.21.15.55:0/2738414349 shutdown_connections 2024-05-10T03:34:16.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.846+0000 7efcbae00640 1 -- 172.21.15.55:0/2738414349 wait complete. 2024-05-10T03:34:16.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.849+0000 7efcbae00640 1 Processor -- start 2024-05-10T03:34:16.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.849+0000 7efcbae00640 1 -- start start 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb40b1bc0 0x7efcb4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb40b2110 0x7efcb4160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4152290 con 0x7efcb40ce260 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb414ef80 con 0x7efcb40b1bc0 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcbae00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c8bc0 con 0x7efcb40b2110 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.850+0000 7efcba400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58210/0 (socket says 172.21.15.55:58210) 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 -- 172.21.15.55:0/2556385819 learned_addr learned my addr 172.21.15.55:0/2556385819 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcb9a00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb40b1bc0 0x7efcb4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcb9000640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb40b2110 0x7efcb4160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 -- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb40b2110 msgr2=0x7efcb4160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb40b2110 0x7efcb4160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 -- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb40b1bc0 msgr2=0x7efcb4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:16.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb40b1bc0 0x7efcb4151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:16.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 -- 172.21.15.55:0/2556385819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0009400 con 0x7efcb40ce260 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.851+0000 7efcba400640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c7c0 tx=0x7efcac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.852+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac004230 con 0x7efcb40ce260 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.852+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac004390 con 0x7efcb40ce260 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.852+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac007860 con 0x7efcb40ce260 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.852+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40c8e10 con 0x7efcb40ce260 2024-05-10T03:34:16.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.852+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb414d160 con 0x7efcb40ce260 2024-05-10T03:34:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.853+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb404e290 con 0x7efcb40ce260 2024-05-10T03:34:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.854+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efcac0079c0 con 0x7efcb40ce260 2024-05-10T03:34:16.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.856+0000 7efcb1600640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 0x7efc9406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:16.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.856+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efcac0896f0 con 0x7efcb40ce260 2024-05-10T03:34:16.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.856+0000 7efcb9a00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 0x7efc9406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:16.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.858+0000 7efcb9a00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 0x7efc9406d110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efca80012e0 tx=0x7efca8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:16.861+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efcac054850 con 0x7efcb40ce260 2024-05-10T03:34:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.101+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efcb40cd490 con 0x7efcb40ce260 2024-05-10T03:34:17.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:34:17.907 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:34:17.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.970+0000 7efcb1600640 1 -- 172.21.15.55:0/2556385819 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7efcac057e70 con 0x7efcb40ce260 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 msgr2=0x7efc9406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 0x7efc9406d110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efca80012e0 tx=0x7efca8009040 comp rx=0 tx=0).stop 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 msgr2=0x7efcb415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c7c0 tx=0x7efcac00cc80 comp rx=0 tx=0).stop 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 shutdown_connections 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcb40ce260 0x7efcb415a680 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:17.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb40b2110 0x7efcb4160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:17.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efc9406ac00 0x7efc9406d110 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:17.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 --2- 172.21.15.55:0/2556385819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcb40b1bc0 0x7efcb4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:17.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 >> 172.21.15.55:0/2556385819 conn(0x7efcb40d30d0 msgr2=0x7efcb40b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:17.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 shutdown_connections 2024-05-10T03:34:17.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:17.973+0000 7efcbae00640 1 -- 172.21.15.55:0/2556385819 wait complete. 2024-05-10T03:34:18.004 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:34:18.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:34:18.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.269+0000 7f0dec400640 1 Processor -- start 2024-05-10T03:34:18.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.269+0000 7f0dec400640 1 -- start start 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0dec400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0de40b23f0 0x7f0de4161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0dec400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de40ce740 0x7f0de415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0dec400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de40d29f0 con 0x7f0de40b8d70 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0dec400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de41623d0 con 0x7f0de40b23f0 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0dec400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40d1de0 con 0x7f0de40ce740 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0de9000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0de40b8d70 0x7f0de40ce030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52782/0 (socket says 172.21.15.55:52782) 2024-05-10T03:34:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.272+0000 7f0de9000640 1 -- 172.21.15.55:0/4175650947 learned_addr learned my addr 172.21.15.55:0/4175650947 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:18.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0dea400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de40ce740 0x7f0de415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4175650947 >> v2:172.21.15.162:3300/0 conn(0x7f0de40b23f0 0x7f0de4161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389137730 0 0) 0x7f0de40d29f0 con 0x7f0de40b8d70 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dcc003520 con 0x7f0de40b8d70 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155238682 0 0) 0x7f0dcc003520 con 0x7f0de40b8d70 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de40ce740 msgr2=0x7f0de415cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de40ce740 0x7f0de415cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 >> v2:172.21.15.162:3300/0 conn(0x7f0de40b23f0 msgr2=0x7f0de4161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 --2- 172.21.15.55:0/4175650947 >> v2:172.21.15.162:3300/0 conn(0x7f0de40b23f0 0x7f0de4161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40d20c0 con 0x7f0de40b8d70 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0dd0001670 con 0x7f0de40b8d70 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.273+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 >> v1:172.21.15.55:6789/0 conn(0x7f0de40b8d70 legacy=0x7f0de40ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dcc003970 0x7f0dcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dcc0047c0 0x7f0dcc004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc005110 0x7f0dcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de41623d0 con 0x7f0dcc005110 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40d1de0 con 0x7f0dcc003970 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc005680 con 0x7f0dcc0047c0 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0dea400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc005110 0x7f0dcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dcc003970 0x7f0dcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.274+0000 7f0de9000640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dcc0047c0 0x7f0dcc004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0dea400640 1 -- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dcc0047c0 msgr2=0x7f0dcc004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0dea400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dcc0047c0 0x7f0dcc004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0dea400640 1 -- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dcc003970 msgr2=0x7f0dcc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:18.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0dea400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dcc003970 0x7f0dcc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0dea400640 1 -- 172.21.15.55:0/4175650947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc003520 con 0x7f0dcc005110 2024-05-10T03:34:18.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.275+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dcc003970 0x7f0dcc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:18.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.276+0000 7f0dea400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc005110 0x7f0dcc00b980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009ac0 tx=0x7f0dcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c8aa7b054142f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:18.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.276+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc031400 con 0x7f0dcc005110 2024-05-10T03:34:18.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.276+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc02d050 con 0x7f0dcc005110 2024-05-10T03:34:18.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.276+0000 7f0de3e00640 1 -- 172.21.15.55:0/4175650947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc031780 con 0x7f0dcc005110 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 -- 172.21.15.55:0/4175650947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc005110 msgr2=0x7f0dcc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 --2- 172.21.15.55:0/4175650947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc005110 0x7f0dcc00b980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009ac0 tx=0x7f0dcc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dea400640 1 -- 172.21.15.55:0/4175650947 reap_dead start 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 -- 172.21.15.55:0/4175650947 shutdown_connections 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 -- 172.21.15.55:0/4175650947 >> 172.21.15.55:0/4175650947 conn(0x7f0de40d0850 msgr2=0x7f0de40d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 -- 172.21.15.55:0/4175650947 shutdown_connections 2024-05-10T03:34:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.277+0000 7f0dec400640 1 -- 172.21.15.55:0/4175650947 wait complete. 2024-05-10T03:34:18.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.279+0000 7f0dec400640 1 Processor -- start 2024-05-10T03:34:18.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.279+0000 7f0dec400640 1 -- start start 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de41f3930 0x7f0de4155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de41f3d40 0x7f0de40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de415bfc0 con 0x7f0de40c7360 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40c7770 con 0x7f0de41f3d40 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dec400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de4157130 con 0x7f0de41f3930 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dea400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dea400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58236/0 (socket says 172.21.15.55:58236) 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.281+0000 7f0dea400640 1 -- 172.21.15.55:0/4112677403 learned_addr learned my addr 172.21.15.55:0/4112677403 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0de9000640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de41f3d40 0x7f0de40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de41f3930 0x7f0de4155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 -- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de41f3930 msgr2=0x7f0de4155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de41f3930 0x7f0de4155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 -- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de41f3d40 msgr2=0x7f0de40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de41f3d40 0x7f0de40c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 -- 172.21.15.55:0/4112677403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40b9300 con 0x7f0de40c7360 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.282+0000 7f0dea400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc02abb0 tx=0x7f0ddc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.283+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc004030 con 0x7f0de40c7360 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.283+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc004190 con 0x7f0de40c7360 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.283+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc004540 con 0x7f0de40c7360 2024-05-10T03:34:18.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.283+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de4157410 con 0x7f0de40c7360 2024-05-10T03:34:18.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.283+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de41579d0 con 0x7f0de40c7360 2024-05-10T03:34:18.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.284+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de4157b90 con 0x7f0de40c7360 2024-05-10T03:34:18.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.285+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ddc038070 con 0x7f0de40c7360 2024-05-10T03:34:18.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.286+0000 7f0de1600640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 0x7f0dc406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:18.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.286+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0ddc000e40 con 0x7f0de40c7360 2024-05-10T03:34:18.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.287+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 0x7f0dc406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:18.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.288+0000 7f0de9a00640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 0x7f0dc406d140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8007440 tx=0x7f0dd800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:18.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.291+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ddc074020 con 0x7f0de40c7360 2024-05-10T03:34:18.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.529+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0de41612a0 con 0x7f0de40c7360 2024-05-10T03:34:18.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.978+0000 7f0de1600640 1 -- 172.21.15.55:0/4112677403 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ddc036020 con 0x7f0de40c7360 2024-05-10T03:34:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.982+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 msgr2=0x7f0dc406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.982+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 0x7f0dc406d140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8007440 tx=0x7f0dd800b040 comp rx=0 tx=0).stop 2024-05-10T03:34:18.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.982+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 msgr2=0x7f0de415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.982+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc02abb0 tx=0x7f0ddc004730 comp rx=0 tx=0).stop 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 shutdown_connections 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0de40c7360 0x7f0de415b2b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0dc406ac30 0x7f0dc406d140 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0de41f3d40 0x7f0de40c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 --2- 172.21.15.55:0/4112677403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0de41f3930 0x7f0de4155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 >> 172.21.15.55:0/4112677403 conn(0x7f0de40d0850 msgr2=0x7f0de406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 shutdown_connections 2024-05-10T03:34:18.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:18.983+0000 7f0dec400640 1 -- 172.21.15.55:0/4112677403 wait complete. 2024-05-10T03:34:19.017 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:34:19.018 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.298+0000 7fce29c00640 1 Processor -- start 2024-05-10T03:34:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.298+0000 7fce29c00640 1 -- start start 2024-05-10T03:34:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.300+0000 7fce29c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fce241659c0 0x7fce24165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.300+0000 7fce29c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce240b2700 0x7fce240b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce29c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce240715e0 con 0x7fce2406b6e0 2024-05-10T03:34:19.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce29c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240711c0 con 0x7fce241659c0 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce29c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2406baf0 con 0x7fce240b2700 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce1be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fce241659c0 0x7fce24165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce22a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fce2406b6e0 0x7fce240ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52788/0 (socket says 172.21.15.55:52788) 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce22a00640 1 -- 172.21.15.55:0/1751412390 learned_addr learned my addr 172.21.15.55:0/1751412390 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.301+0000 7fce23400640 1 --2- 172.21.15.55:0/1751412390 >> v2:172.21.15.162:3300/0 conn(0x7fce240b2700 0x7fce240b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.302+0000 7fce1be00640 1 -- 172.21.15.55:0/1751412390 >> v2:172.21.15.162:3300/0 conn(0x7fce240b2700 msgr2=0x7fce240b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.302+0000 7fce1be00640 1 --2- 172.21.15.55:0/1751412390 >> v2:172.21.15.162:3300/0 conn(0x7fce240b2700 0x7fce240b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.302+0000 7fce1be00640 1 -- 172.21.15.55:0/1751412390 >> v1:172.21.15.55:6789/0 conn(0x7fce2406b6e0 legacy=0x7fce240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.302+0000 7fce1be00640 1 -- 172.21.15.55:0/1751412390 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24166310 con 0x7fce241659c0 2024-05-10T03:34:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.302+0000 7fce22000640 1 -- 172.21.15.55:0/1751412390 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950139762 0 0) 0x7fce14001110 con 0x7fce2406b6e0 2024-05-10T03:34:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.303+0000 7fce1be00640 1 --2- 172.21.15.55:0/1751412390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce24165dd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fce240cd1d0 tx=0x7fce0c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57322c73614153ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.303+0000 7fce22000640 1 -- 172.21.15.55:0/1751412390 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c02ae60 con 0x7fce241659c0 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.303+0000 7fce22000640 1 -- 172.21.15.55:0/1751412390 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c032440 con 0x7fce241659c0 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.303+0000 7fce22000640 1 -- 172.21.15.55:0/1751412390 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0327d0 con 0x7fce241659c0 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 -- 172.21.15.55:0/1751412390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 msgr2=0x7fce24165dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 --2- 172.21.15.55:0/1751412390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce24165dd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fce240cd1d0 tx=0x7fce0c02a110 comp rx=0 tx=0).stop 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 -- 172.21.15.55:0/1751412390 shutdown_connections 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 --2- 172.21.15.55:0/1751412390 >> v2:172.21.15.162:3300/0 conn(0x7fce240b2700 0x7fce240b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 --2- 172.21.15.55:0/1751412390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce24165dd0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 -- 172.21.15.55:0/1751412390 >> 172.21.15.55:0/1751412390 conn(0x7fce240cbb90 msgr2=0x7fce240cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 -- 172.21.15.55:0/1751412390 shutdown_connections 2024-05-10T03:34:19.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.304+0000 7fce29c00640 1 -- 172.21.15.55:0/1751412390 wait complete. 2024-05-10T03:34:19.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.306+0000 7fce29c00640 1 Processor -- start 2024-05-10T03:34:19.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.306+0000 7fce29c00640 1 -- start start 2024-05-10T03:34:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 0x7fce241612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce240b2700 0x7fce24159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce241f8c40 con 0x7fce241659c0 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415e2b0 con 0x7fce240b2700 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce29c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce241602c0 con 0x7fce2406b6e0 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce23400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce23400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58260/0 (socket says 172.21.15.55:58260) 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.308+0000 7fce23400640 1 -- 172.21.15.55:0/4146070768 learned_addr learned my addr 172.21.15.55:0/4146070768 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce22a00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 0x7fce241612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce1be00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce240b2700 0x7fce24159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 -- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 msgr2=0x7fce241612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 0x7fce241612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 -- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce240b2700 msgr2=0x7fce24159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce240b2700 0x7fce24159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 -- 172.21.15.55:0/4146070768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c009400 con 0x7fce241659c0 2024-05-10T03:34:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce22a00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 0x7fce241612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.309+0000 7fce23400640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007e90 tx=0x7fce1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.310+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c014420 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.310+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c014580 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.310+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c014930 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.310+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce24160510 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.310+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce24160ad0 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.311+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce24066f60 con 0x7fce241659c0 2024-05-10T03:34:19.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.312+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fce1c014a90 con 0x7fce241659c0 2024-05-10T03:34:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.313+0000 7fce1aa00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 0x7fce0006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.314+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce1c089ea0 con 0x7fce241659c0 2024-05-10T03:34:19.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.314+0000 7fce22a00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 0x7fce0006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.316+0000 7fce22a00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 0x7fce0006d110 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fce14000f60 tx=0x7fce14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.318+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fce1c054ea0 con 0x7fce241659c0 2024-05-10T03:34:19.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.560+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce24067220 con 0x7fce241659c0 2024-05-10T03:34:19.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.561+0000 7fce1aa00640 1 -- 172.21.15.55:0/4146070768 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fce1c0584c0 con 0x7fce241659c0 2024-05-10T03:34:19.565 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:19.565 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:19.566 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 msgr2=0x7fce0006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 0x7fce0006d110 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fce14000f60 tx=0x7fce14009040 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 msgr2=0x7fce241f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007e90 tx=0x7fce1c00e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 shutdown_connections 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce241659c0 0x7fce241f8030 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fce0006ac00 0x7fce0006d110 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce240b2700 0x7fce24159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 --2- 172.21.15.55:0/4146070768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce2406b6e0 0x7fce241612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 >> 172.21.15.55:0/4146070768 conn(0x7fce240cbb90 msgr2=0x7fce2415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 shutdown_connections 2024-05-10T03:34:19.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.564+0000 7fce29c00640 1 -- 172.21.15.55:0/4146070768 wait complete. 2024-05-10T03:34:19.594 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:19.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.873+0000 7f5b19000640 1 Processor -- start 2024-05-10T03:34:19.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.873+0000 7f5b19000640 1 -- start start 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.875+0000 7f5b19000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b1415fe50 0x7f5b14141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b19000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5b140b2140 0x7f5b14142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b19000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1406ad20 con 0x7f5b140b2550 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b19000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b14141f50 con 0x7f5b140b2140 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b19000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1406a110 con 0x7f5b1415fe50 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b12800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5b140b2550 0x7f5b1406b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52802/0 (socket says 172.21.15.55:52802) 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b12800640 1 -- 172.21.15.55:0/2242768203 learned_addr learned my addr 172.21.15.55:0/2242768203 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:19.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.876+0000 7f5b09400640 1 --2- 172.21.15.55:0/2242768203 >> v2:172.21.15.55:3300/0 conn(0x7f5b140b2140 0x7f5b14142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b11e00640 1 --2- 172.21.15.55:0/2242768203 >> v2:172.21.15.162:3300/0 conn(0x7f5b1415fe50 0x7f5b14141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b11400640 1 -- 172.21.15.55:0/2242768203 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874008796 0 0) 0x7f5b1406ad20 con 0x7f5b140b2550 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b11400640 1 -- 172.21.15.55:0/2242768203 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af8003520 con 0x7f5b140b2550 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b09400640 1 -- 172.21.15.55:0/2242768203 >> v2:172.21.15.162:3300/0 conn(0x7f5b1415fe50 msgr2=0x7f5b14141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b09400640 1 --2- 172.21.15.55:0/2242768203 >> v2:172.21.15.162:3300/0 conn(0x7f5b1415fe50 0x7f5b14141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b09400640 1 -- 172.21.15.55:0/2242768203 >> v1:172.21.15.55:6789/0 conn(0x7f5b140b2550 legacy=0x7f5b1406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.877+0000 7f5b09400640 1 -- 172.21.15.55:0/2242768203 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1406a3f0 con 0x7f5b140b2140 2024-05-10T03:34:19.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.878+0000 7f5b09400640 1 --2- 172.21.15.55:0/2242768203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b140b2140 0x7f5b14142440 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406e520 tx=0x7f5afc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f7a0cf84e1017e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.879+0000 7f5b11400640 1 -- 172.21.15.55:0/2242768203 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc032440 con 0x7f5b140b2140 2024-05-10T03:34:19.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.879+0000 7f5b11400640 1 -- 172.21.15.55:0/2242768203 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc0325a0 con 0x7f5b140b2140 2024-05-10T03:34:19.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.879+0000 7f5b11400640 1 -- 172.21.15.55:0/2242768203 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc032930 con 0x7f5b140b2140 2024-05-10T03:34:19.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.879+0000 7f5b19000640 1 -- 172.21.15.55:0/2242768203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b140b2140 msgr2=0x7f5b14142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.879+0000 7f5b19000640 1 --2- 172.21.15.55:0/2242768203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b140b2140 0x7f5b14142440 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406e520 tx=0x7f5afc02a250 comp rx=0 tx=0).stop 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 -- 172.21.15.55:0/2242768203 shutdown_connections 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 --2- 172.21.15.55:0/2242768203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b140b2140 0x7f5b14142440 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 --2- 172.21.15.55:0/2242768203 >> v2:172.21.15.162:3300/0 conn(0x7f5b1415fe50 0x7f5b14141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 -- 172.21.15.55:0/2242768203 >> 172.21.15.55:0/2242768203 conn(0x7f5b140cbb90 msgr2=0x7f5b140cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 -- 172.21.15.55:0/2242768203 shutdown_connections 2024-05-10T03:34:19.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.880+0000 7f5b19000640 1 -- 172.21.15.55:0/2242768203 wait complete. 2024-05-10T03:34:19.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.882+0000 7f5b19000640 1 Processor -- start 2024-05-10T03:34:19.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.882+0000 7f5b19000640 1 -- start start 2024-05-10T03:34:19.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b140b2140 0x7f5b1415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b140b2550 0x7f5b1413e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14159370 con 0x7f5b1415fe50 2024-05-10T03:34:19.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1415db80 con 0x7f5b140b2550 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b19000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14142b10 con 0x7f5b140b2140 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b12800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b12800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58276/0 (socket says 172.21.15.55:58276) 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.884+0000 7f5b12800640 1 -- 172.21.15.55:0/269251600 learned_addr learned my addr 172.21.15.55:0/269251600 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b11e00640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b140b2140 0x7f5b1415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b09400640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b140b2550 0x7f5b1413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 -- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b140b2140 msgr2=0x7f5b1415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b140b2140 0x7f5b1415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 -- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b140b2550 msgr2=0x7f5b1413e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b140b2550 0x7f5b1413e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 -- 172.21.15.55:0/269251600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009400 con 0x7f5b1415fe50 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b12800640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00caa0 tx=0x7f5b0c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00ac00 con 0x7f5b1415fe50 2024-05-10T03:34:19.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00ad60 con 0x7f5b1415fe50 2024-05-10T03:34:19.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c012620 con 0x7f5b1415fe50 2024-05-10T03:34:19.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.885+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1415e400 con 0x7f5b1415fe50 2024-05-10T03:34:19.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.886+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14142220 con 0x7f5b1415fe50 2024-05-10T03:34:19.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.887+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14066f60 con 0x7f5b1415fe50 2024-05-10T03:34:19.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.888+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5b0c007070 con 0x7f5b1415fe50 2024-05-10T03:34:19.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.889+0000 7f5b0aa00640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 0x7f5af006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:19.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.889+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5b0c0894d0 con 0x7f5b1415fe50 2024-05-10T03:34:19.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.890+0000 7f5b11e00640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 0x7f5af006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:19.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.891+0000 7f5b11e00640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 0x7f5af006d110 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009a30 tx=0x7f5b04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:19.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:19.894+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5b0c0544d0 con 0x7f5b1415fe50 2024-05-10T03:34:20.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.099+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b14141300 con 0x7f5b1415fe50 2024-05-10T03:34:20.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.102+0000 7f5b0aa00640 1 -- 172.21.15.55:0/269251600 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16249 (secure 0 0 0) 0x7f5b0c057af0 con 0x7f5b1415fe50 2024-05-10T03:34:20.106 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:20.106 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":90,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:16.492872+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 msgr2=0x7f5af006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 0x7f5af006d110 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009a30 tx=0x7f5b04008040 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 msgr2=0x7f5b1413e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00caa0 tx=0x7f5b0c00cf60 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 shutdown_connections 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b1415fe50 0x7f5b1413e890 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5af006ac00 0x7f5af006d110 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b140b2550 0x7f5b1413e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 --2- 172.21.15.55:0/269251600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b140b2140 0x7f5b1415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 >> 172.21.15.55:0/269251600 conn(0x7f5b140cbb90 msgr2=0x7f5b140cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 shutdown_connections 2024-05-10T03:34:20.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.105+0000 7f5b19000640 1 -- 172.21.15.55:0/269251600 wait complete. 2024-05-10T03:34:20.137 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:34:20.137 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:34:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.412+0000 7f46a2000640 1 Processor -- start 2024-05-10T03:34:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.413+0000 7f46a2000640 1 -- start start 2024-05-10T03:34:20.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.415+0000 7f46a2000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f469c1614b0 0x7f469c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:20.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.415+0000 7f46a2000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0cf580 con 0x7f469c06b830 2024-05-10T03:34:20.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.415+0000 7f46a2000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c15e5c0 con 0x7f469c15dec0 2024-05-10T03:34:20.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.415+0000 7f46a2000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c15ee60 con 0x7f469c1614b0 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f469c1614b0 0x7f469c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f469c06b830 0x7f469c0b2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52818/0 (socket says 172.21.15.55:52818) 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469be00640 1 -- 172.21.15.55:0/2811261352 learned_addr learned my addr 172.21.15.55:0/2811261352 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469b400640 1 -- 172.21.15.55:0/2811261352 >> v1:172.21.15.162:6789/0 conn(0x7f469c15dec0 legacy=0x7f469c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469a000640 1 -- 172.21.15.55:0/2811261352 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259283968 0 0) 0x7f469c0cf580 con 0x7f469c06b830 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469b400640 1 -- 172.21.15.55:0/2811261352 >> v1:172.21.15.55:6789/0 conn(0x7f469c06b830 legacy=0x7f469c0b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.416+0000 7f469b400640 1 -- 172.21.15.55:0/2811261352 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c15e8a0 con 0x7f469c1614b0 2024-05-10T03:34:20.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.418+0000 7f469b400640 1 --2- 172.21.15.55:0/2811261352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0aedd0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24e12c95e32035e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.418+0000 7f469a000640 1 -- 172.21.15.55:0/2811261352 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684032440 con 0x7f469c1614b0 2024-05-10T03:34:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.418+0000 7f469a000640 1 -- 172.21.15.55:0/2811261352 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840325a0 con 0x7f469c1614b0 2024-05-10T03:34:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.418+0000 7f469a000640 1 -- 172.21.15.55:0/2811261352 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684032930 con 0x7f469c1614b0 2024-05-10T03:34:20.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 -- 172.21.15.55:0/2811261352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 msgr2=0x7f469c0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 --2- 172.21.15.55:0/2811261352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0aedd0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402a230 comp rx=0 tx=0).stop 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 -- 172.21.15.55:0/2811261352 shutdown_connections 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 --2- 172.21.15.55:0/2811261352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0aedd0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 -- 172.21.15.55:0/2811261352 >> 172.21.15.55:0/2811261352 conn(0x7f469c070900 msgr2=0x7f469c070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 -- 172.21.15.55:0/2811261352 shutdown_connections 2024-05-10T03:34:20.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.419+0000 7f46a2000640 1 -- 172.21.15.55:0/2811261352 wait complete. 2024-05-10T03:34:20.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.421+0000 7f46a2000640 1 Processor -- start 2024-05-10T03:34:20.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.421+0000 7f46a2000640 1 -- start start 2024-05-10T03:34:20.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469c15dec0 0x7f469c14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:20.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469c0afe10 0x7f469c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0b0e20 con 0x7f469c1614b0 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c149360 con 0x7f469c0afe10 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f46a2000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0b0330 con 0x7f469c15dec0 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f469aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f469aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58294/0 (socket says 172.21.15.55:58294) 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.423+0000 7f469aa00640 1 -- 172.21.15.55:0/3404064068 learned_addr learned my addr 172.21.15.55:0/3404064068 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469b400640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469c15dec0 0x7f469c14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469be00640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469c0afe10 0x7f469c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 -- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469c15dec0 msgr2=0x7f469c14d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469c15dec0 0x7f469c14d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 -- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469c0afe10 msgr2=0x7f469c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469c0afe10 0x7f469c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 -- 172.21.15.55:0/3404064068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684009400 con 0x7f469c1614b0 2024-05-10T03:34:20.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.424+0000 7f469aa00640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f468c00c730 tx=0x7f468c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:20.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.425+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c003b70 con 0x7f469c1614b0 2024-05-10T03:34:20.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.425+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c003cf0 con 0x7f469c1614b0 2024-05-10T03:34:20.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.425+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c003ea0 con 0x7f469c1614b0 2024-05-10T03:34:20.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.425+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c14d850 con 0x7f469c1614b0 2024-05-10T03:34:20.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.425+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c147b90 con 0x7f469c1614b0 2024-05-10T03:34:20.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.426+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c147d20 con 0x7f469c1614b0 2024-05-10T03:34:20.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.427+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f468c00aa90 con 0x7f469c1614b0 2024-05-10T03:34:20.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.428+0000 7f4693400640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 0x7f467806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:20.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.428+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f468c0893f0 con 0x7f469c1614b0 2024-05-10T03:34:20.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.429+0000 7f469b400640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 0x7f467806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:20.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.430+0000 7f469b400640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 0x7f467806d110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4684034000 tx=0x7f468402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:20.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.433+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f468c054360 con 0x7f469c1614b0 2024-05-10T03:34:20.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.628+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f469c15e2d0 con 0x7f467806ac00 2024-05-10T03:34:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.652+0000 7f4693400640 1 -- 172.21.15.55:0/3404064068 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286873 (secure 0 0 0) 0x7f4684032b40 con 0x7f467806ac00 2024-05-10T03:34:20.660 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:34:20.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.658+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 msgr2=0x7f467806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.658+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 0x7f467806d110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4684034000 tx=0x7f468402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 msgr2=0x7f469c0c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f468c00c730 tx=0x7f468c00cbf0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 shutdown_connections 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f467806ac00 0x7f467806d110 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f469c0afe10 0x7f469c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c1614b0 0x7f469c0c4800 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 --2- 172.21.15.55:0/3404064068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469c15dec0 0x7f469c14d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 >> 172.21.15.55:0/3404064068 conn(0x7f469c070900 msgr2=0x7f469c15d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 shutdown_connections 2024-05-10T03:34:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:20.659+0000 7f46a2000640 1 -- 172.21.15.55:0/3404064068 wait complete. 2024-05-10T03:34:20.691 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-05-10T03:34:19.056430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258812,"kb_used_data":9860,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715908,"statfs":{"total":773094113280,"available":772829089792,"internally_reserved":0,"allocated":10096640,"data_stored":5176288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.155772"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513889+0000","last_change":"2024-05-10T03:34:15.528797+0000","last_active":"2024-05-10T03:34:16.513889+0000","last_peered":"2024-05-10T03:34:16.513889+0000","last_clean":"2024-05-10T03:34:16.513889+0000","last_became_active":"2024-05-10T03:34:15.528428+0000","last_became_peered":"2024-05-10T03:34:15.528428+0000","last_unstale":"2024-05-10T03:34:16.513889+0000","last_undegraded":"2024-05-10T03:34:16.513889+0000","last_fullsized":"2024-05-10T03:34:16.513889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:29.222932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516928+0000","last_change":"2024-05-10T03:34:15.511392+0000","last_active":"2024-05-10T03:34:16.516928+0000","last_peered":"2024-05-10T03:34:16.516928+0000","last_clean":"2024-05-10T03:34:16.516928+0000","last_became_active":"2024-05-10T03:34:15.510472+0000","last_became_peered":"2024-05-10T03:34:15.510472+0000","last_unstale":"2024-05-10T03:34:16.516928+0000","last_undegraded":"2024-05-10T03:34:16.516928+0000","last_fullsized":"2024-05-10T03:34:16.516928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:06.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513542+0000","last_change":"2024-05-10T03:34:15.535756+0000","last_active":"2024-05-10T03:34:16.513542+0000","last_peered":"2024-05-10T03:34:16.513542+0000","last_clean":"2024-05-10T03:34:16.513542+0000","last_became_active":"2024-05-10T03:34:15.535449+0000","last_became_peered":"2024-05-10T03:34:15.535449+0000","last_unstale":"2024-05-10T03:34:16.513542+0000","last_undegraded":"2024-05-10T03:34:16.513542+0000","last_fullsized":"2024-05-10T03:34:16.513542+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:34.133109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818517+0000","last_change":"2024-05-10T03:34:15.537352+0000","last_active":"2024-05-10T03:34:16.818517+0000","last_peered":"2024-05-10T03:34:16.818517+0000","last_clean":"2024-05-10T03:34:16.818517+0000","last_became_active":"2024-05-10T03:34:15.536804+0000","last_became_peered":"2024-05-10T03:34:15.536804+0000","last_unstale":"2024-05-10T03:34:16.818517+0000","last_undegraded":"2024-05-10T03:34:16.818517+0000","last_fullsized":"2024-05-10T03:34:16.818517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:30.223450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516680+0000","last_change":"2024-05-10T03:34:15.536521+0000","last_active":"2024-05-10T03:34:16.516680+0000","last_peered":"2024-05-10T03:34:16.516680+0000","last_clean":"2024-05-10T03:34:16.516680+0000","last_became_active":"2024-05-10T03:34:15.535865+0000","last_became_peered":"2024-05-10T03:34:15.535865+0000","last_unstale":"2024-05-10T03:34:16.516680+0000","last_undegraded":"2024-05-10T03:34:16.516680+0000","last_fullsized":"2024-05-10T03:34:16.516680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:18:15.083196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815557+0000","last_change":"2024-05-10T03:34:15.524624+0000","last_active":"2024-05-10T03:34:16.815557+0000","last_peered":"2024-05-10T03:34:16.815557+0000","last_clean":"2024-05-10T03:34:16.815557+0000","last_became_active":"2024-05-10T03:34:15.524200+0000","last_became_peered":"2024-05-10T03:34:15.524200+0000","last_unstale":"2024-05-10T03:34:16.815557+0000","last_undegraded":"2024-05-10T03:34:16.815557+0000","last_fullsized":"2024-05-10T03:34:16.815557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:34.133109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815619+0000","last_change":"2024-05-10T03:34:15.539051+0000","last_active":"2024-05-10T03:34:16.815619+0000","last_peered":"2024-05-10T03:34:16.815619+0000","last_clean":"2024-05-10T03:34:16.815619+0000","last_became_active":"2024-05-10T03:34:15.538774+0000","last_became_peered":"2024-05-10T03:34:15.538774+0000","last_unstale":"2024-05-10T03:34:16.815619+0000","last_undegraded":"2024-05-10T03:34:16.815619+0000","last_fullsized":"2024-05-10T03:34:16.815619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816057+0000","last_change":"2024-05-10T03:34:15.540854+0000","last_active":"2024-05-10T03:34:16.816057+0000","last_peered":"2024-05-10T03:34:16.816057+0000","last_clean":"2024-05-10T03:34:16.816057+0000","last_became_active":"2024-05-10T03:34:15.539780+0000","last_became_peered":"2024-05-10T03:34:15.539780+0000","last_unstale":"2024-05-10T03:34:16.816057+0000","last_undegraded":"2024-05-10T03:34:16.816057+0000","last_fullsized":"2024-05-10T03:34:16.816057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517846+0000","last_change":"2024-05-10T03:34:15.526743+0000","last_active":"2024-05-10T03:34:16.517846+0000","last_peered":"2024-05-10T03:34:16.517846+0000","last_clean":"2024-05-10T03:34:16.517846+0000","last_became_active":"2024-05-10T03:34:15.526262+0000","last_became_peered":"2024-05-10T03:34:15.526262+0000","last_unstale":"2024-05-10T03:34:16.517846+0000","last_undegraded":"2024-05-10T03:34:16.517846+0000","last_fullsized":"2024-05-10T03:34:16.517846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816003+0000","last_change":"2024-05-10T03:34:15.525674+0000","last_active":"2024-05-10T03:34:16.816003+0000","last_peered":"2024-05-10T03:34:16.816003+0000","last_clean":"2024-05-10T03:34:16.816003+0000","last_became_active":"2024-05-10T03:34:15.525361+0000","last_became_peered":"2024-05-10T03:34:15.525361+0000","last_unstale":"2024-05-10T03:34:16.816003+0000","last_undegraded":"2024-05-10T03:34:16.816003+0000","last_fullsized":"2024-05-10T03:34:16.816003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:05.431760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517276+0000","last_change":"2024-05-10T03:34:15.528030+0000","last_active":"2024-05-10T03:34:16.517276+0000","last_peered":"2024-05-10T03:34:16.517276+0000","last_clean":"2024-05-10T03:34:16.517276+0000","last_became_active":"2024-05-10T03:34:15.527476+0000","last_became_peered":"2024-05-10T03:34:15.527476+0000","last_unstale":"2024-05-10T03:34:16.517276+0000","last_undegraded":"2024-05-10T03:34:16.517276+0000","last_fullsized":"2024-05-10T03:34:16.517276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815895+0000","last_change":"2024-05-10T03:34:15.515921+0000","last_active":"2024-05-10T03:34:16.815895+0000","last_peered":"2024-05-10T03:34:16.815895+0000","last_clean":"2024-05-10T03:34:16.815895+0000","last_became_active":"2024-05-10T03:34:15.515621+0000","last_became_peered":"2024-05-10T03:34:15.515621+0000","last_unstale":"2024-05-10T03:34:16.815895+0000","last_undegraded":"2024-05-10T03:34:16.815895+0000","last_fullsized":"2024-05-10T03:34:16.815895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:37.543793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817642+0000","last_change":"2024-05-10T03:34:15.538984+0000","last_active":"2024-05-10T03:34:16.817642+0000","last_peered":"2024-05-10T03:34:16.817642+0000","last_clean":"2024-05-10T03:34:16.817642+0000","last_became_active":"2024-05-10T03:34:15.538647+0000","last_became_peered":"2024-05-10T03:34:15.538647+0000","last_unstale":"2024-05-10T03:34:16.817642+0000","last_undegraded":"2024-05-10T03:34:16.817642+0000","last_fullsized":"2024-05-10T03:34:16.817642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515432+0000","last_change":"2024-05-10T03:34:15.525465+0000","last_active":"2024-05-10T03:34:16.515432+0000","last_peered":"2024-05-10T03:34:16.515432+0000","last_clean":"2024-05-10T03:34:16.515432+0000","last_became_active":"2024-05-10T03:34:15.525121+0000","last_became_peered":"2024-05-10T03:34:15.525121+0000","last_unstale":"2024-05-10T03:34:16.515432+0000","last_undegraded":"2024-05-10T03:34:16.515432+0000","last_fullsized":"2024-05-10T03:34:16.515432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:06.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817147+0000","last_change":"2024-05-10T03:34:15.520850+0000","last_active":"2024-05-10T03:34:16.817147+0000","last_peered":"2024-05-10T03:34:16.817147+0000","last_clean":"2024-05-10T03:34:16.817147+0000","last_became_active":"2024-05-10T03:34:15.520325+0000","last_became_peered":"2024-05-10T03:34:15.520325+0000","last_unstale":"2024-05-10T03:34:16.817147+0000","last_undegraded":"2024-05-10T03:34:16.817147+0000","last_fullsized":"2024-05-10T03:34:16.817147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:11:39.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815071+0000","last_change":"2024-05-10T03:34:15.511944+0000","last_active":"2024-05-10T03:34:16.815071+0000","last_peered":"2024-05-10T03:34:16.815071+0000","last_clean":"2024-05-10T03:34:16.815071+0000","last_became_active":"2024-05-10T03:34:15.511636+0000","last_became_peered":"2024-05-10T03:34:15.511636+0000","last_unstale":"2024-05-10T03:34:16.815071+0000","last_undegraded":"2024-05-10T03:34:16.815071+0000","last_fullsized":"2024-05-10T03:34:16.815071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:17.560672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817183+0000","last_change":"2024-05-10T03:34:15.520885+0000","last_active":"2024-05-10T03:34:16.817183+0000","last_peered":"2024-05-10T03:34:16.817183+0000","last_clean":"2024-05-10T03:34:16.817183+0000","last_became_active":"2024-05-10T03:34:15.520621+0000","last_became_peered":"2024-05-10T03:34:15.520621+0000","last_unstale":"2024-05-10T03:34:16.817183+0000","last_undegraded":"2024-05-10T03:34:16.817183+0000","last_fullsized":"2024-05-10T03:34:16.817183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:02.412699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513638+0000","last_change":"2024-05-10T03:34:15.518504+0000","last_active":"2024-05-10T03:34:16.513638+0000","last_peered":"2024-05-10T03:34:16.513638+0000","last_clean":"2024-05-10T03:34:16.513638+0000","last_became_active":"2024-05-10T03:34:15.516749+0000","last_became_peered":"2024-05-10T03:34:15.516749+0000","last_unstale":"2024-05-10T03:34:16.513638+0000","last_undegraded":"2024-05-10T03:34:16.513638+0000","last_fullsized":"2024-05-10T03:34:16.513638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:19.298176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517431+0000","last_change":"2024-05-10T03:34:15.525319+0000","last_active":"2024-05-10T03:34:16.517431+0000","last_peered":"2024-05-10T03:34:16.517431+0000","last_clean":"2024-05-10T03:34:16.517431+0000","last_became_active":"2024-05-10T03:34:15.524851+0000","last_became_peered":"2024-05-10T03:34:15.524851+0000","last_unstale":"2024-05-10T03:34:16.517431+0000","last_undegraded":"2024-05-10T03:34:16.517431+0000","last_fullsized":"2024-05-10T03:34:16.517431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818778+0000","last_change":"2024-05-10T03:34:15.525829+0000","last_active":"2024-05-10T03:34:16.818778+0000","last_peered":"2024-05-10T03:34:16.818778+0000","last_clean":"2024-05-10T03:34:16.818778+0000","last_became_active":"2024-05-10T03:34:15.521290+0000","last_became_peered":"2024-05-10T03:34:15.521290+0000","last_unstale":"2024-05-10T03:34:16.818778+0000","last_undegraded":"2024-05-10T03:34:16.818778+0000","last_fullsized":"2024-05-10T03:34:16.818778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:31.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515966+0000","last_change":"2024-05-10T03:34:15.538836+0000","last_active":"2024-05-10T03:34:16.515966+0000","last_peered":"2024-05-10T03:34:16.515966+0000","last_clean":"2024-05-10T03:34:16.515966+0000","last_became_active":"2024-05-10T03:34:15.536144+0000","last_became_peered":"2024-05-10T03:34:15.536144+0000","last_unstale":"2024-05-10T03:34:16.515966+0000","last_undegraded":"2024-05-10T03:34:16.515966+0000","last_fullsized":"2024-05-10T03:34:16.515966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:00.783781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515759+0000","last_change":"2024-05-10T03:34:15.524619+0000","last_active":"2024-05-10T03:34:16.515759+0000","last_peered":"2024-05-10T03:34:16.515759+0000","last_clean":"2024-05-10T03:34:16.515759+0000","last_became_active":"2024-05-10T03:34:15.522451+0000","last_became_peered":"2024-05-10T03:34:15.522451+0000","last_unstale":"2024-05-10T03:34:16.515759+0000","last_undegraded":"2024-05-10T03:34:16.515759+0000","last_fullsized":"2024-05-10T03:34:16.515759+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:51:49.117770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814544+0000","last_change":"2024-05-10T03:34:15.543575+0000","last_active":"2024-05-10T03:34:16.814544+0000","last_peered":"2024-05-10T03:34:16.814544+0000","last_clean":"2024-05-10T03:34:16.814544+0000","last_became_active":"2024-05-10T03:34:15.542382+0000","last_became_peered":"2024-05-10T03:34:15.542382+0000","last_unstale":"2024-05-10T03:34:16.814544+0000","last_undegraded":"2024-05-10T03:34:16.814544+0000","last_fullsized":"2024-05-10T03:34:16.814544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:55.158777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.512832+0000","last_change":"2024-05-10T03:34:15.533288+0000","last_active":"2024-05-10T03:34:16.512832+0000","last_peered":"2024-05-10T03:34:16.512832+0000","last_clean":"2024-05-10T03:34:16.512832+0000","last_became_active":"2024-05-10T03:34:15.532987+0000","last_became_peered":"2024-05-10T03:34:15.532987+0000","last_unstale":"2024-05-10T03:34:16.512832+0000","last_undegraded":"2024-05-10T03:34:16.512832+0000","last_fullsized":"2024-05-10T03:34:16.512832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:16.495944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517937+0000","last_change":"2024-05-10T03:34:15.540823+0000","last_active":"2024-05-10T03:34:16.517937+0000","last_peered":"2024-05-10T03:34:16.517937+0000","last_clean":"2024-05-10T03:34:16.517937+0000","last_became_active":"2024-05-10T03:34:15.540476+0000","last_became_peered":"2024-05-10T03:34:15.540476+0000","last_unstale":"2024-05-10T03:34:16.517937+0000","last_undegraded":"2024-05-10T03:34:16.517937+0000","last_fullsized":"2024-05-10T03:34:16.517937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517091+0000","last_change":"2024-05-10T03:34:15.525170+0000","last_active":"2024-05-10T03:34:16.517091+0000","last_peered":"2024-05-10T03:34:16.517091+0000","last_clean":"2024-05-10T03:34:16.517091+0000","last_became_active":"2024-05-10T03:34:15.524797+0000","last_became_peered":"2024-05-10T03:34:15.524797+0000","last_unstale":"2024-05-10T03:34:16.517091+0000","last_undegraded":"2024-05-10T03:34:16.517091+0000","last_fullsized":"2024-05-10T03:34:16.517091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815455+0000","last_change":"2024-05-10T03:34:15.535727+0000","last_active":"2024-05-10T03:34:16.815455+0000","last_peered":"2024-05-10T03:34:16.815455+0000","last_clean":"2024-05-10T03:34:16.815455+0000","last_became_active":"2024-05-10T03:34:15.535387+0000","last_became_peered":"2024-05-10T03:34:15.535387+0000","last_unstale":"2024-05-10T03:34:16.815455+0000","last_undegraded":"2024-05-10T03:34:16.815455+0000","last_fullsized":"2024-05-10T03:34:16.815455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:17:21.537793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818835+0000","last_change":"2024-05-10T03:34:15.535799+0000","last_active":"2024-05-10T03:34:16.818835+0000","last_peered":"2024-05-10T03:34:16.818835+0000","last_clean":"2024-05-10T03:34:16.818835+0000","last_became_active":"2024-05-10T03:34:15.535470+0000","last_became_peered":"2024-05-10T03:34:15.535470+0000","last_unstale":"2024-05-10T03:34:16.818835+0000","last_undegraded":"2024-05-10T03:34:16.818835+0000","last_fullsized":"2024-05-10T03:34:16.818835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:38.415468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816112+0000","last_change":"2024-05-10T03:34:15.537333+0000","last_active":"2024-05-10T03:34:16.816112+0000","last_peered":"2024-05-10T03:34:16.816112+0000","last_clean":"2024-05-10T03:34:16.816112+0000","last_became_active":"2024-05-10T03:34:15.536963+0000","last_became_peered":"2024-05-10T03:34:15.536963+0000","last_unstale":"2024-05-10T03:34:16.816112+0000","last_undegraded":"2024-05-10T03:34:16.816112+0000","last_fullsized":"2024-05-10T03:34:16.816112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516051+0000","last_change":"2024-05-10T03:34:15.529645+0000","last_active":"2024-05-10T03:34:16.516051+0000","last_peered":"2024-05-10T03:34:16.516051+0000","last_clean":"2024-05-10T03:34:16.516051+0000","last_became_active":"2024-05-10T03:34:15.529093+0000","last_became_peered":"2024-05-10T03:34:15.529093+0000","last_unstale":"2024-05-10T03:34:16.516051+0000","last_undegraded":"2024-05-10T03:34:16.516051+0000","last_fullsized":"2024-05-10T03:34:16.516051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515005+0000","last_change":"2024-05-10T03:34:15.524199+0000","last_active":"2024-05-10T03:34:16.515005+0000","last_peered":"2024-05-10T03:34:16.515005+0000","last_clean":"2024-05-10T03:34:16.515005+0000","last_became_active":"2024-05-10T03:34:15.523749+0000","last_became_peered":"2024-05-10T03:34:15.523749+0000","last_unstale":"2024-05-10T03:34:16.515005+0000","last_undegraded":"2024-05-10T03:34:16.515005+0000","last_fullsized":"2024-05-10T03:34:16.515005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814806+0000","last_change":"2024-05-10T03:34:15.509139+0000","last_active":"2024-05-10T03:34:16.814806+0000","last_peered":"2024-05-10T03:34:16.814806+0000","last_clean":"2024-05-10T03:34:16.814806+0000","last_became_active":"2024-05-10T03:34:15.508477+0000","last_became_peered":"2024-05-10T03:34:15.508477+0000","last_unstale":"2024-05-10T03:34:16.814806+0000","last_undegraded":"2024-05-10T03:34:16.814806+0000","last_fullsized":"2024-05-10T03:34:16.814806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:19.788399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816164+0000","last_change":"2024-05-10T03:34:15.539401+0000","last_active":"2024-05-10T03:34:16.816164+0000","last_peered":"2024-05-10T03:34:16.816164+0000","last_clean":"2024-05-10T03:34:16.816164+0000","last_became_active":"2024-05-10T03:34:15.538963+0000","last_became_peered":"2024-05-10T03:34:15.538963+0000","last_unstale":"2024-05-10T03:34:16.816164+0000","last_undegraded":"2024-05-10T03:34:16.816164+0000","last_fullsized":"2024-05-10T03:34:16.816164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:49:02.226571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815238+0000","last_change":"2024-05-10T03:34:14.950337+0000","last_active":"2024-05-10T03:34:16.815238+0000","last_peered":"2024-05-10T03:34:16.815238+0000","last_clean":"2024-05-10T03:34:16.815238+0000","last_became_active":"2024-05-10T03:34:14.949683+0000","last_became_peered":"2024-05-10T03:34:14.949683+0000","last_unstale":"2024-05-10T03:34:16.815238+0000","last_undegraded":"2024-05-10T03:34:16.815238+0000","last_fullsized":"2024-05-10T03:34:16.815238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:36.397376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816640+0000","last_change":"2024-05-10T03:34:15.521234+0000","last_active":"2024-05-10T03:34:16.816640+0000","last_peered":"2024-05-10T03:34:16.816640+0000","last_clean":"2024-05-10T03:34:16.816640+0000","last_became_active":"2024-05-10T03:34:15.517964+0000","last_became_peered":"2024-05-10T03:34:15.517964+0000","last_unstale":"2024-05-10T03:34:16.816640+0000","last_undegraded":"2024-05-10T03:34:16.816640+0000","last_fullsized":"2024-05-10T03:34:16.816640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:08.850179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517397+0000","last_change":"2024-05-10T03:34:14.942647+0000","last_active":"2024-05-10T03:34:16.517397+0000","last_peered":"2024-05-10T03:34:16.517397+0000","last_clean":"2024-05-10T03:34:16.517397+0000","last_became_active":"2024-05-10T03:34:14.942299+0000","last_became_peered":"2024-05-10T03:34:14.942299+0000","last_unstale":"2024-05-10T03:34:16.517397+0000","last_undegraded":"2024-05-10T03:34:16.517397+0000","last_fullsized":"2024-05-10T03:34:16.517397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:55:04.387279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816643+0000","last_change":"2024-05-10T03:34:15.536769+0000","last_active":"2024-05-10T03:34:16.816643+0000","last_peered":"2024-05-10T03:34:16.816643+0000","last_clean":"2024-05-10T03:34:16.816643+0000","last_became_active":"2024-05-10T03:34:15.536232+0000","last_became_peered":"2024-05-10T03:34:15.536232+0000","last_unstale":"2024-05-10T03:34:16.816643+0000","last_undegraded":"2024-05-10T03:34:16.816643+0000","last_fullsized":"2024-05-10T03:34:16.816643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:59.177046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816298+0000","last_change":"2024-05-10T03:34:14.943064+0000","last_active":"2024-05-10T03:34:16.816298+0000","last_peered":"2024-05-10T03:34:16.816298+0000","last_clean":"2024-05-10T03:34:16.816298+0000","last_became_active":"2024-05-10T03:34:14.942420+0000","last_became_peered":"2024-05-10T03:34:14.942420+0000","last_unstale":"2024-05-10T03:34:16.816298+0000","last_undegraded":"2024-05-10T03:34:16.816298+0000","last_fullsized":"2024-05-10T03:34:16.816298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:30.425448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817295+0000","last_change":"2024-05-10T03:34:15.541426+0000","last_active":"2024-05-10T03:34:16.817295+0000","last_peered":"2024-05-10T03:34:16.817295+0000","last_clean":"2024-05-10T03:34:16.817295+0000","last_became_active":"2024-05-10T03:34:15.541111+0000","last_became_peered":"2024-05-10T03:34:15.541111+0000","last_unstale":"2024-05-10T03:34:16.817295+0000","last_undegraded":"2024-05-10T03:34:16.817295+0000","last_fullsized":"2024-05-10T03:34:16.817295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:21:31.673939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815455+0000","last_change":"2024-05-10T03:34:14.947233+0000","last_active":"2024-05-10T03:34:16.815455+0000","last_peered":"2024-05-10T03:34:16.815455+0000","last_clean":"2024-05-10T03:34:16.815455+0000","last_became_active":"2024-05-10T03:34:14.946945+0000","last_became_peered":"2024-05-10T03:34:14.946945+0000","last_unstale":"2024-05-10T03:34:16.815455+0000","last_undegraded":"2024-05-10T03:34:16.815455+0000","last_fullsized":"2024-05-10T03:34:16.815455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:05.308556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816343+0000","last_change":"2024-05-10T03:34:15.518416+0000","last_active":"2024-05-10T03:34:16.816343+0000","last_peered":"2024-05-10T03:34:16.816343+0000","last_clean":"2024-05-10T03:34:16.816343+0000","last_became_active":"2024-05-10T03:34:15.518135+0000","last_became_peered":"2024-05-10T03:34:15.518135+0000","last_unstale":"2024-05-10T03:34:16.816343+0000","last_undegraded":"2024-05-10T03:34:16.816343+0000","last_fullsized":"2024-05-10T03:34:16.816343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:04:17.699300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516654+0000","last_change":"2024-05-10T03:34:14.935163+0000","last_active":"2024-05-10T03:34:16.516654+0000","last_peered":"2024-05-10T03:34:16.516654+0000","last_clean":"2024-05-10T03:34:16.516654+0000","last_became_active":"2024-05-10T03:34:14.934811+0000","last_became_peered":"2024-05-10T03:34:14.934811+0000","last_unstale":"2024-05-10T03:34:16.516654+0000","last_undegraded":"2024-05-10T03:34:16.516654+0000","last_fullsized":"2024-05-10T03:34:16.516654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:10:14.518870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814643+0000","last_change":"2024-05-10T03:34:15.540784+0000","last_active":"2024-05-10T03:34:16.814643+0000","last_peered":"2024-05-10T03:34:16.814643+0000","last_clean":"2024-05-10T03:34:16.814643+0000","last_became_active":"2024-05-10T03:34:15.540440+0000","last_became_peered":"2024-05-10T03:34:15.540440+0000","last_unstale":"2024-05-10T03:34:16.814643+0000","last_undegraded":"2024-05-10T03:34:16.814643+0000","last_fullsized":"2024-05-10T03:34:16.814643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:31.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.819168+0000","last_change":"2024-05-10T03:34:14.946744+0000","last_active":"2024-05-10T03:34:16.819168+0000","last_peered":"2024-05-10T03:34:16.819168+0000","last_clean":"2024-05-10T03:34:16.819168+0000","last_became_active":"2024-05-10T03:34:14.946389+0000","last_became_peered":"2024-05-10T03:34:14.946389+0000","last_unstale":"2024-05-10T03:34:16.819168+0000","last_undegraded":"2024-05-10T03:34:16.819168+0000","last_fullsized":"2024-05-10T03:34:16.819168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:38.426482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815674+0000","last_change":"2024-05-10T03:34:15.524424+0000","last_active":"2024-05-10T03:34:16.815674+0000","last_peered":"2024-05-10T03:34:16.815674+0000","last_clean":"2024-05-10T03:34:16.815674+0000","last_became_active":"2024-05-10T03:34:15.523863+0000","last_became_peered":"2024-05-10T03:34:15.523863+0000","last_unstale":"2024-05-10T03:34:16.815674+0000","last_undegraded":"2024-05-10T03:34:16.815674+0000","last_fullsized":"2024-05-10T03:34:16.815674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:16.495944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513161+0000","last_change":"2024-05-10T03:34:14.939189+0000","last_active":"2024-05-10T03:34:16.513161+0000","last_peered":"2024-05-10T03:34:16.513161+0000","last_clean":"2024-05-10T03:34:16.513161+0000","last_became_active":"2024-05-10T03:34:14.938479+0000","last_became_peered":"2024-05-10T03:34:14.938479+0000","last_unstale":"2024-05-10T03:34:16.513161+0000","last_undegraded":"2024-05-10T03:34:16.513161+0000","last_fullsized":"2024-05-10T03:34:16.513161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:58.157256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817927+0000","last_change":"2024-05-10T03:34:15.542682+0000","last_active":"2024-05-10T03:34:16.817927+0000","last_peered":"2024-05-10T03:34:16.817927+0000","last_clean":"2024-05-10T03:34:16.817927+0000","last_became_active":"2024-05-10T03:34:15.542353+0000","last_became_peered":"2024-05-10T03:34:15.542353+0000","last_unstale":"2024-05-10T03:34:16.817927+0000","last_undegraded":"2024-05-10T03:34:16.817927+0000","last_fullsized":"2024-05-10T03:34:16.817927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:11.640179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814705+0000","last_change":"2024-05-10T03:34:14.948746+0000","last_active":"2024-05-10T03:34:16.814705+0000","last_peered":"2024-05-10T03:34:16.814705+0000","last_clean":"2024-05-10T03:34:16.814705+0000","last_became_active":"2024-05-10T03:34:14.948370+0000","last_became_peered":"2024-05-10T03:34:14.948370+0000","last_unstale":"2024-05-10T03:34:16.814705+0000","last_undegraded":"2024-05-10T03:34:16.814705+0000","last_fullsized":"2024-05-10T03:34:16.814705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:01.001923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517123+0000","last_change":"2024-05-10T03:34:15.529851+0000","last_active":"2024-05-10T03:34:16.517123+0000","last_peered":"2024-05-10T03:34:16.517123+0000","last_clean":"2024-05-10T03:34:16.517123+0000","last_became_active":"2024-05-10T03:34:15.529453+0000","last_became_peered":"2024-05-10T03:34:15.529453+0000","last_unstale":"2024-05-10T03:34:16.517123+0000","last_undegraded":"2024-05-10T03:34:16.517123+0000","last_fullsized":"2024-05-10T03:34:16.517123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:34.133376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.518090+0000","last_change":"2024-05-10T03:34:14.923811+0000","last_active":"2024-05-10T03:34:16.518090+0000","last_peered":"2024-05-10T03:34:16.518090+0000","last_clean":"2024-05-10T03:34:16.518090+0000","last_became_active":"2024-05-10T03:34:14.923077+0000","last_became_peered":"2024-05-10T03:34:14.923077+0000","last_unstale":"2024-05-10T03:34:16.518090+0000","last_undegraded":"2024-05-10T03:34:16.518090+0000","last_fullsized":"2024-05-10T03:34:16.518090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:30.425448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816270+0000","last_change":"2024-05-10T03:34:15.537663+0000","last_active":"2024-05-10T03:34:16.816270+0000","last_peered":"2024-05-10T03:34:16.816270+0000","last_clean":"2024-05-10T03:34:16.816270+0000","last_became_active":"2024-05-10T03:34:15.537333+0000","last_became_peered":"2024-05-10T03:34:15.537333+0000","last_unstale":"2024-05-10T03:34:16.816270+0000","last_undegraded":"2024-05-10T03:34:16.816270+0000","last_fullsized":"2024-05-10T03:34:16.816270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513789+0000","last_change":"2024-05-10T03:34:14.951924+0000","last_active":"2024-05-10T03:34:16.513789+0000","last_peered":"2024-05-10T03:34:16.513789+0000","last_clean":"2024-05-10T03:34:16.513789+0000","last_became_active":"2024-05-10T03:34:14.951529+0000","last_became_peered":"2024-05-10T03:34:14.951529+0000","last_unstale":"2024-05-10T03:34:16.513789+0000","last_undegraded":"2024-05-10T03:34:16.513789+0000","last_fullsized":"2024-05-10T03:34:16.513789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:23.726303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513685+0000","last_change":"2024-05-10T03:34:15.517382+0000","last_active":"2024-05-10T03:34:16.513685+0000","last_peered":"2024-05-10T03:34:16.513685+0000","last_clean":"2024-05-10T03:34:16.513685+0000","last_became_active":"2024-05-10T03:34:15.516079+0000","last_became_peered":"2024-05-10T03:34:15.516079+0000","last_unstale":"2024-05-10T03:34:16.513685+0000","last_undegraded":"2024-05-10T03:34:16.513685+0000","last_fullsized":"2024-05-10T03:34:16.513685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:39:30.876200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517699+0000","last_change":"2024-05-10T03:34:14.951098+0000","last_active":"2024-05-10T03:34:16.517699+0000","last_peered":"2024-05-10T03:34:16.517699+0000","last_clean":"2024-05-10T03:34:16.517699+0000","last_became_active":"2024-05-10T03:34:14.950678+0000","last_became_peered":"2024-05-10T03:34:14.950678+0000","last_unstale":"2024-05-10T03:34:16.517699+0000","last_undegraded":"2024-05-10T03:34:16.517699+0000","last_fullsized":"2024-05-10T03:34:16.517699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:05:39.826217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818763+0000","last_change":"2024-05-10T03:34:15.523322+0000","last_active":"2024-05-10T03:34:16.818763+0000","last_peered":"2024-05-10T03:34:16.818763+0000","last_clean":"2024-05-10T03:34:16.818763+0000","last_became_active":"2024-05-10T03:34:15.522986+0000","last_became_peered":"2024-05-10T03:34:15.522986+0000","last_unstale":"2024-05-10T03:34:16.818763+0000","last_undegraded":"2024-05-10T03:34:16.818763+0000","last_fullsized":"2024-05-10T03:34:16.818763+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:15.387271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816400+0000","last_change":"2024-05-10T03:34:14.949482+0000","last_active":"2024-05-10T03:34:16.816400+0000","last_peered":"2024-05-10T03:34:16.816400+0000","last_clean":"2024-05-10T03:34:16.816400+0000","last_became_active":"2024-05-10T03:34:14.949154+0000","last_became_peered":"2024-05-10T03:34:14.949154+0000","last_unstale":"2024-05-10T03:34:16.816400+0000","last_undegraded":"2024-05-10T03:34:16.816400+0000","last_fullsized":"2024-05-10T03:34:16.816400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:05.166784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816659+0000","last_change":"2024-05-10T03:34:15.533682+0000","last_active":"2024-05-10T03:34:16.816659+0000","last_peered":"2024-05-10T03:34:16.816659+0000","last_clean":"2024-05-10T03:34:16.816659+0000","last_became_active":"2024-05-10T03:34:15.533342+0000","last_became_peered":"2024-05-10T03:34:15.533342+0000","last_unstale":"2024-05-10T03:34:16.816659+0000","last_undegraded":"2024-05-10T03:34:16.816659+0000","last_fullsized":"2024-05-10T03:34:16.816659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:55.158777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816276+0000","last_change":"2024-05-10T03:34:14.946042+0000","last_active":"2024-05-10T03:34:16.816276+0000","last_peered":"2024-05-10T03:34:16.816276+0000","last_clean":"2024-05-10T03:34:16.816276+0000","last_became_active":"2024-05-10T03:34:14.945633+0000","last_became_peered":"2024-05-10T03:34:14.945633+0000","last_unstale":"2024-05-10T03:34:16.816276+0000","last_undegraded":"2024-05-10T03:34:16.816276+0000","last_fullsized":"2024-05-10T03:34:16.816276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:29.203659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515851+0000","last_change":"2024-05-10T03:34:15.538963+0000","last_active":"2024-05-10T03:34:16.515851+0000","last_peered":"2024-05-10T03:34:16.515851+0000","last_clean":"2024-05-10T03:34:16.515851+0000","last_became_active":"2024-05-10T03:34:15.536797+0000","last_became_peered":"2024-05-10T03:34:15.536797+0000","last_unstale":"2024-05-10T03:34:16.515851+0000","last_undegraded":"2024-05-10T03:34:16.515851+0000","last_fullsized":"2024-05-10T03:34:16.515851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815776+0000","last_change":"2024-05-10T03:34:14.947524+0000","last_active":"2024-05-10T03:34:16.815776+0000","last_peered":"2024-05-10T03:34:16.815776+0000","last_clean":"2024-05-10T03:34:16.815776+0000","last_became_active":"2024-05-10T03:34:14.947137+0000","last_became_peered":"2024-05-10T03:34:14.947137+0000","last_unstale":"2024-05-10T03:34:16.815776+0000","last_undegraded":"2024-05-10T03:34:16.815776+0000","last_fullsized":"2024-05-10T03:34:16.815776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:12.647164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814632+0000","last_change":"2024-05-10T03:34:15.524948+0000","last_active":"2024-05-10T03:34:16.814632+0000","last_peered":"2024-05-10T03:34:16.814632+0000","last_clean":"2024-05-10T03:34:16.814632+0000","last_became_active":"2024-05-10T03:34:15.523451+0000","last_became_peered":"2024-05-10T03:34:15.523451+0000","last_unstale":"2024-05-10T03:34:16.814632+0000","last_undegraded":"2024-05-10T03:34:16.814632+0000","last_fullsized":"2024-05-10T03:34:16.814632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:05.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817646+0000","last_change":"2024-05-10T03:34:14.952816+0000","last_active":"2024-05-10T03:34:16.817646+0000","last_peered":"2024-05-10T03:34:16.817646+0000","last_clean":"2024-05-10T03:34:16.817646+0000","last_became_active":"2024-05-10T03:34:14.951269+0000","last_became_peered":"2024-05-10T03:34:14.951269+0000","last_unstale":"2024-05-10T03:34:16.817646+0000","last_undegraded":"2024-05-10T03:34:16.817646+0000","last_fullsized":"2024-05-10T03:34:16.817646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:10:14.518870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814928+0000","last_change":"2024-05-10T03:34:15.527299+0000","last_active":"2024-05-10T03:34:16.814928+0000","last_peered":"2024-05-10T03:34:16.814928+0000","last_clean":"2024-05-10T03:34:16.814928+0000","last_became_active":"2024-05-10T03:34:15.527007+0000","last_became_peered":"2024-05-10T03:34:15.527007+0000","last_unstale":"2024-05-10T03:34:16.814928+0000","last_undegraded":"2024-05-10T03:34:16.814928+0000","last_fullsized":"2024-05-10T03:34:16.814928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:40.864469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816945+0000","last_change":"2024-05-10T03:34:14.948249+0000","last_active":"2024-05-10T03:34:16.816945+0000","last_peered":"2024-05-10T03:34:16.816945+0000","last_clean":"2024-05-10T03:34:16.816945+0000","last_became_active":"2024-05-10T03:34:14.947920+0000","last_became_peered":"2024-05-10T03:34:14.947920+0000","last_unstale":"2024-05-10T03:34:16.816945+0000","last_undegraded":"2024-05-10T03:34:16.816945+0000","last_fullsized":"2024-05-10T03:34:16.816945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:09.353930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816286+0000","last_change":"2024-05-10T03:34:15.530186+0000","last_active":"2024-05-10T03:34:16.816286+0000","last_peered":"2024-05-10T03:34:16.816286+0000","last_clean":"2024-05-10T03:34:16.816286+0000","last_became_active":"2024-05-10T03:34:15.529905+0000","last_became_peered":"2024-05-10T03:34:15.529905+0000","last_unstale":"2024-05-10T03:34:16.816286+0000","last_undegraded":"2024-05-10T03:34:16.816286+0000","last_fullsized":"2024-05-10T03:34:16.816286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:47.424391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815627+0000","last_change":"2024-05-10T03:34:14.942751+0000","last_active":"2024-05-10T03:34:16.815627+0000","last_peered":"2024-05-10T03:34:16.815627+0000","last_clean":"2024-05-10T03:34:16.815627+0000","last_became_active":"2024-05-10T03:34:14.942362+0000","last_became_peered":"2024-05-10T03:34:14.942362+0000","last_unstale":"2024-05-10T03:34:16.815627+0000","last_undegraded":"2024-05-10T03:34:16.815627+0000","last_fullsized":"2024-05-10T03:34:16.815627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:52.207470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814957+0000","last_change":"2024-05-10T03:34:15.533943+0000","last_active":"2024-05-10T03:34:16.814957+0000","last_peered":"2024-05-10T03:34:16.814957+0000","last_clean":"2024-05-10T03:34:16.814957+0000","last_became_active":"2024-05-10T03:34:15.533503+0000","last_became_peered":"2024-05-10T03:34:15.533503+0000","last_unstale":"2024-05-10T03:34:16.814957+0000","last_undegraded":"2024-05-10T03:34:16.814957+0000","last_fullsized":"2024-05-10T03:34:16.814957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816598+0000","last_change":"2024-05-10T03:34:14.944530+0000","last_active":"2024-05-10T03:34:16.816598+0000","last_peered":"2024-05-10T03:34:16.816598+0000","last_clean":"2024-05-10T03:34:16.816598+0000","last_became_active":"2024-05-10T03:34:14.944190+0000","last_became_peered":"2024-05-10T03:34:14.944190+0000","last_unstale":"2024-05-10T03:34:16.816598+0000","last_undegraded":"2024-05-10T03:34:16.816598+0000","last_fullsized":"2024-05-10T03:34:16.816598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:28.679478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517280+0000","last_change":"2024-05-10T03:34:15.517291+0000","last_active":"2024-05-10T03:34:16.517280+0000","last_peered":"2024-05-10T03:34:16.517280+0000","last_clean":"2024-05-10T03:34:16.517280+0000","last_became_active":"2024-05-10T03:34:15.516848+0000","last_became_peered":"2024-05-10T03:34:15.516848+0000","last_unstale":"2024-05-10T03:34:16.517280+0000","last_undegraded":"2024-05-10T03:34:16.517280+0000","last_fullsized":"2024-05-10T03:34:16.517280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:39.075357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517803+0000","last_change":"2024-05-10T03:34:14.945673+0000","last_active":"2024-05-10T03:34:16.517803+0000","last_peered":"2024-05-10T03:34:16.517803+0000","last_clean":"2024-05-10T03:34:16.517803+0000","last_became_active":"2024-05-10T03:34:14.945197+0000","last_became_peered":"2024-05-10T03:34:14.945197+0000","last_unstale":"2024-05-10T03:34:16.517803+0000","last_undegraded":"2024-05-10T03:34:16.517803+0000","last_fullsized":"2024-05-10T03:34:16.517803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:59.387389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815139+0000","last_change":"2024-05-10T03:34:15.524917+0000","last_active":"2024-05-10T03:34:16.815139+0000","last_peered":"2024-05-10T03:34:16.815139+0000","last_clean":"2024-05-10T03:34:16.815139+0000","last_became_active":"2024-05-10T03:34:15.522122+0000","last_became_peered":"2024-05-10T03:34:15.522122+0000","last_unstale":"2024-05-10T03:34:16.815139+0000","last_undegraded":"2024-05-10T03:34:16.815139+0000","last_fullsized":"2024-05-10T03:34:16.815139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:00.050088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515344+0000","last_change":"2024-05-10T03:34:14.925937+0000","last_active":"2024-05-10T03:34:16.515344+0000","last_peered":"2024-05-10T03:34:16.515344+0000","last_clean":"2024-05-10T03:34:16.515344+0000","last_became_active":"2024-05-10T03:34:14.925378+0000","last_became_peered":"2024-05-10T03:34:14.925378+0000","last_unstale":"2024-05-10T03:34:16.515344+0000","last_undegraded":"2024-05-10T03:34:16.515344+0000","last_fullsized":"2024-05-10T03:34:16.515344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:14.367480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515645+0000","last_change":"2024-05-10T03:34:15.513493+0000","last_active":"2024-05-10T03:34:16.515645+0000","last_peered":"2024-05-10T03:34:16.515645+0000","last_clean":"2024-05-10T03:34:16.515645+0000","last_became_active":"2024-05-10T03:34:15.512327+0000","last_became_peered":"2024-05-10T03:34:15.512327+0000","last_unstale":"2024-05-10T03:34:16.515645+0000","last_undegraded":"2024-05-10T03:34:16.515645+0000","last_fullsized":"2024-05-10T03:34:16.515645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:51:49.117770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814626+0000","last_change":"2024-05-10T03:34:14.941946+0000","last_active":"2024-05-10T03:34:16.814626+0000","last_peered":"2024-05-10T03:34:16.814626+0000","last_clean":"2024-05-10T03:34:16.814626+0000","last_became_active":"2024-05-10T03:34:14.941608+0000","last_became_peered":"2024-05-10T03:34:14.941608+0000","last_unstale":"2024-05-10T03:34:16.814626+0000","last_undegraded":"2024-05-10T03:34:16.814626+0000","last_fullsized":"2024-05-10T03:34:16.814626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:19.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515431+0000","last_change":"2024-05-10T03:34:15.523875+0000","last_active":"2024-05-10T03:34:16.515431+0000","last_peered":"2024-05-10T03:34:16.515431+0000","last_clean":"2024-05-10T03:34:16.515431+0000","last_became_active":"2024-05-10T03:34:15.523455+0000","last_became_peered":"2024-05-10T03:34:15.523455+0000","last_unstale":"2024-05-10T03:34:16.515431+0000","last_undegraded":"2024-05-10T03:34:16.515431+0000","last_fullsized":"2024-05-10T03:34:16.515431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515909+0000","last_change":"2024-05-10T03:34:14.944524+0000","last_active":"2024-05-10T03:34:16.515909+0000","last_peered":"2024-05-10T03:34:16.515909+0000","last_clean":"2024-05-10T03:34:16.515909+0000","last_became_active":"2024-05-10T03:34:14.944003+0000","last_became_peered":"2024-05-10T03:34:14.944003+0000","last_unstale":"2024-05-10T03:34:16.515909+0000","last_undegraded":"2024-05-10T03:34:16.515909+0000","last_fullsized":"2024-05-10T03:34:16.515909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:29.203659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817123+0000","last_change":"2024-05-10T03:34:15.520647+0000","last_active":"2024-05-10T03:34:16.817123+0000","last_peered":"2024-05-10T03:34:16.817123+0000","last_clean":"2024-05-10T03:34:16.817123+0000","last_became_active":"2024-05-10T03:34:15.518850+0000","last_became_peered":"2024-05-10T03:34:15.518850+0000","last_unstale":"2024-05-10T03:34:16.817123+0000","last_undegraded":"2024-05-10T03:34:16.817123+0000","last_fullsized":"2024-05-10T03:34:16.817123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:44.549294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814118+0000","last_change":"2024-05-10T03:34:14.927895+0000","last_active":"2024-05-10T03:34:16.814118+0000","last_peered":"2024-05-10T03:34:16.814118+0000","last_clean":"2024-05-10T03:34:16.814118+0000","last_became_active":"2024-05-10T03:34:14.927588+0000","last_became_peered":"2024-05-10T03:34:14.927588+0000","last_unstale":"2024-05-10T03:34:16.814118+0000","last_undegraded":"2024-05-10T03:34:16.814118+0000","last_fullsized":"2024-05-10T03:34:16.814118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:00.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815228+0000","last_change":"2024-05-10T03:34:15.543638+0000","last_active":"2024-05-10T03:34:16.815228+0000","last_peered":"2024-05-10T03:34:16.815228+0000","last_clean":"2024-05-10T03:34:16.815228+0000","last_became_active":"2024-05-10T03:34:15.541482+0000","last_became_peered":"2024-05-10T03:34:15.541482+0000","last_unstale":"2024-05-10T03:34:16.815228+0000","last_undegraded":"2024-05-10T03:34:16.815228+0000","last_fullsized":"2024-05-10T03:34:16.815228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:30.223450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816696+0000","last_change":"2024-05-10T03:34:14.942982+0000","last_active":"2024-05-10T03:34:16.816696+0000","last_peered":"2024-05-10T03:34:16.816696+0000","last_clean":"2024-05-10T03:34:16.816696+0000","last_became_active":"2024-05-10T03:34:14.942638+0000","last_became_peered":"2024-05-10T03:34:14.942638+0000","last_unstale":"2024-05-10T03:34:16.816696+0000","last_undegraded":"2024-05-10T03:34:16.816696+0000","last_fullsized":"2024-05-10T03:34:16.816696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:54.138986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816752+0000","last_change":"2024-05-10T03:34:15.519514+0000","last_active":"2024-05-10T03:34:16.816752+0000","last_peered":"2024-05-10T03:34:16.816752+0000","last_clean":"2024-05-10T03:34:16.816752+0000","last_became_active":"2024-05-10T03:34:15.517817+0000","last_became_peered":"2024-05-10T03:34:15.517817+0000","last_unstale":"2024-05-10T03:34:16.816752+0000","last_undegraded":"2024-05-10T03:34:16.816752+0000","last_fullsized":"2024-05-10T03:34:16.816752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:37:03.119608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517426+0000","last_change":"2024-05-10T03:34:14.939598+0000","last_active":"2024-05-10T03:34:16.517426+0000","last_peered":"2024-05-10T03:34:16.517426+0000","last_clean":"2024-05-10T03:34:16.517426+0000","last_became_active":"2024-05-10T03:34:14.939316+0000","last_became_peered":"2024-05-10T03:34:14.939316+0000","last_unstale":"2024-05-10T03:34:16.517426+0000","last_undegraded":"2024-05-10T03:34:16.517426+0000","last_fullsized":"2024-05-10T03:34:16.517426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:11.802006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816596+0000","last_change":"2024-05-10T03:34:15.518344+0000","last_active":"2024-05-10T03:34:16.816596+0000","last_peered":"2024-05-10T03:34:16.816596+0000","last_clean":"2024-05-10T03:34:16.816596+0000","last_became_active":"2024-05-10T03:34:15.517408+0000","last_became_peered":"2024-05-10T03:34:15.517408+0000","last_unstale":"2024-05-10T03:34:16.816596+0000","last_undegraded":"2024-05-10T03:34:16.816596+0000","last_fullsized":"2024-05-10T03:34:16.816596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:19.286396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517843+0000","last_change":"2024-05-10T03:34:14.950882+0000","last_active":"2024-05-10T03:34:16.517843+0000","last_peered":"2024-05-10T03:34:16.517843+0000","last_clean":"2024-05-10T03:34:16.517843+0000","last_became_active":"2024-05-10T03:34:14.950594+0000","last_became_peered":"2024-05-10T03:34:14.950594+0000","last_unstale":"2024-05-10T03:34:16.517843+0000","last_undegraded":"2024-05-10T03:34:16.517843+0000","last_fullsized":"2024-05-10T03:34:16.517843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:17.244818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517946+0000","last_change":"2024-05-10T03:34:15.530844+0000","last_active":"2024-05-10T03:34:16.517946+0000","last_peered":"2024-05-10T03:34:16.517946+0000","last_clean":"2024-05-10T03:34:16.517946+0000","last_became_active":"2024-05-10T03:34:15.530430+0000","last_became_peered":"2024-05-10T03:34:15.530430+0000","last_unstale":"2024-05-10T03:34:16.517946+0000","last_undegraded":"2024-05-10T03:34:16.517946+0000","last_fullsized":"2024-05-10T03:34:16.517946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815509+0000","last_change":"2024-05-10T03:34:14.941122+0000","last_active":"2024-05-10T03:34:16.815509+0000","last_peered":"2024-05-10T03:34:16.815509+0000","last_clean":"2024-05-10T03:34:16.815509+0000","last_became_active":"2024-05-10T03:34:14.940760+0000","last_became_peered":"2024-05-10T03:34:14.940760+0000","last_unstale":"2024-05-10T03:34:16.815509+0000","last_undegraded":"2024-05-10T03:34:16.815509+0000","last_fullsized":"2024-05-10T03:34:16.815509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:24.213926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814173+0000","last_change":"2024-05-10T03:34:15.512024+0000","last_active":"2024-05-10T03:34:16.814173+0000","last_peered":"2024-05-10T03:34:16.814173+0000","last_clean":"2024-05-10T03:34:16.814173+0000","last_became_active":"2024-05-10T03:34:15.511452+0000","last_became_peered":"2024-05-10T03:34:15.511452+0000","last_unstale":"2024-05-10T03:34:16.814173+0000","last_undegraded":"2024-05-10T03:34:16.814173+0000","last_fullsized":"2024-05-10T03:34:16.814173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:03:03.547694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817646+0000","last_change":"2024-05-10T03:34:14.941460+0000","last_active":"2024-05-10T03:34:16.817646+0000","last_peered":"2024-05-10T03:34:16.817646+0000","last_clean":"2024-05-10T03:34:16.817646+0000","last_became_active":"2024-05-10T03:34:14.940822+0000","last_became_peered":"2024-05-10T03:34:14.940822+0000","last_unstale":"2024-05-10T03:34:16.817646+0000","last_undegraded":"2024-05-10T03:34:16.817646+0000","last_fullsized":"2024-05-10T03:34:16.817646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:33.532220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513046+0000","last_change":"2024-05-10T03:34:15.525028+0000","last_active":"2024-05-10T03:34:16.513046+0000","last_peered":"2024-05-10T03:34:16.513046+0000","last_clean":"2024-05-10T03:34:16.513046+0000","last_became_active":"2024-05-10T03:34:15.524744+0000","last_became_peered":"2024-05-10T03:34:15.524744+0000","last_unstale":"2024-05-10T03:34:16.513046+0000","last_undegraded":"2024-05-10T03:34:16.513046+0000","last_fullsized":"2024-05-10T03:34:16.513046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:05.704532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515509+0000","last_change":"2024-05-10T03:34:14.930636+0000","last_active":"2024-05-10T03:34:16.515509+0000","last_peered":"2024-05-10T03:34:16.515509+0000","last_clean":"2024-05-10T03:34:16.515509+0000","last_became_active":"2024-05-10T03:34:14.930302+0000","last_became_peered":"2024-05-10T03:34:14.930302+0000","last_unstale":"2024-05-10T03:34:16.515509+0000","last_undegraded":"2024-05-10T03:34:16.515509+0000","last_fullsized":"2024-05-10T03:34:16.515509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:37.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814554+0000","last_change":"2024-05-10T03:34:15.545444+0000","last_active":"2024-05-10T03:34:16.814554+0000","last_peered":"2024-05-10T03:34:16.814554+0000","last_clean":"2024-05-10T03:34:16.814554+0000","last_became_active":"2024-05-10T03:34:15.543724+0000","last_became_peered":"2024-05-10T03:34:15.543724+0000","last_unstale":"2024-05-10T03:34:16.814554+0000","last_undegraded":"2024-05-10T03:34:16.814554+0000","last_fullsized":"2024-05-10T03:34:16.814554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815546+0000","last_change":"2024-05-10T03:34:14.940987+0000","last_active":"2024-05-10T03:34:16.815546+0000","last_peered":"2024-05-10T03:34:16.815546+0000","last_clean":"2024-05-10T03:34:16.815546+0000","last_became_active":"2024-05-10T03:34:14.940553+0000","last_became_peered":"2024-05-10T03:34:14.940553+0000","last_unstale":"2024-05-10T03:34:16.815546+0000","last_undegraded":"2024-05-10T03:34:16.815546+0000","last_fullsized":"2024-05-10T03:34:16.815546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:43.934723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817395+0000","last_change":"2024-05-10T03:34:15.508750+0000","last_active":"2024-05-10T03:34:16.817395+0000","last_peered":"2024-05-10T03:34:16.817395+0000","last_clean":"2024-05-10T03:34:16.817395+0000","last_became_active":"2024-05-10T03:34:15.508457+0000","last_became_peered":"2024-05-10T03:34:15.508457+0000","last_unstale":"2024-05-10T03:34:16.817395+0000","last_undegraded":"2024-05-10T03:34:16.817395+0000","last_fullsized":"2024-05-10T03:34:16.817395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:41.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815704+0000","last_change":"2024-05-10T03:34:14.949941+0000","last_active":"2024-05-10T03:34:16.815704+0000","last_peered":"2024-05-10T03:34:16.815704+0000","last_clean":"2024-05-10T03:34:16.815704+0000","last_became_active":"2024-05-10T03:34:14.949657+0000","last_became_peered":"2024-05-10T03:34:14.949657+0000","last_unstale":"2024-05-10T03:34:16.815704+0000","last_undegraded":"2024-05-10T03:34:16.815704+0000","last_fullsized":"2024-05-10T03:34:16.815704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:14.367480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816406+0000","last_change":"2024-05-10T03:34:15.539253+0000","last_active":"2024-05-10T03:34:16.816406+0000","last_peered":"2024-05-10T03:34:16.816406+0000","last_clean":"2024-05-10T03:34:16.816406+0000","last_became_active":"2024-05-10T03:34:15.537177+0000","last_became_peered":"2024-05-10T03:34:15.537177+0000","last_unstale":"2024-05-10T03:34:16.816406+0000","last_undegraded":"2024-05-10T03:34:16.816406+0000","last_fullsized":"2024-05-10T03:34:16.816406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818889+0000","last_change":"2024-05-10T03:34:14.943389+0000","last_active":"2024-05-10T03:34:16.818889+0000","last_peered":"2024-05-10T03:34:16.818889+0000","last_clean":"2024-05-10T03:34:16.818889+0000","last_became_active":"2024-05-10T03:34:14.942985+0000","last_became_peered":"2024-05-10T03:34:14.942985+0000","last_unstale":"2024-05-10T03:34:16.818889+0000","last_undegraded":"2024-05-10T03:34:16.818889+0000","last_fullsized":"2024-05-10T03:34:16.818889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:41.926795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T03:34:15.520527+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:34:15.520527+0000","last_peered":"2024-05-10T03:34:15.520527+0000","last_clean":"2024-05-10T03:34:15.520527+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:34:15.520527+0000","last_undegraded":"2024-05-10T03:34:15.520527+0000","last_fullsized":"2024-05-10T03:34:15.520527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1712128,"data_stored":1091760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.2350000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.651,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.484},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3380000000000001,"15min":1.355},"min":{"1min":0.83099999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.696,"5min":1.877,"15min":1.877},"last":1.752}]},{"osd":1,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.83499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.798,"5min":1.853,"15min":1.853},"last":1.034},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.746,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6990000000000001,"5min":1.925,"15min":1.925},"last":1.5329999999999999}]},{"osd":2,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.3180000000000001,"15min":1.3680000000000001},"min":{"1min":0.89900000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.581,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3720000000000001,"15min":1.4259999999999999},"min":{"1min":0.98199999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3450000000000002,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3600000000000001,"15min":1.3819999999999999},"min":{"1min":0.96299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.8660000000000001,"5min":2.222,"15min":2.222},"last":1.377},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.369,"15min":1.339},"min":{"1min":0.85599999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":1.4339999999999999}]},{"osd":4,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.421,"15min":1.4610000000000001},"min":{"1min":0.83399999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.923,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.518,"15min":1.506},"min":{"1min":0.80400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.978,"5min":2.323,"15min":2.323},"last":1.593}]},{"osd":5,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5309999999999999,"15min":1.5109999999999999},"min":{"1min":0.78400000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4700000000000002,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.6659999999999999,"15min":1.6879999999999999},"min":{"1min":0.88600000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.117,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.8759999999999999}]},{"osd":6,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.698,"15min":1.72},"min":{"1min":1.002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4100000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.681,"5min":1.661,"15min":1.643},"min":{"1min":1.0309999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.298,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.7150000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1130496,"data_stored":503504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2849999999999999,"15min":1.337},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7689999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.282,"15min":1.3080000000000001},"min":{"1min":0.84199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.556,"5min":3.113,"15min":3.113},"last":1.331},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3720000000000001,"15min":1.4630000000000001},"min":{"1min":0.74099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0350000000000001,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.319,"15min":1.2569999999999999},"min":{"1min":0.80400000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7909999999999999,"5min":2.02,"15min":2.02},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.52,"5min":1.4259999999999999,"15min":1.502},"min":{"1min":0.98299999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.202,"5min":3.569,"15min":3.569},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1230000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.367,"15min":1.3240000000000001},"min":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.92000000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.766,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.137},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.552,"15min":1.581},"min":{"1min":0.89100000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0579999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.53}]},{"osd":5,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.4590000000000001,"15min":1.4099999999999999},"min":{"1min":1.0089999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.617,"15min":1.621},"min":{"1min":1.1759999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8799999999999999,"5min":2.472,"15min":2.472},"last":1.246}]},{"osd":7,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.1919999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4590000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.841,"5min":1.663,"15min":1.659},"min":{"1min":1.05,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1110016,"data_stored":501336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0920000000000001,"15min":1.071},"min":{"1min":0.72999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6950000000000001,"5min":1.851,"15min":1.851},"last":1.2969999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.145,"15min":1.141},"min":{"1min":0.72899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7190000000000001,"5min":1.855,"15min":1.855},"last":1.143},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.702,"5min":1.954,"15min":1.954},"last":1.76}]},{"osd":2,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.232,"15min":1.222},"min":{"1min":0.749,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.669,"5min":2.359,"15min":2.359},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.331},"min":{"1min":1.0069999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.6559999999999999}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.361,"15min":1.357},"min":{"1min":0.98799999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7210000000000001,"5min":1.881,"15min":1.881},"last":1.615},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.357,"15min":1.3049999999999999},"min":{"1min":1.0489999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.909,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.573}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9470000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.5920000000000001,"15min":1.603},"min":{"1min":1.246,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1600000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.71}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.534,"15min":1.518},"min":{"1min":1.3480000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0190000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.8},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5600000000000001,"15min":1.5549999999999999},"min":{"1min":1.129,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.869}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.7829999999999999,"15min":1.8069999999999999},"min":{"1min":1.2330000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.3969999999999998,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.915},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.6459999999999999,"15min":1.6220000000000001},"min":{"1min":0.96599999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.4299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":494876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.169},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5940000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1850000000000001,"15min":1.163},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.24,"15min":1.27},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.75,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.03},{"interface":"front","average":{"1min":1.335,"5min":1.3540000000000001,"15min":1.3919999999999999},"min":{"1min":0.80700000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.647,"5min":2.29,"15min":2.29},"last":1.764}]},{"osd":3,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3899999999999999,"15min":1.454},"min":{"1min":0.85199999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8180000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5549999999999999,"15min":1.577},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0800000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.546}]},{"osd":5,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4850000000000001,"15min":1.5},"min":{"1min":0.93000000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.877,"5min":2.512,"15min":2.512},"last":1.504},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4730000000000001,"15min":1.5409999999999999},"min":{"1min":0.70399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.4129999999999998,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.585}]},{"osd":6,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5589999999999999,"15min":1.583},"min":{"1min":0.75900000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1859999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6910000000000001,"15min":1.766},"min":{"1min":0.879,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.069,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.657}]},{"osd":7,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.6379999999999999,"15min":1.6850000000000001},"min":{"1min":0.78400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.5099999999999998,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.7330000000000001,"15min":1.742},"min":{"1min":0.82899999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3130000000000002,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.6240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1110016,"data_stored":501336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.395,"5min":1.754,"15min":1.754},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94399999999999995,"15min":0.97199999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6200000000000001,"5min":1.708,"15min":1.708},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3069999999999999,"5min":1.603,"15min":1.603},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.179,"5min":1.2110000000000001,"15min":1.2450000000000001},"min":{"1min":0.50700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":1.964,"15min":1.964},"last":1.5429999999999999}]},{"osd":2,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1699999999999999,"15min":1.1819999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.278,"15min":1.3480000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7130000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.621}]},{"osd":4,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.355,"15min":1.371},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9710000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4179999999999999,"15min":1.3879999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9650000000000001,"5min":2.464,"15min":2.464},"last":1.379}]},{"osd":5,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.583,"15min":1.6639999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.4399999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.524,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.298}]},{"osd":6,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5549999999999999,"15min":1.573},"min":{"1min":0.90300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.1480000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.579},{"interface":"front","average":{"1min":1.73,"5min":1.7290000000000001,"15min":1.7889999999999999},"min":{"1min":0.92300000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4329999999999998,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.7470000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.542,"15min":1.583},"min":{"1min":0.97699999999999998,"5min":0.87,"15min":0.87},"max":{"1min":2.2229999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.607,"15min":1.647},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0089999999999999,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.647}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1097728,"data_stored":494716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93400000000000005,"15min":0.98299999999999998},"min":{"1min":0.504,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.389,"5min":1.772,"15min":1.772},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.97899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.635,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.349}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.56399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.425,"5min":1.746,"15min":1.746},"last":1.28},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6879999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.085,"15min":1.117},"min":{"1min":0.63600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.526,"5min":2.169,"15min":2.169},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.21,"15min":1.25},"min":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8520000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.714}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.5269999999999999},"min":{"1min":0.92600000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.024,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8600000000000001,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":2.0489999999999999}]},{"osd":5,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.464,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.173,"5min":2.508,"15min":2.508},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5900000000000001,"15min":1.617},"min":{"1min":0.97999999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.0859999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.454}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.512,"15min":1.5449999999999999},"min":{"1min":0.90700000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1339999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":2.1989999999999998},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5489999999999999,"15min":1.587},"min":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.331,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.607,"15min":1.649},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2749999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":2.3149999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.619,"15min":1.665},"min":{"1min":0.95599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.266,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.673}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1101824,"data_stored":494786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.407,"5min":1.607,"15min":1.607},"last":0.626},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0069999999999999,"15min":1.0720000000000001},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.583,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.54100000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.748,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.996},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1819999999999999},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.0720000000000001,"5min":2.444,"15min":2.444},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.256,"15min":1.3129999999999999},"min":{"1min":0.59099999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6799999999999999,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.677},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2350000000000001,"15min":1.288},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.772,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4650000000000001,"15min":1.5109999999999999},"min":{"1min":0.95099999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.2650000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.5089999999999999,"15min":1.577},"min":{"1min":0.92500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1139999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.347}]},{"osd":5,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4890000000000001,"15min":1.575},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0539999999999998,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.6200000000000001},"min":{"1min":1.004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0470000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.601}]},{"osd":6,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.601,"15min":1.704},"min":{"1min":0.96499999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.347,"5min":3.48,"15min":3.48},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.6180000000000001,"15min":1.716},"min":{"1min":1.054,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.1230000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.6539999999999999}]},{"osd":7,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.665,"15min":1.7270000000000001},"min":{"1min":1.089,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.2709999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.79},{"interface":"front","average":{"1min":1.581,"5min":1.6739999999999999,"15min":1.7669999999999999},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.52,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.206}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1728512,"data_stored":1093974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92800000000000005,"15min":1.012},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.222,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.046,"15min":1.173},"min":{"1min":0.69699999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6120000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.21},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4910000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.1990000000000001},"min":{"1min":0.57899999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.627,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0309999999999999,"15min":1.095},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.7050000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.877},{"interface":"front","average":{"1min":1.28,"5min":1.264,"15min":1.319},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8240000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.464,"15min":1.6080000000000001},"min":{"1min":0.83699999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.7769999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.383,"15min":1.4299999999999999},"min":{"1min":0.84099999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.778,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.177}]},{"osd":5,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.405,"15min":1.4670000000000001},"min":{"1min":0.99399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9239999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.552},{"interface":"front","average":{"1min":1.506,"5min":1.5169999999999999,"15min":1.617},"min":{"1min":0.90900000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8109999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.6120000000000001}]},{"osd":6,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5700000000000001,"15min":1.6339999999999999},"min":{"1min":1.0960000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.966,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.5960000000000001,"15min":1.7230000000000001},"min":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.129,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.323}]},{"osd":7,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.6240000000000001,"15min":1.7709999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.972,"5min":3.149,"15min":3.149},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.536,"15min":1.573},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.0600000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:34:20.697 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:34:20.876 INFO:tasks.ceph.mds.c.smithi055.stderr:2024-05-10T03:34:20.872+0000 7fbfa8a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:34:21.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.018+0000 7f31fcc00640 1 Processor -- start 2024-05-10T03:34:21.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.018+0000 7f31fcc00640 1 -- start start 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.020+0000 7f31fcc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31f8071db0 0x7f31f8158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.020+0000 7f31fcc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 0x7f31f80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.020+0000 7f31fcc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f80708b0 con 0x7f31f80b3860 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.020+0000 7f31fcc00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f80b2490 con 0x7f31f8071db0 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31fcc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f8070050 con 0x7f31f806a4b0 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31f5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31f8071db0 0x7f31f8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31f6800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f31f80b3860 0x7f31f80b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52824/0 (socket says 172.21.15.55:52824) 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31f6800640 1 -- 172.21.15.55:0/3703462802 learned_addr learned my addr 172.21.15.55:0/3703462802 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:21.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31f5400640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 0x7f31f80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.021+0000 7f31f4a00640 1 -- 172.21.15.55:0/3703462802 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623970476 0 0) 0x7f31f80708b0 con 0x7f31f80b3860 2024-05-10T03:34:21.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f4a00640 1 -- 172.21.15.55:0/3703462802 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31dc005730 con 0x7f31f80b3860 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f5e00640 1 -- 172.21.15.55:0/3703462802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 msgr2=0x7f31f80ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f5e00640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 0x7f31f80ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f5e00640 1 -- 172.21.15.55:0/3703462802 >> v1:172.21.15.55:6789/0 conn(0x7f31f80b3860 legacy=0x7f31f80b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f5e00640 1 -- 172.21.15.55:0/3703462802 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f815b940 con 0x7f31f8071db0 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.022+0000 7f31f5400640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 0x7f31f80ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:21.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.023+0000 7f31f5e00640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f8071db0 0x7f31f8158150 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009d90 tx=0x7f31e802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965dece8076398d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.023+0000 7f31f4a00640 1 -- 172.21.15.55:0/3703462802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8031600 con 0x7f31f8071db0 2024-05-10T03:34:21.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.023+0000 7f31f4a00640 1 -- 172.21.15.55:0/3703462802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e802d050 con 0x7f31f8071db0 2024-05-10T03:34:21.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.023+0000 7f31f4a00640 1 -- 172.21.15.55:0/3703462802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8031980 con 0x7f31f8071db0 2024-05-10T03:34:21.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 -- 172.21.15.55:0/3703462802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f8071db0 msgr2=0x7f31f8158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f8071db0 0x7f31f8158150 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009d90 tx=0x7f31e802a5d0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 -- 172.21.15.55:0/3703462802 shutdown_connections 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f806a4b0 0x7f31f80ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3703462802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f8071db0 0x7f31f8158150 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 -- 172.21.15.55:0/3703462802 >> 172.21.15.55:0/3703462802 conn(0x7f31f80cbb90 msgr2=0x7f31f80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 -- 172.21.15.55:0/3703462802 shutdown_connections 2024-05-10T03:34:21.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.024+0000 7f31fcc00640 1 -- 172.21.15.55:0/3703462802 wait complete. 2024-05-10T03:34:21.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.028+0000 7f31fcc00640 1 Processor -- start 2024-05-10T03:34:21.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.028+0000 7f31fcc00640 1 -- start start 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31f806a4b0 0x7f31f81ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f8071db0 0x7f31f80c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80ad9b0 con 0x7f31f80b3860 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80c4ab0 con 0x7f31f806a4b0 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31fcc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80c4840 con 0x7f31f8071db0 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.030+0000 7f31f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58320/0 (socket says 172.21.15.55:58320) 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 -- 172.21.15.55:0/3169072942 learned_addr learned my addr 172.21.15.55:0/3169072942 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:21.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f5e00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31f806a4b0 0x7f31f81ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f5400640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f8071db0 0x7f31f80c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 -- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f8071db0 msgr2=0x7f31f80c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f8071db0 0x7f31f80c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 -- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31f806a4b0 msgr2=0x7f31f81ed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31f806a4b0 0x7f31f81ed540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 -- 172.21.15.55:0/3169072942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8009a70 con 0x7f31f80b3860 2024-05-10T03:34:21.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.031+0000 7f31f6800640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f31f0001eb0 tx=0x7f31f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.032+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0005ce0 con 0x7f31f80b3860 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.032+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0005e40 con 0x7f31f80b3860 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.032+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f00072d0 con 0x7f31f80b3860 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.032+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f80c4d00 con 0x7f31f80b3860 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.032+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f80c52c0 con 0x7f31f80b3860 2024-05-10T03:34:21.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.033+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f8066f60 con 0x7f31f80b3860 2024-05-10T03:34:21.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.034+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f31f0007430 con 0x7f31f80b3860 2024-05-10T03:34:21.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.035+0000 7f31ee000640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 0x7f31d806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.036+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f31f0088960 con 0x7f31f80b3860 2024-05-10T03:34:21.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.036+0000 7f31f5e00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 0x7f31d806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.037+0000 7f31f5e00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 0x7f31d806d140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31e8034000 tx=0x7f31e8033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.040+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f31f0053a10 con 0x7f31f80b3860 2024-05-10T03:34:21.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.236+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31f815b5d0 con 0x7f31d806ac30 2024-05-10T03:34:21.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.260+0000 7f31ee000640 1 -- 172.21.15.55:0/3169072942 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286873 (secure 0 0 0) 0x7f31e8031720 con 0x7f31d806ac30 2024-05-10T03:34:21.265 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:21.270 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:34:21.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 msgr2=0x7f31d806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 0x7f31d806d140 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31e8034000 tx=0x7f31e8033040 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 msgr2=0x7f31f81f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f31f0001eb0 tx=0x7f31f000c820 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 shutdown_connections 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31f80b3860 0x7f31f81f6cf0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31f8071db0 0x7f31f80c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f31d806ac30 0x7f31d806d140 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 --2- 172.21.15.55:0/3169072942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31f806a4b0 0x7f31f81ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.267+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 >> 172.21.15.55:0/3169072942 conn(0x7f31f80cbb90 msgr2=0x7f31f806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.268+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 shutdown_connections 2024-05-10T03:34:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.268+0000 7f31fcc00640 1 -- 172.21.15.55:0/3169072942 wait complete. 2024-05-10T03:34:21.302 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-05-10T03:34:21.057383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258812,"kb_used_data":9860,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715908,"statfs":{"total":773094113280,"available":772829089792,"internally_reserved":0,"allocated":10096640,"data_stored":5176288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.150323"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513889+0000","last_change":"2024-05-10T03:34:15.528797+0000","last_active":"2024-05-10T03:34:16.513889+0000","last_peered":"2024-05-10T03:34:16.513889+0000","last_clean":"2024-05-10T03:34:16.513889+0000","last_became_active":"2024-05-10T03:34:15.528428+0000","last_became_peered":"2024-05-10T03:34:15.528428+0000","last_unstale":"2024-05-10T03:34:16.513889+0000","last_undegraded":"2024-05-10T03:34:16.513889+0000","last_fullsized":"2024-05-10T03:34:16.513889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:29.222932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516928+0000","last_change":"2024-05-10T03:34:15.511392+0000","last_active":"2024-05-10T03:34:16.516928+0000","last_peered":"2024-05-10T03:34:16.516928+0000","last_clean":"2024-05-10T03:34:16.516928+0000","last_became_active":"2024-05-10T03:34:15.510472+0000","last_became_peered":"2024-05-10T03:34:15.510472+0000","last_unstale":"2024-05-10T03:34:16.516928+0000","last_undegraded":"2024-05-10T03:34:16.516928+0000","last_fullsized":"2024-05-10T03:34:16.516928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:06.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513542+0000","last_change":"2024-05-10T03:34:15.535756+0000","last_active":"2024-05-10T03:34:16.513542+0000","last_peered":"2024-05-10T03:34:16.513542+0000","last_clean":"2024-05-10T03:34:16.513542+0000","last_became_active":"2024-05-10T03:34:15.535449+0000","last_became_peered":"2024-05-10T03:34:15.535449+0000","last_unstale":"2024-05-10T03:34:16.513542+0000","last_undegraded":"2024-05-10T03:34:16.513542+0000","last_fullsized":"2024-05-10T03:34:16.513542+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:34.133109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818517+0000","last_change":"2024-05-10T03:34:15.537352+0000","last_active":"2024-05-10T03:34:16.818517+0000","last_peered":"2024-05-10T03:34:16.818517+0000","last_clean":"2024-05-10T03:34:16.818517+0000","last_became_active":"2024-05-10T03:34:15.536804+0000","last_became_peered":"2024-05-10T03:34:15.536804+0000","last_unstale":"2024-05-10T03:34:16.818517+0000","last_undegraded":"2024-05-10T03:34:16.818517+0000","last_fullsized":"2024-05-10T03:34:16.818517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:30.223450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516680+0000","last_change":"2024-05-10T03:34:15.536521+0000","last_active":"2024-05-10T03:34:16.516680+0000","last_peered":"2024-05-10T03:34:16.516680+0000","last_clean":"2024-05-10T03:34:16.516680+0000","last_became_active":"2024-05-10T03:34:15.535865+0000","last_became_peered":"2024-05-10T03:34:15.535865+0000","last_unstale":"2024-05-10T03:34:16.516680+0000","last_undegraded":"2024-05-10T03:34:16.516680+0000","last_fullsized":"2024-05-10T03:34:16.516680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:18:15.083196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815557+0000","last_change":"2024-05-10T03:34:15.524624+0000","last_active":"2024-05-10T03:34:16.815557+0000","last_peered":"2024-05-10T03:34:16.815557+0000","last_clean":"2024-05-10T03:34:16.815557+0000","last_became_active":"2024-05-10T03:34:15.524200+0000","last_became_peered":"2024-05-10T03:34:15.524200+0000","last_unstale":"2024-05-10T03:34:16.815557+0000","last_undegraded":"2024-05-10T03:34:16.815557+0000","last_fullsized":"2024-05-10T03:34:16.815557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:34.133109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815619+0000","last_change":"2024-05-10T03:34:15.539051+0000","last_active":"2024-05-10T03:34:16.815619+0000","last_peered":"2024-05-10T03:34:16.815619+0000","last_clean":"2024-05-10T03:34:16.815619+0000","last_became_active":"2024-05-10T03:34:15.538774+0000","last_became_peered":"2024-05-10T03:34:15.538774+0000","last_unstale":"2024-05-10T03:34:16.815619+0000","last_undegraded":"2024-05-10T03:34:16.815619+0000","last_fullsized":"2024-05-10T03:34:16.815619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816057+0000","last_change":"2024-05-10T03:34:15.540854+0000","last_active":"2024-05-10T03:34:16.816057+0000","last_peered":"2024-05-10T03:34:16.816057+0000","last_clean":"2024-05-10T03:34:16.816057+0000","last_became_active":"2024-05-10T03:34:15.539780+0000","last_became_peered":"2024-05-10T03:34:15.539780+0000","last_unstale":"2024-05-10T03:34:16.816057+0000","last_undegraded":"2024-05-10T03:34:16.816057+0000","last_fullsized":"2024-05-10T03:34:16.816057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517846+0000","last_change":"2024-05-10T03:34:15.526743+0000","last_active":"2024-05-10T03:34:16.517846+0000","last_peered":"2024-05-10T03:34:16.517846+0000","last_clean":"2024-05-10T03:34:16.517846+0000","last_became_active":"2024-05-10T03:34:15.526262+0000","last_became_peered":"2024-05-10T03:34:15.526262+0000","last_unstale":"2024-05-10T03:34:16.517846+0000","last_undegraded":"2024-05-10T03:34:16.517846+0000","last_fullsized":"2024-05-10T03:34:16.517846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816003+0000","last_change":"2024-05-10T03:34:15.525674+0000","last_active":"2024-05-10T03:34:16.816003+0000","last_peered":"2024-05-10T03:34:16.816003+0000","last_clean":"2024-05-10T03:34:16.816003+0000","last_became_active":"2024-05-10T03:34:15.525361+0000","last_became_peered":"2024-05-10T03:34:15.525361+0000","last_unstale":"2024-05-10T03:34:16.816003+0000","last_undegraded":"2024-05-10T03:34:16.816003+0000","last_fullsized":"2024-05-10T03:34:16.816003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:27:05.431760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517276+0000","last_change":"2024-05-10T03:34:15.528030+0000","last_active":"2024-05-10T03:34:16.517276+0000","last_peered":"2024-05-10T03:34:16.517276+0000","last_clean":"2024-05-10T03:34:16.517276+0000","last_became_active":"2024-05-10T03:34:15.527476+0000","last_became_peered":"2024-05-10T03:34:15.527476+0000","last_unstale":"2024-05-10T03:34:16.517276+0000","last_undegraded":"2024-05-10T03:34:16.517276+0000","last_fullsized":"2024-05-10T03:34:16.517276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815895+0000","last_change":"2024-05-10T03:34:15.515921+0000","last_active":"2024-05-10T03:34:16.815895+0000","last_peered":"2024-05-10T03:34:16.815895+0000","last_clean":"2024-05-10T03:34:16.815895+0000","last_became_active":"2024-05-10T03:34:15.515621+0000","last_became_peered":"2024-05-10T03:34:15.515621+0000","last_unstale":"2024-05-10T03:34:16.815895+0000","last_undegraded":"2024-05-10T03:34:16.815895+0000","last_fullsized":"2024-05-10T03:34:16.815895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:55:37.543793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817642+0000","last_change":"2024-05-10T03:34:15.538984+0000","last_active":"2024-05-10T03:34:16.817642+0000","last_peered":"2024-05-10T03:34:16.817642+0000","last_clean":"2024-05-10T03:34:16.817642+0000","last_became_active":"2024-05-10T03:34:15.538647+0000","last_became_peered":"2024-05-10T03:34:15.538647+0000","last_unstale":"2024-05-10T03:34:16.817642+0000","last_undegraded":"2024-05-10T03:34:16.817642+0000","last_fullsized":"2024-05-10T03:34:16.817642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515432+0000","last_change":"2024-05-10T03:34:15.525465+0000","last_active":"2024-05-10T03:34:16.515432+0000","last_peered":"2024-05-10T03:34:16.515432+0000","last_clean":"2024-05-10T03:34:16.515432+0000","last_became_active":"2024-05-10T03:34:15.525121+0000","last_became_peered":"2024-05-10T03:34:15.525121+0000","last_unstale":"2024-05-10T03:34:16.515432+0000","last_undegraded":"2024-05-10T03:34:16.515432+0000","last_fullsized":"2024-05-10T03:34:16.515432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:06.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817147+0000","last_change":"2024-05-10T03:34:15.520850+0000","last_active":"2024-05-10T03:34:16.817147+0000","last_peered":"2024-05-10T03:34:16.817147+0000","last_clean":"2024-05-10T03:34:16.817147+0000","last_became_active":"2024-05-10T03:34:15.520325+0000","last_became_peered":"2024-05-10T03:34:15.520325+0000","last_unstale":"2024-05-10T03:34:16.817147+0000","last_undegraded":"2024-05-10T03:34:16.817147+0000","last_fullsized":"2024-05-10T03:34:16.817147+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:11:39.095670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815071+0000","last_change":"2024-05-10T03:34:15.511944+0000","last_active":"2024-05-10T03:34:16.815071+0000","last_peered":"2024-05-10T03:34:16.815071+0000","last_clean":"2024-05-10T03:34:16.815071+0000","last_became_active":"2024-05-10T03:34:15.511636+0000","last_became_peered":"2024-05-10T03:34:15.511636+0000","last_unstale":"2024-05-10T03:34:16.815071+0000","last_undegraded":"2024-05-10T03:34:16.815071+0000","last_fullsized":"2024-05-10T03:34:16.815071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:17.560672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817183+0000","last_change":"2024-05-10T03:34:15.520885+0000","last_active":"2024-05-10T03:34:16.817183+0000","last_peered":"2024-05-10T03:34:16.817183+0000","last_clean":"2024-05-10T03:34:16.817183+0000","last_became_active":"2024-05-10T03:34:15.520621+0000","last_became_peered":"2024-05-10T03:34:15.520621+0000","last_unstale":"2024-05-10T03:34:16.817183+0000","last_undegraded":"2024-05-10T03:34:16.817183+0000","last_fullsized":"2024-05-10T03:34:16.817183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:02.412699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513638+0000","last_change":"2024-05-10T03:34:15.518504+0000","last_active":"2024-05-10T03:34:16.513638+0000","last_peered":"2024-05-10T03:34:16.513638+0000","last_clean":"2024-05-10T03:34:16.513638+0000","last_became_active":"2024-05-10T03:34:15.516749+0000","last_became_peered":"2024-05-10T03:34:15.516749+0000","last_unstale":"2024-05-10T03:34:16.513638+0000","last_undegraded":"2024-05-10T03:34:16.513638+0000","last_fullsized":"2024-05-10T03:34:16.513638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:06:19.298176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517431+0000","last_change":"2024-05-10T03:34:15.525319+0000","last_active":"2024-05-10T03:34:16.517431+0000","last_peered":"2024-05-10T03:34:16.517431+0000","last_clean":"2024-05-10T03:34:16.517431+0000","last_became_active":"2024-05-10T03:34:15.524851+0000","last_became_peered":"2024-05-10T03:34:15.524851+0000","last_unstale":"2024-05-10T03:34:16.517431+0000","last_undegraded":"2024-05-10T03:34:16.517431+0000","last_fullsized":"2024-05-10T03:34:16.517431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818778+0000","last_change":"2024-05-10T03:34:15.525829+0000","last_active":"2024-05-10T03:34:16.818778+0000","last_peered":"2024-05-10T03:34:16.818778+0000","last_clean":"2024-05-10T03:34:16.818778+0000","last_became_active":"2024-05-10T03:34:15.521290+0000","last_became_peered":"2024-05-10T03:34:15.521290+0000","last_unstale":"2024-05-10T03:34:16.818778+0000","last_undegraded":"2024-05-10T03:34:16.818778+0000","last_fullsized":"2024-05-10T03:34:16.818778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:31.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515966+0000","last_change":"2024-05-10T03:34:15.538836+0000","last_active":"2024-05-10T03:34:16.515966+0000","last_peered":"2024-05-10T03:34:16.515966+0000","last_clean":"2024-05-10T03:34:16.515966+0000","last_became_active":"2024-05-10T03:34:15.536144+0000","last_became_peered":"2024-05-10T03:34:15.536144+0000","last_unstale":"2024-05-10T03:34:16.515966+0000","last_undegraded":"2024-05-10T03:34:16.515966+0000","last_fullsized":"2024-05-10T03:34:16.515966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:21:00.783781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515759+0000","last_change":"2024-05-10T03:34:15.524619+0000","last_active":"2024-05-10T03:34:16.515759+0000","last_peered":"2024-05-10T03:34:16.515759+0000","last_clean":"2024-05-10T03:34:16.515759+0000","last_became_active":"2024-05-10T03:34:15.522451+0000","last_became_peered":"2024-05-10T03:34:15.522451+0000","last_unstale":"2024-05-10T03:34:16.515759+0000","last_undegraded":"2024-05-10T03:34:16.515759+0000","last_fullsized":"2024-05-10T03:34:16.515759+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:51:49.117770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814544+0000","last_change":"2024-05-10T03:34:15.543575+0000","last_active":"2024-05-10T03:34:16.814544+0000","last_peered":"2024-05-10T03:34:16.814544+0000","last_clean":"2024-05-10T03:34:16.814544+0000","last_became_active":"2024-05-10T03:34:15.542382+0000","last_became_peered":"2024-05-10T03:34:15.542382+0000","last_unstale":"2024-05-10T03:34:16.814544+0000","last_undegraded":"2024-05-10T03:34:16.814544+0000","last_fullsized":"2024-05-10T03:34:16.814544+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:55.158777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.512832+0000","last_change":"2024-05-10T03:34:15.533288+0000","last_active":"2024-05-10T03:34:16.512832+0000","last_peered":"2024-05-10T03:34:16.512832+0000","last_clean":"2024-05-10T03:34:16.512832+0000","last_became_active":"2024-05-10T03:34:15.532987+0000","last_became_peered":"2024-05-10T03:34:15.532987+0000","last_unstale":"2024-05-10T03:34:16.512832+0000","last_undegraded":"2024-05-10T03:34:16.512832+0000","last_fullsized":"2024-05-10T03:34:16.512832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:16.495944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517937+0000","last_change":"2024-05-10T03:34:15.540823+0000","last_active":"2024-05-10T03:34:16.517937+0000","last_peered":"2024-05-10T03:34:16.517937+0000","last_clean":"2024-05-10T03:34:16.517937+0000","last_became_active":"2024-05-10T03:34:15.540476+0000","last_became_peered":"2024-05-10T03:34:15.540476+0000","last_unstale":"2024-05-10T03:34:16.517937+0000","last_undegraded":"2024-05-10T03:34:16.517937+0000","last_fullsized":"2024-05-10T03:34:16.517937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517091+0000","last_change":"2024-05-10T03:34:15.525170+0000","last_active":"2024-05-10T03:34:16.517091+0000","last_peered":"2024-05-10T03:34:16.517091+0000","last_clean":"2024-05-10T03:34:16.517091+0000","last_became_active":"2024-05-10T03:34:15.524797+0000","last_became_peered":"2024-05-10T03:34:15.524797+0000","last_unstale":"2024-05-10T03:34:16.517091+0000","last_undegraded":"2024-05-10T03:34:16.517091+0000","last_fullsized":"2024-05-10T03:34:16.517091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815455+0000","last_change":"2024-05-10T03:34:15.535727+0000","last_active":"2024-05-10T03:34:16.815455+0000","last_peered":"2024-05-10T03:34:16.815455+0000","last_clean":"2024-05-10T03:34:16.815455+0000","last_became_active":"2024-05-10T03:34:15.535387+0000","last_became_peered":"2024-05-10T03:34:15.535387+0000","last_unstale":"2024-05-10T03:34:16.815455+0000","last_undegraded":"2024-05-10T03:34:16.815455+0000","last_fullsized":"2024-05-10T03:34:16.815455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:17:21.537793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818835+0000","last_change":"2024-05-10T03:34:15.535799+0000","last_active":"2024-05-10T03:34:16.818835+0000","last_peered":"2024-05-10T03:34:16.818835+0000","last_clean":"2024-05-10T03:34:16.818835+0000","last_became_active":"2024-05-10T03:34:15.535470+0000","last_became_peered":"2024-05-10T03:34:15.535470+0000","last_unstale":"2024-05-10T03:34:16.818835+0000","last_undegraded":"2024-05-10T03:34:16.818835+0000","last_fullsized":"2024-05-10T03:34:16.818835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:38.415468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816112+0000","last_change":"2024-05-10T03:34:15.537333+0000","last_active":"2024-05-10T03:34:16.816112+0000","last_peered":"2024-05-10T03:34:16.816112+0000","last_clean":"2024-05-10T03:34:16.816112+0000","last_became_active":"2024-05-10T03:34:15.536963+0000","last_became_peered":"2024-05-10T03:34:15.536963+0000","last_unstale":"2024-05-10T03:34:16.816112+0000","last_undegraded":"2024-05-10T03:34:16.816112+0000","last_fullsized":"2024-05-10T03:34:16.816112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516051+0000","last_change":"2024-05-10T03:34:15.529645+0000","last_active":"2024-05-10T03:34:16.516051+0000","last_peered":"2024-05-10T03:34:16.516051+0000","last_clean":"2024-05-10T03:34:16.516051+0000","last_became_active":"2024-05-10T03:34:15.529093+0000","last_became_peered":"2024-05-10T03:34:15.529093+0000","last_unstale":"2024-05-10T03:34:16.516051+0000","last_undegraded":"2024-05-10T03:34:16.516051+0000","last_fullsized":"2024-05-10T03:34:16.516051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515005+0000","last_change":"2024-05-10T03:34:15.524199+0000","last_active":"2024-05-10T03:34:16.515005+0000","last_peered":"2024-05-10T03:34:16.515005+0000","last_clean":"2024-05-10T03:34:16.515005+0000","last_became_active":"2024-05-10T03:34:15.523749+0000","last_became_peered":"2024-05-10T03:34:15.523749+0000","last_unstale":"2024-05-10T03:34:16.515005+0000","last_undegraded":"2024-05-10T03:34:16.515005+0000","last_fullsized":"2024-05-10T03:34:16.515005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814806+0000","last_change":"2024-05-10T03:34:15.509139+0000","last_active":"2024-05-10T03:34:16.814806+0000","last_peered":"2024-05-10T03:34:16.814806+0000","last_clean":"2024-05-10T03:34:16.814806+0000","last_became_active":"2024-05-10T03:34:15.508477+0000","last_became_peered":"2024-05-10T03:34:15.508477+0000","last_unstale":"2024-05-10T03:34:16.814806+0000","last_undegraded":"2024-05-10T03:34:16.814806+0000","last_fullsized":"2024-05-10T03:34:16.814806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:36:19.788399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816164+0000","last_change":"2024-05-10T03:34:15.539401+0000","last_active":"2024-05-10T03:34:16.816164+0000","last_peered":"2024-05-10T03:34:16.816164+0000","last_clean":"2024-05-10T03:34:16.816164+0000","last_became_active":"2024-05-10T03:34:15.538963+0000","last_became_peered":"2024-05-10T03:34:15.538963+0000","last_unstale":"2024-05-10T03:34:16.816164+0000","last_undegraded":"2024-05-10T03:34:16.816164+0000","last_fullsized":"2024-05-10T03:34:16.816164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:49:02.226571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815238+0000","last_change":"2024-05-10T03:34:14.950337+0000","last_active":"2024-05-10T03:34:16.815238+0000","last_peered":"2024-05-10T03:34:16.815238+0000","last_clean":"2024-05-10T03:34:16.815238+0000","last_became_active":"2024-05-10T03:34:14.949683+0000","last_became_peered":"2024-05-10T03:34:14.949683+0000","last_unstale":"2024-05-10T03:34:16.815238+0000","last_undegraded":"2024-05-10T03:34:16.815238+0000","last_fullsized":"2024-05-10T03:34:16.815238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:14:36.397376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816640+0000","last_change":"2024-05-10T03:34:15.521234+0000","last_active":"2024-05-10T03:34:16.816640+0000","last_peered":"2024-05-10T03:34:16.816640+0000","last_clean":"2024-05-10T03:34:16.816640+0000","last_became_active":"2024-05-10T03:34:15.517964+0000","last_became_peered":"2024-05-10T03:34:15.517964+0000","last_unstale":"2024-05-10T03:34:16.816640+0000","last_undegraded":"2024-05-10T03:34:16.816640+0000","last_fullsized":"2024-05-10T03:34:16.816640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:08.850179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517397+0000","last_change":"2024-05-10T03:34:14.942647+0000","last_active":"2024-05-10T03:34:16.517397+0000","last_peered":"2024-05-10T03:34:16.517397+0000","last_clean":"2024-05-10T03:34:16.517397+0000","last_became_active":"2024-05-10T03:34:14.942299+0000","last_became_peered":"2024-05-10T03:34:14.942299+0000","last_unstale":"2024-05-10T03:34:16.517397+0000","last_undegraded":"2024-05-10T03:34:16.517397+0000","last_fullsized":"2024-05-10T03:34:16.517397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:55:04.387279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816643+0000","last_change":"2024-05-10T03:34:15.536769+0000","last_active":"2024-05-10T03:34:16.816643+0000","last_peered":"2024-05-10T03:34:16.816643+0000","last_clean":"2024-05-10T03:34:16.816643+0000","last_became_active":"2024-05-10T03:34:15.536232+0000","last_became_peered":"2024-05-10T03:34:15.536232+0000","last_unstale":"2024-05-10T03:34:16.816643+0000","last_undegraded":"2024-05-10T03:34:16.816643+0000","last_fullsized":"2024-05-10T03:34:16.816643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:59.177046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816298+0000","last_change":"2024-05-10T03:34:14.943064+0000","last_active":"2024-05-10T03:34:16.816298+0000","last_peered":"2024-05-10T03:34:16.816298+0000","last_clean":"2024-05-10T03:34:16.816298+0000","last_became_active":"2024-05-10T03:34:14.942420+0000","last_became_peered":"2024-05-10T03:34:14.942420+0000","last_unstale":"2024-05-10T03:34:16.816298+0000","last_undegraded":"2024-05-10T03:34:16.816298+0000","last_fullsized":"2024-05-10T03:34:16.816298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:30.425448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817295+0000","last_change":"2024-05-10T03:34:15.541426+0000","last_active":"2024-05-10T03:34:16.817295+0000","last_peered":"2024-05-10T03:34:16.817295+0000","last_clean":"2024-05-10T03:34:16.817295+0000","last_became_active":"2024-05-10T03:34:15.541111+0000","last_became_peered":"2024-05-10T03:34:15.541111+0000","last_unstale":"2024-05-10T03:34:16.817295+0000","last_undegraded":"2024-05-10T03:34:16.817295+0000","last_fullsized":"2024-05-10T03:34:16.817295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:21:31.673939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815455+0000","last_change":"2024-05-10T03:34:14.947233+0000","last_active":"2024-05-10T03:34:16.815455+0000","last_peered":"2024-05-10T03:34:16.815455+0000","last_clean":"2024-05-10T03:34:16.815455+0000","last_became_active":"2024-05-10T03:34:14.946945+0000","last_became_peered":"2024-05-10T03:34:14.946945+0000","last_unstale":"2024-05-10T03:34:16.815455+0000","last_undegraded":"2024-05-10T03:34:16.815455+0000","last_fullsized":"2024-05-10T03:34:16.815455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:05.308556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816343+0000","last_change":"2024-05-10T03:34:15.518416+0000","last_active":"2024-05-10T03:34:16.816343+0000","last_peered":"2024-05-10T03:34:16.816343+0000","last_clean":"2024-05-10T03:34:16.816343+0000","last_became_active":"2024-05-10T03:34:15.518135+0000","last_became_peered":"2024-05-10T03:34:15.518135+0000","last_unstale":"2024-05-10T03:34:16.816343+0000","last_undegraded":"2024-05-10T03:34:16.816343+0000","last_fullsized":"2024-05-10T03:34:16.816343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:04:17.699300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.516654+0000","last_change":"2024-05-10T03:34:14.935163+0000","last_active":"2024-05-10T03:34:16.516654+0000","last_peered":"2024-05-10T03:34:16.516654+0000","last_clean":"2024-05-10T03:34:16.516654+0000","last_became_active":"2024-05-10T03:34:14.934811+0000","last_became_peered":"2024-05-10T03:34:14.934811+0000","last_unstale":"2024-05-10T03:34:16.516654+0000","last_undegraded":"2024-05-10T03:34:16.516654+0000","last_fullsized":"2024-05-10T03:34:16.516654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:10:14.518870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814643+0000","last_change":"2024-05-10T03:34:15.540784+0000","last_active":"2024-05-10T03:34:16.814643+0000","last_peered":"2024-05-10T03:34:16.814643+0000","last_clean":"2024-05-10T03:34:16.814643+0000","last_became_active":"2024-05-10T03:34:15.540440+0000","last_became_peered":"2024-05-10T03:34:15.540440+0000","last_unstale":"2024-05-10T03:34:16.814643+0000","last_undegraded":"2024-05-10T03:34:16.814643+0000","last_fullsized":"2024-05-10T03:34:16.814643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:31.445239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.819168+0000","last_change":"2024-05-10T03:34:14.946744+0000","last_active":"2024-05-10T03:34:16.819168+0000","last_peered":"2024-05-10T03:34:16.819168+0000","last_clean":"2024-05-10T03:34:16.819168+0000","last_became_active":"2024-05-10T03:34:14.946389+0000","last_became_peered":"2024-05-10T03:34:14.946389+0000","last_unstale":"2024-05-10T03:34:16.819168+0000","last_undegraded":"2024-05-10T03:34:16.819168+0000","last_fullsized":"2024-05-10T03:34:16.819168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:38.426482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815674+0000","last_change":"2024-05-10T03:34:15.524424+0000","last_active":"2024-05-10T03:34:16.815674+0000","last_peered":"2024-05-10T03:34:16.815674+0000","last_clean":"2024-05-10T03:34:16.815674+0000","last_became_active":"2024-05-10T03:34:15.523863+0000","last_became_peered":"2024-05-10T03:34:15.523863+0000","last_unstale":"2024-05-10T03:34:16.815674+0000","last_undegraded":"2024-05-10T03:34:16.815674+0000","last_fullsized":"2024-05-10T03:34:16.815674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:36:16.495944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513161+0000","last_change":"2024-05-10T03:34:14.939189+0000","last_active":"2024-05-10T03:34:16.513161+0000","last_peered":"2024-05-10T03:34:16.513161+0000","last_clean":"2024-05-10T03:34:16.513161+0000","last_became_active":"2024-05-10T03:34:14.938479+0000","last_became_peered":"2024-05-10T03:34:14.938479+0000","last_unstale":"2024-05-10T03:34:16.513161+0000","last_undegraded":"2024-05-10T03:34:16.513161+0000","last_fullsized":"2024-05-10T03:34:16.513161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:10:58.157256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817927+0000","last_change":"2024-05-10T03:34:15.542682+0000","last_active":"2024-05-10T03:34:16.817927+0000","last_peered":"2024-05-10T03:34:16.817927+0000","last_clean":"2024-05-10T03:34:16.817927+0000","last_became_active":"2024-05-10T03:34:15.542353+0000","last_became_peered":"2024-05-10T03:34:15.542353+0000","last_unstale":"2024-05-10T03:34:16.817927+0000","last_undegraded":"2024-05-10T03:34:16.817927+0000","last_fullsized":"2024-05-10T03:34:16.817927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:11.640179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814705+0000","last_change":"2024-05-10T03:34:14.948746+0000","last_active":"2024-05-10T03:34:16.814705+0000","last_peered":"2024-05-10T03:34:16.814705+0000","last_clean":"2024-05-10T03:34:16.814705+0000","last_became_active":"2024-05-10T03:34:14.948370+0000","last_became_peered":"2024-05-10T03:34:14.948370+0000","last_unstale":"2024-05-10T03:34:16.814705+0000","last_undegraded":"2024-05-10T03:34:16.814705+0000","last_fullsized":"2024-05-10T03:34:16.814705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:01.001923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517123+0000","last_change":"2024-05-10T03:34:15.529851+0000","last_active":"2024-05-10T03:34:16.517123+0000","last_peered":"2024-05-10T03:34:16.517123+0000","last_clean":"2024-05-10T03:34:16.517123+0000","last_became_active":"2024-05-10T03:34:15.529453+0000","last_became_peered":"2024-05-10T03:34:15.529453+0000","last_unstale":"2024-05-10T03:34:16.517123+0000","last_undegraded":"2024-05-10T03:34:16.517123+0000","last_fullsized":"2024-05-10T03:34:16.517123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:34.133376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.518090+0000","last_change":"2024-05-10T03:34:14.923811+0000","last_active":"2024-05-10T03:34:16.518090+0000","last_peered":"2024-05-10T03:34:16.518090+0000","last_clean":"2024-05-10T03:34:16.518090+0000","last_became_active":"2024-05-10T03:34:14.923077+0000","last_became_peered":"2024-05-10T03:34:14.923077+0000","last_unstale":"2024-05-10T03:34:16.518090+0000","last_undegraded":"2024-05-10T03:34:16.518090+0000","last_fullsized":"2024-05-10T03:34:16.518090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:04:30.425448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816270+0000","last_change":"2024-05-10T03:34:15.537663+0000","last_active":"2024-05-10T03:34:16.816270+0000","last_peered":"2024-05-10T03:34:16.816270+0000","last_clean":"2024-05-10T03:34:16.816270+0000","last_became_active":"2024-05-10T03:34:15.537333+0000","last_became_peered":"2024-05-10T03:34:15.537333+0000","last_unstale":"2024-05-10T03:34:16.816270+0000","last_undegraded":"2024-05-10T03:34:16.816270+0000","last_fullsized":"2024-05-10T03:34:16.816270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513789+0000","last_change":"2024-05-10T03:34:14.951924+0000","last_active":"2024-05-10T03:34:16.513789+0000","last_peered":"2024-05-10T03:34:16.513789+0000","last_clean":"2024-05-10T03:34:16.513789+0000","last_became_active":"2024-05-10T03:34:14.951529+0000","last_became_peered":"2024-05-10T03:34:14.951529+0000","last_unstale":"2024-05-10T03:34:16.513789+0000","last_undegraded":"2024-05-10T03:34:16.513789+0000","last_fullsized":"2024-05-10T03:34:16.513789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:01:23.726303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513685+0000","last_change":"2024-05-10T03:34:15.517382+0000","last_active":"2024-05-10T03:34:16.513685+0000","last_peered":"2024-05-10T03:34:16.513685+0000","last_clean":"2024-05-10T03:34:16.513685+0000","last_became_active":"2024-05-10T03:34:15.516079+0000","last_became_peered":"2024-05-10T03:34:15.516079+0000","last_unstale":"2024-05-10T03:34:16.513685+0000","last_undegraded":"2024-05-10T03:34:16.513685+0000","last_fullsized":"2024-05-10T03:34:16.513685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:39:30.876200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517699+0000","last_change":"2024-05-10T03:34:14.951098+0000","last_active":"2024-05-10T03:34:16.517699+0000","last_peered":"2024-05-10T03:34:16.517699+0000","last_clean":"2024-05-10T03:34:16.517699+0000","last_became_active":"2024-05-10T03:34:14.950678+0000","last_became_peered":"2024-05-10T03:34:14.950678+0000","last_unstale":"2024-05-10T03:34:16.517699+0000","last_undegraded":"2024-05-10T03:34:16.517699+0000","last_fullsized":"2024-05-10T03:34:16.517699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:05:39.826217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818763+0000","last_change":"2024-05-10T03:34:15.523322+0000","last_active":"2024-05-10T03:34:16.818763+0000","last_peered":"2024-05-10T03:34:16.818763+0000","last_clean":"2024-05-10T03:34:16.818763+0000","last_became_active":"2024-05-10T03:34:15.522986+0000","last_became_peered":"2024-05-10T03:34:15.522986+0000","last_unstale":"2024-05-10T03:34:16.818763+0000","last_undegraded":"2024-05-10T03:34:16.818763+0000","last_fullsized":"2024-05-10T03:34:16.818763+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:15.387271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816400+0000","last_change":"2024-05-10T03:34:14.949482+0000","last_active":"2024-05-10T03:34:16.816400+0000","last_peered":"2024-05-10T03:34:16.816400+0000","last_clean":"2024-05-10T03:34:16.816400+0000","last_became_active":"2024-05-10T03:34:14.949154+0000","last_became_peered":"2024-05-10T03:34:14.949154+0000","last_unstale":"2024-05-10T03:34:16.816400+0000","last_undegraded":"2024-05-10T03:34:16.816400+0000","last_fullsized":"2024-05-10T03:34:16.816400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:05.166784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816659+0000","last_change":"2024-05-10T03:34:15.533682+0000","last_active":"2024-05-10T03:34:16.816659+0000","last_peered":"2024-05-10T03:34:16.816659+0000","last_clean":"2024-05-10T03:34:16.816659+0000","last_became_active":"2024-05-10T03:34:15.533342+0000","last_became_peered":"2024-05-10T03:34:15.533342+0000","last_unstale":"2024-05-10T03:34:16.816659+0000","last_undegraded":"2024-05-10T03:34:16.816659+0000","last_fullsized":"2024-05-10T03:34:16.816659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:55.158777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816276+0000","last_change":"2024-05-10T03:34:14.946042+0000","last_active":"2024-05-10T03:34:16.816276+0000","last_peered":"2024-05-10T03:34:16.816276+0000","last_clean":"2024-05-10T03:34:16.816276+0000","last_became_active":"2024-05-10T03:34:14.945633+0000","last_became_peered":"2024-05-10T03:34:14.945633+0000","last_unstale":"2024-05-10T03:34:16.816276+0000","last_undegraded":"2024-05-10T03:34:16.816276+0000","last_fullsized":"2024-05-10T03:34:16.816276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:29.203659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515851+0000","last_change":"2024-05-10T03:34:15.538963+0000","last_active":"2024-05-10T03:34:16.515851+0000","last_peered":"2024-05-10T03:34:16.515851+0000","last_clean":"2024-05-10T03:34:16.515851+0000","last_became_active":"2024-05-10T03:34:15.536797+0000","last_became_peered":"2024-05-10T03:34:15.536797+0000","last_unstale":"2024-05-10T03:34:16.515851+0000","last_undegraded":"2024-05-10T03:34:16.515851+0000","last_fullsized":"2024-05-10T03:34:16.515851+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:40:34.236092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815776+0000","last_change":"2024-05-10T03:34:14.947524+0000","last_active":"2024-05-10T03:34:16.815776+0000","last_peered":"2024-05-10T03:34:16.815776+0000","last_clean":"2024-05-10T03:34:16.815776+0000","last_became_active":"2024-05-10T03:34:14.947137+0000","last_became_peered":"2024-05-10T03:34:14.947137+0000","last_unstale":"2024-05-10T03:34:16.815776+0000","last_undegraded":"2024-05-10T03:34:16.815776+0000","last_fullsized":"2024-05-10T03:34:16.815776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:12.647164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814632+0000","last_change":"2024-05-10T03:34:15.524948+0000","last_active":"2024-05-10T03:34:16.814632+0000","last_peered":"2024-05-10T03:34:16.814632+0000","last_clean":"2024-05-10T03:34:16.814632+0000","last_became_active":"2024-05-10T03:34:15.523451+0000","last_became_peered":"2024-05-10T03:34:15.523451+0000","last_unstale":"2024-05-10T03:34:16.814632+0000","last_undegraded":"2024-05-10T03:34:16.814632+0000","last_fullsized":"2024-05-10T03:34:16.814632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:05.411698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817646+0000","last_change":"2024-05-10T03:34:14.952816+0000","last_active":"2024-05-10T03:34:16.817646+0000","last_peered":"2024-05-10T03:34:16.817646+0000","last_clean":"2024-05-10T03:34:16.817646+0000","last_became_active":"2024-05-10T03:34:14.951269+0000","last_became_peered":"2024-05-10T03:34:14.951269+0000","last_unstale":"2024-05-10T03:34:16.817646+0000","last_undegraded":"2024-05-10T03:34:16.817646+0000","last_fullsized":"2024-05-10T03:34:16.817646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:10:14.518870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814928+0000","last_change":"2024-05-10T03:34:15.527299+0000","last_active":"2024-05-10T03:34:16.814928+0000","last_peered":"2024-05-10T03:34:16.814928+0000","last_clean":"2024-05-10T03:34:16.814928+0000","last_became_active":"2024-05-10T03:34:15.527007+0000","last_became_peered":"2024-05-10T03:34:15.527007+0000","last_unstale":"2024-05-10T03:34:16.814928+0000","last_undegraded":"2024-05-10T03:34:16.814928+0000","last_fullsized":"2024-05-10T03:34:16.814928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:40.864469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816945+0000","last_change":"2024-05-10T03:34:14.948249+0000","last_active":"2024-05-10T03:34:16.816945+0000","last_peered":"2024-05-10T03:34:16.816945+0000","last_clean":"2024-05-10T03:34:16.816945+0000","last_became_active":"2024-05-10T03:34:14.947920+0000","last_became_peered":"2024-05-10T03:34:14.947920+0000","last_unstale":"2024-05-10T03:34:16.816945+0000","last_undegraded":"2024-05-10T03:34:16.816945+0000","last_fullsized":"2024-05-10T03:34:16.816945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:09.353930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816286+0000","last_change":"2024-05-10T03:34:15.530186+0000","last_active":"2024-05-10T03:34:16.816286+0000","last_peered":"2024-05-10T03:34:16.816286+0000","last_clean":"2024-05-10T03:34:16.816286+0000","last_became_active":"2024-05-10T03:34:15.529905+0000","last_became_peered":"2024-05-10T03:34:15.529905+0000","last_unstale":"2024-05-10T03:34:16.816286+0000","last_undegraded":"2024-05-10T03:34:16.816286+0000","last_fullsized":"2024-05-10T03:34:16.816286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:09:47.424391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815627+0000","last_change":"2024-05-10T03:34:14.942751+0000","last_active":"2024-05-10T03:34:16.815627+0000","last_peered":"2024-05-10T03:34:16.815627+0000","last_clean":"2024-05-10T03:34:16.815627+0000","last_became_active":"2024-05-10T03:34:14.942362+0000","last_became_peered":"2024-05-10T03:34:14.942362+0000","last_unstale":"2024-05-10T03:34:16.815627+0000","last_undegraded":"2024-05-10T03:34:16.815627+0000","last_fullsized":"2024-05-10T03:34:16.815627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:55:52.207470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814957+0000","last_change":"2024-05-10T03:34:15.533943+0000","last_active":"2024-05-10T03:34:16.814957+0000","last_peered":"2024-05-10T03:34:16.814957+0000","last_clean":"2024-05-10T03:34:16.814957+0000","last_became_active":"2024-05-10T03:34:15.533503+0000","last_became_peered":"2024-05-10T03:34:15.533503+0000","last_unstale":"2024-05-10T03:34:16.814957+0000","last_undegraded":"2024-05-10T03:34:16.814957+0000","last_fullsized":"2024-05-10T03:34:16.814957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:35:02.021713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816598+0000","last_change":"2024-05-10T03:34:14.944530+0000","last_active":"2024-05-10T03:34:16.816598+0000","last_peered":"2024-05-10T03:34:16.816598+0000","last_clean":"2024-05-10T03:34:16.816598+0000","last_became_active":"2024-05-10T03:34:14.944190+0000","last_became_peered":"2024-05-10T03:34:14.944190+0000","last_unstale":"2024-05-10T03:34:16.816598+0000","last_undegraded":"2024-05-10T03:34:16.816598+0000","last_fullsized":"2024-05-10T03:34:16.816598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:28.679478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517280+0000","last_change":"2024-05-10T03:34:15.517291+0000","last_active":"2024-05-10T03:34:16.517280+0000","last_peered":"2024-05-10T03:34:16.517280+0000","last_clean":"2024-05-10T03:34:16.517280+0000","last_became_active":"2024-05-10T03:34:15.516848+0000","last_became_peered":"2024-05-10T03:34:15.516848+0000","last_unstale":"2024-05-10T03:34:16.517280+0000","last_undegraded":"2024-05-10T03:34:16.517280+0000","last_fullsized":"2024-05-10T03:34:16.517280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:21:39.075357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517803+0000","last_change":"2024-05-10T03:34:14.945673+0000","last_active":"2024-05-10T03:34:16.517803+0000","last_peered":"2024-05-10T03:34:16.517803+0000","last_clean":"2024-05-10T03:34:16.517803+0000","last_became_active":"2024-05-10T03:34:14.945197+0000","last_became_peered":"2024-05-10T03:34:14.945197+0000","last_unstale":"2024-05-10T03:34:16.517803+0000","last_undegraded":"2024-05-10T03:34:16.517803+0000","last_fullsized":"2024-05-10T03:34:16.517803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:16:59.387389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815139+0000","last_change":"2024-05-10T03:34:15.524917+0000","last_active":"2024-05-10T03:34:16.815139+0000","last_peered":"2024-05-10T03:34:16.815139+0000","last_clean":"2024-05-10T03:34:16.815139+0000","last_became_active":"2024-05-10T03:34:15.522122+0000","last_became_peered":"2024-05-10T03:34:15.522122+0000","last_unstale":"2024-05-10T03:34:16.815139+0000","last_undegraded":"2024-05-10T03:34:16.815139+0000","last_fullsized":"2024-05-10T03:34:16.815139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:22:00.050088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515344+0000","last_change":"2024-05-10T03:34:14.925937+0000","last_active":"2024-05-10T03:34:16.515344+0000","last_peered":"2024-05-10T03:34:16.515344+0000","last_clean":"2024-05-10T03:34:16.515344+0000","last_became_active":"2024-05-10T03:34:14.925378+0000","last_became_peered":"2024-05-10T03:34:14.925378+0000","last_unstale":"2024-05-10T03:34:16.515344+0000","last_undegraded":"2024-05-10T03:34:16.515344+0000","last_fullsized":"2024-05-10T03:34:16.515344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:14.367480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515645+0000","last_change":"2024-05-10T03:34:15.513493+0000","last_active":"2024-05-10T03:34:16.515645+0000","last_peered":"2024-05-10T03:34:16.515645+0000","last_clean":"2024-05-10T03:34:16.515645+0000","last_became_active":"2024-05-10T03:34:15.512327+0000","last_became_peered":"2024-05-10T03:34:15.512327+0000","last_unstale":"2024-05-10T03:34:16.515645+0000","last_undegraded":"2024-05-10T03:34:16.515645+0000","last_fullsized":"2024-05-10T03:34:16.515645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:51:49.117770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814626+0000","last_change":"2024-05-10T03:34:14.941946+0000","last_active":"2024-05-10T03:34:16.814626+0000","last_peered":"2024-05-10T03:34:16.814626+0000","last_clean":"2024-05-10T03:34:16.814626+0000","last_became_active":"2024-05-10T03:34:14.941608+0000","last_became_peered":"2024-05-10T03:34:14.941608+0000","last_unstale":"2024-05-10T03:34:16.814626+0000","last_undegraded":"2024-05-10T03:34:16.814626+0000","last_fullsized":"2024-05-10T03:34:16.814626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:19.055958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515431+0000","last_change":"2024-05-10T03:34:15.523875+0000","last_active":"2024-05-10T03:34:16.515431+0000","last_peered":"2024-05-10T03:34:16.515431+0000","last_clean":"2024-05-10T03:34:16.515431+0000","last_became_active":"2024-05-10T03:34:15.523455+0000","last_became_peered":"2024-05-10T03:34:15.523455+0000","last_unstale":"2024-05-10T03:34:16.515431+0000","last_undegraded":"2024-05-10T03:34:16.515431+0000","last_fullsized":"2024-05-10T03:34:16.515431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:28:13.666954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515909+0000","last_change":"2024-05-10T03:34:14.944524+0000","last_active":"2024-05-10T03:34:16.515909+0000","last_peered":"2024-05-10T03:34:16.515909+0000","last_clean":"2024-05-10T03:34:16.515909+0000","last_became_active":"2024-05-10T03:34:14.944003+0000","last_became_peered":"2024-05-10T03:34:14.944003+0000","last_unstale":"2024-05-10T03:34:16.515909+0000","last_undegraded":"2024-05-10T03:34:16.515909+0000","last_fullsized":"2024-05-10T03:34:16.515909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:29.203659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817123+0000","last_change":"2024-05-10T03:34:15.520647+0000","last_active":"2024-05-10T03:34:16.817123+0000","last_peered":"2024-05-10T03:34:16.817123+0000","last_clean":"2024-05-10T03:34:16.817123+0000","last_became_active":"2024-05-10T03:34:15.518850+0000","last_became_peered":"2024-05-10T03:34:15.518850+0000","last_unstale":"2024-05-10T03:34:16.817123+0000","last_undegraded":"2024-05-10T03:34:16.817123+0000","last_fullsized":"2024-05-10T03:34:16.817123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:44.549294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814118+0000","last_change":"2024-05-10T03:34:14.927895+0000","last_active":"2024-05-10T03:34:16.814118+0000","last_peered":"2024-05-10T03:34:16.814118+0000","last_clean":"2024-05-10T03:34:16.814118+0000","last_became_active":"2024-05-10T03:34:14.927588+0000","last_became_peered":"2024-05-10T03:34:14.927588+0000","last_unstale":"2024-05-10T03:34:16.814118+0000","last_undegraded":"2024-05-10T03:34:16.814118+0000","last_fullsized":"2024-05-10T03:34:16.814118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:40:00.194295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815228+0000","last_change":"2024-05-10T03:34:15.543638+0000","last_active":"2024-05-10T03:34:16.815228+0000","last_peered":"2024-05-10T03:34:16.815228+0000","last_clean":"2024-05-10T03:34:16.815228+0000","last_became_active":"2024-05-10T03:34:15.541482+0000","last_became_peered":"2024-05-10T03:34:15.541482+0000","last_unstale":"2024-05-10T03:34:16.815228+0000","last_undegraded":"2024-05-10T03:34:16.815228+0000","last_fullsized":"2024-05-10T03:34:16.815228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:18:30.223450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816696+0000","last_change":"2024-05-10T03:34:14.942982+0000","last_active":"2024-05-10T03:34:16.816696+0000","last_peered":"2024-05-10T03:34:16.816696+0000","last_clean":"2024-05-10T03:34:16.816696+0000","last_became_active":"2024-05-10T03:34:14.942638+0000","last_became_peered":"2024-05-10T03:34:14.942638+0000","last_unstale":"2024-05-10T03:34:16.816696+0000","last_undegraded":"2024-05-10T03:34:16.816696+0000","last_fullsized":"2024-05-10T03:34:16.816696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:19:54.138986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816752+0000","last_change":"2024-05-10T03:34:15.519514+0000","last_active":"2024-05-10T03:34:16.816752+0000","last_peered":"2024-05-10T03:34:16.816752+0000","last_clean":"2024-05-10T03:34:16.816752+0000","last_became_active":"2024-05-10T03:34:15.517817+0000","last_became_peered":"2024-05-10T03:34:15.517817+0000","last_unstale":"2024-05-10T03:34:16.816752+0000","last_undegraded":"2024-05-10T03:34:16.816752+0000","last_fullsized":"2024-05-10T03:34:16.816752+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:37:03.119608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517426+0000","last_change":"2024-05-10T03:34:14.939598+0000","last_active":"2024-05-10T03:34:16.517426+0000","last_peered":"2024-05-10T03:34:16.517426+0000","last_clean":"2024-05-10T03:34:16.517426+0000","last_became_active":"2024-05-10T03:34:14.939316+0000","last_became_peered":"2024-05-10T03:34:14.939316+0000","last_unstale":"2024-05-10T03:34:16.517426+0000","last_undegraded":"2024-05-10T03:34:16.517426+0000","last_fullsized":"2024-05-10T03:34:16.517426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:11.802006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816596+0000","last_change":"2024-05-10T03:34:15.518344+0000","last_active":"2024-05-10T03:34:16.816596+0000","last_peered":"2024-05-10T03:34:16.816596+0000","last_clean":"2024-05-10T03:34:16.816596+0000","last_became_active":"2024-05-10T03:34:15.517408+0000","last_became_peered":"2024-05-10T03:34:15.517408+0000","last_unstale":"2024-05-10T03:34:16.816596+0000","last_undegraded":"2024-05-10T03:34:16.816596+0000","last_fullsized":"2024-05-10T03:34:16.816596+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:19.286396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517843+0000","last_change":"2024-05-10T03:34:14.950882+0000","last_active":"2024-05-10T03:34:16.517843+0000","last_peered":"2024-05-10T03:34:16.517843+0000","last_clean":"2024-05-10T03:34:16.517843+0000","last_became_active":"2024-05-10T03:34:14.950594+0000","last_became_peered":"2024-05-10T03:34:14.950594+0000","last_unstale":"2024-05-10T03:34:16.517843+0000","last_undegraded":"2024-05-10T03:34:16.517843+0000","last_fullsized":"2024-05-10T03:34:16.517843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:43:17.244818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.517946+0000","last_change":"2024-05-10T03:34:15.530844+0000","last_active":"2024-05-10T03:34:16.517946+0000","last_peered":"2024-05-10T03:34:16.517946+0000","last_clean":"2024-05-10T03:34:16.517946+0000","last_became_active":"2024-05-10T03:34:15.530430+0000","last_became_peered":"2024-05-10T03:34:15.530430+0000","last_unstale":"2024-05-10T03:34:16.517946+0000","last_undegraded":"2024-05-10T03:34:16.517946+0000","last_fullsized":"2024-05-10T03:34:16.517946+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:34.552010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815509+0000","last_change":"2024-05-10T03:34:14.941122+0000","last_active":"2024-05-10T03:34:16.815509+0000","last_peered":"2024-05-10T03:34:16.815509+0000","last_clean":"2024-05-10T03:34:16.815509+0000","last_became_active":"2024-05-10T03:34:14.940760+0000","last_became_peered":"2024-05-10T03:34:14.940760+0000","last_unstale":"2024-05-10T03:34:16.815509+0000","last_undegraded":"2024-05-10T03:34:16.815509+0000","last_fullsized":"2024-05-10T03:34:16.815509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:10:24.213926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814173+0000","last_change":"2024-05-10T03:34:15.512024+0000","last_active":"2024-05-10T03:34:16.814173+0000","last_peered":"2024-05-10T03:34:16.814173+0000","last_clean":"2024-05-10T03:34:16.814173+0000","last_became_active":"2024-05-10T03:34:15.511452+0000","last_became_peered":"2024-05-10T03:34:15.511452+0000","last_unstale":"2024-05-10T03:34:16.814173+0000","last_undegraded":"2024-05-10T03:34:16.814173+0000","last_fullsized":"2024-05-10T03:34:16.814173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:03:03.547694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817646+0000","last_change":"2024-05-10T03:34:14.941460+0000","last_active":"2024-05-10T03:34:16.817646+0000","last_peered":"2024-05-10T03:34:16.817646+0000","last_clean":"2024-05-10T03:34:16.817646+0000","last_became_active":"2024-05-10T03:34:14.940822+0000","last_became_peered":"2024-05-10T03:34:14.940822+0000","last_unstale":"2024-05-10T03:34:16.817646+0000","last_undegraded":"2024-05-10T03:34:16.817646+0000","last_fullsized":"2024-05-10T03:34:16.817646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:10:33.532220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.513046+0000","last_change":"2024-05-10T03:34:15.525028+0000","last_active":"2024-05-10T03:34:16.513046+0000","last_peered":"2024-05-10T03:34:16.513046+0000","last_clean":"2024-05-10T03:34:16.513046+0000","last_became_active":"2024-05-10T03:34:15.524744+0000","last_became_peered":"2024-05-10T03:34:15.524744+0000","last_unstale":"2024-05-10T03:34:16.513046+0000","last_undegraded":"2024-05-10T03:34:16.513046+0000","last_fullsized":"2024-05-10T03:34:16.513046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:11:05.704532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.515509+0000","last_change":"2024-05-10T03:34:14.930636+0000","last_active":"2024-05-10T03:34:16.515509+0000","last_peered":"2024-05-10T03:34:16.515509+0000","last_clean":"2024-05-10T03:34:16.515509+0000","last_became_active":"2024-05-10T03:34:14.930302+0000","last_became_peered":"2024-05-10T03:34:14.930302+0000","last_unstale":"2024-05-10T03:34:16.515509+0000","last_undegraded":"2024-05-10T03:34:16.515509+0000","last_fullsized":"2024-05-10T03:34:16.515509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:59:37.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.814554+0000","last_change":"2024-05-10T03:34:15.545444+0000","last_active":"2024-05-10T03:34:16.814554+0000","last_peered":"2024-05-10T03:34:16.814554+0000","last_clean":"2024-05-10T03:34:16.814554+0000","last_became_active":"2024-05-10T03:34:15.543724+0000","last_became_peered":"2024-05-10T03:34:15.543724+0000","last_unstale":"2024-05-10T03:34:16.814554+0000","last_undegraded":"2024-05-10T03:34:16.814554+0000","last_fullsized":"2024-05-10T03:34:16.814554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:29.699268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815546+0000","last_change":"2024-05-10T03:34:14.940987+0000","last_active":"2024-05-10T03:34:16.815546+0000","last_peered":"2024-05-10T03:34:16.815546+0000","last_clean":"2024-05-10T03:34:16.815546+0000","last_became_active":"2024-05-10T03:34:14.940553+0000","last_became_peered":"2024-05-10T03:34:14.940553+0000","last_unstale":"2024-05-10T03:34:16.815546+0000","last_undegraded":"2024-05-10T03:34:16.815546+0000","last_fullsized":"2024-05-10T03:34:16.815546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:20:43.934723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.817395+0000","last_change":"2024-05-10T03:34:15.508750+0000","last_active":"2024-05-10T03:34:16.817395+0000","last_peered":"2024-05-10T03:34:16.817395+0000","last_clean":"2024-05-10T03:34:16.817395+0000","last_became_active":"2024-05-10T03:34:15.508457+0000","last_became_peered":"2024-05-10T03:34:15.508457+0000","last_unstale":"2024-05-10T03:34:16.817395+0000","last_undegraded":"2024-05-10T03:34:16.817395+0000","last_fullsized":"2024-05-10T03:34:16.817395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:41.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.815704+0000","last_change":"2024-05-10T03:34:14.949941+0000","last_active":"2024-05-10T03:34:16.815704+0000","last_peered":"2024-05-10T03:34:16.815704+0000","last_clean":"2024-05-10T03:34:16.815704+0000","last_became_active":"2024-05-10T03:34:14.949657+0000","last_became_peered":"2024-05-10T03:34:14.949657+0000","last_unstale":"2024-05-10T03:34:16.815704+0000","last_undegraded":"2024-05-10T03:34:16.815704+0000","last_fullsized":"2024-05-10T03:34:16.815704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:14.367480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.816406+0000","last_change":"2024-05-10T03:34:15.539253+0000","last_active":"2024-05-10T03:34:16.816406+0000","last_peered":"2024-05-10T03:34:16.816406+0000","last_clean":"2024-05-10T03:34:16.816406+0000","last_became_active":"2024-05-10T03:34:15.537177+0000","last_became_peered":"2024-05-10T03:34:15.537177+0000","last_unstale":"2024-05-10T03:34:16.816406+0000","last_undegraded":"2024-05-10T03:34:16.816406+0000","last_fullsized":"2024-05-10T03:34:16.816406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:14.897068+0000","last_clean_scrub_stamp":"2024-05-10T03:34:14.897068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:46:35.193603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-10T03:34:16.818889+0000","last_change":"2024-05-10T03:34:14.943389+0000","last_active":"2024-05-10T03:34:16.818889+0000","last_peered":"2024-05-10T03:34:16.818889+0000","last_clean":"2024-05-10T03:34:16.818889+0000","last_became_active":"2024-05-10T03:34:14.942985+0000","last_became_peered":"2024-05-10T03:34:14.942985+0000","last_unstale":"2024-05-10T03:34:16.818889+0000","last_undegraded":"2024-05-10T03:34:16.818889+0000","last_fullsized":"2024-05-10T03:34:16.818889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:13.877277+0000","last_clean_scrub_stamp":"2024-05-10T03:34:13.877277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:41.926795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-10T03:34:15.520527+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:34:15.520527+0000","last_peered":"2024-05-10T03:34:15.520527+0000","last_clean":"2024-05-10T03:34:15.520527+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:34:15.520527+0000","last_undegraded":"2024-05-10T03:34:15.520527+0000","last_fullsized":"2024-05-10T03:34:15.520527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1712128,"data_stored":1091760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.2350000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.651,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.484},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3380000000000001,"15min":1.355},"min":{"1min":0.83099999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.696,"5min":1.877,"15min":1.877},"last":1.752}]},{"osd":1,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.83499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.798,"5min":1.853,"15min":1.853},"last":1.034},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.226,"15min":1.26},"min":{"1min":0.746,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6990000000000001,"5min":1.925,"15min":1.925},"last":1.5329999999999999}]},{"osd":2,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.3180000000000001,"15min":1.3680000000000001},"min":{"1min":0.89900000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.581,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3720000000000001,"15min":1.4259999999999999},"min":{"1min":0.98199999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3450000000000002,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3600000000000001,"15min":1.3819999999999999},"min":{"1min":0.96299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.8660000000000001,"5min":2.222,"15min":2.222},"last":1.377},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.369,"15min":1.339},"min":{"1min":0.85599999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":1.4339999999999999}]},{"osd":4,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.421,"15min":1.4610000000000001},"min":{"1min":0.83399999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.923,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.518,"15min":1.506},"min":{"1min":0.80400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.978,"5min":2.323,"15min":2.323},"last":1.593}]},{"osd":5,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5309999999999999,"15min":1.5109999999999999},"min":{"1min":0.78400000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4700000000000002,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.6659999999999999,"15min":1.6879999999999999},"min":{"1min":0.88600000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.117,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.8759999999999999}]},{"osd":6,"last update":"Fri May 10 03:33:41 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.698,"15min":1.72},"min":{"1min":1.002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4100000000000001,"5min":2.5590000000000002,"15min":2.5590000000000002},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.681,"5min":1.661,"15min":1.643},"min":{"1min":1.0309999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.298,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.7150000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1130496,"data_stored":503504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2849999999999999,"15min":1.337},"min":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7689999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.282,"15min":1.3080000000000001},"min":{"1min":0.84199999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.556,"5min":3.113,"15min":3.113},"last":1.331},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3720000000000001,"15min":1.4630000000000001},"min":{"1min":0.74099999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0350000000000001,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.319,"15min":1.2569999999999999},"min":{"1min":0.80400000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7909999999999999,"5min":2.02,"15min":2.02},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.52,"5min":1.4259999999999999,"15min":1.502},"min":{"1min":0.98299999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.202,"5min":3.569,"15min":3.569},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1230000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.367,"15min":1.3240000000000001},"min":{"1min":0.82099999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4279999999999999},"min":{"1min":0.92000000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.766,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.137},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.552,"15min":1.581},"min":{"1min":0.89100000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0579999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.53}]},{"osd":5,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.4590000000000001,"15min":1.4099999999999999},"min":{"1min":1.0089999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.617,"15min":1.621},"min":{"1min":1.1759999999999999,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8799999999999999,"5min":2.472,"15min":2.472},"last":1.246}]},{"osd":7,"last update":"Fri May 10 03:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.752,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.1919999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.4590000000000001,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.841,"5min":1.663,"15min":1.659},"min":{"1min":1.05,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1110016,"data_stored":501336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.0920000000000001,"15min":1.071},"min":{"1min":0.72999999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1890000000000001,"15min":1.1990000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6950000000000001,"5min":1.851,"15min":1.851},"last":1.2969999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.145,"15min":1.141},"min":{"1min":0.72899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.7190000000000001,"5min":1.855,"15min":1.855},"last":1.143},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.702,"5min":1.954,"15min":1.954},"last":1.76}]},{"osd":2,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.232,"15min":1.222},"min":{"1min":0.749,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.669,"5min":2.359,"15min":2.359},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.337,"5min":1.3080000000000001,"15min":1.331},"min":{"1min":1.0069999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.6559999999999999}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.361,"15min":1.357},"min":{"1min":0.98799999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7210000000000001,"5min":1.881,"15min":1.881},"last":1.615},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.357,"15min":1.3049999999999999},"min":{"1min":1.0489999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.909,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.573}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.79200000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9470000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.5920000000000001,"15min":1.603},"min":{"1min":1.246,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1600000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":1.71}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.534,"15min":1.518},"min":{"1min":1.3480000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0190000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.8},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5600000000000001,"15min":1.5549999999999999},"min":{"1min":1.129,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.869}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.7829999999999999,"15min":1.8069999999999999},"min":{"1min":1.2330000000000001,"5min":0.753,"15min":0.753},"max":{"1min":2.3969999999999998,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.915},{"interface":"front","average":{"1min":1.7390000000000001,"5min":1.6459999999999999,"15min":1.6220000000000001},"min":{"1min":0.96599999999999997,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.4299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1105920,"data_stored":494876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.169},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5940000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1279999999999999,"15min":1.1359999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1850000000000001,"15min":1.163},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.24,"15min":1.27},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.75,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2529999999999999,"15min":1.2749999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.03},{"interface":"front","average":{"1min":1.335,"5min":1.3540000000000001,"15min":1.3919999999999999},"min":{"1min":0.80700000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.647,"5min":2.29,"15min":2.29},"last":1.764}]},{"osd":3,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3899999999999999,"15min":1.454},"min":{"1min":0.85199999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8180000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5549999999999999,"15min":1.577},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0800000000000001,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.546}]},{"osd":5,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4850000000000001,"15min":1.5},"min":{"1min":0.93000000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.877,"5min":2.512,"15min":2.512},"last":1.504},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4730000000000001,"15min":1.5409999999999999},"min":{"1min":0.70399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.4129999999999998,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.585}]},{"osd":6,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5589999999999999,"15min":1.583},"min":{"1min":0.75900000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.1859999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6910000000000001,"15min":1.766},"min":{"1min":0.879,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.069,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.657}]},{"osd":7,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.6379999999999999,"15min":1.6850000000000001},"min":{"1min":0.78400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":2.5099999999999998,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.7330000000000001,"15min":1.742},"min":{"1min":0.82899999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3130000000000002,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.6240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1110016,"data_stored":501336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.45700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.395,"5min":1.754,"15min":1.754},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94399999999999995,"15min":0.97199999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.6200000000000001,"5min":1.708,"15min":1.708},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3069999999999999,"5min":1.603,"15min":1.603},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.179,"5min":1.2110000000000001,"15min":1.2450000000000001},"min":{"1min":0.50700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":1.964,"15min":1.964},"last":1.5429999999999999}]},{"osd":2,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1699999999999999,"15min":1.1819999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.278,"15min":1.3480000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7130000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.621}]},{"osd":4,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.355,"15min":1.371},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9710000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4179999999999999,"15min":1.3879999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9650000000000001,"5min":2.464,"15min":2.464},"last":1.379}]},{"osd":5,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.583,"15min":1.6639999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.4399999999999999,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.524,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.298}]},{"osd":6,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5549999999999999,"15min":1.573},"min":{"1min":0.90300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.1480000000000001,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.579},{"interface":"front","average":{"1min":1.73,"5min":1.7290000000000001,"15min":1.7889999999999999},"min":{"1min":0.92300000000000004,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.4329999999999998,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.7470000000000001}]},{"osd":7,"last update":"Fri May 10 03:33:38 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.542,"15min":1.583},"min":{"1min":0.97699999999999998,"5min":0.87,"15min":0.87},"max":{"1min":2.2229999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.607,"15min":1.647},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.0089999999999999,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.647}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1097728,"data_stored":494716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93400000000000005,"15min":0.98299999999999998},"min":{"1min":0.504,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.389,"5min":1.772,"15min":1.772},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.94099999999999995,"15min":0.97899999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.635,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.349}]},{"osd":1,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.56399999999999995,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.425,"5min":1.746,"15min":1.746},"last":1.28},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.61699999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.6879999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.085,"15min":1.117},"min":{"1min":0.63600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.526,"5min":2.169,"15min":2.169},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.21,"15min":1.25},"min":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8520000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.714}]},{"osd":4,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.5269999999999999},"min":{"1min":0.92600000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.024,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8600000000000001,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":2.0489999999999999}]},{"osd":5,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.464,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.173,"5min":2.508,"15min":2.508},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5900000000000001,"15min":1.617},"min":{"1min":0.97999999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":2.0859999999999999,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.454}]},{"osd":6,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.512,"15min":1.5449999999999999},"min":{"1min":0.90700000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1339999999999999,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":2.1989999999999998},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5489999999999999,"15min":1.587},"min":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.331,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri May 10 03:33:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.607,"15min":1.649},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2749999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":2.3149999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.619,"15min":1.665},"min":{"1min":0.95599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.266,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.673}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1101824,"data_stored":494786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91000000000000003,"15min":0.92200000000000004},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.407,"5min":1.607,"15min":1.607},"last":0.626},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0069999999999999,"15min":1.0720000000000001},"min":{"1min":0.54100000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.583,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.98599999999999999,"15min":1.006},"min":{"1min":0.54100000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.748,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.996},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.143,"15min":1.1819999999999999},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.0720000000000001,"5min":2.444,"15min":2.444},"last":1.4430000000000001}]},{"osd":3,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.256,"15min":1.3129999999999999},"min":{"1min":0.59099999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6799999999999999,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.677},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2350000000000001,"15min":1.288},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.772,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4650000000000001,"15min":1.5109999999999999},"min":{"1min":0.95099999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.2650000000000001,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.5089999999999999,"15min":1.577},"min":{"1min":0.92500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1139999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.347}]},{"osd":5,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4890000000000001,"15min":1.575},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0539999999999998,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.538,"15min":1.6200000000000001},"min":{"1min":1.004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0470000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.601}]},{"osd":6,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.601,"15min":1.704},"min":{"1min":0.96499999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.347,"5min":3.48,"15min":3.48},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.6180000000000001,"15min":1.716},"min":{"1min":1.054,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.1230000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.6539999999999999}]},{"osd":7,"last update":"Fri May 10 03:33:42 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.665,"15min":1.7270000000000001},"min":{"1min":1.089,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.2709999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.79},{"interface":"front","average":{"1min":1.581,"5min":1.6739999999999999,"15min":1.7669999999999999},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.52,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.206}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1728512,"data_stored":1093974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92800000000000005,"15min":1.012},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.222,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.046,"15min":1.173},"min":{"1min":0.69699999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6120000000000001,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":0.95199999999999996}]},{"osd":2,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1200000000000001,"15min":1.21},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4910000000000001,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.097,"15min":1.1990000000000001},"min":{"1min":0.57899999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.627,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0309999999999999,"15min":1.095},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.7050000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.877},{"interface":"front","average":{"1min":1.28,"5min":1.264,"15min":1.319},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8240000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.464,"15min":1.6080000000000001},"min":{"1min":0.83699999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.7769999999999999,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.383,"15min":1.4299999999999999},"min":{"1min":0.84099999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.778,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.177}]},{"osd":5,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.405,"15min":1.4670000000000001},"min":{"1min":0.99399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9239999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.552},{"interface":"front","average":{"1min":1.506,"5min":1.5169999999999999,"15min":1.617},"min":{"1min":0.90900000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8109999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.6120000000000001}]},{"osd":6,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5700000000000001,"15min":1.6339999999999999},"min":{"1min":1.0960000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.966,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.5960000000000001,"15min":1.7230000000000001},"min":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.129,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.323}]},{"osd":7,"last update":"Fri May 10 03:33:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.6240000000000001,"15min":1.7709999999999999},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.972,"5min":3.149,"15min":3.149},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.536,"15min":1.573},"min":{"1min":0.89800000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.0600000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:34:21.307 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:34:21.307 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:34:21.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.639+0000 7f6c3e200640 1 Processor -- start 2024-05-10T03:34:21.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.639+0000 7f6c3e200640 1 -- start start 2024-05-10T03:34:21.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.641+0000 7f6c3e200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6c380ce5f0 0x7f6c3813cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.641+0000 7f6c3e200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3815ae50 con 0x7f6c3806b830 2024-05-10T03:34:21.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.641+0000 7f6c3e200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3813d0b0 con 0x7f6c380b1410 2024-05-10T03:34:21.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.641+0000 7f6c3e200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3815a520 con 0x7f6c380ce5f0 2024-05-10T03:34:21.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.642+0000 7f6c3d800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6c3806b830 0x7f6c3815db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52836/0 (socket says 172.21.15.55:52836) 2024-05-10T03:34:21.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.642+0000 7f6c3d800640 1 -- 172.21.15.55:0/3296396998 learned_addr learned my addr 172.21.15.55:0/3296396998 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:21.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.642+0000 7f6c3ce00640 1 --2- 172.21.15.55:0/3296396998 >> v2:172.21.15.162:3300/0 conn(0x7f6c380ce5f0 0x7f6c3813cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.642+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764649814 0 0) 0x7f6c3815ae50 con 0x7f6c3806b830 2024-05-10T03:34:21.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.642+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c20003520 con 0x7f6c3806b830 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521121987 0 0) 0x7f6c20003520 con 0x7f6c3806b830 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 >> v2:172.21.15.162:3300/0 conn(0x7f6c380ce5f0 msgr2=0x7f6c3813cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 --2- 172.21.15.55:0/3296396998 >> v2:172.21.15.162:3300/0 conn(0x7f6c380ce5f0 0x7f6c3813cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 >> v1:172.21.15.162:6789/0 conn(0x7f6c380b1410 legacy=0x7f6c3806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3815a800 con 0x7f6c3806b830 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f6c300019f0 con 0x7f6c3806b830 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 >> v1:172.21.15.55:6789/0 conn(0x7f6c3806b830 legacy=0x7f6c3815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c20003970 0x7f6c20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c200047c0 0x7f6c20004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 0x7f6c2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3815a520 con 0x7f6c20005180 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2000c120 con 0x7f6c20003970 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2000c2e0 con 0x7f6c200047c0 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.643+0000 7f6c3d800640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 0x7f6c2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3ce00640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c20003970 0x7f6c20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c37e00640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c200047c0 0x7f6c20004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3d800640 1 -- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c200047c0 msgr2=0x7f6c20004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3d800640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c200047c0 0x7f6c20004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3d800640 1 -- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c20003970 msgr2=0x7f6c20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3d800640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c20003970 0x7f6c20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c3d800640 1 -- 172.21.15.55:0/3296396998 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20003520 con 0x7f6c20005180 2024-05-10T03:34:21.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.644+0000 7f6c37e00640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c200047c0 0x7f6c20004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.645+0000 7f6c3d800640 1 -- 172.21.15.55:0/3296396998 reap_dead start 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.645+0000 7f6c3d800640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 0x7f6c2000b980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000a800 tx=0x7f6c200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ccd6c98922f1a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.645+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3002b960 con 0x7f6c20005180 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.645+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3002bae0 con 0x7f6c20005180 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.645+0000 7f6c37400640 1 -- 172.21.15.55:0/3296396998 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3002be90 con 0x7f6c20005180 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 -- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 msgr2=0x7f6c2000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 0x7f6c2000b980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000a800 tx=0x7f6c200057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 -- 172.21.15.55:0/3296396998 shutdown_connections 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 --2- 172.21.15.55:0/3296396998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c20005180 0x7f6c2000b980 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 -- 172.21.15.55:0/3296396998 >> 172.21.15.55:0/3296396998 conn(0x7f6c3806c8f0 msgr2=0x7f6c380b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 -- 172.21.15.55:0/3296396998 shutdown_connections 2024-05-10T03:34:21.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.646+0000 7f6c3e200640 1 -- 172.21.15.55:0/3296396998 wait complete. 2024-05-10T03:34:21.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.649+0000 7f6c3e200640 1 Processor -- start 2024-05-10T03:34:21.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.649+0000 7f6c3e200640 1 -- start start 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c20005180 0x7f6c38138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c381f6100 0x7f6c381f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3814fd50 con 0x7f6c38153d80 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c381501b0 con 0x7f6c381f6100 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3e200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380cba60 con 0x7f6c20005180 2024-05-10T03:34:21.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58336/0 (socket says 172.21.15.55:58336) 2024-05-10T03:34:21.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.651+0000 7f6c3d800640 1 -- 172.21.15.55:0/2456728633 learned_addr learned my addr 172.21.15.55:0/2456728633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c37e00640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c381f6100 0x7f6c381f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3ce00640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c20005180 0x7f6c38138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 -- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c20005180 msgr2=0x7f6c38138930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c20005180 0x7f6c38138930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 -- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c381f6100 msgr2=0x7f6c381f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c381f6100 0x7f6c381f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 -- 172.21.15.55:0/2456728633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380cbf90 con 0x7f6c38153d80 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.652+0000 7f6c3d800640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000a860 tx=0x7f6c3002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.653+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3003b040 con 0x7f6c38153d80 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.653+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30034880 con 0x7f6c38153d80 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.653+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30034bf0 con 0x7f6c38153d80 2024-05-10T03:34:21.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.653+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c381538f0 con 0x7f6c38153d80 2024-05-10T03:34:21.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.653+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380cbcb0 con 0x7f6c38153d80 2024-05-10T03:34:21.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.654+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c38153a80 con 0x7f6c38153d80 2024-05-10T03:34:21.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.655+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6c30031070 con 0x7f6c38153d80 2024-05-10T03:34:21.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.656+0000 7f6c34c00640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 0x7f6c1806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:21.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.656+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6c300a9190 con 0x7f6c38153d80 2024-05-10T03:34:21.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.657+0000 7f6c3ce00640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 0x7f6c1806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:21.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.658+0000 7f6c3ce00640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 0x7f6c1806d0a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009990 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:21.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.661+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6c30074220 con 0x7f6c38153d80 2024-05-10T03:34:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.935+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6c38067220 con 0x7f6c38153d80 2024-05-10T03:34:21.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.944+0000 7f6c34c00640 1 -- 172.21.15.55:0/2456728633 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6c3000a450 con 0x7f6c38153d80 2024-05-10T03:34:21.948 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 msgr2=0x7f6c1806d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 0x7f6c1806d0a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009990 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 msgr2=0x7f6c38154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000a860 tx=0x7f6c3002d070 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 shutdown_connections 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c38153d80 0x7f6c38154190 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6c1806ab90 0x7f6c1806d0a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c381f6100 0x7f6c381f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 --2- 172.21.15.55:0/2456728633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c20005180 0x7f6c38138930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:21.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 >> 172.21.15.55:0/2456728633 conn(0x7f6c3806c8f0 msgr2=0x7f6c380cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:21.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 shutdown_connections 2024-05-10T03:34:21.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:21.947+0000 7f6c3e200640 1 -- 172.21.15.55:0/2456728633 wait complete. 2024-05-10T03:34:21.978 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:22.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.249+0000 7fb007e00640 1 Processor -- start 2024-05-10T03:34:22.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.249+0000 7fb007e00640 1 -- start start 2024-05-10T03:34:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.251+0000 7fb007e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 0x7fb0080b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.251+0000 7fb007e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.251+0000 7fb007e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080b2aa0 con 0x7fb0080b3d30 2024-05-10T03:34:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.251+0000 7fb007e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080b6190 con 0x7fb0080ce3a0 2024-05-10T03:34:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.251+0000 7fb007e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b6c30 con 0x7fb00815d970 2024-05-10T03:34:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.252+0000 7fb006000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.252+0000 7fb006a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 0x7fb0080b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.252+0000 7fb006000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:46716/0 (socket says 172.21.15.55:46716) 2024-05-10T03:34:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.252+0000 7fb006000640 1 -- 172.21.15.55:0/1114814125 learned_addr learned my addr 172.21.15.55:0/1114814125 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.253+0000 7fb006a00640 1 -- 172.21.15.55:0/1114814125 >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 msgr2=0x7fb0080b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.253+0000 7fb006a00640 1 --2- 172.21.15.55:0/1114814125 >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.253+0000 7fb006a00640 1 -- 172.21.15.55:0/1114814125 >> v1:172.21.15.162:6789/0 conn(0x7fb0080b3d30 legacy=0x7fb008161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.253+0000 7fb006a00640 1 -- 172.21.15.55:0/1114814125 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080b2d10 con 0x7fb00815d970 2024-05-10T03:34:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.253+0000 7fb006000640 1 --2- 172.21.15.55:0/1114814125 >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:22.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.254+0000 7fb006a00640 1 --2- 172.21.15.55:0/1114814125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 0x7fb0080b66f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faff80097a0 tx=0x7faff802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189271400a48d75b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:22.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb005600640 1 -- 172.21.15.55:0/1114814125 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8032440 con 0x7fb00815d970 2024-05-10T03:34:22.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb005600640 1 -- 172.21.15.55:0/1114814125 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff80325a0 con 0x7fb00815d970 2024-05-10T03:34:22.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb005600640 1 -- 172.21.15.55:0/1114814125 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8032930 con 0x7fb00815d970 2024-05-10T03:34:22.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 -- 172.21.15.55:0/1114814125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 msgr2=0x7fb0080b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 --2- 172.21.15.55:0/1114814125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 0x7fb0080b66f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faff80097a0 tx=0x7faff802a180 comp rx=0 tx=0).stop 2024-05-10T03:34:22.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 -- 172.21.15.55:0/1114814125 shutdown_connections 2024-05-10T03:34:22.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 --2- 172.21.15.55:0/1114814125 >> v2:172.21.15.162:3300/0 conn(0x7fb0080ce3a0 0x7fb0080b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 --2- 172.21.15.55:0/1114814125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb00815d970 0x7fb0080b66f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.255+0000 7fb007e00640 1 -- 172.21.15.55:0/1114814125 >> 172.21.15.55:0/1114814125 conn(0x7fb0080d0370 msgr2=0x7fb0080d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:22.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.256+0000 7fb007e00640 1 -- 172.21.15.55:0/1114814125 shutdown_connections 2024-05-10T03:34:22.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.256+0000 7fb007e00640 1 -- 172.21.15.55:0/1114814125 wait complete. 2024-05-10T03:34:22.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.258+0000 7fb007e00640 1 Processor -- start 2024-05-10T03:34:22.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.258+0000 7fb007e00640 1 -- start start 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0080b3d30 0x7fb0080c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 0x7fb0080c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0081540b0 con 0x7fb00815d970 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c9a50 con 0x7fb0080b3d30 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0081550f0 con 0x7fb0080ce3a0 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58346/0 (socket says 172.21.15.55:58346) 2024-05-10T03:34:22.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb007400640 1 -- 172.21.15.55:0/1806856411 learned_addr learned my addr 172.21.15.55:0/1806856411 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:22.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb006000640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 0x7fb0080c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.260+0000 7fb006a00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0080b3d30 0x7fb0080c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 -- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 msgr2=0x7fb0080c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 0x7fb0080c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 -- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0080b3d30 msgr2=0x7fb0080c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0080b3d30 0x7fb0080c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 -- 172.21.15.55:0/1806856411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009b30 con 0x7fb00815d970 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb006000640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 0x7fb0080c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007400640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c3c0 tx=0x7fb00000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000052d0 con 0x7fb00815d970 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000005430 con 0x7fb00815d970 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000003810 con 0x7fb00815d970 2024-05-10T03:34:22.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff8009400 con 0x7fb00815d970 2024-05-10T03:34:22.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.261+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080c9520 con 0x7fb00815d970 2024-05-10T03:34:22.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.263+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0080cb600 con 0x7fb00815d970 2024-05-10T03:34:22.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.264+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb000003970 con 0x7fb00815d970 2024-05-10T03:34:22.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.265+0000 7faffea00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 0x7fafe406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:22.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.265+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb0000889c0 con 0x7fb00815d970 2024-05-10T03:34:22.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.266+0000 7fb006a00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 0x7fafe406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:22.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.267+0000 7fb006a00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 0x7fafe406d140 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faff8034000 tx=0x7faff802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:22.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.269+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb0080cb600 con 0x7fb00815d970 2024-05-10T03:34:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.505+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0080cb600 con 0x7fb00815d970 2024-05-10T03:34:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.506+0000 7faffea00640 1 -- 172.21.15.55:0/1806856411 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fb0080cb600 con 0x7fb00815d970 2024-05-10T03:34:22.511 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:22.511 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 msgr2=0x7fafe406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 0x7fafe406d140 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faff8034000 tx=0x7faff802e000 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 msgr2=0x7fb00815c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c3c0 tx=0x7fb00000c880 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 shutdown_connections 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00815d970 0x7fb00815c710 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0080ce3a0 0x7fb0080c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafe406ac30 0x7fafe406d140 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 --2- 172.21.15.55:0/1806856411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0080b3d30 0x7fb0080c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 >> 172.21.15.55:0/1806856411 conn(0x7fb0080d0370 msgr2=0x7fb0080b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 shutdown_connections 2024-05-10T03:34:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:22.509+0000 7fb007e00640 1 -- 172.21.15.55:0/1806856411 wait complete. 2024-05-10T03:34:22.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:15.487725+0000', 'modified': '2024-05-10T03:34:18.971182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6930': {'gid': 6930, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/709318521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 709318521}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 709318521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:34:22.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:34:22.943 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T03:34:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.225+0000 7fd8f7200640 1 Processor -- start 2024-05-10T03:34:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.225+0000 7fd8f7200640 1 -- start start 2024-05-10T03:34:23.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f813e390 0x7fd8f815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8f806b830 0x7fd8f813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f813eae0 con 0x7fd8f81414c0 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f813e7a0 con 0x7fd8f806b830 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f815c020 con 0x7fd8f813e390 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8ed400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd8f81414c0 0x7fd8f806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44838/0 (socket says 172.21.15.55:44838) 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8ed400640 1 -- 172.21.15.55:0/3392387628 learned_addr learned my addr 172.21.15.55:0/3392387628 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3392387628 >> v2:172.21.15.55:3300/0 conn(0x7fd8f806b830 0x7fd8f813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.228+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f813e390 0x7fd8f815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f5400640 1 -- 172.21.15.55:0/3392387628 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250149101 0 0) 0x7fd8f813eae0 con 0x7fd8f81414c0 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f5400640 1 -- 172.21.15.55:0/3392387628 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8dc003520 con 0x7fd8f81414c0 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f6800640 1 -- 172.21.15.55:0/3392387628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f813e390 msgr2=0x7fd8f815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f813e390 0x7fd8f815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f6800640 1 -- 172.21.15.55:0/3392387628 >> v1:172.21.15.55:6789/0 conn(0x7fd8f81414c0 legacy=0x7fd8f806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.229+0000 7fd8f6800640 1 -- 172.21.15.55:0/3392387628 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f815c300 con 0x7fd8f806b830 2024-05-10T03:34:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.230+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f806b830 0x7fd8f813ef70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00097a0 tx=0x7fd8f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f841899d4c69e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.230+0000 7fd8f5400640 1 -- 172.21.15.55:0/3392387628 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0032440 con 0x7fd8f806b830 2024-05-10T03:34:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.230+0000 7fd8f5400640 1 -- 172.21.15.55:0/3392387628 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00325a0 con 0x7fd8f806b830 2024-05-10T03:34:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.230+0000 7fd8f5400640 1 -- 172.21.15.55:0/3392387628 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0032930 con 0x7fd8f806b830 2024-05-10T03:34:23.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 -- 172.21.15.55:0/3392387628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f806b830 msgr2=0x7fd8f813ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f806b830 0x7fd8f813ef70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00097a0 tx=0x7fd8f002a180 comp rx=0 tx=0).stop 2024-05-10T03:34:23.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 -- 172.21.15.55:0/3392387628 shutdown_connections 2024-05-10T03:34:23.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f806b830 0x7fd8f813ef70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 --2- 172.21.15.55:0/3392387628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f813e390 0x7fd8f815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 -- 172.21.15.55:0/3392387628 >> 172.21.15.55:0/3392387628 conn(0x7fd8f806c730 msgr2=0x7fd8f80b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 -- 172.21.15.55:0/3392387628 shutdown_connections 2024-05-10T03:34:23.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.231+0000 7fd8f7200640 1 -- 172.21.15.55:0/3392387628 wait complete. 2024-05-10T03:34:23.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.234+0000 7fd8f7200640 1 Processor -- start 2024-05-10T03:34:23.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.234+0000 7fd8f7200640 1 -- start start 2024-05-10T03:34:23.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 0x7fd8f81566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f81414c0 0x7fd8f8151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f81f14f0 con 0x7fd8f81f1050 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8156e80 con 0x7fd8f813e390 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.235+0000 7fd8f7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8156c30 con 0x7fd8f81414c0 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51000/0 (socket says 172.21.15.55:51000) 2024-05-10T03:34:23.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 -- 172.21.15.55:0/855919470 learned_addr learned my addr 172.21.15.55:0/855919470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8ed400640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f81414c0 0x7fd8f8151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 0x7fd8f81566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 -- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f81414c0 msgr2=0x7fd8f8151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f81414c0 0x7fd8f8151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 -- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 msgr2=0x7fd8f81566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 0x7fd8f81566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f6800640 1 -- 172.21.15.55:0/855919470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0009400 con 0x7fd8f81f1050 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.236+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 0x7fd8f81566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8f6800640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00098c0 tx=0x7fd8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8eea00640 1 -- 172.21.15.55:0/855919470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0030b00 con 0x7fd8f81f1050 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8eea00640 1 -- 172.21.15.55:0/855919470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0032e90 con 0x7fd8f81f1050 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8eea00640 1 -- 172.21.15.55:0/855919470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0006e10 con 0x7fd8f81f1050 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8141a50 con 0x7fd8f81f1050 2024-05-10T03:34:23.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.237+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f81573b0 con 0x7fd8f81f1050 2024-05-10T03:34:23.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.239+0000 7fd8eea00640 1 -- 172.21.15.55:0/855919470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd8f0039040 con 0x7fd8f81f1050 2024-05-10T03:34:23.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.240+0000 7fd8eea00640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 0x7fd8d406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.240+0000 7fd8eea00640 1 -- 172.21.15.55:0/855919470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd8f00a9c00 con 0x7fd8f81f1050 2024-05-10T03:34:23.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.241+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 0x7fd8d406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.241+0000 7fd903a25740 1 Processor -- start 2024-05-10T03:34:23.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.241+0000 7fd903a25740 1 -- start start 2024-05-10T03:34:23.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.242+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 0x7fd8d406d140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b4350 tx=0x7fd8e800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af60649c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6064a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6056ca70 con 0x55af60264270 2024-05-10T03:34:23.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6056a200 con 0x55af60243590 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af60565af0 con 0x55af6023b3b0 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51010/0 (socket says 172.21.15.55:51010) 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 -- 172.21.15.55:0/1629609112 learned_addr learned my addr 172.21.15.55:0/1629609112 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 -- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 msgr2=0x55af60649c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af60649c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 -- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 msgr2=0x55af6064a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6064a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.243+0000 7fd8f5e00640 1 -- 172.21.15.55:0/1629609112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af60530b90 con 0x55af60264270 2024-05-10T03:34:23.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.244+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80098a0 tx=0x7fd8e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06c15270e14788d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.244+0000 7fd8e6600640 1 -- 172.21.15.55:0/1629609112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8014a20 con 0x55af60264270 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.244+0000 7fd8e6600640 1 -- 172.21.15.55:0/1629609112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8014b80 con 0x55af60264270 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.244+0000 7fd8e6600640 1 -- 172.21.15.55:0/1629609112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8021400 con 0x55af60264270 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 -- 172.21.15.55:0/1629609112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 msgr2=0x55af6062a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80098a0 tx=0x7fd8e8012000 comp rx=0 tx=0).stop 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 -- 172.21.15.55:0/1629609112 shutdown_connections 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6064a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af60649c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 --2- 172.21.15.55:0/1629609112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062a4a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 -- 172.21.15.55:0/1629609112 >> 172.21.15.55:0/1629609112 conn(0x55af602583c0 msgr2=0x55af60603330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 -- 172.21.15.55:0/1629609112 shutdown_connections 2024-05-10T03:34:23.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.245+0000 7fd903a25740 1 -- 172.21.15.55:0/1629609112 wait complete. 2024-05-10T03:34:23.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.246+0000 7fd903a25740 1 -- start start 2024-05-10T03:34:23.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.246+0000 7fd903a25740 1 Processor -- start 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 0x55af60645500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af60243590 0x55af6062d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60507220 con 0x55af60264270 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60506ec0 con 0x55af6023b3b0 2024-05-10T03:34:23.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af602ccf20 con 0x55af60243590 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd903a25740 10 client.0 _ll_register_callbacks cb 0x55af6055f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51018/0 (socket says 172.21.15.55:51018) 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.248+0000 7fd8f6800640 1 -- 172.21.15.55:0/999298967 learned_addr learned my addr 172.21.15.55:0/999298967 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 0x55af60645500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8ed400640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af60243590 0x55af6062d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 -- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af60243590 msgr2=0x55af6062d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af60243590 0x55af6062d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 -- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 msgr2=0x55af60645500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 0x55af60645500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 -- 172.21.15.55:0/999298967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8016420 con 0x55af60264270 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 0x55af60645500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8f6800640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0078850 tx=0x7fd8f0079100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9598c96e6f11dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8daa00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0080cb0 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd8f0078b20 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.249+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af6058fc90 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f0078ce0 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 10 client.7077 fetch_fsmap learned FSMap version 47 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55af60564170 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af6058fc90 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd8f0077db0 con 0x55af60264270 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 10 client.7077 fetch_fsmap finished waiting for FSMap version 47 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 10 client.7077 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 20 client.7077 populate_metadata read hostname 'smithi055' 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af60222e40 0x55af6065f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 4 client.7077 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:34:23.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 --> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af602593c0 con 0x55af60222e40 2024-05-10T03:34:23.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.250+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af60222e40 0x55af6065f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.251+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af60222e40 0x55af6065f790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7a6267dbc0650 server_cookie=6876c3c0f8820c30 in_seq=0 out_seq=0 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.251+0000 7fd8daa00640 10 client.7077 ms_handle_connect on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd8daa00640 1 -- 172.21.15.55:0/999298967 <== mds.0 v2:172.21.15.55:6834/709318521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af602593c0 con 0x55af60222e40 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd8daa00640 10 client.7077 handle_command_reply: tid=0 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 client.7077 shutdown 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 msgr2=0x55af6062ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0078850 tx=0x7fd8f0079100 comp rx=0 tx=0).stop 2024-05-10T03:34:23.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 shutdown_connections 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af60222e40 0x55af6065f790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af6062ddf0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af60243590 0x55af6062d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 --2- 172.21.15.55:0/999298967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af6023b3b0 0x55af60645500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 >> 172.21.15.55:0/999298967 conn(0x55af602583c0 msgr2=0x55af606489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd8daa00640 0 client.7077 ms_handle_reset on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 shutdown_connections 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.252+0000 7fd903a25740 1 -- 172.21.15.55:0/999298967 wait complete. 2024-05-10T03:34:23.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.253+0000 7fd903a25740 20 client.7077 trim_cache size 0 max 16384 2024-05-10T03:34:23.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.269+0000 7fd903a25740 1 Processor -- start 2024-05-10T03:34:23.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.269+0000 7fd903a25740 1 -- start start 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6066f0a0 con 0x55af60264270 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6065c430 con 0x55af60243590 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af605fceb0 con 0x55af6023b3b0 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51030/0 (socket says 172.21.15.55:51030) 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 -- 172.21.15.55:0/2900658497 learned_addr learned my addr 172.21.15.55:0/2900658497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8ed400640 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 -- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 msgr2=0x55af6065b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 -- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 msgr2=0x55af6065bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.271+0000 7fd8f6800640 1 -- 172.21.15.55:0/2900658497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af60599130 con 0x55af60264270 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.272+0000 7fd8f6800640 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002a1c0 tx=0x7fd8f003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b4a5db8f1aab8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.272+0000 7fd8e7000640 1 -- 172.21.15.55:0/2900658497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0078400 con 0x55af60264270 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.272+0000 7fd8e7000640 1 -- 172.21.15.55:0/2900658497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0080cb0 con 0x55af60264270 2024-05-10T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.272+0000 7fd8e7000640 1 -- 172.21.15.55:0/2900658497 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00746b0 con 0x55af60264270 2024-05-10T03:34:23.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 -- 172.21.15.55:0/2900658497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 msgr2=0x55af60601910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002a1c0 tx=0x7fd8f003e000 comp rx=0 tx=0).stop 2024-05-10T03:34:23.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 -- 172.21.15.55:0/2900658497 shutdown_connections 2024-05-10T03:34:23.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60601910 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 --2- 172.21.15.55:0/2900658497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 -- 172.21.15.55:0/2900658497 >> 172.21.15.55:0/2900658497 conn(0x55af602583c0 msgr2=0x55af606657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 -- 172.21.15.55:0/2900658497 shutdown_connections 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.273+0000 7fd903a25740 1 -- 172.21.15.55:0/2900658497 wait complete. 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.274+0000 7fd903a25740 1 -- start start 2024-05-10T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.274+0000 7fd903a25740 1 Processor -- start 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af605f9910 con 0x55af60264270 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af606041c0 con 0x55af60243590 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af606695d0 con 0x55af6023b3b0 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd903a25740 10 client.0 _ll_register_callbacks cb 0x7fd8f0030ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51032/0 (socket says 172.21.15.55:51032) 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 -- 172.21.15.55:0/3537748679 learned_addr learned my addr 172.21.15.55:0/3537748679 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8ed400640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 -- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 msgr2=0x55af6065b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 -- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 msgr2=0x55af6065bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.276+0000 7fd8f6800640 1 -- 172.21.15.55:0/3537748679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af60564170 con 0x55af60264270 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8f6800640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00752e0 tx=0x7fd8f0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c990838e371b104a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8daa00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0084070 con 0x55af60264270 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd8f0078b20 con 0x55af60264270 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e8014b80 con 0x55af60264270 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8f0078ce0 con 0x55af60264270 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd903a25740 10 client.7083 fetch_fsmap learned FSMap version 47 2024-05-10T03:34:23.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55af60506430 con 0x55af60264270 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.277+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8014b80 con 0x55af60264270 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd8f0087020 con 0x55af60264270 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 10 client.7083 fetch_fsmap finished waiting for FSMap version 47 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 10 client.7083 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 20 client.7083 populate_metadata read hostname 'smithi055' 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af606b5e00 0x55af60603850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 4 client.7083 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 --> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af6026a6c0 con 0x55af606b5e00 2024-05-10T03:34:23.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af606b5e00 0x55af60603850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd8f5e00640 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af606b5e00 0x55af60603850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b22d04fa8a9849d server_cookie=e7e2c460120da3be in_seq=0 out_seq=0 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.278+0000 7fd8daa00640 10 client.7083 ms_handle_connect on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd8daa00640 1 -- 172.21.15.55:0/3537748679 <== mds.0 v2:172.21.15.55:6834/709318521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55af6026a6c0 con 0x55af606b5e00 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd8daa00640 10 client.7083 handle_command_reply: tid=0 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 client.7083 shutdown 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 msgr2=0x55af60603310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00752e0 tx=0x7fd8f0047000 comp rx=0 tx=0).stop 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 shutdown_connections 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55af606b5e00 0x55af60603850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55af60264270 0x55af60603310 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55af60243590 0x55af6065bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 --2- 172.21.15.55:0/3537748679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55af6023b3b0 0x55af6065b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 >> 172.21.15.55:0/3537748679 conn(0x55af602583c0 msgr2=0x55af60629e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd8daa00640 0 client.7083 ms_handle_reset on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 shutdown_connections 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 1 -- 172.21.15.55:0/3537748679 wait complete. 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.279+0000 7fd903a25740 20 client.7083 trim_cache size 0 max 16384 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6930, 2024-05-10T03:34:23.287 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.7871213859999999, 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 47, 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 90, 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 90, 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.396404217000001 2024-05-10T03:34:23.288 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 msgr2=0x7fd8d406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 0x7fd8d406d140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b4350 tx=0x7fd8e800b040 comp rx=0 tx=0).stop 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 msgr2=0x7fd8f8158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00098c0 tx=0x7fd8f00046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 shutdown_connections 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8f81f1050 0x7fd8f8158170 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f81414c0 0x7fd8f8151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8d406ac30 0x7fd8d406d140 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 --2- 172.21.15.55:0/855919470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f813e390 0x7fd8f81566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 >> 172.21.15.55:0/855919470 conn(0x7fd8f806c730 msgr2=0x7fd8f806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 shutdown_connections 2024-05-10T03:34:23.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.280+0000 7fd8f7200640 1 -- 172.21.15.55:0/855919470 wait complete. 2024-05-10T03:34:23.311 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:34:23.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.582+0000 7fe2c7200640 1 Processor -- start 2024-05-10T03:34:23.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.582+0000 7fe2c7200640 1 -- start start 2024-05-10T03:34:23.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.584+0000 7fe2c7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c80b2170 0x7fe2c815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.584+0000 7fe2c7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2c81618b0 0x7fe2c806aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.584+0000 7fe2c7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c80d25a0 con 0x7fe2c80b6680 2024-05-10T03:34:23.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.584+0000 7fe2c7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80d2b20 con 0x7fe2c81618b0 2024-05-10T03:34:23.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.584+0000 7fe2c7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815db20 con 0x7fe2c80b2170 2024-05-10T03:34:23.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2c81618b0 0x7fe2c806aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c5e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe2c80b6680 0x7fe2c81611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44846/0 (socket says 172.21.15.55:44846) 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c5e00640 1 -- 172.21.15.55:0/2760796560 learned_addr learned my addr 172.21.15.55:0/2760796560 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c5400640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c80b2170 0x7fe2c815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c6800640 1 -- 172.21.15.55:0/2760796560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c80b2170 msgr2=0x7fe2c815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c6800640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c80b2170 0x7fe2c815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c6800640 1 -- 172.21.15.55:0/2760796560 >> v1:172.21.15.55:6789/0 conn(0x7fe2c80b6680 legacy=0x7fe2c81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c4a00640 1 -- 172.21.15.55:0/2760796560 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105583726 0 0) 0x7fe2c80d25a0 con 0x7fe2c80b6680 2024-05-10T03:34:23.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.585+0000 7fe2c6800640 1 -- 172.21.15.55:0/2760796560 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80b28d0 con 0x7fe2c81618b0 2024-05-10T03:34:23.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.587+0000 7fe2c6800640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c81618b0 0x7fe2c806aab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00097a0 tx=0x7fe2c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659ed7b3284ca339 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.587+0000 7fe2c4a00640 1 -- 172.21.15.55:0/2760796560 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0032440 con 0x7fe2c81618b0 2024-05-10T03:34:23.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.587+0000 7fe2c4a00640 1 -- 172.21.15.55:0/2760796560 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c00325a0 con 0x7fe2c81618b0 2024-05-10T03:34:23.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.587+0000 7fe2c4a00640 1 -- 172.21.15.55:0/2760796560 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0032930 con 0x7fe2c81618b0 2024-05-10T03:34:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 -- 172.21.15.55:0/2760796560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c81618b0 msgr2=0x7fe2c806aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c81618b0 0x7fe2c806aab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00097a0 tx=0x7fe2c002a180 comp rx=0 tx=0).stop 2024-05-10T03:34:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 -- 172.21.15.55:0/2760796560 shutdown_connections 2024-05-10T03:34:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c80b2170 0x7fe2c815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2760796560 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c81618b0 0x7fe2c806aab0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 -- 172.21.15.55:0/2760796560 >> 172.21.15.55:0/2760796560 conn(0x7fe2c80b5730 msgr2=0x7fe2c80d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 -- 172.21.15.55:0/2760796560 shutdown_connections 2024-05-10T03:34:23.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.588+0000 7fe2c7200640 1 -- 172.21.15.55:0/2760796560 wait complete. 2024-05-10T03:34:23.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.590+0000 7fe2c7200640 1 Processor -- start 2024-05-10T03:34:23.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.591+0000 7fe2c7200640 1 -- start start 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c80b2170 0x7fe2c814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c81618b0 0x7fe2c80c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815dec0 con 0x7fe2c80b6680 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c814eae0 con 0x7fe2c80b2170 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.592+0000 7fe2c7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c815f620 con 0x7fe2c81618b0 2024-05-10T03:34:23.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51040/0 (socket says 172.21.15.55:51040) 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 -- 172.21.15.55:0/2046227419 learned_addr learned my addr 172.21.15.55:0/2046227419 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c6800640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c81618b0 0x7fe2c80c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5e00640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c80b2170 0x7fe2c814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 -- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c81618b0 msgr2=0x7fe2c80c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c81618b0 0x7fe2c80c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 -- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c80b2170 msgr2=0x7fe2c814e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c80b2170 0x7fe2c814e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.593+0000 7fe2c5400640 1 -- 172.21.15.55:0/2046227419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0009400 con 0x7fe2c80b6680 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2c5400640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007e70 tx=0x7fe2b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4014620 con 0x7fe2c80b6680 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b4014780 con 0x7fe2c80b6680 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4014b30 con 0x7fe2c80b6680 2024-05-10T03:34:23.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c80c7950 con 0x7fe2c80b6680 2024-05-10T03:34:23.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.594+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c80c7ba0 con 0x7fe2c80b6680 2024-05-10T03:34:23.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.595+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c8067740 con 0x7fe2c80b6680 2024-05-10T03:34:23.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.596+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe2b4014c90 con 0x7fe2c80b6680 2024-05-10T03:34:23.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.597+0000 7fe2be000640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 0x7fe2a806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:23.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.597+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe2b4089ee0 con 0x7fe2c80b6680 2024-05-10T03:34:23.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.598+0000 7fe2c5e00640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 0x7fe2a806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:23.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.599+0000 7fe2c5e00640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 0x7fe2a806d140 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b8007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:23.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.602+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe2b4054e50 con 0x7fe2c80b6680 2024-05-10T03:34:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.844+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2c815dc60 con 0x7fe2c80b6680 2024-05-10T03:34:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.845+0000 7fe2be000640 1 -- 172.21.15.55:0/2046227419 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2b4058470 con 0x7fe2c80b6680 2024-05-10T03:34:23.849 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:34:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 msgr2=0x7fe2a806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 0x7fe2a806d140 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b8007c00 comp rx=0 tx=0).stop 2024-05-10T03:34:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 msgr2=0x7fe2c8160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4007e70 tx=0x7fe2b400e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 shutdown_connections 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2c81618b0 0x7fe2c80c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2c80b6680 0x7fe2c8160680 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2a806ac30 0x7fe2a806d140 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 --2- 172.21.15.55:0/2046227419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2c80b2170 0x7fe2c814e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 >> 172.21.15.55:0/2046227419 conn(0x7fe2c80b5730 msgr2=0x7fe2c80b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 shutdown_connections 2024-05-10T03:34:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:23.848+0000 7fe2c7200640 1 -- 172.21.15.55:0/2046227419 wait complete. 2024-05-10T03:34:23.878 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:34:23.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:34:23.879 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:34:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:23.879 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:34:23.933 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:23.935 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.937 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:34:23.938 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:34:23.938 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:34:23.938 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:23.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:23.938 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:34:24.021 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:34:24.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.026 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:34:24.076 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:34:24.076 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:34:24.077 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:34:24.077 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:34:24.077 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:34:24.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.077 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:34:24.077 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:34:24.077 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:34:24.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:34:24.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:34:24.078 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:34:24.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.222 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:34:24.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:34:24.276 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:34:24.277 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:34:24.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.277 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:34:24.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:34:24.367 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:34:24.425 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:34:24.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:34:24.425 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:34:24.480 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:34:24.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.480 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:34:24.688 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:34:24.688 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:34:24.688 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:34:24.716 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:34:24.716 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:34:24.716 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:34:24.716 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:34:24.716 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:34:24.757 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:34:24.758 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:34:24.759 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:34:24.759 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:34:24.759 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:34:24.759 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:34:24.759 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:34:24.763 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:34:24.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.763 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:34:24.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:24.844 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:34:24.906 INFO:teuthology.orchestra.run.smithi055.stdout:client7095 2024-05-10T03:34:24.906 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:24.906 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:24.960 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:34:25.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.245+0000 7f9991a00640 1 Processor -- start 2024-05-10T03:34:25.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.245+0000 7f9991a00640 1 -- start start 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.247+0000 7f9991a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 0x7f998c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.247+0000 7f9991a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f998c0b3190 0x7f998c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.247+0000 7f9991a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c06d350 con 0x7f998c144710 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.247+0000 7f9991a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c144530 con 0x7f998c0b3190 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.247+0000 7f9991a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c06dd20 con 0x7f998c144120 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.248+0000 7f9991000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f998c0b3190 0x7f998c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.248+0000 7f998b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 0x7f998c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.248+0000 7f998be00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f998c144710 0x7f998c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:57384/0 (socket says 172.21.15.55:57384) 2024-05-10T03:34:25.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.248+0000 7f998be00640 1 -- 172.21.15.55:0/4280829547 learned_addr learned my addr 172.21.15.55:0/4280829547 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.249+0000 7f9991000640 1 -- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 msgr2=0x7f998c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.249+0000 7f9991000640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 0x7f998c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.249+0000 7f9991000640 1 -- 172.21.15.55:0/4280829547 >> v1:172.21.15.162:6789/0 conn(0x7f998c144710 legacy=0x7f998c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.249+0000 7f9991000640 1 -- 172.21.15.55:0/4280829547 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c06c740 con 0x7f998c0b3190 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.249+0000 7f998b400640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 0x7f998c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:25.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.250+0000 7f9991000640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c0b3190 0x7f998c0cd9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99840097a0 tx=0x7f998402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f5223bf5635f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:25.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.250+0000 7f998aa00640 1 -- 172.21.15.55:0/4280829547 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998402adb0 con 0x7f998c0b3190 2024-05-10T03:34:25.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f998aa00640 1 -- 172.21.15.55:0/4280829547 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9984032440 con 0x7f998c0b3190 2024-05-10T03:34:25.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f998aa00640 1 -- 172.21.15.55:0/4280829547 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840327d0 con 0x7f998c0b3190 2024-05-10T03:34:25.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 -- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c0b3190 msgr2=0x7f998c0cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c0b3190 0x7f998c0cd9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99840097a0 tx=0x7f998402a060 comp rx=0 tx=0).stop 2024-05-10T03:34:25.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 -- 172.21.15.55:0/4280829547 shutdown_connections 2024-05-10T03:34:25.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144120 0x7f998c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 --2- 172.21.15.55:0/4280829547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c0b3190 0x7f998c0cd9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.251+0000 7f9991a00640 1 -- 172.21.15.55:0/4280829547 >> 172.21.15.55:0/4280829547 conn(0x7f998c06a110 msgr2=0x7f998c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:25.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.252+0000 7f9991a00640 1 -- 172.21.15.55:0/4280829547 shutdown_connections 2024-05-10T03:34:25.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.252+0000 7f9991a00640 1 -- 172.21.15.55:0/4280829547 wait complete. 2024-05-10T03:34:25.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.254+0000 7f9991a00640 1 Processor -- start 2024-05-10T03:34:25.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.254+0000 7f9991a00640 1 -- start start 2024-05-10T03:34:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 0x7f998c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144710 0x7f998c06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c141180 con 0x7f998c0b3190 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1345b0 con 0x7f998c144120 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f9991a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c0adb60 con 0x7f998c144710 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f998be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f998be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51056/0 (socket says 172.21.15.55:51056) 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.256+0000 7f998be00640 1 -- 172.21.15.55:0/1838540576 learned_addr learned my addr 172.21.15.55:0/1838540576 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f9991000640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144710 0x7f998c06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998b400640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 0x7f998c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 -- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144710 msgr2=0x7f998c06fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144710 0x7f998c06fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 -- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 msgr2=0x7f998c15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 0x7f998c15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 -- 172.21.15.55:0/1838540576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974009400 con 0x7f998c0b3190 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998b400640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 0x7f998c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f998be00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f997c00cad0 tx=0x7f997c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00abd0 con 0x7f998c0b3190 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.257+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c00ad30 con 0x7f998c0b3190 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.258+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9984009400 con 0x7f998c0b3190 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.258+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c012620 con 0x7f998c0b3190 2024-05-10T03:34:25.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.258+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c141700 con 0x7f998c0b3190 2024-05-10T03:34:25.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.259+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c141890 con 0x7f998c0b3190 2024-05-10T03:34:25.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.260+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f997c007070 con 0x7f998c0b3190 2024-05-10T03:34:25.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.261+0000 7f9983e00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 0x7f996806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:25.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.261+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f997c0894e0 con 0x7f998c0b3190 2024-05-10T03:34:25.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.262+0000 7f998b400640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 0x7f996806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:25.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.263+0000 7f998b400640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 0x7f996806d140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f998c141040 tx=0x7f9974008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:25.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.266+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f997c054460 con 0x7f998c0b3190 2024-05-10T03:34:25.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.509+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f998c066ce0 con 0x7f998c0b3190 2024-05-10T03:34:25.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.510+0000 7f9983e00640 1 -- 172.21.15.55:0/1838540576 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f997c057a80 con 0x7f998c0b3190 2024-05-10T03:34:25.514 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:34:25.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 msgr2=0x7f996806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 0x7f996806d140 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f998c141040 tx=0x7f9974008040 comp rx=0 tx=0).stop 2024-05-10T03:34:25.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 msgr2=0x7f998c0ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:25.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f997c00cad0 tx=0x7f997c00cf90 comp rx=0 tx=0).stop 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 shutdown_connections 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f998c144710 0x7f998c06fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f996806ac30 0x7f996806d140 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f998c144120 0x7f998c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 --2- 172.21.15.55:0/1838540576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0b3190 0x7f998c0ad2f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.514+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 >> 172.21.15.55:0/1838540576 conn(0x7f998c06a110 msgr2=0x7f998c143110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.515+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 shutdown_connections 2024-05-10T03:34:25.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:25.515+0000 7f9991a00640 1 -- 172.21.15.55:0/1838540576 wait complete. 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:34:25.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:34:25.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:34:25.548 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:34:25.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.548 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.607 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42211sec preferred_lft 42211sec 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:34:25.608 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:34:25.609 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:34:25.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.610 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:34:25.694 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:34:25.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.698 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:34:25.749 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:34:25.749 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:34:25.750 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:34:25.750 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:34:25.750 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:34:25.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.750 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:34:25.750 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:34:25.750 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:34:25.750 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:34:25.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:34:25.751 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:34:25.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.897 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:34:25.943 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:34:25.943 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:34:25.943 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:34:25.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:25.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:34:26.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:34:26.073 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:34:26.124 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:34:26.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:34:26.124 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:34:26.173 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:34:26.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:26.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:34:26.375 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:34:26.375 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:34:26.375 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:34:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:34:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:34:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:34:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:34:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:34:26.438 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:34:26.439 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:34:26.440 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:34:26.440 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:34:26.441 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:34:26.441 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:34:26.441 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:34:26.445 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:34:26.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:26.446 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:34:26.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:26.515 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:34:26.568 INFO:teuthology.orchestra.run.smithi162.stdout:client6010 2024-05-10T03:34:26.568 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:26.568 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:26.618 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:26.618 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:26.681 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:26.681 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:26.737 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:27.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.022+0000 7efe5b400640 1 Processor -- start 2024-05-10T03:34:27.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.023+0000 7efe5b400640 1 -- start start 2024-05-10T03:34:27.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.025+0000 7efe5b400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efe540b3710 0x7efe5416cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.025+0000 7efe5b400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540715e0 con 0x7efe541659c0 2024-05-10T03:34:27.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.025+0000 7efe5b400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe541667f0 con 0x7efe5406b6e0 2024-05-10T03:34:27.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.025+0000 7efe5b400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540711c0 con 0x7efe540b3710 2024-05-10T03:34:27.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.026+0000 7efe4be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efe541659c0 0x7efe54165dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44848/0 (socket says 172.21.15.55:44848) 2024-05-10T03:34:27.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.026+0000 7efe4be00640 1 -- 172.21.15.55:0/558280519 learned_addr learned my addr 172.21.15.55:0/558280519 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.026+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46107678 0 0) 0x7efe44001110 con 0x7efe541659c0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.026+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe34003520 con 0x7efe541659c0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.026+0000 7efe59400640 1 --2- 172.21.15.55:0/558280519 >> v2:172.21.15.162:3300/0 conn(0x7efe540b3710 0x7efe5416cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025698983 0 0) 0x7efe34003520 con 0x7efe541659c0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 >> v2:172.21.15.162:3300/0 conn(0x7efe540b3710 msgr2=0x7efe5416cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 --2- 172.21.15.55:0/558280519 >> v2:172.21.15.162:3300/0 conn(0x7efe540b3710 0x7efe5416cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 >> v1:172.21.15.162:6789/0 conn(0x7efe5406b6e0 legacy=0x7efe540ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54070260 con 0x7efe541659c0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7efe44001800 con 0x7efe541659c0 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 >> v1:172.21.15.55:6789/0 conn(0x7efe541659c0 legacy=0x7efe54165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 0x7efe34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe340047c0 0x7efe34004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe340052c0 0x7efe3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540711c0 con 0x7efe34003970 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400c260 con 0x7efe340052c0 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3400c420 con 0x7efe340047c0 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.027+0000 7efe58a00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 0x7efe34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe4be00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe340047c0 0x7efe34004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe58a00640 1 -- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe340047c0 msgr2=0x7efe34004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe58a00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe340047c0 0x7efe34004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe58a00640 1 -- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe340052c0 msgr2=0x7efe3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe58a00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe340052c0 0x7efe3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe58a00640 1 -- 172.21.15.55:0/558280519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34003520 con 0x7efe34003970 2024-05-10T03:34:27.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.028+0000 7efe59400640 1 -- 172.21.15.55:0/558280519 reap_dead start 2024-05-10T03:34:27.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.029+0000 7efe58a00640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 0x7efe34003de0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001720 tx=0x7efe340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19189a1257f45c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.029+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0052d0 con 0x7efe34003970 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.029+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c005430 con 0x7efe34003970 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.029+0000 7efe53e00640 1 -- 172.21.15.55:0/558280519 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c02ab90 con 0x7efe34003970 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 -- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 msgr2=0x7efe34003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 0x7efe34003de0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001720 tx=0x7efe340057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 -- 172.21.15.55:0/558280519 shutdown_connections 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 --2- 172.21.15.55:0/558280519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe34003970 0x7efe34003de0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 -- 172.21.15.55:0/558280519 >> 172.21.15.55:0/558280519 conn(0x7efe540cbb90 msgr2=0x7efe540cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 -- 172.21.15.55:0/558280519 shutdown_connections 2024-05-10T03:34:27.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.030+0000 7efe5b400640 1 -- 172.21.15.55:0/558280519 wait complete. 2024-05-10T03:34:27.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.033+0000 7efe5b400640 1 Processor -- start 2024-05-10T03:34:27.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.033+0000 7efe5b400640 1 -- start start 2024-05-10T03:34:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.034+0000 7efe5b400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 0x7efe5414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe5b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 0x7efe540a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe5b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe5b400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe541600d0 con 0x7efe540a9650 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe5b400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe54161a10 con 0x7efe540a84d0 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe5b400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5415e6e0 con 0x7efe34003970 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51070/0 (socket says 172.21.15.55:51070) 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 -- 172.21.15.55:0/1032917473 learned_addr learned my addr 172.21.15.55:0/1032917473 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe58a00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 0x7efe5414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe4be00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 0x7efe540a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 -- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 msgr2=0x7efe5414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 0x7efe5414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 -- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 msgr2=0x7efe540a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 0x7efe540a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe59400640 1 -- 172.21.15.55:0/1032917473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54150370 con 0x7efe540a9650 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe4be00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 0x7efe540a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:27.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.035+0000 7efe58a00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 0x7efe5414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe59400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efe540b4240 tx=0x7efe4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00edc0 con 0x7efe540a9650 2024-05-10T03:34:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c014420 con 0x7efe540a9650 2024-05-10T03:34:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0147d0 con 0x7efe540a9650 2024-05-10T03:34:27.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5415e930 con 0x7efe540a9650 2024-05-10T03:34:27.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.036+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5415eef0 con 0x7efe540a9650 2024-05-10T03:34:27.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.038+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe54066f60 con 0x7efe540a9650 2024-05-10T03:34:27.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.039+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efe4c014930 con 0x7efe540a9650 2024-05-10T03:34:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.039+0000 7efe51600640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 0x7efe3006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.040+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7efe4c089b20 con 0x7efe540a9650 2024-05-10T03:34:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.040+0000 7efe58a00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 0x7efe3006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.041+0000 7efe58a00640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 0x7efe3006d110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efe3c032000 tx=0x7efe3c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.044+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efe4c054b30 con 0x7efe540a9650 2024-05-10T03:34:27.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.246+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe54067220 con 0x7efe540a9650 2024-05-10T03:34:27.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.248+0000 7efe51600640 1 -- 172.21.15.55:0/1032917473 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16443 (secure 0 0 0) 0x7efe4c058150 con 0x7efe540a9650 2024-05-10T03:34:27.253 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:27.253 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:25.020639+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:27.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 msgr2=0x7efe3006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 0x7efe3006d110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efe3c032000 tx=0x7efe3c02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 msgr2=0x7efe541fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efe540b4240 tx=0x7efe4c00e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 shutdown_connections 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe540a9650 0x7efe541fb800 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efe3006ac00 0x7efe3006d110 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe540a84d0 0x7efe540a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 --2- 172.21.15.55:0/1032917473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe34003970 0x7efe5414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.251+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 >> 172.21.15.55:0/1032917473 conn(0x7efe540cbb90 msgr2=0x7efe54165dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.252+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 shutdown_connections 2024-05-10T03:34:27.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.252+0000 7efe5b400640 1 -- 172.21.15.55:0/1032917473 wait complete. 2024-05-10T03:34:27.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:13.758492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:14.516676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:27.287 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:27.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.569+0000 7eff32800640 1 Processor -- start 2024-05-10T03:34:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.569+0000 7eff32800640 1 -- start start 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff32800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7eff2c0d1a40 0x7eff2c0b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff32800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff2c0d2920 0x7eff2c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff32800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c0d2ed0 con 0x7eff2c0b3e10 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff32800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c15db20 con 0x7eff2c0d2920 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff32800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c0b2b50 con 0x7eff2c0d1a40 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.571+0000 7eff30a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff2c0d2920 0x7eff2c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7eff2c0d2920 0x7eff2c0b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51074/0 (socket says 172.21.15.55:51074) 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 -- 172.21.15.55:0/2364891820 learned_addr learned my addr 172.21.15.55:0/2364891820 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff31e00640 1 --1- 172.21.15.55:0/2364891820 >> v1:172.21.15.55:6789/0 conn(0x7eff2c0b3e10 0x7eff2c0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44850/0 (socket says 172.21.15.55:44850) 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff31400640 1 --2- 172.21.15.55:0/2364891820 >> v2:172.21.15.162:3300/0 conn(0x7eff2c0d1a40 0x7eff2c0b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 -- 172.21.15.55:0/2364891820 >> v2:172.21.15.162:3300/0 conn(0x7eff2c0d1a40 msgr2=0x7eff2c0b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 --2- 172.21.15.55:0/2364891820 >> v2:172.21.15.162:3300/0 conn(0x7eff2c0d1a40 0x7eff2c0b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 -- 172.21.15.55:0/2364891820 >> v1:172.21.15.55:6789/0 conn(0x7eff2c0b3e10 legacy=0x7eff2c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff30a00640 1 -- 172.21.15.55:0/2364891820 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0b2df0 con 0x7eff2c0d2920 2024-05-10T03:34:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.572+0000 7eff2be00640 1 -- 172.21.15.55:0/2364891820 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925532841 0 0) 0x7eff2c0d2ed0 con 0x7eff2c0b3e10 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.573+0000 7eff30a00640 1 --2- 172.21.15.55:0/2364891820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d2920 0x7eff2c0b38d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7eff2c08bdb0 tx=0x7eff1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f67c6a4eea69f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.574+0000 7eff2be00640 1 -- 172.21.15.55:0/2364891820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18032440 con 0x7eff2c0d2920 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.574+0000 7eff2be00640 1 -- 172.21.15.55:0/2364891820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180325a0 con 0x7eff2c0d2920 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.574+0000 7eff2be00640 1 -- 172.21.15.55:0/2364891820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18032930 con 0x7eff2c0d2920 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.574+0000 7eff32800640 1 -- 172.21.15.55:0/2364891820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d2920 msgr2=0x7eff2c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.574+0000 7eff32800640 1 --2- 172.21.15.55:0/2364891820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d2920 0x7eff2c0b38d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7eff2c08bdb0 tx=0x7eff1802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 -- 172.21.15.55:0/2364891820 shutdown_connections 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 --2- 172.21.15.55:0/2364891820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d2920 0x7eff2c0b38d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 --2- 172.21.15.55:0/2364891820 >> v2:172.21.15.162:3300/0 conn(0x7eff2c0d1a40 0x7eff2c0b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 -- 172.21.15.55:0/2364891820 >> 172.21.15.55:0/2364891820 conn(0x7eff2c0d04d0 msgr2=0x7eff2c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 -- 172.21.15.55:0/2364891820 shutdown_connections 2024-05-10T03:34:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.575+0000 7eff32800640 1 -- 172.21.15.55:0/2364891820 wait complete. 2024-05-10T03:34:27.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.577+0000 7eff32800640 1 Processor -- start 2024-05-10T03:34:27.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.577+0000 7eff32800640 1 -- start start 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 0x7eff2c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2c0d2920 0x7eff2c06b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1f3690 con 0x7eff2c0d1a40 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1f75c0 con 0x7eff2c0d2920 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff32800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c15b010 con 0x7eff2c0b3e10 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff30a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff30a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51086/0 (socket says 172.21.15.55:51086) 2024-05-10T03:34:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff30a00640 1 -- 172.21.15.55:0/2782795176 learned_addr learned my addr 172.21.15.55:0/2782795176 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.579+0000 7eff31e00640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2c0d2920 0x7eff2c06b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff31400640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 0x7eff2c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 -- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 msgr2=0x7eff2c157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 0x7eff2c157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 -- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2c0d2920 msgr2=0x7eff2c06b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2c0d2920 0x7eff2c06b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 -- 172.21.15.55:0/2782795176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180094d0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff31400640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 0x7eff2c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff30a00640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7eff1802a220 tx=0x7eff180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.580+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18032440 con 0x7eff2c0d1a40 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.581+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18030ba0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.581+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1803d440 con 0x7eff2c0d1a40 2024-05-10T03:34:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.581+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c157aa0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.581+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1f7af0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.582+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c04e290 con 0x7eff2c0d1a40 2024-05-10T03:34:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.583+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7eff18039040 con 0x7eff2c0d1a40 2024-05-10T03:34:27.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.584+0000 7eff29600640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 0x7eff0c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:27.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.584+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7eff180aa170 con 0x7eff2c0d1a40 2024-05-10T03:34:27.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.585+0000 7eff31400640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 0x7eff0c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:27.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.586+0000 7eff31400640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 0x7eff0c06d110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7eff20006fd0 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.589+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7eff180750f0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.785+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff2c066cc0 con 0x7eff2c0d1a40 2024-05-10T03:34:27.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.786+0000 7eff29600640 1 -- 172.21.15.55:0/2782795176 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff18078710 con 0x7eff2c0d1a40 2024-05-10T03:34:27.791 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 msgr2=0x7eff0c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 0x7eff0c06d110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7eff20006fd0 tx=0x7eff20008040 comp rx=0 tx=0).stop 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 msgr2=0x7eff2c1f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7eff1802a220 tx=0x7eff180046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 shutdown_connections 2024-05-10T03:34:27.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0c06ac00 0x7eff0c06d110 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2c0d2920 0x7eff2c06b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff2c0d1a40 0x7eff2c1f6c00 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 --2- 172.21.15.55:0/2782795176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2c0b3e10 0x7eff2c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 >> 172.21.15.55:0/2782795176 conn(0x7eff2c0d04d0 msgr2=0x7eff2c0ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 shutdown_connections 2024-05-10T03:34:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:27.789+0000 7eff32800640 1 -- 172.21.15.55:0/2782795176 wait complete. 2024-05-10T03:34:27.822 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:34:28.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.093+0000 7f3229800640 1 Processor -- start 2024-05-10T03:34:28.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.094+0000 7f3229800640 1 -- start start 2024-05-10T03:34:28.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.096+0000 7f3229800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f322413fb90 0x7f32240cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.096+0000 7f3229800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b530 0x7f32240cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.096+0000 7f3229800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32240ce150 con 0x7f322406b120 2024-05-10T03:34:28.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.096+0000 7f3229800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240ce420 con 0x7f322413fb90 2024-05-10T03:34:28.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.096+0000 7f3229800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32240ce700 con 0x7f322406b530 2024-05-10T03:34:28.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.097+0000 7f3223400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b530 0x7f32240cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.097+0000 7f3222a00640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f322406b120 0x7f322413f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.55:57412/0 (socket says 172.21.15.55:57412) 2024-05-10T03:34:28.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.097+0000 7f3222a00640 1 -- 172.21.15.55:0/157580666 learned_addr learned my addr 172.21.15.55:0/157580666 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:28.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.097+0000 7f321a000640 1 --2- 172.21.15.55:0/157580666 >> v2:172.21.15.162:3300/0 conn(0x7f322413fb90 0x7f32240cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.098+0000 7f321a000640 1 -- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b530 msgr2=0x7f32240cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:28.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.098+0000 7f321a000640 1 --2- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b530 0x7f32240cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.098+0000 7f321a000640 1 -- 172.21.15.55:0/157580666 >> v1:172.21.15.162:6789/0 conn(0x7f322406b120 legacy=0x7f322413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.098+0000 7f321a000640 1 -- 172.21.15.55:0/157580666 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240ce9e0 con 0x7f322413fb90 2024-05-10T03:34:28.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.099+0000 7f321a000640 1 --2- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322413fb90 0x7f32240cd6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097a0 tx=0x7f320c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf868789b4419d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.100+0000 7f3222000640 1 -- 172.21.15.55:0/157580666 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c02ae50 con 0x7f322413fb90 2024-05-10T03:34:28.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.100+0000 7f3222000640 1 -- 172.21.15.55:0/157580666 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c032440 con 0x7f322413fb90 2024-05-10T03:34:28.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.100+0000 7f3222000640 1 -- 172.21.15.55:0/157580666 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c0327d0 con 0x7f322413fb90 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 -- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322413fb90 msgr2=0x7f32240cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 --2- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322413fb90 0x7f32240cd6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097a0 tx=0x7f320c02a100 comp rx=0 tx=0).stop 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 -- 172.21.15.55:0/157580666 shutdown_connections 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 --2- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b530 0x7f32240cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 --2- 172.21.15.55:0/157580666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322413fb90 0x7f32240cd6d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 -- 172.21.15.55:0/157580666 >> 172.21.15.55:0/157580666 conn(0x7f322406c120 msgr2=0x7f322406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 -- 172.21.15.55:0/157580666 shutdown_connections 2024-05-10T03:34:28.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.101+0000 7f3229800640 1 -- 172.21.15.55:0/157580666 wait complete. 2024-05-10T03:34:28.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.103+0000 7f3229800640 1 Processor -- start 2024-05-10T03:34:28.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.104+0000 7f3229800640 1 -- start start 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 0x7f3224136860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322406b530 0x7f32240b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322415a310 con 0x7f3224151600 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224158f20 con 0x7f322406b530 2024-05-10T03:34:28.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3229800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3224153c70 con 0x7f322406b120 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3223400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3223400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51096/0 (socket says 172.21.15.55:51096) 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.105+0000 7f3223400640 1 -- 172.21.15.55:0/700430585 learned_addr learned my addr 172.21.15.55:0/700430585 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f321a000640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322406b530 0x7f32240b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3222a00640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 0x7f3224136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 -- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 msgr2=0x7f3224136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 0x7f3224136860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 -- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322406b530 msgr2=0x7f32240b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322406b530 0x7f32240b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 -- 172.21.15.55:0/700430585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c009400 con 0x7f3224151600 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3222a00640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 0x7f3224136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:28.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.106+0000 7f3223400640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f321c00efc0 tx=0x7f321c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.107+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c009b20 con 0x7f3224151600 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.107+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c00f040 con 0x7f3224151600 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.107+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0146c0 con 0x7f3224151600 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.107+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3224151f50 con 0x7f3224151600 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.107+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3224159450 con 0x7f3224151600 2024-05-10T03:34:28.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.108+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224066f60 con 0x7f3224151600 2024-05-10T03:34:28.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.110+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f321c014820 con 0x7f3224151600 2024-05-10T03:34:28.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.110+0000 7f321b400640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 0x7f320006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.111+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f321c08a2f0 con 0x7f3224151600 2024-05-10T03:34:28.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.111+0000 7f3222a00640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 0x7f320006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.113+0000 7f3222a00640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 0x7f320006d160 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3214005880 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.115+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f321c0552f0 con 0x7f3224151600 2024-05-10T03:34:28.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.310+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3224153de0 con 0x7f3224151600 2024-05-10T03:34:28.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.311+0000 7f321b400640 1 -- 172.21.15.55:0/700430585 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f321c009e60 con 0x7f3224151600 2024-05-10T03:34:28.315 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 msgr2=0x7f320006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 0x7f320006d160 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3214005880 tx=0x7f3214009040 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 msgr2=0x7f3224151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f321c00efc0 tx=0x7f321c00c490 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 shutdown_connections 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224151600 0x7f3224151a10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f320006ac50 0x7f320006d160 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322406b530 0x7f32240b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 --2- 172.21.15.55:0/700430585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322406b120 0x7f3224136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 >> 172.21.15.55:0/700430585 conn(0x7f322406c120 msgr2=0x7f3224139850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 shutdown_connections 2024-05-10T03:34:28.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.314+0000 7f3229800640 1 -- 172.21.15.55:0/700430585 wait complete. 2024-05-10T03:34:28.344 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:34:28.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.615+0000 7f102aa00640 1 Processor -- start 2024-05-10T03:34:28.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.615+0000 7f102aa00640 1 -- start start 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.617+0000 7f102aa00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f10240cdba0 0x7f102406a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.617+0000 7f102aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1024071cf0 0x7f10240b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.617+0000 7f102aa00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240cd710 con 0x7f102415ab30 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.617+0000 7f102aa00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10240cdfb0 con 0x7f1024071cf0 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.617+0000 7f102aa00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102406b4d0 con 0x7f10240cdba0 2024-05-10T03:34:28.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1024071cf0 0x7f10240b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1024071cf0 0x7f10240b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51106/0 (socket says 172.21.15.55:51106) 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 -- 172.21.15.55:0/4029684732 learned_addr learned my addr 172.21.15.55:0/4029684732 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 -- 172.21.15.55:0/4029684732 >> v2:172.21.15.162:3300/0 conn(0x7f10240cdba0 msgr2=0x7f102406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 --2- 172.21.15.55:0/4029684732 >> v2:172.21.15.162:3300/0 conn(0x7f10240cdba0 0x7f102406a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.618+0000 7f1028a00640 1 -- 172.21.15.55:0/4029684732 >> v1:172.21.15.162:6789/0 conn(0x7f102415ab30 legacy=0x7f10240715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:28.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.619+0000 7f1028a00640 1 -- 172.21.15.55:0/4029684732 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102406b1f0 con 0x7f1024071cf0 2024-05-10T03:34:28.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.620+0000 7f1028a00640 1 --2- 172.21.15.55:0/4029684732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1024071cf0 0x7f10240b2fa0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f102406bcb0 tx=0x7f101c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9a2f7860decd7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.620+0000 7f1022a00640 1 -- 172.21.15.55:0/4029684732 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c02ae10 con 0x7f1024071cf0 2024-05-10T03:34:28.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.620+0000 7f1022a00640 1 -- 172.21.15.55:0/4029684732 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c032440 con 0x7f1024071cf0 2024-05-10T03:34:28.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.620+0000 7f1022a00640 1 -- 172.21.15.55:0/4029684732 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0327d0 con 0x7f1024071cf0 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 -- 172.21.15.55:0/4029684732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1024071cf0 msgr2=0x7f10240b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 --2- 172.21.15.55:0/4029684732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1024071cf0 0x7f10240b2fa0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f102406bcb0 tx=0x7f101c02a100 comp rx=0 tx=0).stop 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 -- 172.21.15.55:0/4029684732 shutdown_connections 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 --2- 172.21.15.55:0/4029684732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1024071cf0 0x7f10240b2fa0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 --2- 172.21.15.55:0/4029684732 >> v2:172.21.15.162:3300/0 conn(0x7f10240cdba0 0x7f102406a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 -- 172.21.15.55:0/4029684732 >> 172.21.15.55:0/4029684732 conn(0x7f10240cbb90 msgr2=0x7f10240cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 -- 172.21.15.55:0/4029684732 shutdown_connections 2024-05-10T03:34:28.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.621+0000 7f102aa00640 1 -- 172.21.15.55:0/4029684732 wait complete. 2024-05-10T03:34:28.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.624+0000 7f102aa00640 1 Processor -- start 2024-05-10T03:34:28.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.624+0000 7f102aa00640 1 -- start start 2024-05-10T03:34:28.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 0x7f102415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10240ac480 0x7f10240ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240a9810 con 0x7f10240cdba0 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240ad140 con 0x7f10240ac480 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f102aa00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102415ca80 con 0x7f102415ab30 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f1023e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f1023e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51112/0 (socket says 172.21.15.55:51112) 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f1023e00640 1 -- 172.21.15.55:0/2100301012 learned_addr learned my addr 172.21.15.55:0/2100301012 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.625+0000 7f1023e00640 1 -- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 msgr2=0x7f102415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1028a00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10240ac480 0x7f10240ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023400640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 0x7f102415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023e00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 0x7f102415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023e00640 1 -- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10240ac480 msgr2=0x7f10240ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023e00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10240ac480 0x7f10240ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023e00640 1 -- 172.21.15.55:0/2100301012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0094d0 con 0x7f10240cdba0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023400640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 0x7f102415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f1023e00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f101400ba50 tx=0x7f101400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400cbe0 con 0x7f10240cdba0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400cd60 con 0x7f10240cdba0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10240ad390 con 0x7f10240cdba0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014014680 con 0x7f10240cdba0 2024-05-10T03:34:28.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.626+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102415c710 con 0x7f10240cdba0 2024-05-10T03:34:28.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.627+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024066f60 con 0x7f10240cdba0 2024-05-10T03:34:28.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.629+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1014007370 con 0x7f10240cdba0 2024-05-10T03:34:28.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.629+0000 7f101be00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 0x7f100006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.630+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f10140894c0 con 0x7f10240cdba0 2024-05-10T03:34:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.630+0000 7f1023400640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 0x7f100006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:28.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.632+0000 7f1023400640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 0x7f100006d110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f10240b44a0 tx=0x7f100c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:28.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.634+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10140544e0 con 0x7f10240cdba0 2024-05-10T03:34:28.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.833+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1024067220 con 0x7f10240cdba0 2024-05-10T03:34:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.834+0000 7f101be00640 1 -- 172.21.15.55:0/2100301012 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1014057b00 con 0x7f10240cdba0 2024-05-10T03:34:28.838 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 msgr2=0x7f100006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 0x7f100006d110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f10240b44a0 tx=0x7f100c008040 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 msgr2=0x7f102414fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f101400ba50 tx=0x7f101400bf10 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 shutdown_connections 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f100006ac00 0x7f100006d110 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10240ac480 0x7f10240ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f102415ab30 0x7f102415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 --2- 172.21.15.55:0/2100301012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10240cdba0 0x7f102414fda0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:28.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 >> 172.21.15.55:0/2100301012 conn(0x7f10240cbb90 msgr2=0x7f102406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:28.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 shutdown_connections 2024-05-10T03:34:28.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:28.837+0000 7f102aa00640 1 -- 172.21.15.55:0/2100301012 wait complete. 2024-05-10T03:34:28.867 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:34:29.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.140+0000 7f304f200640 1 Processor -- start 2024-05-10T03:34:29.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.140+0000 7f304f200640 1 -- start start 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f304f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30480ceb50 0x7f304815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f304f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304806bcb0 con 0x7f30480b6bf0 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f304f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304806bf90 con 0x7f30480cdef0 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f304f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304806c270 con 0x7f30480ceb50 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f3047e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f30480b6bf0 0x7f30480ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44860/0 (socket says 172.21.15.55:44860) 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f3047e00640 1 -- 172.21.15.55:0/3080057513 learned_addr learned my addr 172.21.15.55:0/3080057513 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:29.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.143+0000 7f3047400640 1 --2- 172.21.15.55:0/3080057513 >> v2:172.21.15.55:3300/0 conn(0x7f30480ceb50 0x7f304815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.144+0000 7f3046a00640 1 -- 172.21.15.55:0/3080057513 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599020554 0 0) 0x7f304806bcb0 con 0x7f30480b6bf0 2024-05-10T03:34:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.144+0000 7f3046a00640 1 -- 172.21.15.55:0/3080057513 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f302c003520 con 0x7f30480b6bf0 2024-05-10T03:34:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.144+0000 7f3047400640 1 -- 172.21.15.55:0/3080057513 >> v1:172.21.15.162:6789/0 conn(0x7f30480cdef0 legacy=0x7f30480d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.144+0000 7f3047400640 1 -- 172.21.15.55:0/3080057513 >> v1:172.21.15.55:6789/0 conn(0x7f30480b6bf0 legacy=0x7f30480ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.144+0000 7f3047400640 1 -- 172.21.15.55:0/3080057513 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480d1360 con 0x7f30480ceb50 2024-05-10T03:34:29.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.145+0000 7f3047400640 1 --2- 172.21.15.55:0/3080057513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30480ceb50 0x7f304815dd50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f30300097a0 tx=0x7f303002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb92285c62eac403 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.145+0000 7f3046a00640 1 -- 172.21.15.55:0/3080057513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030032640 con 0x7f30480ceb50 2024-05-10T03:34:29.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f3046a00640 1 -- 172.21.15.55:0/3080057513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300327a0 con 0x7f30480ceb50 2024-05-10T03:34:29.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f3046a00640 1 -- 172.21.15.55:0/3080057513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030032b30 con 0x7f30480ceb50 2024-05-10T03:34:29.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f304f200640 1 -- 172.21.15.55:0/3080057513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30480ceb50 msgr2=0x7f304815dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f304f200640 1 --2- 172.21.15.55:0/3080057513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30480ceb50 0x7f304815dd50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f30300097a0 tx=0x7f303002a2c0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f304f200640 1 -- 172.21.15.55:0/3080057513 shutdown_connections 2024-05-10T03:34:29.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f304f200640 1 --2- 172.21.15.55:0/3080057513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30480ceb50 0x7f304815dd50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.146+0000 7f304f200640 1 -- 172.21.15.55:0/3080057513 >> 172.21.15.55:0/3080057513 conn(0x7f30480d32c0 msgr2=0x7f30480d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:29.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.147+0000 7f304f200640 1 -- 172.21.15.55:0/3080057513 shutdown_connections 2024-05-10T03:34:29.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.147+0000 7f304f200640 1 -- 172.21.15.55:0/3080057513 wait complete. 2024-05-10T03:34:29.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.149+0000 7f304f200640 1 Processor -- start 2024-05-10T03:34:29.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.149+0000 7f304f200640 1 -- start start 2024-05-10T03:34:29.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.150+0000 7f304f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30480b6bf0 0x7f304814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.150+0000 7f304f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 0x7f30480b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30481f53d0 con 0x7f30481f4cd0 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480b59e0 con 0x7f30480cdef0 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480b3780 con 0x7f30480b6bf0 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51126/0 (socket says 172.21.15.55:51126) 2024-05-10T03:34:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 -- 172.21.15.55:0/1003253809 learned_addr learned my addr 172.21.15.55:0/1003253809 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:29.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f3047e00640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30480b6bf0 0x7f304814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f3047400640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 0x7f30480b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 -- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30480b6bf0 msgr2=0x7f304814e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30480b6bf0 0x7f304814e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 -- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 msgr2=0x7f30480b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 0x7f30480b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f304d200640 1 -- 172.21.15.55:0/1003253809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3030009400 con 0x7f30481f4cd0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.151+0000 7f3047400640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 0x7f30480b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f304d200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400052d0 con 0x7f30481f4cd0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040005430 con 0x7f30481f4cd0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040003830 con 0x7f30481f4cd0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480b5c30 con 0x7f30481f4cd0 2024-05-10T03:34:29.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.152+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30480b3130 con 0x7f30481f4cd0 2024-05-10T03:34:29.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.154+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048067220 con 0x7f30481f4cd0 2024-05-10T03:34:29.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.154+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3040003990 con 0x7f30481f4cd0 2024-05-10T03:34:29.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.155+0000 7f303fe00640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 0x7f302406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.155+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3040088ba0 con 0x7f30481f4cd0 2024-05-10T03:34:29.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.156+0000 7f3047e00640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 0x7f302406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.157+0000 7f3047e00640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 0x7f302406d110 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3038001560 tx=0x7f3038009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.160+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3040053b20 con 0x7f30481f4cd0 2024-05-10T03:34:29.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.363+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30480b38c0 con 0x7f30481f4cd0 2024-05-10T03:34:29.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.364+0000 7f303fe00640 1 -- 172.21.15.55:0/1003253809 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3040017080 con 0x7f30481f4cd0 2024-05-10T03:34:29.367 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:34:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 msgr2=0x7f302406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 0x7f302406d110 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3038001560 tx=0x7f3038009040 comp rx=0 tx=0).stop 2024-05-10T03:34:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 msgr2=0x7f30481508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f304000c3c0 tx=0x7f304000c7c0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 shutdown_connections 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30481f4cd0 0x7f30481508a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f302406ac00 0x7f302406d110 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30480cdef0 0x7f30480b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 --2- 172.21.15.55:0/1003253809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30480b6bf0 0x7f304814e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 >> 172.21.15.55:0/1003253809 conn(0x7f30480d32c0 msgr2=0x7f3048158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 shutdown_connections 2024-05-10T03:34:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.366+0000 7f304f200640 1 -- 172.21.15.55:0/1003253809 wait complete. 2024-05-10T03:34:29.398 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:34:29.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:29.398 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:34:29.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:34:29.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:29.492 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:34:29.545 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:29.545 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:29.602 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:29.602 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:29.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:29.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.945+0000 7f7492800640 1 Processor -- start 2024-05-10T03:34:29.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.945+0000 7f7492800640 1 -- start start 2024-05-10T03:34:29.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7492800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f748c0b2d00 0x7f748c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7492800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7492800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c162c50 con 0x7f748c162130 2024-05-10T03:34:29.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7492800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0b1f60 con 0x7f748c0b2d00 2024-05-10T03:34:29.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7492800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c162ed0 con 0x7f748c144b60 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.948+0000 7f7491400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f748c0b2d00 0x7f748c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f748c0b2d00 0x7f748c1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51140/0 (socket says 172.21.15.55:51140) 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 -- 172.21.15.55:0/888568046 learned_addr learned my addr 172.21.15.55:0/888568046 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7490a00640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 -- 172.21.15.55:0/888568046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 msgr2=0x7f748c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 -- 172.21.15.55:0/888568046 >> v1:172.21.15.162:6789/0 conn(0x7f748c162130 legacy=0x7f748c162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.949+0000 7f7491400640 1 -- 172.21.15.55:0/888568046 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0b2220 con 0x7f748c0b2d00 2024-05-10T03:34:29.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.951+0000 7f7491400640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c0b2d00 0x7f748c1471f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f747c009e50 tx=0x7f747c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29e4a4821c71538 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.951+0000 7f748be00640 1 -- 172.21.15.55:0/888568046 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c02aea0 con 0x7f748c0b2d00 2024-05-10T03:34:29.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.951+0000 7f748be00640 1 -- 172.21.15.55:0/888568046 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c02d050 con 0x7f748c0b2d00 2024-05-10T03:34:29.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.951+0000 7f748be00640 1 -- 172.21.15.55:0/888568046 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c031820 con 0x7f748c0b2d00 2024-05-10T03:34:29.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 -- 172.21.15.55:0/888568046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c0b2d00 msgr2=0x7f748c1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c0b2d00 0x7f748c1471f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f747c009e50 tx=0x7f747c02a680 comp rx=0 tx=0).stop 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 -- 172.21.15.55:0/888568046 shutdown_connections 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 --2- 172.21.15.55:0/888568046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c0b2d00 0x7f748c1471f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 -- 172.21.15.55:0/888568046 >> 172.21.15.55:0/888568046 conn(0x7f748c06a110 msgr2=0x7f748c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 -- 172.21.15.55:0/888568046 shutdown_connections 2024-05-10T03:34:29.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.952+0000 7f7492800640 1 -- 172.21.15.55:0/888568046 wait complete. 2024-05-10T03:34:29.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.955+0000 7f7492800640 1 Processor -- start 2024-05-10T03:34:29.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.955+0000 7f7492800640 1 -- start start 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.956+0000 7f7492800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.956+0000 7f7492800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7492800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f748c156040 0x7f748c1fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7492800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c13ec40 con 0x7f748c162130 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7492800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c1567b0 con 0x7f748c156040 2024-05-10T03:34:29.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7492800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c13ee50 con 0x7f748c144b60 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51150/0 (socket says 172.21.15.55:51150) 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 -- 172.21.15.55:0/2911904674 learned_addr learned my addr 172.21.15.55:0/2911904674 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7491400640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 -- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 msgr2=0x7f748c1f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c1f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 -- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f748c156040 msgr2=0x7f748c1fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f748c156040 0x7f748c1fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.957+0000 7f7490a00640 1 -- 172.21.15.55:0/2911904674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009b30 con 0x7f748c162130 2024-05-10T03:34:29.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7490a00640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f747800d5f0 tx=0x7f747800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780125f0 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478012750 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478012b00 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c13f130 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.958+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c13f6f0 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.960+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c066ce0 con 0x7f748c162130 2024-05-10T03:34:29.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.961+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7478012c60 con 0x7f748c162130 2024-05-10T03:34:29.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.961+0000 7f7489600640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 0x7f746c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:29.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.962+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7478089a00 con 0x7f748c162130 2024-05-10T03:34:29.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.962+0000 7f7491400640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 0x7f746c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:29.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.964+0000 7f7491400640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 0x7f746c06d110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f747c033000 tx=0x7f747c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:29.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:29.966+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7478054a20 con 0x7f748c162130 2024-05-10T03:34:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.166+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f748c04e290 con 0x7f748c162130 2024-05-10T03:34:30.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.168+0000 7f7489600640 1 -- 172.21.15.55:0/2911904674 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f7478058040 con 0x7f748c162130 2024-05-10T03:34:30.173 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:30.173 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:25.020639+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:30.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 msgr2=0x7f746c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 0x7f746c06d110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f747c033000 tx=0x7f747c009000 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 msgr2=0x7f748c1ec7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f747800d5f0 tx=0x7f747800dab0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 shutdown_connections 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f746c06ac00 0x7f746c06d110 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f748c156040 0x7f748c1fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f748c162130 0x7f748c1ec7c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.171+0000 7f7492800640 1 --2- 172.21.15.55:0/2911904674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f748c144b60 0x7f748c1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.172+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 >> 172.21.15.55:0/2911904674 conn(0x7f748c06a110 msgr2=0x7f748c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.172+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 shutdown_connections 2024-05-10T03:34:30.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.172+0000 7f7492800640 1 -- 172.21.15.55:0/2911904674 wait complete. 2024-05-10T03:34:30.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:13.758492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:14.516676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:30.207 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:30.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.496+0000 7f788e400640 1 Processor -- start 2024-05-10T03:34:30.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.497+0000 7f788e400640 1 -- start start 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.498+0000 7f788e400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 0x7f788806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.498+0000 7f788e400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f78880700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.498+0000 7f788e400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7888071e00 0x7f7888070620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.498+0000 7f788e400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815fdf0 con 0x7f7888071e00 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788e400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815f1e0 con 0x7f78880719f0 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788e400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788815f4c0 con 0x7f788806b840 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7888071e00 0x7f7888070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7888071e00 0x7f7888070620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51152/0 (socket says 172.21.15.55:51152) 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788da00640 1 -- 172.21.15.55:0/3027977730 learned_addr learned my addr 172.21.15.55:0/3027977730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f7887e00640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f78880700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.499+0000 7f788d000640 1 --2- 172.21.15.55:0/3027977730 >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 0x7f788806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788da00640 1 -- 172.21.15.55:0/3027977730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 msgr2=0x7f78880700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788da00640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f78880700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788da00640 1 -- 172.21.15.55:0/3027977730 >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 msgr2=0x7f788806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788da00640 1 --2- 172.21.15.55:0/3027977730 >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 0x7f788806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788da00640 1 -- 172.21.15.55:0/3027977730 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788815f7a0 con 0x7f7888071e00 2024-05-10T03:34:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.500+0000 7f788d000640 1 --2- 172.21.15.55:0/3027977730 >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 0x7f788806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.501+0000 7f788da00640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7888071e00 0x7f7888070620 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f78880b41f0 tx=0x7f788002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce970fd904ca0420 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.502+0000 7f7887400640 1 -- 172.21.15.55:0/3027977730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031600 con 0x7f7888071e00 2024-05-10T03:34:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.502+0000 7f7887400640 1 -- 172.21.15.55:0/3027977730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788002d050 con 0x7f7888071e00 2024-05-10T03:34:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.502+0000 7f7887400640 1 -- 172.21.15.55:0/3027977730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031980 con 0x7f7888071e00 2024-05-10T03:34:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.502+0000 7f788e400640 1 -- 172.21.15.55:0/3027977730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7888071e00 msgr2=0x7f7888070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.502+0000 7f788e400640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7888071e00 0x7f7888070620 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f78880b41f0 tx=0x7f788002a610 comp rx=0 tx=0).stop 2024-05-10T03:34:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 -- 172.21.15.55:0/3027977730 shutdown_connections 2024-05-10T03:34:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7888071e00 0x7f7888070620 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 --2- 172.21.15.55:0/3027977730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f78880700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 --2- 172.21.15.55:0/3027977730 >> v2:172.21.15.162:3300/0 conn(0x7f78880719f0 0x7f788806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 -- 172.21.15.55:0/3027977730 >> 172.21.15.55:0/3027977730 conn(0x7f788806c750 msgr2=0x7f78880b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:30.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 -- 172.21.15.55:0/3027977730 shutdown_connections 2024-05-10T03:34:30.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.503+0000 7f788e400640 1 -- 172.21.15.55:0/3027977730 wait complete. 2024-05-10T03:34:30.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.505+0000 7f788e400640 1 Processor -- start 2024-05-10T03:34:30.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.505+0000 7f788e400640 1 -- start start 2024-05-10T03:34:30.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f7888153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 0x7f7888151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788814e3e0 con 0x7f78880719f0 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788814e580 con 0x7f7888071e00 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788e400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78881522a0 con 0x7f788806b840 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f7887e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f7887e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51154/0 (socket says 172.21.15.55:51154) 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f7887e00640 1 -- 172.21.15.55:0/2022834025 learned_addr learned my addr 172.21.15.55:0/2022834025 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788d000640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f7888153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.507+0000 7f788da00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 0x7f7888151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 -- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 msgr2=0x7f7888153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f7888153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 -- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 msgr2=0x7f7888151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 0x7f7888151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 -- 172.21.15.55:0/2022834025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009b30 con 0x7f78880719f0 2024-05-10T03:34:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f788da00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 0x7f7888151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f788d000640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f7888153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7887e00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f787000d830 tx=0x7f787000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000be10 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78700098b0 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870009a40 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.508+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78881527d0 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.509+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7888154390 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.510+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788804e290 con 0x7f78880719f0 2024-05-10T03:34:30.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.511+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7870009ba0 con 0x7f78880719f0 2024-05-10T03:34:30.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.512+0000 7f7884c00640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 0x7f786406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:30.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.512+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7870089930 con 0x7f78880719f0 2024-05-10T03:34:30.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.513+0000 7f788d000640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 0x7f786406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:30.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.514+0000 7f788d000640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 0x7f786406d110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7878006fd0 tx=0x7f7878008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.517+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f78700548b0 con 0x7f78880719f0 2024-05-10T03:34:30.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.715+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7888071190 con 0x7f78880719f0 2024-05-10T03:34:30.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.716+0000 7f7884c00640 1 -- 172.21.15.55:0/2022834025 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7870018020 con 0x7f78880719f0 2024-05-10T03:34:30.720 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:30.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 msgr2=0x7f786406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 0x7f786406d110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7878006fd0 tx=0x7f7878008040 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 msgr2=0x7f7888151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f787000d830 tx=0x7f787000dcf0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 shutdown_connections 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f786406ac00 0x7f786406d110 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.718+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888071e00 0x7f7888151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.719+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78880719f0 0x7f7888151730 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.719+0000 7f788e400640 1 --2- 172.21.15.55:0/2022834025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f788806b840 0x7f7888153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.719+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 >> 172.21.15.55:0/2022834025 conn(0x7f788806c750 msgr2=0x7f7888142ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.719+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 shutdown_connections 2024-05-10T03:34:30.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:30.719+0000 7f788e400640 1 -- 172.21.15.55:0/2022834025 wait complete. 2024-05-10T03:34:30.748 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:34:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.037+0000 7f734de00640 1 Processor -- start 2024-05-10T03:34:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.037+0000 7f734de00640 1 -- start start 2024-05-10T03:34:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73480719f0 0x7f734806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734de00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7348071e00 0x7f7348141600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734de00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734815fdf0 con 0x7f734806b840 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734de00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734815f1e0 con 0x7f73480719f0 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734de00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734815f4c0 con 0x7f7348071e00 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734ca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73480719f0 0x7f734806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734ca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73480719f0 0x7f734806b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51164/0 (socket says 172.21.15.55:51164) 2024-05-10T03:34:31.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.040+0000 7f734ca00640 1 -- 172.21.15.55:0/1131316140 learned_addr learned my addr 172.21.15.55:0/1131316140 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.041+0000 7f734d400640 1 --2- 172.21.15.55:0/1131316140 >> v2:172.21.15.162:3300/0 conn(0x7f7348071e00 0x7f7348141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.041+0000 7f734ca00640 1 -- 172.21.15.55:0/1131316140 >> v2:172.21.15.162:3300/0 conn(0x7f7348071e00 msgr2=0x7f7348141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.041+0000 7f734ca00640 1 --2- 172.21.15.55:0/1131316140 >> v2:172.21.15.162:3300/0 conn(0x7f7348071e00 0x7f7348141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.041+0000 7f734ca00640 1 -- 172.21.15.55:0/1131316140 >> v1:172.21.15.162:6789/0 conn(0x7f734806b840 legacy=0x7f73480700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.041+0000 7f734ca00640 1 -- 172.21.15.55:0/1131316140 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734815f7a0 con 0x7f73480719f0 2024-05-10T03:34:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.042+0000 7f734ca00640 1 --2- 172.21.15.55:0/1131316140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73480719f0 0x7f734806b300 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f73480ced30 tx=0x7f733c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3275b362d0d23aad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f7347400640 1 -- 172.21.15.55:0/1131316140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c032440 con 0x7f73480719f0 2024-05-10T03:34:31.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f7347400640 1 -- 172.21.15.55:0/1131316140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c0325a0 con 0x7f73480719f0 2024-05-10T03:34:31.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f7347400640 1 -- 172.21.15.55:0/1131316140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c032930 con 0x7f73480719f0 2024-05-10T03:34:31.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 -- 172.21.15.55:0/1131316140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73480719f0 msgr2=0x7f734806b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 --2- 172.21.15.55:0/1131316140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73480719f0 0x7f734806b300 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f73480ced30 tx=0x7f733c02a1a0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 -- 172.21.15.55:0/1131316140 shutdown_connections 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 --2- 172.21.15.55:0/1131316140 >> v2:172.21.15.162:3300/0 conn(0x7f7348071e00 0x7f7348141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 --2- 172.21.15.55:0/1131316140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73480719f0 0x7f734806b300 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.043+0000 7f734de00640 1 -- 172.21.15.55:0/1131316140 >> 172.21.15.55:0/1131316140 conn(0x7f734806c750 msgr2=0x7f73480b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.044+0000 7f734de00640 1 -- 172.21.15.55:0/1131316140 shutdown_connections 2024-05-10T03:34:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.044+0000 7f734de00640 1 -- 172.21.15.55:0/1131316140 wait complete. 2024-05-10T03:34:31.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.046+0000 7f734de00640 1 Processor -- start 2024-05-10T03:34:31.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.046+0000 7f734de00640 1 -- start start 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 0x7f73480cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 0x7f734813edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815c040 con 0x7f7348071e00 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480c51c0 con 0x7f73480719f0 2024-05-10T03:34:31.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480cc670 con 0x7f734806b840 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51176/0 (socket says 172.21.15.55:51176) 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734d400640 1 -- 172.21.15.55:0/1035225418 learned_addr learned my addr 172.21.15.55:0/1035225418 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.048+0000 7f734ca00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 0x7f73480cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f7347e00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 0x7f734813edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 -- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 msgr2=0x7f73480cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 0x7f73480cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 -- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 msgr2=0x7f734813edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 0x7f734813edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 -- 172.21.15.55:0/1035225418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0094d0 con 0x7f7348071e00 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734ca00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 0x7f73480cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f7347e00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 0x7f734813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734d400640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7340007d80 tx=0x7f734000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340014420 con 0x7f7348071e00 2024-05-10T03:34:31.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340014580 con 0x7f7348071e00 2024-05-10T03:34:31.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340014930 con 0x7f7348071e00 2024-05-10T03:34:31.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.049+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734815b3c0 con 0x7f7348071e00 2024-05-10T03:34:31.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.050+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734815b980 con 0x7f7348071e00 2024-05-10T03:34:31.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.051+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734804e290 con 0x7f7348071e00 2024-05-10T03:34:31.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.052+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7340014a90 con 0x7f7348071e00 2024-05-10T03:34:31.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.053+0000 7f7344c00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 0x7f732806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.053+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7340089e20 con 0x7f7348071e00 2024-05-10T03:34:31.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.054+0000 7f734ca00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 0x7f732806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.055+0000 7f734ca00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 0x7f732806d110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f733c034000 tx=0x7f733c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.058+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7340054da0 con 0x7f7348071e00 2024-05-10T03:34:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.261+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f734815ed20 con 0x7f7348071e00 2024-05-10T03:34:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.261+0000 7f7344c00640 1 -- 172.21.15.55:0/1035225418 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73400583c0 con 0x7f7348071e00 2024-05-10T03:34:31.265 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.264+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 msgr2=0x7f732806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.264+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 0x7f732806d110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f733c034000 tx=0x7f733c02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.264+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 msgr2=0x7f734815cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.264+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7340007d80 tx=0x7f734000e4b0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 shutdown_connections 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7348071e00 0x7f734815cad0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f732806ac00 0x7f732806d110 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73480719f0 0x7f734813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 --2- 172.21.15.55:0/1035225418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f734806b840 0x7f73480cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 >> 172.21.15.55:0/1035225418 conn(0x7f734806c750 msgr2=0x7f73480ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:31.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 shutdown_connections 2024-05-10T03:34:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.265+0000 7f734de00640 1 -- 172.21.15.55:0/1035225418 wait complete. 2024-05-10T03:34:31.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:34:31.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.586+0000 7fd82f200640 1 Processor -- start 2024-05-10T03:34:31.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.586+0000 7fd82f200640 1 -- start start 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82f200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8301612f0 0x7fd83015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 0x7fd8300ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300cea20 con 0x7fd8300ce610 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82f200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83015dc80 con 0x7fd8301612f0 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300cfa00 con 0x7fd83006b520 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.588+0000 7fd82de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8301612f0 0x7fd83015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8301612f0 0x7fd83015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51182/0 (socket says 172.21.15.55:51182) 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 -- 172.21.15.55:0/2965171549 learned_addr learned my addr 172.21.15.55:0/2965171549 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd827e00640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 0x7fd8300ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 -- 172.21.15.55:0/2965171549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 msgr2=0x7fd8300ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:31.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 0x7fd8300ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 -- 172.21.15.55:0/2965171549 >> v1:172.21.15.162:6789/0 conn(0x7fd8300ce610 legacy=0x7fd8300b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:31.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd82de00640 1 -- 172.21.15.55:0/2965171549 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83006a480 con 0x7fd8301612f0 2024-05-10T03:34:31.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.589+0000 7fd827e00640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 0x7fd8300ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.590+0000 7fd82de00640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd83015d740 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd82002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60702571439550d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82d400640 1 -- 172.21.15.55:0/2965171549 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820032440 con 0x7fd8301612f0 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82d400640 1 -- 172.21.15.55:0/2965171549 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8200325a0 con 0x7fd8301612f0 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82d400640 1 -- 172.21.15.55:0/2965171549 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820032930 con 0x7fd8301612f0 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 -- 172.21.15.55:0/2965171549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 msgr2=0x7fd83015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd83015d740 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd82002a250 comp rx=0 tx=0).stop 2024-05-10T03:34:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 -- 172.21.15.55:0/2965171549 shutdown_connections 2024-05-10T03:34:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd83006b520 0x7fd8300ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 --2- 172.21.15.55:0/2965171549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd83015d740 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.591+0000 7fd82f200640 1 -- 172.21.15.55:0/2965171549 >> 172.21.15.55:0/2965171549 conn(0x7fd8300d0830 msgr2=0x7fd8300d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.592+0000 7fd82f200640 1 -- 172.21.15.55:0/2965171549 shutdown_connections 2024-05-10T03:34:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.592+0000 7fd82f200640 1 -- 172.21.15.55:0/2965171549 wait complete. 2024-05-10T03:34:31.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.594+0000 7fd82f200640 1 Processor -- start 2024-05-10T03:34:31.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.594+0000 7fd82f200640 1 -- start start 2024-05-10T03:34:31.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83006b520 0x7fd8300ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8300ce610 0x7fd830142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300cd300 con 0x7fd8301612f0 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301436c0 con 0x7fd83006b520 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830150190 con 0x7fd8300ce610 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51198/0 (socket says 172.21.15.55:51198) 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82e800640 1 -- 172.21.15.55:0/2912961722 learned_addr learned my addr 172.21.15.55:0/2912961722 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.596+0000 7fd82de00640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83006b520 0x7fd8300ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd827e00640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8300ce610 0x7fd830142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 -- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8300ce610 msgr2=0x7fd830142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8300ce610 0x7fd830142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 -- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83006b520 msgr2=0x7fd8300ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83006b520 0x7fd8300ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 -- 172.21.15.55:0/2912961722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200094d0 con 0x7fd8301612f0 2024-05-10T03:34:31.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82e800640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd82800bec0 tx=0x7fd828009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003970 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828003ad0 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83014fc00 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003e80 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.597+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830150ff0 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.598+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd830151180 con 0x7fd8301612f0 2024-05-10T03:34:31.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.600+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd828003fe0 con 0x7fd8301612f0 2024-05-10T03:34:31.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.600+0000 7fd826000640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 0x7fd80c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:31.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.601+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd828089490 con 0x7fd8301612f0 2024-05-10T03:34:31.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.601+0000 7fd82de00640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 0x7fd80c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.602+0000 7fd82de00640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 0x7fd80c06d140 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd82002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.605+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd828054420 con 0x7fd8301612f0 2024-05-10T03:34:31.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.802+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd83014fc00 con 0x7fd8301612f0 2024-05-10T03:34:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.803+0000 7fd826000640 1 -- 172.21.15.55:0/2912961722 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd83014fc00 con 0x7fd8301612f0 2024-05-10T03:34:31.807 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:34:31.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 msgr2=0x7fd80c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 0x7fd80c06d140 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd82002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:31.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 msgr2=0x7fd830160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:31.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd82800bec0 tx=0x7fd828009f30 comp rx=0 tx=0).stop 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 shutdown_connections 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8301612f0 0x7fd830160050 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8300ce610 0x7fd830142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd80c06ac30 0x7fd80c06d140 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 --2- 172.21.15.55:0/2912961722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83006b520 0x7fd8300ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 >> 172.21.15.55:0/2912961722 conn(0x7fd8300d0830 msgr2=0x7fd83015b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 shutdown_connections 2024-05-10T03:34:31.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:31.806+0000 7fd82f200640 1 -- 172.21.15.55:0/2912961722 wait complete. 2024-05-10T03:34:31.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:34:32.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.112+0000 7f6537800640 1 Processor -- start 2024-05-10T03:34:32.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.112+0000 7f6537800640 1 -- start start 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.114+0000 7f6537800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f653015fe50 0x7f6530141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.114+0000 7f6537800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65300b2140 0x7f6530142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6537800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653006ad20 con 0x7f65300b2550 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6537800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6530141f50 con 0x7f65300b2140 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6537800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653006a110 con 0x7f653015fe50 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6535800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f65300b2550 0x7f653006b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44874/0 (socket says 172.21.15.55:44874) 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f652fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65300b2140 0x7f6530142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6535800640 1 -- 172.21.15.55:0/1885490858 learned_addr learned my addr 172.21.15.55:0/1885490858 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.115+0000 7f6534e00640 1 --2- 172.21.15.55:0/1885490858 >> v2:172.21.15.162:3300/0 conn(0x7f653015fe50 0x7f6530141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.116+0000 7f652fe00640 1 -- 172.21.15.55:0/1885490858 >> v2:172.21.15.162:3300/0 conn(0x7f653015fe50 msgr2=0x7f6530141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.116+0000 7f652fe00640 1 --2- 172.21.15.55:0/1885490858 >> v2:172.21.15.162:3300/0 conn(0x7f653015fe50 0x7f6530141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.116+0000 7f652fe00640 1 -- 172.21.15.55:0/1885490858 >> v1:172.21.15.55:6789/0 conn(0x7f65300b2550 legacy=0x7f653006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.116+0000 7f652fe00640 1 -- 172.21.15.55:0/1885490858 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653006a3f0 con 0x7f65300b2140 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.116+0000 7f652f400640 1 -- 172.21.15.55:0/1885490858 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535319235 0 0) 0x7f653006ad20 con 0x7f65300b2550 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.117+0000 7f652fe00640 1 --2- 172.21.15.55:0/1885490858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65300b2140 0x7f6530142440 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f653006e520 tx=0x7f651c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d73ec7f7644c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.117+0000 7f652f400640 1 -- 172.21.15.55:0/1885490858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c032440 con 0x7f65300b2140 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.117+0000 7f652f400640 1 -- 172.21.15.55:0/1885490858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c0325a0 con 0x7f65300b2140 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.117+0000 7f652f400640 1 -- 172.21.15.55:0/1885490858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c032930 con 0x7f65300b2140 2024-05-10T03:34:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 -- 172.21.15.55:0/1885490858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65300b2140 msgr2=0x7f6530142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 --2- 172.21.15.55:0/1885490858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65300b2140 0x7f6530142440 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f653006e520 tx=0x7f651c02a230 comp rx=0 tx=0).stop 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 -- 172.21.15.55:0/1885490858 shutdown_connections 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 --2- 172.21.15.55:0/1885490858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65300b2140 0x7f6530142440 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 --2- 172.21.15.55:0/1885490858 >> v2:172.21.15.162:3300/0 conn(0x7f653015fe50 0x7f6530141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 -- 172.21.15.55:0/1885490858 >> 172.21.15.55:0/1885490858 conn(0x7f65300cbb90 msgr2=0x7f65300cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 -- 172.21.15.55:0/1885490858 shutdown_connections 2024-05-10T03:34:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.118+0000 7f6537800640 1 -- 172.21.15.55:0/1885490858 wait complete. 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.121+0000 7f6537800640 1 Processor -- start 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.121+0000 7f6537800640 1 -- start start 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65300b2140 0x7f653013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65300b2550 0x7f65301f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65301f5610 con 0x7f653015fe50 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653015cdf0 con 0x7f65300b2550 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6537800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65301452a0 con 0x7f65300b2140 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51212/0 (socket says 172.21.15.55:51212) 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 -- 172.21.15.55:0/2121850593 learned_addr learned my addr 172.21.15.55:0/2121850593 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f652fe00640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65300b2550 0x7f65301f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6534e00640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65300b2140 0x7f653013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 -- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65300b2140 msgr2=0x7f653013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65300b2140 0x7f653013b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 -- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65300b2550 msgr2=0x7f65301f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65300b2550 0x7f65301f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 -- 172.21.15.55:0/2121850593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c009400 con 0x7f653015fe50 2024-05-10T03:34:32.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.123+0000 7f6535800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f652800c780 tx=0x7f652800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.124+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800ce90 con 0x7f653015fe50 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.124+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800a640 con 0x7f653015fe50 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.124+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800a9d0 con 0x7f653015fe50 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.124+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65301f5100 con 0x7f653015fe50 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.124+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653013e760 con 0x7f653015fe50 2024-05-10T03:34:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.125+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653015cfb0 con 0x7f653015fe50 2024-05-10T03:34:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.127+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f652800ab30 con 0x7f653015fe50 2024-05-10T03:34:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.127+0000 7f652cc00640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 0x7f651006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.127+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6528088b20 con 0x7f653015fe50 2024-05-10T03:34:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.128+0000 7f6534e00640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 0x7f651006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.129+0000 7f6534e00640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 0x7f651006d110 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6524006fd0 tx=0x7f6524008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.132+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f653015cfb0 con 0x7f653015fe50 2024-05-10T03:34:32.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.351+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6530066f60 con 0x7f653015fe50 2024-05-10T03:34:32.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.352+0000 7f652cc00640 1 -- 172.21.15.55:0/2121850593 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6528018070 con 0x7f653015fe50 2024-05-10T03:34:32.356 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.354+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 msgr2=0x7f651006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.354+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 0x7f651006d110 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6524006fd0 tx=0x7f6524008040 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.354+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 msgr2=0x7f653014d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.354+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f652800c780 tx=0x7f652800cc40 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 shutdown_connections 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f653015fe50 0x7f653014d530 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f651006ac00 0x7f651006d110 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65300b2550 0x7f65301f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 --2- 172.21.15.55:0/2121850593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65300b2140 0x7f653013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 >> 172.21.15.55:0/2121850593 conn(0x7f65300cbb90 msgr2=0x7f65300b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 shutdown_connections 2024-05-10T03:34:32.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.355+0000 7f6537800640 1 -- 172.21.15.55:0/2121850593 wait complete. 2024-05-10T03:34:32.386 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:34:32.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:32.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:34:32.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:34:32.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:34:32.483 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:34:32.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:32.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.825+0000 7f7de5800640 1 Processor -- start 2024-05-10T03:34:32.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.825+0000 7f7de5800640 1 -- start start 2024-05-10T03:34:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.827+0000 7f7de5800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7de0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.827+0000 7f7de5800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7de006d7e0 0x7f7de00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7de5800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de006d350 con 0x7f7de00b2fc0 2024-05-10T03:34:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7de5800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de006c740 con 0x7f7de006d7e0 2024-05-10T03:34:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7de5800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de006ca20 con 0x7f7de0144710 2024-05-10T03:34:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7de4e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7de00b2fc0 0x7f7de015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44890/0 (socket says 172.21.15.55:44890) 2024-05-10T03:34:32.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7de4e00640 1 -- 172.21.15.55:0/2054473828 learned_addr learned my addr 172.21.15.55:0/2054473828 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2054473828 >> v2:172.21.15.162:3300/0 conn(0x7f7de006d7e0 0x7f7de00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7ddfe00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7de0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106930527 0 0) 0x7f7de006d350 con 0x7f7de00b2fc0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.828+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc4003520 con 0x7f7de00b2fc0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171996563 0 0) 0x7f7dc4003520 con 0x7f7de00b2fc0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 msgr2=0x7f7de0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7de0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 >> v2:172.21.15.162:3300/0 conn(0x7f7de006d7e0 msgr2=0x7f7de00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 --2- 172.21.15.55:0/2054473828 >> v2:172.21.15.162:3300/0 conn(0x7f7de006d7e0 0x7f7de00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de006cd00 con 0x7f7de00b2fc0 2024-05-10T03:34:32.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7dd80019f0 con 0x7f7de00b2fc0 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 >> v1:172.21.15.55:6789/0 conn(0x7f7de00b2fc0 legacy=0x7f7de015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7dc40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7dc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dc4004d00 0x7f7dc4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de006c740 con 0x7f7dc4003f60 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de006ca20 con 0x7f7dc4004d00 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc400bae0 con 0x7f7de0144710 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.829+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7dc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7de4e00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dc4004d00 0x7f7dc4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddf400640 1 -- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 msgr2=0x7f7dc40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddfe00640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7dc40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de0144710 0x7f7dc40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddf400640 1 -- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dc4004d00 msgr2=0x7f7dc4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:32.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dc4004d00 0x7f7dc4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.830+0000 7f7ddf400640 1 -- 172.21.15.55:0/2054473828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc4003520 con 0x7f7dc4003f60 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.831+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7dc40047c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7de00ae870 tx=0x7f7dc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58acdb7496456ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.831+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc031400 con 0x7f7dc4003f60 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc02d050 con 0x7f7dc4003f60 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7ddea00640 1 -- 172.21.15.55:0/2054473828 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc031780 con 0x7f7dc4003f60 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 -- 172.21.15.55:0/2054473828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 msgr2=0x7f7dc40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 --2- 172.21.15.55:0/2054473828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7dc40047c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7de00ae870 tx=0x7f7dc40057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de4e00640 1 -- 172.21.15.55:0/2054473828 reap_dead start 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 -- 172.21.15.55:0/2054473828 shutdown_connections 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 -- 172.21.15.55:0/2054473828 >> 172.21.15.55:0/2054473828 conn(0x7f7de006a110 msgr2=0x7f7de006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 -- 172.21.15.55:0/2054473828 shutdown_connections 2024-05-10T03:34:32.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.832+0000 7f7de5800640 1 -- 172.21.15.55:0/2054473828 wait complete. 2024-05-10T03:34:32.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.835+0000 7f7de5800640 1 Processor -- start 2024-05-10T03:34:32.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.835+0000 7f7de5800640 1 -- start start 2024-05-10T03:34:32.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de013dae0 0x7f7de00ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7de00adf60 0x7f7de0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0070cb0 con 0x7f7dc4003f60 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0152e60 con 0x7f7de00adf60 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de5800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de00ad590 con 0x7f7de013dae0 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51222/0 (socket says 172.21.15.55:51222) 2024-05-10T03:34:32.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 -- 172.21.15.55:0/2557975276 learned_addr learned my addr 172.21.15.55:0/2557975276 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 -- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de013dae0 msgr2=0x7f7de00ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de013dae0 0x7f7de00ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7de4e00640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7de00adf60 0x7f7de0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 -- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7de00adf60 msgr2=0x7f7de0152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7de00adf60 0x7f7de0152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:32.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.837+0000 7f7ddfe00640 1 -- 172.21.15.55:0/2557975276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de00ac980 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7ddfe00640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00107f0 tx=0x7f7dd0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00095d0 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000eb00 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de00acc60 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000ee90 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.838+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de00ad220 con 0x7f7dc4003f60 2024-05-10T03:34:32.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.839+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de0066ce0 con 0x7f7dc4003f60 2024-05-10T03:34:32.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.841+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7dd001c400 con 0x7f7dc4003f60 2024-05-10T03:34:32.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.842+0000 7f7dd7e00640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 0x7f7dc006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:32.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.842+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7dd0089a10 con 0x7f7dc4003f60 2024-05-10T03:34:32.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.843+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 0x7f7dc006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:32.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.844+0000 7f7ddf400640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 0x7f7dc006d140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc033000 tx=0x7f7dcc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:32.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:32.848+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7dd0054a40 con 0x7f7dc4003f60 2024-05-10T03:34:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.090+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7de004e290 con 0x7f7dc4003f60 2024-05-10T03:34:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.091+0000 7f7dd7e00640 1 -- 172.21.15.55:0/2557975276 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f7dd0058060 con 0x7f7dc4003f60 2024-05-10T03:34:33.094 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:33.095 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 msgr2=0x7f7dc006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 0x7f7dc006d140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc033000 tx=0x7f7dcc02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 msgr2=0x7f7de00703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00107f0 tx=0x7f7dd0010cb0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 shutdown_connections 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7dc006ac30 0x7f7dc006d140 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7de00adf60 0x7f7de0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7de013dae0 0x7f7de00ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 --2- 172.21.15.55:0/2557975276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dc4003f60 0x7f7de00703b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 >> 172.21.15.55:0/2557975276 conn(0x7f7de006a110 msgr2=0x7f7de00aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 shutdown_connections 2024-05-10T03:34:33.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.094+0000 7f7de5800640 1 -- 172.21.15.55:0/2557975276 wait complete. 2024-05-10T03:34:33.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 session ls 2024-05-10T03:34:33.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.397+0000 7f584da00640 1 Processor -- start 2024-05-10T03:34:33.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.397+0000 7f584da00640 1 -- start start 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584da00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58480ce6a0 0x7f58480b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f58480b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584da00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480b3040 con 0x7f58480b2460 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584da00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480b3cc0 con 0x7f58480ce6a0 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b3320 con 0x7f58480d1ac0 2024-05-10T03:34:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f5847400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f58480ce6a0 0x7f58480b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f5847e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f58480b2460 0x7f58480cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49912/0 (socket says 172.21.15.55:49912) 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f5847e00640 1 -- 172.21.15.55:0/3434218922 learned_addr learned my addr 172.21.15.55:0/3434218922 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.400+0000 7f584d000640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f58480b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.401+0000 7f5847400640 1 -- 172.21.15.55:0/3434218922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 msgr2=0x7f58480b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.401+0000 7f5847400640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f58480b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.401+0000 7f5847400640 1 -- 172.21.15.55:0/3434218922 >> v1:172.21.15.55:6789/0 conn(0x7f58480b2460 legacy=0x7f58480cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.401+0000 7f5847400640 1 -- 172.21.15.55:0/3434218922 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480b3600 con 0x7f58480ce6a0 2024-05-10T03:34:33.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.401+0000 7f5846a00640 1 -- 172.21.15.55:0/3434218922 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955683136 0 0) 0x7f58480b3040 con 0x7f58480b2460 2024-05-10T03:34:33.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.402+0000 7f5847400640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480ce6a0 0x7f58480b29b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5838009d90 tx=0x7f583802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40125aef69d7de67 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.402+0000 7f5846a00640 1 -- 172.21.15.55:0/3434218922 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838007740 con 0x7f58480ce6a0 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.402+0000 7f5846a00640 1 -- 172.21.15.55:0/3434218922 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583802d050 con 0x7f58480ce6a0 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.402+0000 7f5846a00640 1 -- 172.21.15.55:0/3434218922 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838007ac0 con 0x7f58480ce6a0 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 -- 172.21.15.55:0/3434218922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480ce6a0 msgr2=0x7f58480b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480ce6a0 0x7f58480b29b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5838009d90 tx=0x7f583802a660 comp rx=0 tx=0).stop 2024-05-10T03:34:33.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 -- 172.21.15.55:0/3434218922 shutdown_connections 2024-05-10T03:34:33.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f58480b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 --2- 172.21.15.55:0/3434218922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480ce6a0 0x7f58480b29b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 -- 172.21.15.55:0/3434218922 >> 172.21.15.55:0/3434218922 conn(0x7f58480d3050 msgr2=0x7f58480d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 -- 172.21.15.55:0/3434218922 shutdown_connections 2024-05-10T03:34:33.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.403+0000 7f584da00640 1 -- 172.21.15.55:0/3434218922 wait complete. 2024-05-10T03:34:33.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.405+0000 7f584da00640 1 Processor -- start 2024-05-10T03:34:33.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.406+0000 7f584da00640 1 -- start start 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f5848156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 0x7f58481e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481f87e0 con 0x7f58480b2460 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584815a070 con 0x7f5848157170 2024-05-10T03:34:33.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.407+0000 7f584da00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58481e73e0 con 0x7f58480d1ac0 2024-05-10T03:34:33.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45758/0 (socket says 172.21.15.55:45758) 2024-05-10T03:34:33.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 -- 172.21.15.55:0/1673477566 learned_addr learned my addr 172.21.15.55:0/1673477566 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:33.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847400640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f5848156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f584d000640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 0x7f58481e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 -- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 msgr2=0x7f5848156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f5848156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 -- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 msgr2=0x7f58481e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 0x7f58481e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 -- 172.21.15.55:0/1673477566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009a70 con 0x7f58480b2460 2024-05-10T03:34:33.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f584d000640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 0x7f58481e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:33.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.408+0000 7f5847e00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f583000c3e0 tx=0x7f583000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.409+0000 7f583fe00640 1 -- 172.21.15.55:0/1673477566 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830005ce0 con 0x7f58480b2460 2024-05-10T03:34:33.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.409+0000 7f583fe00640 1 -- 172.21.15.55:0/1673477566 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830005e40 con 0x7f58480b2460 2024-05-10T03:34:33.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.409+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58481567a0 con 0x7f58480b2460 2024-05-10T03:34:33.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.409+0000 7f583fe00640 1 -- 172.21.15.55:0/1673477566 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830007310 con 0x7f58480b2460 2024-05-10T03:34:33.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.409+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58481e7630 con 0x7f58480b2460 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.412+0000 7f583fe00640 1 -- 172.21.15.55:0/1673477566 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5830007470 con 0x7f58480b2460 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.413+0000 7f583fe00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 0x7f582406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.413+0000 7f583fe00640 1 -- 172.21.15.55:0/1673477566 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5830088ae0 con 0x7f58480b2460 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.414+0000 7f5855e69740 1 Processor -- start 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.414+0000 7f5847400640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 0x7f582406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.414+0000 7f5855e69740 1 -- start start 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.415+0000 7f5847400640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 0x7f582406d140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5838034000 tx=0x7f5838033040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c916c80 con 0x55a90c60e390 2024-05-10T03:34:33.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c914410 con 0x55a90c5e54d0 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a90c90fd00 con 0x55a90c5ed6b0 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45766/0 (socket says 172.21.15.55:45766) 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5847e00640 1 -- 172.21.15.55:0/132092574 learned_addr learned my addr 172.21.15.55:0/132092574 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f584d000640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5847400640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.416+0000 7f5847e00640 1 -- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 msgr2=0x55a90c9ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847e00640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847e00640 1 -- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 msgr2=0x55a90c9abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847e00640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847e00640 1 -- 172.21.15.55:0/132092574 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90c8c1c80 con 0x55a90c60e390 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f584d000640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847400640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:33.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f5847e00640 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5830053c20 tx=0x7f58300077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69edb842c402ff9f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f583cc00640 1 -- 172.21.15.55:0/132092574 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583005b550 con 0x55a90c60e390 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f583cc00640 1 -- 172.21.15.55:0/132092574 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583005fb10 con 0x55a90c60e390 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.417+0000 7f583cc00640 1 -- 172.21.15.55:0/132092574 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830053e30 con 0x55a90c60e390 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 -- 172.21.15.55:0/132092574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 msgr2=0x55a90c9d7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5830053c20 tx=0x7f58300077e0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 -- 172.21.15.55:0/132092574 shutdown_connections 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 --2- 172.21.15.55:0/132092574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c60e390 0x55a90c9d7dc0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 -- 172.21.15.55:0/132092574 >> 172.21.15.55:0/132092574 conn(0x55a90c6024e0 msgr2=0x55a90c9f2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 -- 172.21.15.55:0/132092574 shutdown_connections 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.418+0000 7f5855e69740 1 -- 172.21.15.55:0/132092574 wait complete. 2024-05-10T03:34:33.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.419+0000 7f5855e69740 1 -- start start 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.420+0000 7f5855e69740 1 Processor -- start 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c60e390 0x55a90c9ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c8b11d0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c8b0e70 con 0x55a90c5e54d0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a90c60cf90 con 0x55a90c60e390 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.421+0000 7f5855e69740 10 client.0 _ll_register_callbacks cb 0x55a90c8c5790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45776/0 (socket says 172.21.15.55:45776) 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 -- 172.21.15.55:0/2965219763 learned_addr learned my addr 172.21.15.55:0/2965219763 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f584d000640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c60e390 0x55a90c9ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847e00640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 -- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c60e390 msgr2=0x55a90c9ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c60e390 0x55a90c9ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 -- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 msgr2=0x55a90c9d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847400640 1 -- 172.21.15.55:0/2965219763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58481e7630 con 0x55a90c5ed6b0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.422+0000 7f5847e00640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5847400640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f583803c490 tx=0x7f5838004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27caf07e59f7b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5835600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583803cdd0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5838043040 con 0x55a90c5ed6b0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a90c939db0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5838048430 con 0x55a90c5ed6b0 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5855e69740 10 client.7185 fetch_fsmap learned FSMap version 47 2024-05-10T03:34:33.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.423+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55a90c8d98c0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a90c939db0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5838038040 con 0x55a90c5ed6b0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 10 client.7185 fetch_fsmap finished waiting for FSMap version 47 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 10 client.7185 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 20 client.7185 populate_metadata read hostname 'smithi055' 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90c5ccf60 0x55a90c9d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 4 client.7185 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 --> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a90c902f30 con 0x55a90c5ccf60 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.424+0000 7f5847e00640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90c5ccf60 0x55a90c9d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.425+0000 7f5847e00640 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90c5ccf60 0x55a90c9d9240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b1554c89e8f75aa server_cookie=4ef1feab27467601 in_seq=0 out_seq=0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.425+0000 7f5835600640 10 client.7185 ms_handle_connect on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.425+0000 7f5835600640 1 -- 172.21.15.55:0/2965219763 <== mds.0 v2:172.21.15.55:6834/709318521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a90c902f30 con 0x55a90c5ccf60 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.425+0000 7f5835600640 10 client.7185 handle_command_reply: tid=0 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 client.7185 shutdown 2024-05-10T03:34:33.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 msgr2=0x55a90c9abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f583803c490 tx=0x7f5838004ab0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 shutdown_connections 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90c5ccf60 0x55a90c9d9240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c60e390 0x55a90c9ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9abe10 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 --2- 172.21.15.55:0/2965219763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c5e54d0 0x55a90c9d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 >> 172.21.15.55:0/2965219763 conn(0x55a90c6024e0 msgr2=0x55a90c9a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5835600640 0 client.7185 ms_handle_reset on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 shutdown_connections 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 1 -- 172.21.15.55:0/2965219763 wait complete. 2024-05-10T03:34:33.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.426+0000 7f5855e69740 20 client.7185 trim_cache size 0 max 16384 2024-05-10T03:34:33.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.444+0000 7f5855e69740 1 Processor -- start 2024-05-10T03:34:33.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.444+0000 7f5855e69740 1 -- start start 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 0x55a90c9a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90ca0ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9d4480 con 0x55a90c5ed6b0 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9cd150 con 0x55a90c60e390 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.445+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9f0780 con 0x55a90c5e54d0 2024-05-10T03:34:33.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45778/0 (socket says 172.21.15.55:45778) 2024-05-10T03:34:33.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 -- 172.21.15.55:0/716038050 learned_addr learned my addr 172.21.15.55:0/716038050 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847e00640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 0x55a90c9a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f584d000640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90ca0ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 -- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 msgr2=0x55a90c9a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 0x55a90c9a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 -- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 msgr2=0x55a90ca0ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90ca0ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 -- 172.21.15.55:0/716038050 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a90c8b0580 con 0x55a90c5ed6b0 2024-05-10T03:34:33.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847e00640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 0x55a90c9a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.446+0000 7f5847400640 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f583802ac50 tx=0x7f5838047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e3842174640f96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.447+0000 7f583d600640 1 -- 172.21.15.55:0/716038050 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838051870 con 0x55a90c5ed6b0 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.447+0000 7f583d600640 1 -- 172.21.15.55:0/716038050 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380519d0 con 0x55a90c5ed6b0 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.447+0000 7f583d600640 1 -- 172.21.15.55:0/716038050 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583805c650 con 0x55a90c5ed6b0 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 -- 172.21.15.55:0/716038050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 msgr2=0x55a90c9a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f583802ac50 tx=0x7f5838047000 comp rx=0 tx=0).stop 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 -- 172.21.15.55:0/716038050 shutdown_connections 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90ca0ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5ed6b0 0x55a90c9a8660 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 --2- 172.21.15.55:0/716038050 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5e54d0 0x55a90c9a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 -- 172.21.15.55:0/716038050 >> 172.21.15.55:0/716038050 conn(0x55a90c6024e0 msgr2=0x55a90c9abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 -- 172.21.15.55:0/716038050 shutdown_connections 2024-05-10T03:34:33.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.448+0000 7f5855e69740 1 -- 172.21.15.55:0/716038050 wait complete. 2024-05-10T03:34:33.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.449+0000 7f5855e69740 1 -- start start 2024-05-10T03:34:33.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.449+0000 7f5855e69740 1 Processor -- start 2024-05-10T03:34:33.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90c9ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9f1eb0 con 0x55a90c5e54d0 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9ce3a0 con 0x55a90c60e390 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a90c9f1a20 con 0x55a90c5ed6b0 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5855e69740 10 client.0 _ll_register_callbacks cb 0x55a90c9d8cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45782/0 (socket says 172.21.15.55:45782) 2024-05-10T03:34:33.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 -- 172.21.15.55:0/2067903642 learned_addr learned my addr 172.21.15.55:0/2067903642 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 -- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 msgr2=0x55a90c9abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.451+0000 7f5847e00640 1 -- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 msgr2=0x55a90c9ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5847e00640 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90c9ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5847e00640 1 -- 172.21.15.55:0/2067903642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a90c8d98c0 con 0x55a90c5e54d0 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5847e00640 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f583000cda0 tx=0x7f5830000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2102f5ad0e93302b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5835600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830053e30 con 0x55a90c5e54d0 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5830053f90 con 0x55a90c5e54d0 2024-05-10T03:34:33.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a90c9d02b0 con 0x55a90c5e54d0 2024-05-10T03:34:33.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5830076170 con 0x55a90c5e54d0 2024-05-10T03:34:33.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5855e69740 10 client.7191 fetch_fsmap learned FSMap version 47 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.452+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55a90c8d5a90 con 0x55a90c5e54d0 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a90c9d02b0 con 0x55a90c5e54d0 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f583005b170 con 0x55a90c5e54d0 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 10 client.7191 fetch_fsmap finished waiting for FSMap version 47 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 10 client.7191 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 20 client.7191 populate_metadata read hostname 'smithi055' 2024-05-10T03:34:33.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90ca51ec0 0x55a90ca0ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 4 client.7191 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 --> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] -- command(tid 0: {"prefix": "session ls"}) v1 -- 0x7f583005fb10 con 0x55a90ca51ec0 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.453+0000 7f5847400640 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90ca51ec0 0x55a90ca0ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5847400640 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90ca51ec0 0x55a90ca0ac60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9f7d828157e790 server_cookie=ab8ee4373618da10 in_seq=0 out_seq=0 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5835600640 10 client.7191 ms_handle_connect on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:33.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5835600640 1 -- 172.21.15.55:0/2067903642 <== mds.0 v2:172.21.15.55:6834/709318521 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f583005fb10 con 0x55a90ca51ec0 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5835600640 10 client.7191 handle_command_reply: tid=0 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 client.7191 shutdown 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 msgr2=0x55a90ca517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f583000cda0 tx=0x7f5830000ed0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 shutdown_connections 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:6834/709318521,v1:172.21.15.55:6835/709318521] conn(0x55a90ca51ec0 0x55a90ca0ac60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a90c60e390 0x55a90c9ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a90c5ed6b0 0x55a90c9abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 --2- 172.21.15.55:0/2067903642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a90c5e54d0 0x55a90ca517c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5835600640 0 client.7191 ms_handle_reset on v2:172.21.15.55:6834/709318521 2024-05-10T03:34:33.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.454+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 >> 172.21.15.55:0/2067903642 conn(0x55a90c6024e0 msgr2=0x55a90c9aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 shutdown_connections 2024-05-10T03:34:33.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f5855e69740 1 -- 172.21.15.55:0/2067903642 wait complete. 2024-05-10T03:34:33.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f5855e69740 20 client.7191 trim_cache size 0 max 16384 2024-05-10T03:34:33.463 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-05-10T03:34:33.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 msgr2=0x7f582406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 0x7f582406d140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5838034000 tx=0x7f5838033040 comp rx=0 tx=0).stop 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 msgr2=0x7f58481599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.455+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f583000c3e0 tx=0x7f583000c8a0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 shutdown_connections 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f582406ac30 0x7f582406d140 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5848157170 0x7f58481e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58480d1ac0 0x7f5848156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 --2- 172.21.15.55:0/1673477566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58480b2460 0x7f58481599c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:33.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 >> 172.21.15.55:0/1673477566 conn(0x7f58480d3050 msgr2=0x7f58480b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:33.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 shutdown_connections 2024-05-10T03:34:33.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:33.456+0000 7f584da00640 1 -- 172.21.15.55:0/1673477566 wait complete. 2024-05-10T03:34:33.483 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-05-10T03:34:33.781 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-05-10T03:34:33.781 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-05-10T03:34:33.782 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:34:33.782 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:33.893 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:34.005 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:34:34.005 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:34.005 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:34.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:34.067 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:34.067 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:34.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:34.123 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:34.123 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:34.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:34.186 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:34.186 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:34.243 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:34.243 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.534+0000 7fc5a4a00640 1 Processor -- start 2024-05-10T03:34:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.534+0000 7fc5a4a00640 1 -- start start 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc5a4a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5a00b2550 0x7fc5a006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc5a4a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a006ad20 con 0x7fc5a015fe50 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc5a4a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a006a110 con 0x7fc5a00b2140 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc5a4a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a006a3f0 con 0x7fc5a00b2550 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc59e800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5a00b2550 0x7fc5a006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc59de00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc5a015fe50 0x7fc5a0141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49926/0 (socket says 172.21.15.55:49926) 2024-05-10T03:34:34.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.536+0000 7fc59de00640 1 -- 172.21.15.55:0/3048698300 learned_addr learned my addr 172.21.15.55:0/3048698300 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:34.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.537+0000 7fc59e800640 1 -- 172.21.15.55:0/3048698300 >> v1:172.21.15.162:6789/0 conn(0x7fc5a00b2140 legacy=0x7fc5a0142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:34.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.537+0000 7fc59e800640 1 -- 172.21.15.55:0/3048698300 >> v1:172.21.15.55:6789/0 conn(0x7fc5a015fe50 legacy=0x7fc5a0141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.537+0000 7fc59e800640 1 -- 172.21.15.55:0/3048698300 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a006a6d0 con 0x7fc5a00b2550 2024-05-10T03:34:34.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.537+0000 7fc59ca00640 1 -- 172.21.15.55:0/3048698300 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709927602 0 0) 0x7fc5a006ad20 con 0x7fc5a015fe50 2024-05-10T03:34:34.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.538+0000 7fc59e800640 1 --2- 172.21.15.55:0/3048698300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2550 0x7fc5a006b1b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097a0 tx=0x7fc59802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdb9b34d5ead4e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:34.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.538+0000 7fc59ca00640 1 -- 172.21.15.55:0/3048698300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598032440 con 0x7fc5a00b2550 2024-05-10T03:34:34.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.538+0000 7fc59ca00640 1 -- 172.21.15.55:0/3048698300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5980325a0 con 0x7fc5a00b2550 2024-05-10T03:34:34.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.538+0000 7fc59ca00640 1 -- 172.21.15.55:0/3048698300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598032930 con 0x7fc5a00b2550 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 -- 172.21.15.55:0/3048698300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2550 msgr2=0x7fc5a006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/3048698300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2550 0x7fc5a006b1b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097a0 tx=0x7fc59802a270 comp rx=0 tx=0).stop 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 -- 172.21.15.55:0/3048698300 shutdown_connections 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/3048698300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2550 0x7fc5a006b1b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 -- 172.21.15.55:0/3048698300 >> 172.21.15.55:0/3048698300 conn(0x7fc5a00cbb90 msgr2=0x7fc5a00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 -- 172.21.15.55:0/3048698300 shutdown_connections 2024-05-10T03:34:34.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.539+0000 7fc5a4a00640 1 -- 172.21.15.55:0/3048698300 wait complete. 2024-05-10T03:34:34.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.542+0000 7fc5a4a00640 1 Processor -- start 2024-05-10T03:34:34.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.542+0000 7fc5a4a00640 1 -- start start 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a00b2550 0x7fc5a015b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5a015fe50 0x7fc5a012dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a015e4f0 con 0x7fc5a00b2140 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00adfa0 con 0x7fc5a00b2550 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.543+0000 7fc5a4a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a015c480 con 0x7fc5a015fe50 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45798/0 (socket says 172.21.15.55:45798) 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 -- 172.21.15.55:0/2056672614 learned_addr learned my addr 172.21.15.55:0/2056672614 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:34.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 -- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5a015fe50 msgr2=0x7fc5a012dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5a015fe50 0x7fc5a012dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 -- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a00b2550 msgr2=0x7fc5a015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a00b2550 0x7fc5a015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.544+0000 7fc59de00640 1 -- 172.21.15.55:0/2056672614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009400 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc59de00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c7f0 tx=0x7fc59000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000a640 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59000a7c0 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a015b6c0 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000ab70 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.545+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a015c6d0 con 0x7fc5a00b2140 2024-05-10T03:34:34.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.546+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a0066f60 con 0x7fc5a00b2140 2024-05-10T03:34:34.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.547+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc59000acd0 con 0x7fc5a00b2140 2024-05-10T03:34:34.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.548+0000 7fc596000640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 0x7fc57c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:34.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.548+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc590088b70 con 0x7fc5a00b2140 2024-05-10T03:34:34.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.549+0000 7fc59d400640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 0x7fc57c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:34.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.550+0000 7fc59d400640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 0x7fc57c06d140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc588006fd0 tx=0x7fc588006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:34.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.553+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc590053a60 con 0x7fc5a00b2140 2024-05-10T03:34:34.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.792+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a006b670 con 0x7fc5a00b2140 2024-05-10T03:34:34.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.794+0000 7fc596000640 1 -- 172.21.15.55:0/2056672614 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fc590018070 con 0x7fc5a00b2140 2024-05-10T03:34:34.797 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:34.797 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:34.797 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 msgr2=0x7fc57c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 0x7fc57c06d140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc588006fd0 tx=0x7fc588006bc0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 msgr2=0x7fc5a012eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c7f0 tx=0x7fc59000ccb0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 shutdown_connections 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5a015fe50 0x7fc5a012dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc57c06ac30 0x7fc57c06d140 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a00b2550 0x7fc5a015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 --2- 172.21.15.55:0/2056672614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5a00b2140 0x7fc5a012eb90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.796+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 >> 172.21.15.55:0/2056672614 conn(0x7fc5a00cbb90 msgr2=0x7fc5a00cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.797+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 shutdown_connections 2024-05-10T03:34:34.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:34.797+0000 7fc5a4a00640 1 -- 172.21.15.55:0/2056672614 wait complete. 2024-05-10T03:34:34.825 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.105+0000 7faa07200640 1 Processor -- start 2024-05-10T03:34:35.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.105+0000 7faa07200640 1 -- start start 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa07200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faa0806c940 0x7faa0806db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa07200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa080b42c0 con 0x7faa0815d7b0 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa07200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa080b45a0 con 0x7faa0806e070 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa07200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080b4880 con 0x7faa0806c940 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa05e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faa0815d7b0 0x7faa0806c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49940/0 (socket says 172.21.15.55:49940) 2024-05-10T03:34:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.108+0000 7faa05e00640 1 -- 172.21.15.55:0/1665177462 learned_addr learned my addr 172.21.15.55:0/1665177462 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa05400640 1 --2- 172.21.15.55:0/1665177462 >> v2:172.21.15.162:3300/0 conn(0x7faa0806c940 0x7faa0806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990726912 0 0) 0x7faa080b42c0 con 0x7faa0815d7b0 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec003520 con 0x7faa0815d7b0 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198579321 0 0) 0x7fa9ec003520 con 0x7faa0815d7b0 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 >> v2:172.21.15.162:3300/0 conn(0x7faa0806c940 msgr2=0x7faa0806db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 --2- 172.21.15.55:0/1665177462 >> v2:172.21.15.162:3300/0 conn(0x7faa0806c940 0x7faa0806db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 >> v1:172.21.15.162:6789/0 conn(0x7faa0806e070 legacy=0x7faa080acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080b4b60 con 0x7faa0815d7b0 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa9f80019f0 con 0x7faa0815d7b0 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 >> v1:172.21.15.55:6789/0 conn(0x7faa0815d7b0 legacy=0x7faa0806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 0x7fa9ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0047c0 0x7fa9ec004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9ec009970 0x7fa9ec00bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b4880 con 0x7fa9ec0039e0 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00c380 con 0x7fa9ec009970 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00c540 con 0x7fa9ec0047c0 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.109+0000 7faa05400640 1 --2- 172.21.15.55:0/1665177462 >> v2:172.21.15.162:3300/0 conn(0x7faa0806c940 0x7faa0806db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 0x7fa9ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 -- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0047c0 msgr2=0x7fa9ec004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:35.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa06800640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9ec009970 0x7fa9ec00bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0047c0 0x7fa9ec004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 -- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9ec009970 msgr2=0x7fa9ec00bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9ec009970 0x7fa9ec00bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa06800640 1 -- 172.21.15.55:0/1665177462 reap_dead start 2024-05-10T03:34:35.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.110+0000 7faa05e00640 1 -- 172.21.15.55:0/1665177462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec003520 con 0x7fa9ec0039e0 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.111+0000 7faa05e00640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 0x7fa9ec003e50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800a800 tx=0x7fa9ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7887c8bd52fac328 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.111+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802ba10 con 0x7fa9ec0039e0 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f802bb70 con 0x7fa9ec0039e0 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa04a00640 1 -- 172.21.15.55:0/1665177462 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8034470 con 0x7fa9ec0039e0 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 -- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 msgr2=0x7fa9ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 0x7fa9ec003e50 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800a800 tx=0x7fa9ec0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 -- 172.21.15.55:0/1665177462 shutdown_connections 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 --2- 172.21.15.55:0/1665177462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9ec0039e0 0x7fa9ec003e50 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 -- 172.21.15.55:0/1665177462 >> 172.21.15.55:0/1665177462 conn(0x7faa08070eb0 msgr2=0x7faa080712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 -- 172.21.15.55:0/1665177462 shutdown_connections 2024-05-10T03:34:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.112+0000 7faa07200640 1 -- 172.21.15.55:0/1665177462 wait complete. 2024-05-10T03:34:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.115+0000 7faa07200640 1 Processor -- start 2024-05-10T03:34:35.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.115+0000 7faa07200640 1 -- start start 2024-05-10T03:34:35.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0039e0 0x7faa08154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa08160530 0x7faa081f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08156f70 con 0x7faa081f6ba0 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0815f330 con 0x7faa08160530 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.116+0000 7faa07200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa081561a0 con 0x7fa9ec0039e0 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45812/0 (socket says 172.21.15.55:45812) 2024-05-10T03:34:35.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 -- 172.21.15.55:0/2427439242 learned_addr learned my addr 172.21.15.55:0/2427439242 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:35.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa05e00640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0039e0 0x7faa08154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa05400640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa08160530 0x7faa081f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 -- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0039e0 msgr2=0x7faa08154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0039e0 0x7faa08154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 -- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa08160530 msgr2=0x7faa081f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa08160530 0x7faa081f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.117+0000 7faa06800640 1 -- 172.21.15.55:0/2427439242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa081f7550 con 0x7faa081f6ba0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7faa06800640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faa0806f510 tx=0x7faa0000d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000b070 con 0x7faa081f6ba0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa000052d0 con 0x7faa081f6ba0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00003830 con 0x7faa081f6ba0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa081f6fb0 con 0x7faa081f6ba0 2024-05-10T03:34:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.118+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa081566d0 con 0x7faa081f6ba0 2024-05-10T03:34:35.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.120+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0815dc10 con 0x7faa081f6ba0 2024-05-10T03:34:35.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.121+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7faa00003990 con 0x7faa081f6ba0 2024-05-10T03:34:35.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.122+0000 7fa9fe000640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 0x7fa9e406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.122+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7faa00088a70 con 0x7faa081f6ba0 2024-05-10T03:34:35.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.122+0000 7faa05e00640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 0x7fa9e406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.124+0000 7faa05e00640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 0x7fa9e406d110 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8033000 tx=0x7fa9f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.126+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faa00053a90 con 0x7faa081f6ba0 2024-05-10T03:34:35.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.331+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa080aef30 con 0x7faa081f6ba0 2024-05-10T03:34:35.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.334+0000 7fa9fe000640 1 -- 172.21.15.55:0/2427439242 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16443 (secure 0 0 0) 0x7faa00018020 con 0x7faa081f6ba0 2024-05-10T03:34:35.338 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:35.338 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:25.020639+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:35.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 msgr2=0x7fa9e406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 0x7fa9e406d110 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8033000 tx=0x7fa9f802e000 comp rx=0 tx=0).stop 2024-05-10T03:34:35.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 msgr2=0x7faa0815dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faa0806f510 tx=0x7faa0000d660 comp rx=0 tx=0).stop 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 shutdown_connections 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa081f6ba0 0x7faa0815dff0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa9e406ac00 0x7fa9e406d110 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa08160530 0x7faa081f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 --2- 172.21.15.55:0/2427439242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0039e0 0x7faa08154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.337+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 >> 172.21.15.55:0/2427439242 conn(0x7faa08070eb0 msgr2=0x7faa080712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.338+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 shutdown_connections 2024-05-10T03:34:35.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.338+0000 7faa07200640 1 -- 172.21.15.55:0/2427439242 wait complete. 2024-05-10T03:34:35.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:13.758492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:14.516676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:35.367 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:35.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.650+0000 7fe312e00640 1 Processor -- start 2024-05-10T03:34:35.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.650+0000 7fe312e00640 1 -- start start 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c0ce890 0x7fe30c161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe30c071de0 0x7fe30c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe30c06b660 0x7fe30c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0b1ef0 con 0x7fe30c06b660 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c06ba70 con 0x7fe30c071de0 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.653+0000 7fe312e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cd660 con 0x7fe30c0ce890 2024-05-10T03:34:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe30c06b660 0x7fe30c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe30c06b660 0x7fe30c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45824/0 (socket says 172.21.15.55:45824) 2024-05-10T03:34:35.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 -- 172.21.15.55:0/3263417947 learned_addr learned my addr 172.21.15.55:0/3263417947 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:35.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe311a00640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c0ce890 0x7fe30c161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe311000640 1 --2- 172.21.15.55:0/3263417947 >> v2:172.21.15.162:3300/0 conn(0x7fe30c071de0 0x7fe30c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 -- 172.21.15.55:0/3263417947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c0ce890 msgr2=0x7fe30c161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c0ce890 0x7fe30c161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 -- 172.21.15.55:0/3263417947 >> v2:172.21.15.162:3300/0 conn(0x7fe30c071de0 msgr2=0x7fe30c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 --2- 172.21.15.55:0/3263417947 >> v2:172.21.15.162:3300/0 conn(0x7fe30c071de0 0x7fe30c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.654+0000 7fe312400640 1 -- 172.21.15.55:0/3263417947 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0cd920 con 0x7fe30c06b660 2024-05-10T03:34:35.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.655+0000 7fe312400640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c06ffe0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e1e0 tx=0x7fe30402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa36757e31fa232 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.656+0000 7fe30be00640 1 -- 172.21.15.55:0/3263417947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304031600 con 0x7fe30c06b660 2024-05-10T03:34:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.656+0000 7fe30be00640 1 -- 172.21.15.55:0/3263417947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30402d050 con 0x7fe30c06b660 2024-05-10T03:34:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.656+0000 7fe30be00640 1 -- 172.21.15.55:0/3263417947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304031980 con 0x7fe30c06b660 2024-05-10T03:34:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.656+0000 7fe312e00640 1 -- 172.21.15.55:0/3263417947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 msgr2=0x7fe30c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.656+0000 7fe312e00640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c06ffe0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e1e0 tx=0x7fe30402a610 comp rx=0 tx=0).stop 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 -- 172.21.15.55:0/3263417947 shutdown_connections 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c06ffe0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 --2- 172.21.15.55:0/3263417947 >> v2:172.21.15.162:3300/0 conn(0x7fe30c071de0 0x7fe30c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 --2- 172.21.15.55:0/3263417947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c0ce890 0x7fe30c161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 -- 172.21.15.55:0/3263417947 >> 172.21.15.55:0/3263417947 conn(0x7fe30c06c750 msgr2=0x7fe30c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 -- 172.21.15.55:0/3263417947 shutdown_connections 2024-05-10T03:34:35.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.657+0000 7fe312e00640 1 -- 172.21.15.55:0/3263417947 wait complete. 2024-05-10T03:34:35.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.659+0000 7fe312e00640 1 Processor -- start 2024-05-10T03:34:35.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.659+0000 7fe312e00640 1 -- start start 2024-05-10T03:34:35.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c071de0 0x7fe30c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe30c0ce890 0x7fe30c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15d600 con 0x7fe30c06b660 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1f8610 con 0x7fe30c0ce890 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe312e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c158fb0 con 0x7fe30c071de0 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe311a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe311a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45828/0 (socket says 172.21.15.55:45828) 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe311a00640 1 -- 172.21.15.55:0/2170572565 learned_addr learned my addr 172.21.15.55:0/2170572565 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe311000640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c071de0 0x7fe30c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.661+0000 7fe311a00640 1 -- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c071de0 msgr2=0x7fe30c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe311a00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c071de0 0x7fe30c15a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe311a00640 1 -- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe30c0ce890 msgr2=0x7fe30c1f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe312400640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe30c0ce890 0x7fe30c1f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe311a00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe30c0ce890 0x7fe30c1f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe311a00640 1 -- 172.21.15.55:0/2170572565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304009b30 con 0x7fe30c06b660 2024-05-10T03:34:35.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe311a00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d620 tx=0x7fe30000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000b620 con 0x7fe30c06b660 2024-05-10T03:34:35.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000b780 con 0x7fe30c06b660 2024-05-10T03:34:35.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c159e10 con 0x7fe30c06b660 2024-05-10T03:34:35.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000bb30 con 0x7fe30c06b660 2024-05-10T03:34:35.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.662+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c1594e0 con 0x7fe30c06b660 2024-05-10T03:34:35.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.664+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04e290 con 0x7fe30c06b660 2024-05-10T03:34:35.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.665+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe30000bc90 con 0x7fe30c06b660 2024-05-10T03:34:35.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.666+0000 7fe309600640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 0x7fe2ec06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:35.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.666+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe300089a60 con 0x7fe30c06b660 2024-05-10T03:34:35.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.666+0000 7fe311000640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 0x7fe2ec06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:35.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.668+0000 7fe311000640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 0x7fe2ec06d140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:35.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.671+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3000549e0 con 0x7fe30c06b660 2024-05-10T03:34:35.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.868+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe30c161120 con 0x7fe30c06b660 2024-05-10T03:34:35.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.870+0000 7fe309600640 1 -- 172.21.15.55:0/2170572565 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe300058000 con 0x7fe30c06b660 2024-05-10T03:34:35.873 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 msgr2=0x7fe2ec06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 0x7fe2ec06d140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8008040 comp rx=0 tx=0).stop 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 msgr2=0x7fe30c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d620 tx=0x7fe30000dae0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 shutdown_connections 2024-05-10T03:34:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe2ec06ac30 0x7fe2ec06d140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe30c0ce890 0x7fe30c1f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe30c071de0 0x7fe30c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 --2- 172.21.15.55:0/2170572565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06b660 0x7fe30c157610 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 >> 172.21.15.55:0/2170572565 conn(0x7fe30c06c750 msgr2=0x7fe30c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 shutdown_connections 2024-05-10T03:34:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:35.873+0000 7fe312e00640 1 -- 172.21.15.55:0/2170572565 wait complete. 2024-05-10T03:34:35.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:34:36.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.187+0000 7f2371000640 1 Processor -- start 2024-05-10T03:34:36.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.187+0000 7f2371000640 1 -- start start 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2371000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f236c141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2371000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c071ce0 con 0x7f236c0cd750 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2371000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c070460 con 0x7f236c15bce0 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2371000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c070f40 con 0x7f236c070050 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2369400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f236c0cd750 0x7f236c15f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49954/0 (socket says 172.21.15.55:49954) 2024-05-10T03:34:36.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.190+0000 7f2369400640 1 -- 172.21.15.55:0/408642927 learned_addr learned my addr 172.21.15.55:0/408642927 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:36.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2369e00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f236c141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509979762 0 0) 0x7f2358001110 con 0x7f236c0cd750 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003520 con 0x7f236c0cd750 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734605695 0 0) 0x7f2350003520 con 0x7f236c0cd750 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 msgr2=0x7f236c141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f236c141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 >> v1:172.21.15.162:6789/0 conn(0x7f236c15bce0 legacy=0x7f236c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0cdb60 con 0x7f236c0cd750 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f2358001800 con 0x7f236c0cd750 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 >> v1:172.21.15.55:6789/0 conn(0x7f236c0cd750 legacy=0x7f236c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.191+0000 7f2368a00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f23500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2368a00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f23500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2368a00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2350004d00 0x7f2350005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c070f40 con 0x7f2350003f20 2024-05-10T03:34:36.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235000bb10 con 0x7f2350004d00 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235000bcd0 con 0x7f236c070050 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f23500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369e00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f23500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 -- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 msgr2=0x7f23500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f236a800640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2350004d00 0x7f2350005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f23500039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 -- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2350004d00 msgr2=0x7f2350005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2350004d00 0x7f2350005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369400640 1 -- 172.21.15.55:0/408642927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350003520 con 0x7f2350003f20 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f2369e00640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c070050 0x7f23500039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.192+0000 7f236a800640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2350004d00 0x7f2350005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:36.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.193+0000 7f2369400640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f23500047c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f235800c240 tx=0x7f23500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb12e4c4beb0345 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.194+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235802e070 con 0x7f2350003f20 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.194+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580058d0 con 0x7f2350003f20 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.194+0000 7f2368a00640 1 -- 172.21.15.55:0/408642927 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358005ca0 con 0x7f2350003f20 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.194+0000 7f2371000640 1 -- 172.21.15.55:0/408642927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 msgr2=0x7f23500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.194+0000 7f2371000640 1 --2- 172.21.15.55:0/408642927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f23500047c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f235800c240 tx=0x7f23500057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.195+0000 7f236a800640 1 -- 172.21.15.55:0/408642927 reap_dead start 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.195+0000 7f2371000640 1 -- 172.21.15.55:0/408642927 shutdown_connections 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.195+0000 7f2371000640 1 -- 172.21.15.55:0/408642927 >> 172.21.15.55:0/408642927 conn(0x7f236c06c120 msgr2=0x7f236c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.195+0000 7f2371000640 1 -- 172.21.15.55:0/408642927 shutdown_connections 2024-05-10T03:34:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.195+0000 7f2371000640 1 -- 172.21.15.55:0/408642927 wait complete. 2024-05-10T03:34:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.197+0000 7f2371000640 1 Processor -- start 2024-05-10T03:34:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.197+0000 7f2371000640 1 -- start start 2024-05-10T03:34:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f236c13d6d0 0x7f236c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c13dc20 0x7f236c0ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0ad530 con 0x7f2350003f20 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0a8600 con 0x7f236c13d6d0 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2371000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c06b450 con 0x7f236c13dc20 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45854/0 (socket says 172.21.15.55:45854) 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 -- 172.21.15.55:0/3966555654 learned_addr learned my addr 172.21.15.55:0/3966555654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 -- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c13dc20 msgr2=0x7f236c0ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c13dc20 0x7f236c0ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 -- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f236c13d6d0 msgr2=0x7f236c158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f236c13d6d0 0x7f236c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.199+0000 7f2369e00640 1 -- 172.21.15.55:0/3966555654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0a8850 con 0x7f2350003f20 2024-05-10T03:34:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2369e00640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f235c0108e0 tx=0x7f235c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00ec20 con 0x7f2350003f20 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00ed80 con 0x7f2350003f20 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c01c620 con 0x7f2350003f20 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c0a8b30 con 0x7f2350003f20 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.200+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c0a90f0 con 0x7f2350003f20 2024-05-10T03:34:36.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.201+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c066f60 con 0x7f2350003f20 2024-05-10T03:34:36.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.202+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f235c01c780 con 0x7f2350003f20 2024-05-10T03:34:36.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.203+0000 7f2362000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 0x7f234c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.203+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f235c089be0 con 0x7f2350003f20 2024-05-10T03:34:36.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.204+0000 7f2369400640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 0x7f234c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.205+0000 7f2369400640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 0x7f234c06d160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f235800c240 tx=0x7f2358009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.208+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f235c054be0 con 0x7f2350003f20 2024-05-10T03:34:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.404+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f236c067220 con 0x7f2350003f20 2024-05-10T03:34:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.405+0000 7f2362000640 1 -- 172.21.15.55:0/3966555654 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f235c058200 con 0x7f2350003f20 2024-05-10T03:34:36.407 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 msgr2=0x7f234c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 0x7f234c06d160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f235800c240 tx=0x7f2358009030 comp rx=0 tx=0).stop 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 msgr2=0x7f236c0a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f235c0108e0 tx=0x7f235c010da0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 shutdown_connections 2024-05-10T03:34:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f236c13dc20 0x7f236c0ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f234c06ac50 0x7f234c06d160 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f236c13d6d0 0x7f236c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 --2- 172.21.15.55:0/3966555654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2350003f20 0x7f236c0a98f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 >> 172.21.15.55:0/3966555654 conn(0x7f236c06c120 msgr2=0x7f236c13e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 shutdown_connections 2024-05-10T03:34:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.407+0000 7f2371000640 1 -- 172.21.15.55:0/3966555654 wait complete. 2024-05-10T03:34:36.434 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:34:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.699+0000 7fed8f000640 1 Processor -- start 2024-05-10T03:34:36.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.700+0000 7fed8f000640 1 -- start start 2024-05-10T03:34:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.702+0000 7fed8f000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed88071db0 0x7fed88158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.702+0000 7fed8f000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed880b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.702+0000 7fed8f000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880708b0 con 0x7fed8806a4b0 2024-05-10T03:34:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.702+0000 7fed8f000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880b2490 con 0x7fed88071db0 2024-05-10T03:34:36.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.702+0000 7fed8f000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed88070050 con 0x7fed880b3860 2024-05-10T03:34:36.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed7fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fed8806a4b0 0x7fed880ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49966/0 (socket says 172.21.15.55:49966) 2024-05-10T03:34:36.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed7fe00640 1 -- 172.21.15.55:0/2709527618 learned_addr learned my addr 172.21.15.55:0/2709527618 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed87e00640 1 --2- 172.21.15.55:0/2709527618 >> v2:172.21.15.162:3300/0 conn(0x7fed88071db0 0x7fed88158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed8d000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed880b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343346797 0 0) 0x7fed880708b0 con 0x7fed8806a4b0 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.703+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed6c003520 con 0x7fed8806a4b0 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 29497039 0 0) 0x7fed6c003520 con 0x7fed8806a4b0 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 msgr2=0x7fed880b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed880b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 >> v2:172.21.15.162:3300/0 conn(0x7fed88071db0 msgr2=0x7fed88158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 --2- 172.21.15.55:0/2709527618 >> v2:172.21.15.162:3300/0 conn(0x7fed88071db0 0x7fed88158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815b940 con 0x7fed8806a4b0 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fed70001670 con 0x7fed8806a4b0 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 >> v1:172.21.15.55:6789/0 conn(0x7fed8806a4b0 legacy=0x7fed880ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c003970 0x7fed6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed6c004d00 0x7fed6c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed88070050 con 0x7fed6c004d00 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c00b980 con 0x7fed6c003970 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c00bb40 con 0x7fed880b3860 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed8d000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed6c004d00 0x7fed6c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.704+0000 7fed7fe00640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed87e00640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c003970 0x7fed6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed8d000640 1 -- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 msgr2=0x7fed6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed8d000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed880b3860 0x7fed6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed8d000640 1 -- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c003970 msgr2=0x7fed6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:36.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed8d000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c003970 0x7fed6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.705+0000 7fed8d000640 1 -- 172.21.15.55:0/2709527618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c003520 con 0x7fed6c004d00 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.706+0000 7fed8d000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed6c004d00 0x7fed6c005110 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fed80009f90 tx=0x7fed6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675b6b2c284630fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.706+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80031610 con 0x7fed6c004d00 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.706+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000f040 con 0x7fed6c004d00 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.706+0000 7fed87400640 1 -- 172.21.15.55:0/2709527618 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80031970 con 0x7fed6c004d00 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 -- 172.21.15.55:0/2709527618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed6c004d00 msgr2=0x7fed6c005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 --2- 172.21.15.55:0/2709527618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed6c004d00 0x7fed6c005110 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fed80009f90 tx=0x7fed6c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8d000640 1 -- 172.21.15.55:0/2709527618 reap_dead start 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 -- 172.21.15.55:0/2709527618 shutdown_connections 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 -- 172.21.15.55:0/2709527618 >> 172.21.15.55:0/2709527618 conn(0x7fed880cbb90 msgr2=0x7fed880cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 -- 172.21.15.55:0/2709527618 shutdown_connections 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.707+0000 7fed8f000640 1 -- 172.21.15.55:0/2709527618 wait complete. 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.710+0000 7fed8f000640 1 Processor -- start 2024-05-10T03:34:36.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.710+0000 7fed8f000640 1 -- start start 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.711+0000 7fed8f000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 0x7fed881f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.711+0000 7fed8f000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed8813bd50 0x7fed880ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8f000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8f000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c5c90 con 0x7fed880afe50 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8f000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed881f74a0 con 0x7fed6c004d00 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8f000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c44c0 con 0x7fed8813bd50 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45868/0 (socket says 172.21.15.55:45868) 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 -- 172.21.15.55:0/3282246111 learned_addr learned my addr 172.21.15.55:0/3282246111 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed87e00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 0x7fed881f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 -- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed8813bd50 msgr2=0x7fed880ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed7fe00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed8813bd50 0x7fed880ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed8813bd50 0x7fed880ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 -- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 msgr2=0x7fed881f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 0x7fed881f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed8d000640 1 -- 172.21.15.55:0/3282246111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880b07a0 con 0x7fed880afe50 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.712+0000 7fed87e00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 0x7fed881f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed8d000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fed80009f90 tx=0x7fed80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80045430 con 0x7fed880afe50 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80040070 con 0x7fed880afe50 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800457d0 con 0x7fed880afe50 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed881f76f0 con 0x7fed880afe50 2024-05-10T03:34:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.713+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed881f7cb0 con 0x7fed880afe50 2024-05-10T03:34:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.714+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed88066f60 con 0x7fed880afe50 2024-05-10T03:34:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.716+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed80042030 con 0x7fed880afe50 2024-05-10T03:34:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.717+0000 7fed84c00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 0x7fed6406d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.717+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fed800aa8f0 con 0x7fed880afe50 2024-05-10T03:34:36.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.717+0000 7fed87e00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 0x7fed6406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:36.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.719+0000 7fed87e00640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 0x7fed6406d070 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fed880b4830 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:36.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.721+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed800759a0 con 0x7fed880afe50 2024-05-10T03:34:36.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.926+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed88067220 con 0x7fed880afe50 2024-05-10T03:34:36.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.927+0000 7fed84c00640 1 -- 172.21.15.55:0/3282246111 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed8004d340 con 0x7fed880afe50 2024-05-10T03:34:36.930 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:34:36.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.929+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 msgr2=0x7fed6406d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.929+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 0x7fed6406d070 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fed880b4830 tx=0x7fed78008040 comp rx=0 tx=0).stop 2024-05-10T03:34:36.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.929+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 msgr2=0x7fed880b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:36.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.929+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fed80009f90 tx=0x7fed80009750 comp rx=0 tx=0).stop 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 shutdown_connections 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed880afe50 0x7fed880b0260 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed8813bd50 0x7fed880ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed6406ab60 0x7fed6406d070 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 --2- 172.21.15.55:0/3282246111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed6c004d00 0x7fed881f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:36.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.930+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 >> 172.21.15.55:0/3282246111 conn(0x7fed880cbb90 msgr2=0x7fed88143d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:36.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.931+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 shutdown_connections 2024-05-10T03:34:36.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:36.931+0000 7fed8f000640 1 -- 172.21.15.55:0/3282246111 wait complete. 2024-05-10T03:34:36.962 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.244+0000 7fa4aac00640 1 Processor -- start 2024-05-10T03:34:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.244+0000 7fa4aac00640 1 -- start start 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4aac00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4a4071db0 0x7fa4a4158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4aac00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40708b0 con 0x7fa4a40b3860 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4aac00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a415b940 con 0x7fa4a406a4b0 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4aac00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a40b2490 con 0x7fa4a4071db0 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4a8c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4a40b3860 0x7fa4a40b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49974/0 (socket says 172.21.15.55:49974) 2024-05-10T03:34:37.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4a8c00640 1 -- 172.21.15.55:0/3919340057 learned_addr learned my addr 172.21.15.55:0/3919340057 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.247+0000 7fa4a3e00640 1 --2- 172.21.15.55:0/3919340057 >> v2:172.21.15.162:3300/0 conn(0x7fa4a4071db0 0x7fa4a4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737936345 0 0) 0x7fa4a40708b0 con 0x7fa4a40b3860 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa488005730 con 0x7fa4a40b3860 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261893613 0 0) 0x7fa488005730 con 0x7fa4a40b3860 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 >> v2:172.21.15.162:3300/0 conn(0x7fa4a4071db0 msgr2=0x7fa4a4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 --2- 172.21.15.55:0/3919340057 >> v2:172.21.15.162:3300/0 conn(0x7fa4a4071db0 0x7fa4a4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 >> v1:172.21.15.162:6789/0 conn(0x7fa4a406a4b0 legacy=0x7fa4a40ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4070260 con 0x7fa4a40b3860 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa49c0019f0 con 0x7fa4a40b3860 2024-05-10T03:34:37.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 >> v1:172.21.15.55:6789/0 conn(0x7fa4a40b3860 legacy=0x7fa4a40b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa488005b80 0x7fa488005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4880069d0 0x7fa488006de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa48800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48800e420 con 0x7fa488007520 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48800e590 con 0x7fa488005b80 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.248+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48800e750 con 0x7fa4880069d0 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa48800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a3e00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa488005b80 0x7fa488005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a3400640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4880069d0 0x7fa488006de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 -- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4880069d0 msgr2=0x7fa488006de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4880069d0 0x7fa488006de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 -- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa488005b80 msgr2=0x7fa488005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:37.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa488005b80 0x7fa488005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a8c00640 1 -- 172.21.15.55:0/3919340057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa488005730 con 0x7fa488007520 2024-05-10T03:34:37.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.249+0000 7fa4a3400640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4880069d0 0x7fa488006de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.250+0000 7fa4a8c00640 1 -- 172.21.15.55:0/3919340057 reap_dead start 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.250+0000 7fa4a8c00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa48800db90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00a830 tx=0x7fa488007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92219b4a70306479 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02ba50 con 0x7fa488007520 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c02bbf0 con 0x7fa488007520 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4a2a00640 1 -- 172.21.15.55:0/3919340057 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c034470 con 0x7fa488007520 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 -- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 msgr2=0x7fa48800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa48800db90 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00a830 tx=0x7fa488007a00 comp rx=0 tx=0).stop 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 -- 172.21.15.55:0/3919340057 shutdown_connections 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 --2- 172.21.15.55:0/3919340057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa48800db90 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 -- 172.21.15.55:0/3919340057 >> 172.21.15.55:0/3919340057 conn(0x7fa4a40cbb90 msgr2=0x7fa4a40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:37.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 -- 172.21.15.55:0/3919340057 shutdown_connections 2024-05-10T03:34:37.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.251+0000 7fa4aac00640 1 -- 172.21.15.55:0/3919340057 wait complete. 2024-05-10T03:34:37.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.254+0000 7fa4aac00640 1 Processor -- start 2024-05-10T03:34:37.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.254+0000 7fa4aac00640 1 -- start start 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 0x7fa4a40a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4a40a9f40 0x7fa4a4158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ac190 con 0x7fa488007520 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40a84d0 con 0x7fa4a40a9f40 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4aac00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40a8710 con 0x7fa4a40aba90 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45892/0 (socket says 172.21.15.55:45892) 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 -- 172.21.15.55:0/2731916260 learned_addr learned my addr 172.21.15.55:0/2731916260 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3400640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 0x7fa4a40a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a8c00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4a40a9f40 0x7fa4a4158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 -- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 msgr2=0x7fa4a40a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 0x7fa4a40a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.256+0000 7fa4a3e00640 1 -- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4a40a9f40 msgr2=0x7fa4a4158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa4a3e00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4a40a9f40 0x7fa4a4158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa4a3e00640 1 -- 172.21.15.55:0/2731916260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a40a8960 con 0x7fa488007520 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa4a3400640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 0x7fa4a40a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa4a3e00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa494011590 tx=0x7fa494011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494004500 con 0x7fa488007520 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494003610 con 0x7fa488007520 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494003980 con 0x7fa488007520 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.257+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40a8c40 con 0x7fa488007520 2024-05-10T03:34:37.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.258+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40a9200 con 0x7fa488007520 2024-05-10T03:34:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.259+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4066f60 con 0x7fa488007520 2024-05-10T03:34:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.260+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa494003ae0 con 0x7fa488007520 2024-05-10T03:34:37.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.261+0000 7fa49be00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 0x7fa48406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.262+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa494089ce0 con 0x7fa488007520 2024-05-10T03:34:37.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.262+0000 7fa4a3400640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 0x7fa48406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.264+0000 7fa4a3400640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 0x7fa48406d110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40718e0 tx=0x7fa490009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.266+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa494054b50 con 0x7fa488007520 2024-05-10T03:34:37.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.503+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4a4067220 con 0x7fa488007520 2024-05-10T03:34:37.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.504+0000 7fa49be00640 1 -- 172.21.15.55:0/2731916260 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7fa494058170 con 0x7fa488007520 2024-05-10T03:34:37.507 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:37.507 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:37.507 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 msgr2=0x7fa48406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 0x7fa48406d110 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40718e0 tx=0x7fa490009040 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 msgr2=0x7fa4a415cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa494011590 tx=0x7fa494011a50 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 shutdown_connections 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa48406ac00 0x7fa48406d110 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4a40a9f40 0x7fa4a4158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4a40aba90 0x7fa4a40a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 --2- 172.21.15.55:0/2731916260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa488007520 0x7fa4a415cbb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 >> 172.21.15.55:0/2731916260 conn(0x7fa4a40cbb90 msgr2=0x7fa4a40ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:37.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 shutdown_connections 2024-05-10T03:34:37.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.507+0000 7fa4aac00640 1 -- 172.21.15.55:0/2731916260 wait complete. 2024-05-10T03:34:37.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:37.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.817+0000 7f94b8e00640 1 Processor -- start 2024-05-10T03:34:37.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.817+0000 7f94b8e00640 1 -- start start 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b8e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94b41549e0 0x7f94b40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b8e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94b4161a00 0x7f94b4071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b8e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b40b37a0 con 0x7f94b40b3390 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b8e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40b39f0 con 0x7f94b41549e0 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b8e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40b3cd0 con 0x7f94b4161a00 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94b3400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94b41549e0 0x7f94b40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94abe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f94b40b3390 0x7f94b41612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49980/0 (socket says 172.21.15.55:49980) 2024-05-10T03:34:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.820+0000 7f94abe00640 1 -- 172.21.15.55:0/2709333977 learned_addr learned my addr 172.21.15.55:0/2709333977 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b3400640 1 -- 172.21.15.55:0/2709333977 >> v2:172.21.15.162:3300/0 conn(0x7f94b4161a00 msgr2=0x7f94b4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b3e00640 1 --2- 172.21.15.55:0/2709333977 >> v2:172.21.15.162:3300/0 conn(0x7f94b4161a00 0x7f94b4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b3400640 1 --2- 172.21.15.55:0/2709333977 >> v2:172.21.15.162:3300/0 conn(0x7f94b4161a00 0x7f94b4071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b3400640 1 -- 172.21.15.55:0/2709333977 >> v1:172.21.15.55:6789/0 conn(0x7f94b40b3390 legacy=0x7f94b41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b2a00640 1 -- 172.21.15.55:0/2709333977 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975197196 0 0) 0x7f94b40b37a0 con 0x7f94b40b3390 2024-05-10T03:34:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.821+0000 7f94b3400640 1 -- 172.21.15.55:0/2709333977 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40b3fb0 con 0x7f94b41549e0 2024-05-10T03:34:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.822+0000 7f94b3400640 1 --2- 172.21.15.55:0/2709333977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b40b2e50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f94a40097a0 tx=0x7f94a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7520568f67e1b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.822+0000 7f94b2a00640 1 -- 172.21.15.55:0/2709333977 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a402ae50 con 0x7f94b41549e0 2024-05-10T03:34:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.822+0000 7f94b2a00640 1 -- 172.21.15.55:0/2709333977 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a4032440 con 0x7f94b41549e0 2024-05-10T03:34:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.822+0000 7f94b2a00640 1 -- 172.21.15.55:0/2709333977 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40327d0 con 0x7f94b41549e0 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 -- 172.21.15.55:0/2709333977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 msgr2=0x7f94b40b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2709333977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b40b2e50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f94a40097a0 tx=0x7f94a402a140 comp rx=0 tx=0).stop 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 -- 172.21.15.55:0/2709333977 shutdown_connections 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2709333977 >> v2:172.21.15.162:3300/0 conn(0x7f94b4161a00 0x7f94b4071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2709333977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b40b2e50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 -- 172.21.15.55:0/2709333977 >> 172.21.15.55:0/2709333977 conn(0x7f94b406a110 msgr2=0x7f94b406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 -- 172.21.15.55:0/2709333977 shutdown_connections 2024-05-10T03:34:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.823+0000 7f94b8e00640 1 -- 172.21.15.55:0/2709333977 wait complete. 2024-05-10T03:34:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.826+0000 7f94b8e00640 1 Processor -- start 2024-05-10T03:34:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.826+0000 7f94b8e00640 1 -- start start 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.827+0000 7f94b8e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b40b3390 0x7f94b41f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.827+0000 7f94b8e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.827+0000 7f94b8e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 0x7f94b414eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b8e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41f3900 con 0x7f94b41549e0 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b8e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40cc9d0 con 0x7f94b40b3390 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b8e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b413ff40 con 0x7f94b414d370 2024-05-10T03:34:37.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45896/0 (socket says 172.21.15.55:45896) 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 -- 172.21.15.55:0/2772306717 learned_addr learned my addr 172.21.15.55:0/2772306717 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b3e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 0x7f94b414eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b3400640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b40b3390 0x7f94b41f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 -- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 msgr2=0x7f94b414eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 0x7f94b414eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 -- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b40b3390 msgr2=0x7f94b41f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b40b3390 0x7f94b41f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94abe00640 1 -- 172.21.15.55:0/2772306717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4009400 con 0x7f94b41549e0 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.828+0000 7f94b3e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 0x7f94b414eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94abe00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c3e0 tx=0x7f949c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0074e0 con 0x7f94b41549e0 2024-05-10T03:34:37.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c007660 con 0x7f94b41549e0 2024-05-10T03:34:37.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c004a20 con 0x7f94b41549e0 2024-05-10T03:34:37.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b40ccc20 con 0x7f94b41549e0 2024-05-10T03:34:37.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.829+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b40cd1e0 con 0x7f94b41549e0 2024-05-10T03:34:37.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.831+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b404e290 con 0x7f94b41549e0 2024-05-10T03:34:37.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.832+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f949c004b80 con 0x7f94b41549e0 2024-05-10T03:34:37.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.833+0000 7f94ab400640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 0x7f949006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:37.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.833+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f949c089350 con 0x7f94b41549e0 2024-05-10T03:34:37.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.833+0000 7f94b3400640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 0x7f949006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:37.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.835+0000 7f94b3400640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 0x7f949006d110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94a4034000 tx=0x7f94a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:37.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:37.837+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f949c057e00 con 0x7f94b41549e0 2024-05-10T03:34:38.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.039+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94b406db30 con 0x7f94b41549e0 2024-05-10T03:34:38.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.041+0000 7f94ab400640 1 -- 172.21.15.55:0/2772306717 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f949c057990 con 0x7f94b41549e0 2024-05-10T03:34:38.044 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:38.044 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:25.020639+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.043+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 msgr2=0x7f949006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 0x7f949006d110 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94a4034000 tx=0x7f94a402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 msgr2=0x7f94b413fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c3e0 tx=0x7f949c00c8a0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 shutdown_connections 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b414d370 0x7f94b414eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94b41549e0 0x7f94b413fa00 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f949006ac00 0x7f949006d110 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 --2- 172.21.15.55:0/2772306717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b40b3390 0x7f94b41f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 >> 172.21.15.55:0/2772306717 conn(0x7f94b406a110 msgr2=0x7f94b4153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 shutdown_connections 2024-05-10T03:34:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.044+0000 7f94b8e00640 1 -- 172.21.15.55:0/2772306717 wait complete. 2024-05-10T03:34:38.074 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:34:38.075 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:38.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.360+0000 7f8527600640 1 Processor -- start 2024-05-10T03:34:38.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.360+0000 7f8527600640 1 -- start start 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8527600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85200b2140 0x7f8520142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8527600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852006ad20 con 0x7f852015fe50 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8527600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852006a110 con 0x7f85200b2550 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8527600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852006a3f0 con 0x7f85200b2140 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8524c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f852015fe50 0x7f8520141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49988/0 (socket says 172.21.15.55:49988) 2024-05-10T03:34:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.363+0000 7f8524c00640 1 -- 172.21.15.55:0/3411457430 learned_addr learned my addr 172.21.15.55:0/3411457430 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851fe00640 1 --2- 172.21.15.55:0/3411457430 >> v2:172.21.15.162:3300/0 conn(0x7f85200b2140 0x7f8520142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839236682 0 0) 0x7f852006ad20 con 0x7f852015fe50 2024-05-10T03:34:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003520 con 0x7f852015fe50 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167608978 0 0) 0x7f8504003520 con 0x7f852015fe50 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 >> v2:172.21.15.162:3300/0 conn(0x7f85200b2140 msgr2=0x7f8520142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 --2- 172.21.15.55:0/3411457430 >> v2:172.21.15.162:3300/0 conn(0x7f85200b2140 0x7f8520142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 >> v1:172.21.15.162:6789/0 conn(0x7f85200b2550 legacy=0x7f852006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852006a6d0 con 0x7f852015fe50 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8508001670 con 0x7f852015fe50 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 >> v1:172.21.15.55:6789/0 conn(0x7f852015fe50 legacy=0x7f8520141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.364+0000 7f851f400640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8504003970 0x7f8504003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851f400640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040047c0 0x7f8504004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851f400640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 0x7f850400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852006a3f0 con 0x7f8504005180 2024-05-10T03:34:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c110 con 0x7f85040047c0 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c2d0 con 0x7f8504003970 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8525600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 0x7f850400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8524c00640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8504003970 0x7f8504003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8525600640 1 -- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8504003970 msgr2=0x7f8504003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f851fe00640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040047c0 0x7f8504004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8525600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8504003970 0x7f8504003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8525600640 1 -- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040047c0 msgr2=0x7f8504004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.365+0000 7f8525600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040047c0 0x7f8504004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.366+0000 7f8525600640 1 -- 172.21.15.55:0/3411457430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504003520 con 0x7f8504005180 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.366+0000 7f8525600640 1 -- 172.21.15.55:0/3411457430 reap_dead start 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.367+0000 7f8525600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 0x7f850400b980 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8518001750 tx=0x7f85040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff0302224120113 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.367+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180052d0 con 0x7f8504005180 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.367+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518005470 con 0x7f8504005180 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.367+0000 7f851f400640 1 -- 172.21.15.55:0/3411457430 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851802ab20 con 0x7f8504005180 2024-05-10T03:34:38.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 -- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 msgr2=0x7f850400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 0x7f850400b980 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8518001750 tx=0x7f85040057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 -- 172.21.15.55:0/3411457430 shutdown_connections 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 --2- 172.21.15.55:0/3411457430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8504005180 0x7f850400b980 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 -- 172.21.15.55:0/3411457430 >> 172.21.15.55:0/3411457430 conn(0x7f85200cbb90 msgr2=0x7f85200cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 -- 172.21.15.55:0/3411457430 shutdown_connections 2024-05-10T03:34:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.368+0000 7f8527600640 1 -- 172.21.15.55:0/3411457430 wait complete. 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.370+0000 7f8527600640 1 Processor -- start 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.371+0000 7f8527600640 1 -- start start 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504005180 0x7f852015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852015b4e0 0x7f852013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852012d960 con 0x7f852012d180 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852015e800 con 0x7f8504005180 2024-05-10T03:34:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.372+0000 7f8527600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852013d560 con 0x7f852015b4e0 2024-05-10T03:34:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45908/0 (socket says 172.21.15.55:45908) 2024-05-10T03:34:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 -- 172.21.15.55:0/2582663238 learned_addr learned my addr 172.21.15.55:0/2582663238 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 -- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852015b4e0 msgr2=0x7f852013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852015b4e0 0x7f852013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 -- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504005180 msgr2=0x7f852015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8524c00640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504005180 0x7f852015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504005180 0x7f852015dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 -- 172.21.15.55:0/2582663238 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852013d800 con 0x7f852012d180 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.373+0000 7f8525600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8518005040 tx=0x7f8518001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.374+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518038070 con 0x7f852012d180 2024-05-10T03:34:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.374+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518007240 con 0x7f852012d180 2024-05-10T03:34:38.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.374+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180075f0 con 0x7f852012d180 2024-05-10T03:34:38.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.374+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852013dae0 con 0x7f852012d180 2024-05-10T03:34:38.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.374+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852013e0a0 con 0x7f852012d180 2024-05-10T03:34:38.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.375+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520066f60 con 0x7f852012d180 2024-05-10T03:34:38.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.376+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f85180067d0 con 0x7f852012d180 2024-05-10T03:34:38.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.377+0000 7f851cc00640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 0x7f84fc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.377+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f85180a8550 con 0x7f852012d180 2024-05-10T03:34:38.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.378+0000 7f8524c00640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 0x7f84fc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.379+0000 7f8524c00640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 0x7f84fc06d110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8508001eb0 tx=0x7f8508000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.382+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f85180734d0 con 0x7f852012d180 2024-05-10T03:34:38.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.616+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8520067220 con 0x7f852012d180 2024-05-10T03:34:38.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.617+0000 7f851cc00640 1 -- 172.21.15.55:0/2582663238 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f8518001370 con 0x7f852012d180 2024-05-10T03:34:38.620 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:38.621 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:38.621 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 msgr2=0x7f84fc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 0x7f84fc06d110 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8508001eb0 tx=0x7f8508000ed0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 msgr2=0x7f852013eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8518005040 tx=0x7f8518001120 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 shutdown_connections 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f852012d180 0x7f852013eb00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852015b4e0 0x7f852013e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f84fc06ac00 0x7f84fc06d110 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 --2- 172.21.15.55:0/2582663238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504005180 0x7f852015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 >> 172.21.15.55:0/2582663238 conn(0x7f85200cbb90 msgr2=0x7f85200cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 shutdown_connections 2024-05-10T03:34:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.620+0000 7f8527600640 1 -- 172.21.15.55:0/2582663238 wait complete. 2024-05-10T03:34:38.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:38.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.926+0000 7faca5600640 1 Processor -- start 2024-05-10T03:34:38.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.926+0000 7faca5600640 1 -- start start 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.929+0000 7faca5600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 0x7faca00cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.929+0000 7faca5600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.929+0000 7faca5600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca015ad30 con 0x7faca00cb730 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.929+0000 7faca5600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca015b010 con 0x7faca00b4880 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.929+0000 7faca5600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca015b2f0 con 0x7faca015a5d0 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.930+0000 7faca4c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.930+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 0x7faca00cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.930+0000 7faca4c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:35058/0 (socket says 172.21.15.55:35058) 2024-05-10T03:34:38.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.930+0000 7faca4c00640 1 -- 172.21.15.55:0/3242566381 learned_addr learned my addr 172.21.15.55:0/3242566381 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:38.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.931+0000 7fac9fe00640 1 -- 172.21.15.55:0/3242566381 >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 msgr2=0x7faca00ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:38.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.931+0000 7fac9fe00640 1 --2- 172.21.15.55:0/3242566381 >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.931+0000 7fac9fe00640 1 -- 172.21.15.55:0/3242566381 >> v1:172.21.15.162:6789/0 conn(0x7faca00cb730 legacy=0x7faca00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.931+0000 7fac9fe00640 1 -- 172.21.15.55:0/3242566381 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca015b5d0 con 0x7faca015a5d0 2024-05-10T03:34:38.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.931+0000 7faca4c00640 1 --2- 172.21.15.55:0/3242566381 >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:38.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.932+0000 7fac9fe00640 1 --2- 172.21.15.55:0/3242566381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 0x7faca00cb1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac9000ba40 tx=0x7fac9002c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1459cbbdd7c396 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.933+0000 7fac9ea00640 1 -- 172.21.15.55:0/3242566381 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90032750 con 0x7faca015a5d0 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.933+0000 7fac9ea00640 1 -- 172.21.15.55:0/3242566381 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac900328b0 con 0x7faca015a5d0 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.933+0000 7fac9ea00640 1 -- 172.21.15.55:0/3242566381 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90032c40 con 0x7faca015a5d0 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 -- 172.21.15.55:0/3242566381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 msgr2=0x7faca00cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 --2- 172.21.15.55:0/3242566381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 0x7faca00cb1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac9000ba40 tx=0x7fac9002c850 comp rx=0 tx=0).stop 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 -- 172.21.15.55:0/3242566381 shutdown_connections 2024-05-10T03:34:38.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 --2- 172.21.15.55:0/3242566381 >> v2:172.21.15.162:3300/0 conn(0x7faca00b4880 0x7faca00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 --2- 172.21.15.55:0/3242566381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca015a5d0 0x7faca00cb1f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 -- 172.21.15.55:0/3242566381 >> 172.21.15.55:0/3242566381 conn(0x7faca006a110 msgr2=0x7faca006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:38.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 -- 172.21.15.55:0/3242566381 shutdown_connections 2024-05-10T03:34:38.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.934+0000 7faca5600640 1 -- 172.21.15.55:0/3242566381 wait complete. 2024-05-10T03:34:38.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.936+0000 7faca5600640 1 Processor -- start 2024-05-10T03:34:38.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.936+0000 7faca5600640 1 -- start start 2024-05-10T03:34:38.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca00cb730 0x7faca0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca00cdee0 0x7faca01426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0158c20 con 0x7faca015a5d0 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca01432e0 con 0x7faca00cdee0 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7faca5600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca0143480 con 0x7faca00cb730 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7fac9f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7fac9f400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45916/0 (socket says 172.21.15.55:45916) 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.938+0000 7fac9f400640 1 -- 172.21.15.55:0/2298893588 learned_addr learned my addr 172.21.15.55:0/2298893588 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9fe00640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca00cb730 0x7faca0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7faca4c00640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca00cdee0 0x7faca01426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 -- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca00cb730 msgr2=0x7faca0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca00cb730 0x7faca0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 -- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca00cdee0 msgr2=0x7faca01426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:38.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca00cdee0 0x7faca01426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:38.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 -- 172.21.15.55:0/2298893588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980094d0 con 0x7faca015a5d0 2024-05-10T03:34:38.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.939+0000 7fac9f400640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c730 tx=0x7fac8800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.940+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88003b90 con 0x7faca015a5d0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.940+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88003d10 con 0x7faca015a5d0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.940+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88005310 con 0x7faca015a5d0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.940+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9000b6a0 con 0x7faca015a5d0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.940+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca0158280 con 0x7faca015a5d0 2024-05-10T03:34:38.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.941+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca0155670 con 0x7faca015a5d0 2024-05-10T03:34:38.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.943+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fac88005470 con 0x7faca015a5d0 2024-05-10T03:34:38.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.944+0000 7fac97e00640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 0x7fac7c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:38.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.944+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fac880892a0 con 0x7faca015a5d0 2024-05-10T03:34:38.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.945+0000 7fac9fe00640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 0x7fac7c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:38.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.946+0000 7fac9fe00640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 0x7fac7c06d110 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac90037000 tx=0x7fac90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:38.949+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7faca0155670 con 0x7faca015a5d0 2024-05-10T03:34:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.188+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faca0155670 con 0x7faca015a5d0 2024-05-10T03:34:39.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.190+0000 7fac97e00640 1 -- 172.21.15.55:0/2298893588 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4736 (secure 0 0 0) 0x7faca0155670 con 0x7faca015a5d0 2024-05-10T03:34:39.193 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:39.193 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:15.487725+0000","modified":"2024-05-10T03:34:18.971182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/709318521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":709318521},{"type":"v1","addr":"172.21.15.55:6835","nonce":709318521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-05-10T03:34:39.193 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 47 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 msgr2=0x7fac7c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 0x7fac7c06d110 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac90037000 tx=0x7fac90008040 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 msgr2=0x7faca00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fac8800c730 tx=0x7fac8800cb30 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 shutdown_connections 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fac7c06ac00 0x7fac7c06d110 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca00cdee0 0x7faca01426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faca015a5d0 0x7faca00cd9a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 --2- 172.21.15.55:0/2298893588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca00cb730 0x7faca0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.192+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 >> 172.21.15.55:0/2298893588 conn(0x7faca006a110 msgr2=0x7faca015e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.193+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 shutdown_connections 2024-05-10T03:34:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.193+0000 7faca5600640 1 -- 172.21.15.55:0/2298893588 wait complete. 2024-05-10T03:34:39.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.500+0000 7f26eb200640 1 Processor -- start 2024-05-10T03:34:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.500+0000 7f26eb200640 1 -- start start 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26eb200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f26ec0b6680 0x7f26ec15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26eb200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26ec0b2170 0x7f26ec1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26eb200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ec06bac0 con 0x7f26ec1616e0 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26eb200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec0d2b20 con 0x7f26ec0b2170 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26eb200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ec15db20 con 0x7f26ec0b6680 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26e9400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26ec0b2170 0x7f26ec1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26ea800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f26ec1616e0 0x7f26ec0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49998/0 (socket says 172.21.15.55:49998) 2024-05-10T03:34:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.503+0000 7f26ea800640 1 -- 172.21.15.55:0/2338814006 learned_addr learned my addr 172.21.15.55:0/2338814006 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e9e00640 1 --2- 172.21.15.55:0/2338814006 >> v2:172.21.15.162:3300/0 conn(0x7f26ec0b6680 0x7f26ec15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e9400640 1 -- 172.21.15.55:0/2338814006 >> v2:172.21.15.162:3300/0 conn(0x7f26ec0b6680 msgr2=0x7f26ec15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e9400640 1 --2- 172.21.15.55:0/2338814006 >> v2:172.21.15.162:3300/0 conn(0x7f26ec0b6680 0x7f26ec15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e9400640 1 -- 172.21.15.55:0/2338814006 >> v1:172.21.15.55:6789/0 conn(0x7f26ec1616e0 legacy=0x7f26ec0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e9400640 1 -- 172.21.15.55:0/2338814006 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0d25a0 con 0x7f26ec0b2170 2024-05-10T03:34:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.504+0000 7f26e8a00640 1 -- 172.21.15.55:0/2338814006 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703101343 0 0) 0x7f26ec06bac0 con 0x7f26ec1616e0 2024-05-10T03:34:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.505+0000 7f26e9400640 1 --2- 172.21.15.55:0/2338814006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec1611a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f26ec08c070 tx=0x7f26d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420ac8fb9bf8158a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.505+0000 7f26e8a00640 1 -- 172.21.15.55:0/2338814006 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8032440 con 0x7f26ec0b2170 2024-05-10T03:34:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.505+0000 7f26e8a00640 1 -- 172.21.15.55:0/2338814006 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80325a0 con 0x7f26ec0b2170 2024-05-10T03:34:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.505+0000 7f26e8a00640 1 -- 172.21.15.55:0/2338814006 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8032930 con 0x7f26ec0b2170 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 -- 172.21.15.55:0/2338814006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 msgr2=0x7f26ec1611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 --2- 172.21.15.55:0/2338814006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec1611a0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f26ec08c070 tx=0x7f26d802a1a0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 -- 172.21.15.55:0/2338814006 shutdown_connections 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 --2- 172.21.15.55:0/2338814006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec1611a0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 --2- 172.21.15.55:0/2338814006 >> v2:172.21.15.162:3300/0 conn(0x7f26ec0b6680 0x7f26ec15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 -- 172.21.15.55:0/2338814006 >> 172.21.15.55:0/2338814006 conn(0x7f26ec0b5730 msgr2=0x7f26ec0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 -- 172.21.15.55:0/2338814006 shutdown_connections 2024-05-10T03:34:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.506+0000 7f26eb200640 1 -- 172.21.15.55:0/2338814006 wait complete. 2024-05-10T03:34:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.509+0000 7f26eb200640 1 Processor -- start 2024-05-10T03:34:39.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.509+0000 7f26eb200640 1 -- start start 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26ec0b6680 0x7f26ec1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26ec1616e0 0x7f26ec0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec160c20 con 0x7f26ec0b2170 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec1f4760 con 0x7f26ec0b6680 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.510+0000 7f26eb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec15f930 con 0x7f26ec1616e0 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45922/0 (socket says 172.21.15.55:45922) 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 -- 172.21.15.55:0/361208917 learned_addr learned my addr 172.21.15.55:0/361208917 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26ea800640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26ec1616e0 0x7f26ec0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9400640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26ec0b6680 0x7f26ec1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 -- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26ec1616e0 msgr2=0x7f26ec0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26ec1616e0 0x7f26ec0c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 -- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26ec0b6680 msgr2=0x7f26ec1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26ec0b6680 0x7f26ec1592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.511+0000 7f26e9e00640 1 -- 172.21.15.55:0/361208917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009400 con 0x7f26ec0b2170 2024-05-10T03:34:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26e9e00640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00d620 tx=0x7f26dc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:39.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00b620 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc00b780 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00bb30 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec160790 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.512+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec15fb80 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.513+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec067740 con 0x7f26ec0b2170 2024-05-10T03:34:39.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.514+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f26dc00bc90 con 0x7f26ec0b2170 2024-05-10T03:34:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.515+0000 7f26e2000640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 0x7f26cc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.516+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f26dc089ad0 con 0x7f26ec0b2170 2024-05-10T03:34:39.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.516+0000 7f26e9400640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 0x7f26cc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:39.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.518+0000 7f26e9400640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 0x7f26cc06d140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f26d8034000 tx=0x7f26d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:39.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.520+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f26dc054a50 con 0x7f26ec0b2170 2024-05-10T03:34:39.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.730+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26ec04e290 con 0x7f26ec0b2170 2024-05-10T03:34:39.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.734+0000 7f26e2000640 1 -- 172.21.15.55:0/361208917 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f26dc058070 con 0x7f26ec0b2170 2024-05-10T03:34:39.737 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:39.737 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:25.020639+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:13.758492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:14.516676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.24","mappings":[{"from":1,"to":5}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 msgr2=0x7f26cc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 0x7f26cc06d140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f26d8034000 tx=0x7f26d802e000 comp rx=0 tx=0).stop 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 msgr2=0x7f26ec14d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00d620 tx=0x7f26dc00dae0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 shutdown_connections 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26ec1616e0 0x7f26ec0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f26cc06ac30 0x7f26cc06d140 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26ec0b6680 0x7f26ec1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 --2- 172.21.15.55:0/361208917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26ec0b2170 0x7f26ec14d500 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:39.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 >> 172.21.15.55:0/361208917 conn(0x7f26ec0b5730 msgr2=0x7f26ec0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:39.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 shutdown_connections 2024-05-10T03:34:39.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:39.737+0000 7f26eb200640 1 -- 172.21.15.55:0/361208917 wait complete. 2024-05-10T03:34:39.767 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:34:40.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.053+0000 7efd2ca00640 1 Processor -- start 2024-05-10T03:34:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.054+0000 7efd2ca00640 1 -- start start 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd2ca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efd280b2040 0x7efd28143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd2ca00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cdbe0 con 0x7efd2806b120 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd2ca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2815e8b0 con 0x7efd281435d0 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd2ca00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd280ce140 con 0x7efd280b2040 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd26800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efd280b2040 0x7efd28143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd27200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efd2806b120 0x7efd2806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50002/0 (socket says 172.21.15.55:50002) 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.057+0000 7efd27200640 1 -- 172.21.15.55:0/1990023418 learned_addr learned my addr 172.21.15.55:0/1990023418 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:40.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.058+0000 7efd26800640 1 -- 172.21.15.55:0/1990023418 >> v1:172.21.15.162:6789/0 conn(0x7efd281435d0 legacy=0x7efd280ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.058+0000 7efd26800640 1 -- 172.21.15.55:0/1990023418 >> v1:172.21.15.55:6789/0 conn(0x7efd2806b120 legacy=0x7efd2806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.058+0000 7efd26800640 1 -- 172.21.15.55:0/1990023418 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2815ede0 con 0x7efd280b2040 2024-05-10T03:34:40.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.058+0000 7efd25400640 1 -- 172.21.15.55:0/1990023418 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302347959 0 0) 0x7efd280cdbe0 con 0x7efd2806b120 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.059+0000 7efd26800640 1 --2- 172.21.15.55:0/1990023418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd280b2040 0x7efd28143020 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7efd180097d0 tx=0x7efd1802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d232c340bc9a1be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.059+0000 7efd25400640 1 -- 172.21.15.55:0/1990023418 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18032440 con 0x7efd280b2040 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd25400640 1 -- 172.21.15.55:0/1990023418 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd180325a0 con 0x7efd280b2040 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd25400640 1 -- 172.21.15.55:0/1990023418 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18032930 con 0x7efd280b2040 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 -- 172.21.15.55:0/1990023418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd280b2040 msgr2=0x7efd28143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 --2- 172.21.15.55:0/1990023418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd280b2040 0x7efd28143020 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7efd180097d0 tx=0x7efd1802a1c0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 -- 172.21.15.55:0/1990023418 shutdown_connections 2024-05-10T03:34:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 --2- 172.21.15.55:0/1990023418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd280b2040 0x7efd28143020 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 -- 172.21.15.55:0/1990023418 >> 172.21.15.55:0/1990023418 conn(0x7efd2806c730 msgr2=0x7efd280b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 -- 172.21.15.55:0/1990023418 shutdown_connections 2024-05-10T03:34:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.060+0000 7efd2ca00640 1 -- 172.21.15.55:0/1990023418 wait complete. 2024-05-10T03:34:40.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.063+0000 7efd2ca00640 1 Processor -- start 2024-05-10T03:34:40.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.063+0000 7efd2ca00640 1 -- start start 2024-05-10T03:34:40.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 0x7efd2813e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 0x7efd2815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd28157260 con 0x7efd281435d0 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280ad7c0 con 0x7efd280b2040 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd2ca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280adaa0 con 0x7efd2806b120 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd27200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd27200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45942/0 (socket says 172.21.15.55:45942) 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd27200640 1 -- 172.21.15.55:0/2910005221 learned_addr learned my addr 172.21.15.55:0/2910005221 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.065+0000 7efd26800640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 0x7efd2813e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd25e00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 0x7efd2815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 -- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 msgr2=0x7efd2813e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 0x7efd2813e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 -- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 msgr2=0x7efd2815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 0x7efd2815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 -- 172.21.15.55:0/2910005221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18009400 con 0x7efd281435d0 2024-05-10T03:34:40.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd26800640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 0x7efd2813e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:40.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd25e00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 0x7efd2815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:40.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd27200640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c780 tx=0x7efd2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000ce30 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2000a640 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2000aa30 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.066+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280add80 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.067+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd280ad490 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.068+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd280b1680 con 0x7efd281435d0 2024-05-10T03:34:40.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.069+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efd2000ab90 con 0x7efd281435d0 2024-05-10T03:34:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.070+0000 7efd1ea00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 0x7efd0806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.070+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6145+0+0 (secure 0 0 0) 0x7efd20088ae0 con 0x7efd281435d0 2024-05-10T03:34:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.071+0000 7efd26800640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 0x7efd0806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.072+0000 7efd26800640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 0x7efd0806d110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efd18034000 tx=0x7efd1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.075+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7efd200539d0 con 0x7efd281435d0 2024-05-10T03:34:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.322+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efd2804e290 con 0x7efd281435d0 2024-05-10T03:34:40.327 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:34:40.324+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:34:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.341+0000 7efd1ea00640 1 -- 172.21.15.55:0/2910005221 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd20018030 con 0x7efd281435d0 2024-05-10T03:34:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:34:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 msgr2=0x7efd0806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 0x7efd0806d110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efd18034000 tx=0x7efd1802e000 comp rx=0 tx=0).stop 2024-05-10T03:34:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 msgr2=0x7efd2815b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7efd2000c780 tx=0x7efd2000cc40 comp rx=0 tx=0).stop 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 shutdown_connections 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd281435d0 0x7efd2815b300 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7efd0806ac00 0x7efd0806d110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd280b2040 0x7efd2815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 --2- 172.21.15.55:0/2910005221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd2806b120 0x7efd2813e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.344+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 >> 172.21.15.55:0/2910005221 conn(0x7efd2806c730 msgr2=0x7efd280cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.345+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 shutdown_connections 2024-05-10T03:34:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.345+0000 7efd2ca00640 1 -- 172.21.15.55:0/2910005221 wait complete. 2024-05-10T03:34:40.359 INFO:tasks.ceph.mds.c.smithi055.stderr: -1221> 2024-05-10T03:34:20.872+0000 7fbfa8a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:34:40.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:34:40.504 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:34:40.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.656+0000 7f2443e00640 1 Processor -- start 2024-05-10T03:34:40.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.657+0000 7f2443e00640 1 -- start start 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2443e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 0x7f243c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2443e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f243c0b2140 0x7f243c142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2443e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c06ad20 con 0x7f243c0b2550 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2443e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c141f50 con 0x7f243c0b2140 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2443e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c06a110 con 0x7f243c15fe50 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2440a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f243c0b2140 0x7f243c142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2440a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f243c0b2140 0x7f243c142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45972/0 (socket says 172.21.15.55:45972) 2024-05-10T03:34:40.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.659+0000 7f2440a00640 1 -- 172.21.15.55:0/248350825 learned_addr learned my addr 172.21.15.55:0/248350825 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2441400640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 0x7f243c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2440a00640 1 -- 172.21.15.55:0/248350825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 msgr2=0x7f243c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2440a00640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 0x7f243c141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2440a00640 1 -- 172.21.15.55:0/248350825 >> v1:172.21.15.162:6789/0 conn(0x7f243c0b2550 legacy=0x7f243c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2440a00640 1 -- 172.21.15.55:0/248350825 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c06a3f0 con 0x7f243c0b2140 2024-05-10T03:34:40.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.660+0000 7f2441400640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 0x7f243c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:40.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.661+0000 7f2440a00640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2140 0x7f243c142440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2428009870 tx=0x7f242802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190b3045f6f02193 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.661+0000 7f243be00640 1 -- 172.21.15.55:0/248350825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428032440 con 0x7f243c0b2140 2024-05-10T03:34:40.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.661+0000 7f243be00640 1 -- 172.21.15.55:0/248350825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24280325a0 con 0x7f243c0b2140 2024-05-10T03:34:40.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.661+0000 7f243be00640 1 -- 172.21.15.55:0/248350825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428032930 con 0x7f243c0b2140 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 -- 172.21.15.55:0/248350825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2140 msgr2=0x7f243c142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2140 0x7f243c142440 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2428009870 tx=0x7f242802a290 comp rx=0 tx=0).stop 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 -- 172.21.15.55:0/248350825 shutdown_connections 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2140 0x7f243c142440 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 --2- 172.21.15.55:0/248350825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c15fe50 0x7f243c141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 -- 172.21.15.55:0/248350825 >> 172.21.15.55:0/248350825 conn(0x7f243c0cbb90 msgr2=0x7f243c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:40.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.662+0000 7f2443e00640 1 -- 172.21.15.55:0/248350825 shutdown_connections 2024-05-10T03:34:40.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.663+0000 7f2443e00640 1 -- 172.21.15.55:0/248350825 wait complete. 2024-05-10T03:34:40.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.665+0000 7f2443e00640 1 Processor -- start 2024-05-10T03:34:40.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.665+0000 7f2443e00640 1 -- start start 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.666+0000 7f2443e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c0b2140 0x7f243c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.666+0000 7f2443e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.666+0000 7f2443e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 0x7f243c1426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.666+0000 7f2443e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c15dc10 con 0x7f243c0b2550 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2443e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c143330 con 0x7f243c15fe50 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2443e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c15a7c0 con 0x7f243c0b2140 2024-05-10T03:34:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45974/0 (socket says 172.21.15.55:45974) 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 -- 172.21.15.55:0/1305445389 learned_addr learned my addr 172.21.15.55:0/1305445389 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2441e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 0x7f243c1426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2441400640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c0b2140 0x7f243c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 -- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c0b2140 msgr2=0x7f243c144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c0b2140 0x7f243c144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 -- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 msgr2=0x7f243c1426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 0x7f243c1426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2440a00640 1 -- 172.21.15.55:0/1305445389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280094d0 con 0x7f243c0b2550 2024-05-10T03:34:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.667+0000 7f2441e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 0x7f243c1426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2440a00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2428009870 tx=0x7f24280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428030b00 con 0x7f243c0b2550 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2428030c60 con 0x7f243c0b2550 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242803d440 con 0x7f243c0b2550 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c15ce50 con 0x7f243c0b2550 2024-05-10T03:34:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.668+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c15d410 con 0x7f243c0b2550 2024-05-10T03:34:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.669+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c066f60 con 0x7f243c0b2550 2024-05-10T03:34:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.671+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2428039040 con 0x7f243c0b2550 2024-05-10T03:34:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.672+0000 7f2439600640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 0x7f241c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.672+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6231+0+0 (secure 0 0 0) 0x7f24280aa250 con 0x7f243c0b2550 2024-05-10T03:34:40.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.672+0000 7f2441400640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 0x7f241c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:40.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.673+0000 7f2441400640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 0x7f241c06d110 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f243000b840 tx=0x7f2430008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:40.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.676+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2428075210 con 0x7f243c0b2550 2024-05-10T03:34:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:40.920+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f243c067220 con 0x7f243c0b2550 2024-05-10T03:34:41.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.348+0000 7f2439600640 1 -- 172.21.15.55:0/1305445389 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2428078830 con 0x7f243c0b2550 2024-05-10T03:34:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 msgr2=0x7f241c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 0x7f241c06d110 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f243000b840 tx=0x7f2430008040 comp rx=0 tx=0).stop 2024-05-10T03:34:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 msgr2=0x7f243c142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2428009870 tx=0x7f24280046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 shutdown_connections 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f241c06ac00 0x7f241c06d110 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f243c15fe50 0x7f243c1426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f243c0b2550 0x7f243c142170 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 --2- 172.21.15.55:0/1305445389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f243c0b2140 0x7f243c144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 >> 172.21.15.55:0/1305445389 conn(0x7f243c0cbb90 msgr2=0x7f243c0b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.352+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 shutdown_connections 2024-05-10T03:34:41.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.353+0000 7f2443e00640 1 -- 172.21.15.55:0/1305445389 wait complete. 2024-05-10T03:34:41.385 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:34:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.666+0000 7fed6dc00640 1 Processor -- start 2024-05-10T03:34:41.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.666+0000 7fed6dc00640 1 -- start start 2024-05-10T03:34:41.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.668+0000 7fed6dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed6815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed6dc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed68071b80 con 0x7fed6815c760 2024-05-10T03:34:41.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed6dc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6815f910 con 0x7fed6815c350 2024-05-10T03:34:41.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed6dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed68072280 con 0x7fed68071e70 2024-05-10T03:34:41.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed6d200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fed6815c760 0x7fed680cb900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50016/0 (socket says 172.21.15.55:50016) 2024-05-10T03:34:41.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed6d200640 1 -- 172.21.15.55:0/267017000 learned_addr learned my addr 172.21.15.55:0/267017000 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.669+0000 7fed67e00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed6815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492856364 0 0) 0x7fed68071b80 con 0x7fed6815c760 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c003520 con 0x7fed6815c760 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217281503 0 0) 0x7fed4c003520 con 0x7fed6815c760 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 msgr2=0x7fed6815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:41.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed6815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 >> v1:172.21.15.162:6789/0 conn(0x7fed6815c350 legacy=0x7fed6806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6815fbf0 con 0x7fed6815c760 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fed600019f0 con 0x7fed6815c760 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 >> v1:172.21.15.55:6789/0 conn(0x7fed6815c760 legacy=0x7fed680cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed4c0039a0 0x7fed4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c004d00 0x7fed4c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00bb10 con 0x7fed4c0039a0 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00bc80 con 0x7fed4c004d00 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.670+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00be40 con 0x7fed68071e70 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed4c0039a0 0x7fed4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67400640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed6d200640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c004d00 0x7fed4c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 -- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 msgr2=0x7fed4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed68071e70 0x7fed4c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 -- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c004d00 msgr2=0x7fed4c005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c004d00 0x7fed4c005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed67e00640 1 -- 172.21.15.55:0/267017000 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c003520 con 0x7fed4c0039a0 2024-05-10T03:34:41.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.671+0000 7fed6d200640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c004d00 0x7fed4c005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.672+0000 7fed67e00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed4c0039a0 0x7fed4c003e10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ac0 tx=0x7fed4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2981b3355e6b04e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58031400 con 0x7fed4c0039a0 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5802d050 con 0x7fed4c0039a0 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed66a00640 1 -- 172.21.15.55:0/267017000 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58031780 con 0x7fed4c0039a0 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed6dc00640 1 -- 172.21.15.55:0/267017000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed4c0039a0 msgr2=0x7fed4c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed6dc00640 1 --2- 172.21.15.55:0/267017000 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed4c0039a0 0x7fed4c003e10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ac0 tx=0x7fed4c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.673+0000 7fed6d200640 1 -- 172.21.15.55:0/267017000 reap_dead start 2024-05-10T03:34:41.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.674+0000 7fed6dc00640 1 -- 172.21.15.55:0/267017000 shutdown_connections 2024-05-10T03:34:41.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.674+0000 7fed6dc00640 1 -- 172.21.15.55:0/267017000 >> 172.21.15.55:0/267017000 conn(0x7fed6806c070 msgr2=0x7fed6806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:41.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.674+0000 7fed6dc00640 1 -- 172.21.15.55:0/267017000 shutdown_connections 2024-05-10T03:34:41.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.674+0000 7fed6dc00640 1 -- 172.21.15.55:0/267017000 wait complete. 2024-05-10T03:34:41.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.676+0000 7fed6dc00640 1 Processor -- start 2024-05-10T03:34:41.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.676+0000 7fed6dc00640 1 -- start start 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0039a0 0x7fed6815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed68157540 0x7fed681ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68158970 con 0x7fed68156bf0 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68157950 con 0x7fed68157540 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed6dc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed68142860 con 0x7fed4c0039a0 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed67400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed67400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45986/0 (socket says 172.21.15.55:45986) 2024-05-10T03:34:41.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.678+0000 7fed67400640 1 -- 172.21.15.55:0/224224825 learned_addr learned my addr 172.21.15.55:0/224224825 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed6d200640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed68157540 0x7fed681ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67e00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0039a0 0x7fed6815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 -- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0039a0 msgr2=0x7fed6815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0039a0 0x7fed6815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 -- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed68157540 msgr2=0x7fed681ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed68157540 0x7fed681ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 -- 172.21.15.55:0/224224825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68158b70 con 0x7fed68156bf0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed67400640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e580 tx=0x7fed5400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed540052e0 con 0x7fed68156bf0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54005440 con 0x7fed68156bf0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54003810 con 0x7fed68156bf0 2024-05-10T03:34:41.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.679+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed68159380 con 0x7fed68156bf0 2024-05-10T03:34:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.680+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6813dd40 con 0x7fed68156bf0 2024-05-10T03:34:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.681+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed68067220 con 0x7fed68156bf0 2024-05-10T03:34:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.682+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed54003970 con 0x7fed68156bf0 2024-05-10T03:34:41.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.683+0000 7fed5fe00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 0x7fed4806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:41.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.684+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6231+0+0 (secure 0 0 0) 0x7fed54089400 con 0x7fed68156bf0 2024-05-10T03:34:41.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.684+0000 7fed67e00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 0x7fed4806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:41.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.685+0000 7fed67e00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 0x7fed4806d110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fed58033000 tx=0x7fed58008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:41.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.688+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed540542b0 con 0x7fed68156bf0 2024-05-10T03:34:41.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:41.890+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed680713d0 con 0x7fed68156bf0 2024-05-10T03:34:42.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.364+0000 7fed5fe00640 1 -- 172.21.15.55:0/224224825 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7fed540578d0 con 0x7fed68156bf0 2024-05-10T03:34:42.368 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:34:42.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 msgr2=0x7fed4806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 0x7fed4806d110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fed58033000 tx=0x7fed58008030 comp rx=0 tx=0).stop 2024-05-10T03:34:42.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 msgr2=0x7fed68157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e580 tx=0x7fed5400ea40 comp rx=0 tx=0).stop 2024-05-10T03:34:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 shutdown_connections 2024-05-10T03:34:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed4806ac00 0x7fed4806d110 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed68157540 0x7fed681ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed68156bf0 0x7fed68157000 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.370+0000 7fed6dc00640 1 --2- 172.21.15.55:0/224224825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0039a0 0x7fed6815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.371+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 >> 172.21.15.55:0/224224825 conn(0x7fed6806c070 msgr2=0x7fed6806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:42.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.371+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 shutdown_connections 2024-05-10T03:34:42.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.371+0000 7fed6dc00640 1 -- 172.21.15.55:0/224224825 wait complete. 2024-05-10T03:34:42.408 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:34:42.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.691+0000 7fe16be00640 1 Processor -- start 2024-05-10T03:34:42.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.692+0000 7fe16be00640 1 -- start start 2024-05-10T03:34:42.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.694+0000 7fe16be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe16c06b830 0x7fe16c168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.694+0000 7fe16be00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c070f80 con 0x7fe16c071cb0 2024-05-10T03:34:42.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.694+0000 7fe16be00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c072220 con 0x7fe16c0ce840 2024-05-10T03:34:42.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.694+0000 7fe16be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c0ac980 con 0x7fe16c06b830 2024-05-10T03:34:42.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.695+0000 7fe16aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe16c071cb0 0x7fe16c15d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:50028/0 (socket says 172.21.15.55:50028) 2024-05-10T03:34:42.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.695+0000 7fe16aa00640 1 -- 172.21.15.55:0/4062937943 learned_addr learned my addr 172.21.15.55:0/4062937943 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:42.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.695+0000 7fe16b400640 1 --2- 172.21.15.55:0/4062937943 >> v2:172.21.15.162:3300/0 conn(0x7fe16c06b830 0x7fe16c168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.695+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844020934 0 0) 0x7fe16c070f80 con 0x7fe16c071cb0 2024-05-10T03:34:42.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.695+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe150003520 con 0x7fe16c071cb0 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 40549822 0 0) 0x7fe150003520 con 0x7fe16c071cb0 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 >> v2:172.21.15.162:3300/0 conn(0x7fe16c06b830 msgr2=0x7fe16c168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 --2- 172.21.15.55:0/4062937943 >> v2:172.21.15.162:3300/0 conn(0x7fe16c06b830 0x7fe16c168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 >> v1:172.21.15.162:6789/0 conn(0x7fe16c0ce840 legacy=0x7fe16c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0acc60 con 0x7fe16c071cb0 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe15c001670 con 0x7fe16c071cb0 2024-05-10T03:34:42.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 >> v1:172.21.15.55:6789/0 conn(0x7fe16c071cb0 legacy=0x7fe16c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 0x7fe150003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500047c0 0x7fe150004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150005180 0x7fe15000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0ac980 con 0x7fe150003970 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000c120 con 0x7fe1500047c0 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000c2e0 con 0x7fe150005180 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16b400640 1 --2- 172.21.15.55:0/4062937943 >> v2:172.21.15.162:3300/0 conn(0x7fe16c06b830 0x7fe16c168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.696+0000 7fe16aa00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 0x7fe150003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe162000640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500047c0 0x7fe150004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16aa00640 1 -- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150005180 msgr2=0x7fe15000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16aa00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150005180 0x7fe15000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16aa00640 1 -- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500047c0 msgr2=0x7fe150004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16aa00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1500047c0 0x7fe150004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16aa00640 1 -- 172.21.15.55:0/4062937943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150003520 con 0x7fe150003970 2024-05-10T03:34:42.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.697+0000 7fe16b400640 1 -- 172.21.15.55:0/4062937943 reap_dead start 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.698+0000 7fe16aa00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 0x7fe150003de0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00c210 tx=0x7fe1500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22392f2b27ab49d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.698+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c005ad0 con 0x7fe150003970 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.698+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c02d070 con 0x7fe150003970 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.698+0000 7fe16a000640 1 -- 172.21.15.55:0/4062937943 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c007070 con 0x7fe150003970 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 -- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 msgr2=0x7fe150003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 0x7fe150003de0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00c210 tx=0x7fe1500057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 -- 172.21.15.55:0/4062937943 shutdown_connections 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 --2- 172.21.15.55:0/4062937943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe150003970 0x7fe150003de0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 -- 172.21.15.55:0/4062937943 >> 172.21.15.55:0/4062937943 conn(0x7fe16c06c070 msgr2=0x7fe16c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:42.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 -- 172.21.15.55:0/4062937943 shutdown_connections 2024-05-10T03:34:42.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.699+0000 7fe16be00640 1 -- 172.21.15.55:0/4062937943 wait complete. 2024-05-10T03:34:42.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.702+0000 7fe16be00640 1 Processor -- start 2024-05-10T03:34:42.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.702+0000 7fe16be00640 1 -- start start 2024-05-10T03:34:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.703+0000 7fe16be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150003970 0x7fe16c147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.703+0000 7fe16be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe16c15de80 0x7fe16c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c1f4930 con 0x7fe16c153800 2024-05-10T03:34:42.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0ccb80 con 0x7fe16c15de80 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c153c10 con 0x7fe150003970 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46004/0 (socket says 172.21.15.55:46004) 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 -- 172.21.15.55:0/884068944 learned_addr learned my addr 172.21.15.55:0/884068944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16aa00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150003970 0x7fe16c147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe162000640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe16c15de80 0x7fe16c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 -- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150003970 msgr2=0x7fe16c147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150003970 0x7fe16c147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 -- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe16c15de80 msgr2=0x7fe16c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe16c15de80 0x7fe16c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.704+0000 7fe16b400640 1 -- 172.21.15.55:0/884068944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0cde70 con 0x7fe16c153800 2024-05-10T03:34:42.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe16b400640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ade10 tx=0x7fe16400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164010040 con 0x7fe16c153800 2024-05-10T03:34:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400be90 con 0x7fe16c153800 2024-05-10T03:34:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164003bd0 con 0x7fe16c153800 2024-05-10T03:34:42.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0ccdd0 con 0x7fe16c153800 2024-05-10T03:34:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.705+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c0cd390 con 0x7fe16c153800 2024-05-10T03:34:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.706+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c067220 con 0x7fe16c153800 2024-05-10T03:34:42.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.708+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe164003d30 con 0x7fe16c153800 2024-05-10T03:34:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.709+0000 7fe163400640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 0x7fe14806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:42.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.709+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fe164089100 con 0x7fe16c153800 2024-05-10T03:34:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.710+0000 7fe16aa00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 0x7fe14806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:42.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.711+0000 7fe16aa00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 0x7fe14806d110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe15c032000 tx=0x7fe15c02d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:42.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.714+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe1640541e0 con 0x7fe16c153800 2024-05-10T03:34:42.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:42.911+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe16c0ae100 con 0x7fe16c153800 2024-05-10T03:34:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.378+0000 7fe163400640 1 -- 172.21.15.55:0/884068944 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe164057800 con 0x7fe16c153800 2024-05-10T03:34:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:34:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.383+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 msgr2=0x7fe14806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.383+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 0x7fe14806d110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe15c032000 tx=0x7fe15c02d000 comp rx=0 tx=0).stop 2024-05-10T03:34:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.383+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 msgr2=0x7fe16c1f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.383+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ade10 tx=0x7fe16400b480 comp rx=0 tx=0).stop 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 shutdown_connections 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe16c153800 0x7fe16c1f37d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe14806ac00 0x7fe14806d110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe16c15de80 0x7fe16c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 --2- 172.21.15.55:0/884068944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe150003970 0x7fe16c147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 >> 172.21.15.55:0/884068944 conn(0x7fe16c06c070 msgr2=0x7fe16c071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.384+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 shutdown_connections 2024-05-10T03:34:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.385+0000 7fe16be00640 1 -- 172.21.15.55:0/884068944 wait complete. 2024-05-10T03:34:43.420 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:34:43.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.695+0000 7f3b40a00640 1 Processor -- start 2024-05-10T03:34:43.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.695+0000 7f3b40a00640 1 -- start start 2024-05-10T03:34:43.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.697+0000 7f3b40a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 0x7f3b3c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.697+0000 7f3b40a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b3c0b1ff0 0x7f3b3c161210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b40a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c162120 con 0x7f3b3c161750 2024-05-10T03:34:43.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b40a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cded0 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b40a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c0b2950 con 0x7f3b3c0d2a80 2024-05-10T03:34:43.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b3c0b1ff0 0x7f3b3c161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b3c0b1ff0 0x7f3b3c161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48332/0 (socket says 172.21.15.55:48332) 2024-05-10T03:34:43.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 -- 172.21.15.55:0/2141576056 learned_addr learned my addr 172.21.15.55:0/2141576056 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3b400640 1 --2- 172.21.15.55:0/2141576056 >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 0x7f3b3c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 -- 172.21.15.55:0/2141576056 >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 msgr2=0x7f3b3c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 --2- 172.21.15.55:0/2141576056 >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 0x7f3b3c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 -- 172.21.15.55:0/2141576056 >> v1:172.21.15.162:6789/0 conn(0x7f3b3c161750 legacy=0x7f3b3c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.698+0000 7f3b3aa00640 1 -- 172.21.15.55:0/2141576056 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0b2670 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.699+0000 7f3b3b400640 1 --2- 172.21.15.55:0/2141576056 >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 0x7f3b3c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:43.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.700+0000 7f3b3aa00640 1 --2- 172.21.15.55:0/2141576056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c0b1ff0 0x7f3b3c161210 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c08bb40 tx=0x7f3b2802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20750fe0c8f71cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:43.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.700+0000 7f3b3a000640 1 -- 172.21.15.55:0/2141576056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28031600 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.700+0000 7f3b3a000640 1 -- 172.21.15.55:0/2141576056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2802d050 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.700+0000 7f3b3a000640 1 -- 172.21.15.55:0/2141576056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28031980 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 -- 172.21.15.55:0/2141576056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c0b1ff0 msgr2=0x7f3b3c161210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 --2- 172.21.15.55:0/2141576056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c0b1ff0 0x7f3b3c161210 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c08bb40 tx=0x7f3b2802a5d0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 -- 172.21.15.55:0/2141576056 shutdown_connections 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 --2- 172.21.15.55:0/2141576056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c0b1ff0 0x7f3b3c161210 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 --2- 172.21.15.55:0/2141576056 >> v2:172.21.15.162:3300/0 conn(0x7f3b3c0d2a80 0x7f3b3c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 -- 172.21.15.55:0/2141576056 >> 172.21.15.55:0/2141576056 conn(0x7f3b3c0d30b0 msgr2=0x7f3b3c0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 -- 172.21.15.55:0/2141576056 shutdown_connections 2024-05-10T03:34:43.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.701+0000 7f3b40a00640 1 -- 172.21.15.55:0/2141576056 wait complete. 2024-05-10T03:34:43.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.704+0000 7f3b40a00640 1 Processor -- start 2024-05-10T03:34:43.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.705+0000 7f3b40a00640 1 -- start start 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b3c0b1ff0 0x7f3b3c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b3c0d2a80 0x7f3b3c1f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0a9bc0 con 0x7f3b3c161750 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1f5320 con 0x7f3b3c0d2a80 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b40a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0ca9b0 con 0x7f3b3c0b1ff0 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b3be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48346/0 (socket says 172.21.15.55:48346) 2024-05-10T03:34:43.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b3be00640 1 -- 172.21.15.55:0/1046662016 learned_addr learned my addr 172.21.15.55:0/1046662016 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:43.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.707+0000 7f3b3aa00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b3c0d2a80 0x7f3b3c1f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3b400640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b3c0b1ff0 0x7f3b3c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 -- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b3c0b1ff0 msgr2=0x7f3b3c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b3c0b1ff0 0x7f3b3c0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 -- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b3c0d2a80 msgr2=0x7f3b3c1f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b3c0d2a80 0x7f3b3c1f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 -- 172.21.15.55:0/1046662016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009b30 con 0x7f3b3c161750 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b3be00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b410 tx=0x7f3b3400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.708+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34004020 con 0x7f3b3c161750 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.709+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34004180 con 0x7f3b3c161750 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.709+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34011630 con 0x7f3b3c161750 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.709+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0cac00 con 0x7f3b3c161750 2024-05-10T03:34:43.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.709+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c0cb1c0 con 0x7f3b3c161750 2024-05-10T03:34:43.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.710+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c04e290 con 0x7f3b3c161750 2024-05-10T03:34:43.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.711+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3b34011790 con 0x7f3b3c161750 2024-05-10T03:34:43.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.712+0000 7f3b33400640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 0x7f3b1c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:43.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.712+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3b34088f10 con 0x7f3b3c161750 2024-05-10T03:34:43.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.713+0000 7f3b3b400640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 0x7f3b1c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:43.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.714+0000 7f3b3b400640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 0x7f3b1c06d140 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00a950 tx=0x7f3b2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:43.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.717+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3b34054180 con 0x7f3b3c161750 2024-05-10T03:34:43.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.941+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b3c0d12d0 con 0x7f3b3c161750 2024-05-10T03:34:43.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.949+0000 7f3b33400640 1 -- 172.21.15.55:0/1046662016 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b340150a0 con 0x7f3b3c161750 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 msgr2=0x7f3b1c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 0x7f3b1c06d140 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00a950 tx=0x7f3b2c008040 comp rx=0 tx=0).stop 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 msgr2=0x7f3b3c15fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b410 tx=0x7f3b3400b8d0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 shutdown_connections 2024-05-10T03:34:43.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b3c161750 0x7f3b3c15fe00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3b1c06ac30 0x7f3b1c06d140 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b3c0d2a80 0x7f3b3c1f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 --2- 172.21.15.55:0/1046662016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b3c0b1ff0 0x7f3b3c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 >> 172.21.15.55:0/1046662016 conn(0x7f3b3c0d30b0 msgr2=0x7f3b3c0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.952+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 shutdown_connections 2024-05-10T03:34:43.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:43.953+0000 7f3b40a00640 1 -- 172.21.15.55:0/1046662016 wait complete. 2024-05-10T03:34:43.981 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-05-10T03:34:44.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.254+0000 7f94eca00640 1 Processor -- start 2024-05-10T03:34:44.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.254+0000 7f94eca00640 1 -- start start 2024-05-10T03:34:44.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.256+0000 7f94eca00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94e815fe50 0x7f94e8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94eca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94e80b2140 0x7f94e8142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94eca00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e806ad20 con 0x7f94e80b2550 2024-05-10T03:34:44.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94eca00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e8141f50 con 0x7f94e80b2140 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94eca00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e806a110 con 0x7f94e815fe50 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94e5400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f94e80b2140 0x7f94e8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94e6800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f94e80b2550 0x7f94e806b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36570/0 (socket says 172.21.15.55:36570) 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94e6800640 1 -- 172.21.15.55:0/3195516053 learned_addr learned my addr 172.21.15.55:0/3195516053 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.257+0000 7f94e5e00640 1 --2- 172.21.15.55:0/3195516053 >> v2:172.21.15.162:3300/0 conn(0x7f94e815fe50 0x7f94e8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.258+0000 7f94e5400640 1 -- 172.21.15.55:0/3195516053 >> v2:172.21.15.162:3300/0 conn(0x7f94e815fe50 msgr2=0x7f94e8141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.258+0000 7f94e5400640 1 --2- 172.21.15.55:0/3195516053 >> v2:172.21.15.162:3300/0 conn(0x7f94e815fe50 0x7f94e8141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.258+0000 7f94e5400640 1 -- 172.21.15.55:0/3195516053 >> v1:172.21.15.55:6789/0 conn(0x7f94e80b2550 legacy=0x7f94e806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:44.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.258+0000 7f94e5400640 1 -- 172.21.15.55:0/3195516053 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e806a3f0 con 0x7f94e80b2140 2024-05-10T03:34:44.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.259+0000 7f94e5400640 1 --2- 172.21.15.55:0/3195516053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e8142440 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f94e806e520 tx=0x7f94d402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef54a007fc5e21f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:44.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.259+0000 7f94e4a00640 1 -- 172.21.15.55:0/3195516053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4032440 con 0x7f94e80b2140 2024-05-10T03:34:44.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.259+0000 7f94e4a00640 1 -- 172.21.15.55:0/3195516053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d40325a0 con 0x7f94e80b2140 2024-05-10T03:34:44.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.259+0000 7f94e4a00640 1 -- 172.21.15.55:0/3195516053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4032930 con 0x7f94e80b2140 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 -- 172.21.15.55:0/3195516053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 msgr2=0x7f94e8142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 --2- 172.21.15.55:0/3195516053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e8142440 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f94e806e520 tx=0x7f94d402a250 comp rx=0 tx=0).stop 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 -- 172.21.15.55:0/3195516053 shutdown_connections 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 --2- 172.21.15.55:0/3195516053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e8142440 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 --2- 172.21.15.55:0/3195516053 >> v2:172.21.15.162:3300/0 conn(0x7f94e815fe50 0x7f94e8141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 -- 172.21.15.55:0/3195516053 >> 172.21.15.55:0/3195516053 conn(0x7f94e80cbb90 msgr2=0x7f94e80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 -- 172.21.15.55:0/3195516053 shutdown_connections 2024-05-10T03:34:44.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.260+0000 7f94eca00640 1 -- 172.21.15.55:0/3195516053 wait complete. 2024-05-10T03:34:44.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.262+0000 7f94eca00640 1 Processor -- start 2024-05-10T03:34:44.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.262+0000 7f94eca00640 1 -- start start 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e80b2550 0x7f94e81e7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94e815fe50 0x7f94e81420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80a8890 con 0x7f94e80b2140 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e813dd80 con 0x7f94e80b2550 2024-05-10T03:34:44.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.264+0000 7f94eca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8143e90 con 0x7f94e815fe50 2024-05-10T03:34:44.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48356/0 (socket says 172.21.15.55:48356) 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 -- 172.21.15.55:0/634055780 learned_addr learned my addr 172.21.15.55:0/634055780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e6800640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94e815fe50 0x7f94e81420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5400640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e80b2550 0x7f94e81e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 -- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94e815fe50 msgr2=0x7f94e81420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94e815fe50 0x7f94e81420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 -- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e80b2550 msgr2=0x7f94e81e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e80b2550 0x7f94e81e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 -- 172.21.15.55:0/634055780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4009400 con 0x7f94e80b2140 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.265+0000 7f94e5e00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f94d800d620 tx=0x7f94d800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.266+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800b620 con 0x7f94e80b2140 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.266+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d800b780 con 0x7f94e80b2140 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.266+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d800bb30 con 0x7f94e80b2140 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.266+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e80a8680 con 0x7f94e80b2140 2024-05-10T03:34:44.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.266+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e813e2b0 con 0x7f94e80b2140 2024-05-10T03:34:44.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.267+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e8066f60 con 0x7f94e80b2140 2024-05-10T03:34:44.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.268+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f94d800bc90 con 0x7f94e80b2140 2024-05-10T03:34:44.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.269+0000 7f94de000640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 0x7f94c806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:44.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.269+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f94d80896b0 con 0x7f94e80b2140 2024-05-10T03:34:44.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.270+0000 7f94e5400640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 0x7f94c806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:44.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.271+0000 7f94e5400640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 0x7f94c806d140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94d4034000 tx=0x7f94d402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:44.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.274+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f94d80549c0 con 0x7f94e80b2140 2024-05-10T03:34:44.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:44.549+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) v1 -- 0x7f94e80b08f0 con 0x7f94e80b2140 2024-05-10T03:34:45.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.392+0000 7f94de000640 1 -- 172.21.15.55:0/634055780 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v315) v1 ==== 145+0+0 (secure 0 0 0) 0x7f94d8057fe0 con 0x7f94e80b2140 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 msgr2=0x7f94c806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 0x7f94c806d140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f94d4034000 tx=0x7f94d402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 msgr2=0x7f94e80ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f94d800d620 tx=0x7f94d800dae0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 shutdown_connections 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94e815fe50 0x7f94e81420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f94c806ac30 0x7f94c806d140 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94e80b2550 0x7f94e81e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 --2- 172.21.15.55:0/634055780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94e80b2140 0x7f94e80ae0d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 >> 172.21.15.55:0/634055780 conn(0x7f94e80cbb90 msgr2=0x7f94e80cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:45.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 shutdown_connections 2024-05-10T03:34:45.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.396+0000 7f94eca00640 1 -- 172.21.15.55:0/634055780 wait complete. 2024-05-10T03:34:45.436 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-05-10T03:34:45.436 INFO:tasks.cephfs_test_runner:Starting test: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) That when many sessions are updated, they are correctly 2024-05-10T03:34:45.436 INFO:tasks.cephfs_test_runner:test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-05-10T03:34:45.436 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:45.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.719+0000 7f08f8200640 1 Processor -- start 2024-05-10T03:34:45.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.720+0000 7f08f8200640 1 -- start start 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.722+0000 7f08f8200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f0141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.722+0000 7f08f8200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08f00b2550 0x7f08f006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.722+0000 7f08f8200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f006ad20 con 0x7f08f00b2140 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.722+0000 7f08f8200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f0141f50 con 0x7f08f00b2550 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.722+0000 7f08f8200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f006a110 con 0x7f08f015fe50 2024-05-10T03:34:45.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08f4e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f08f00b2140 0x7f08f0142440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36578/0 (socket says 172.21.15.55:36578) 2024-05-10T03:34:45.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08f4e00640 1 -- 172.21.15.55:0/2685212022 learned_addr learned my addr 172.21.15.55:0/2685212022 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:45.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08f6200640 1 --2- 172.21.15.55:0/2685212022 >> v2:172.21.15.55:3300/0 conn(0x7f08f00b2550 0x7f08f006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08f5800640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f0141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08efe00640 1 -- 172.21.15.55:0/2685212022 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614375837 0 0) 0x7f08f006ad20 con 0x7f08f00b2140 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.723+0000 7f08efe00640 1 -- 172.21.15.55:0/2685212022 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d4003520 con 0x7f08f00b2140 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.724+0000 7f08f6200640 1 -- 172.21.15.55:0/2685212022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 msgr2=0x7f08f0141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.724+0000 7f08f6200640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f0141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.724+0000 7f08f6200640 1 -- 172.21.15.55:0/2685212022 >> v1:172.21.15.55:6789/0 conn(0x7f08f00b2140 legacy=0x7f08f0142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.724+0000 7f08f6200640 1 -- 172.21.15.55:0/2685212022 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f006a3f0 con 0x7f08f00b2550 2024-05-10T03:34:45.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.725+0000 7f08f6200640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f006b1b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009d90 tx=0x7f08e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951fd92be758af3a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:45.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.725+0000 7f08efe00640 1 -- 172.21.15.55:0/2685212022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8007740 con 0x7f08f00b2550 2024-05-10T03:34:45.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.725+0000 7f08efe00640 1 -- 172.21.15.55:0/2685212022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e802d050 con 0x7f08f00b2550 2024-05-10T03:34:45.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.725+0000 7f08efe00640 1 -- 172.21.15.55:0/2685212022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8007aa0 con 0x7f08f00b2550 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 -- 172.21.15.55:0/2685212022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 msgr2=0x7f08f006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f006b1b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009d90 tx=0x7f08e802a6f0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 -- 172.21.15.55:0/2685212022 shutdown_connections 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f006b1b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 --2- 172.21.15.55:0/2685212022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f0141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 -- 172.21.15.55:0/2685212022 >> 172.21.15.55:0/2685212022 conn(0x7f08f00cbb90 msgr2=0x7f08f00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 -- 172.21.15.55:0/2685212022 shutdown_connections 2024-05-10T03:34:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.726+0000 7f08f8200640 1 -- 172.21.15.55:0/2685212022 wait complete. 2024-05-10T03:34:45.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.729+0000 7f08f8200640 1 Processor -- start 2024-05-10T03:34:45.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.729+0000 7f08f8200640 1 -- start start 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 0x7f08f013c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f01f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f015c480 con 0x7f08f00b2550 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f015e460 con 0x7f08f00b2140 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f8200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f01e51f0 con 0x7f08f015fe50 2024-05-10T03:34:45.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48364/0 (socket says 172.21.15.55:48364) 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f4e00640 1 -- 172.21.15.55:0/3771087367 learned_addr learned my addr 172.21.15.55:0/3771087367 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f5800640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 0x7f08f013c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f6200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f01f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f4e00640 1 -- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 msgr2=0x7f08f01f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.732+0000 7f08f4e00640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f01f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f4e00640 1 -- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 msgr2=0x7f08f013c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f4e00640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 0x7f08f013c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f4e00640 1 -- 172.21.15.55:0/3771087367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e8009a70 con 0x7f08f00b2550 2024-05-10T03:34:45.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f5800640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 0x7f08f013c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f4e00640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c3c0 tx=0x7f08dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc005ce0 con 0x7f08f00b2550 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc005e40 con 0x7f08f00b2550 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0072f0 con 0x7f08f00b2550 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.733+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f015e5d0 con 0x7f08f00b2550 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.734+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f015e820 con 0x7f08f00b2550 2024-05-10T03:34:45.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.735+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f0066f60 con 0x7f08f00b2550 2024-05-10T03:34:45.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.736+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08dc007450 con 0x7f08f00b2550 2024-05-10T03:34:45.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.737+0000 7f08ed600640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 0x7f08d006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:45.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.737+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f08dc088790 con 0x7f08f00b2550 2024-05-10T03:34:45.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.738+0000 7f08f5800640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 0x7f08d006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:45.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.739+0000 7f08f5800640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 0x7f08d006d110 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009a20 tx=0x7f08e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:45.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.743+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f08dc053b20 con 0x7f08f00b2550 2024-05-10T03:34:45.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.947+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08f00ce7e0 con 0x7f08f00b2550 2024-05-10T03:34:45.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.949+0000 7f08ed600640 1 -- 172.21.15.55:0/3771087367 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+12722 (secure 0 0 0) 0x7f08dc017020 con 0x7f08f00b2550 2024-05-10T03:34:45.952 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:45.953 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":95,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:43.357841+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6835/709318521":"2024-05-11T03:34:40.324087+0000","172.21.15.55:6834/709318521":"2024-05-11T03:34:40.324087+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:45.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 msgr2=0x7f08d006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 0x7f08d006d110 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009a20 tx=0x7f08e0008040 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 msgr2=0x7f08f015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c3c0 tx=0x7f08dc00c880 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 shutdown_connections 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08f015fe50 0x7f08f01f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.952+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08f00b2550 0x7f08f015ad10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.953+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08d006ac00 0x7f08d006d110 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.953+0000 7f08f8200640 1 --2- 172.21.15.55:0/3771087367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08f00b2140 0x7f08f013c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.953+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 >> 172.21.15.55:0/3771087367 conn(0x7f08f00cbb90 msgr2=0x7f08f00aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.953+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 shutdown_connections 2024-05-10T03:34:45.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:45.953+0000 7f08f8200640 1 -- 172.21.15.55:0/3771087367 wait complete. 2024-05-10T03:34:45.983 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:45.983 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:46.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.257+0000 7ff28be00640 1 Processor -- start 2024-05-10T03:34:46.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.258+0000 7ff28be00640 1 -- start start 2024-05-10T03:34:46.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.260+0000 7ff28be00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff28c1546f0 0x7ff28c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.260+0000 7ff28be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff28c06b660 0x7ff28c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.260+0000 7ff28be00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c072270 con 0x7ff28c161e70 2024-05-10T03:34:46.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.260+0000 7ff28be00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c161a50 con 0x7ff28c06b660 2024-05-10T03:34:46.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.260+0000 7ff28be00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c06ba70 con 0x7ff28c1546f0 2024-05-10T03:34:46.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28b400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff28c06b660 0x7ff28c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff28c161e70 0x7ff28c0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36594/0 (socket says 172.21.15.55:36594) 2024-05-10T03:34:46.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28aa00640 1 -- 172.21.15.55:0/2376597116 learned_addr learned my addr 172.21.15.55:0/2376597116 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:46.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff283e00640 1 --2- 172.21.15.55:0/2376597116 >> v2:172.21.15.162:3300/0 conn(0x7ff28c1546f0 0x7ff28c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28b400640 1 -- 172.21.15.55:0/2376597116 >> v2:172.21.15.162:3300/0 conn(0x7ff28c1546f0 msgr2=0x7ff28c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28b400640 1 --2- 172.21.15.55:0/2376597116 >> v2:172.21.15.162:3300/0 conn(0x7ff28c1546f0 0x7ff28c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28b400640 1 -- 172.21.15.55:0/2376597116 >> v1:172.21.15.55:6789/0 conn(0x7ff28c161e70 legacy=0x7ff28c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.261+0000 7ff28a000640 1 -- 172.21.15.55:0/2376597116 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289379357 0 0) 0x7ff28c072270 con 0x7ff28c161e70 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.262+0000 7ff28b400640 1 -- 172.21.15.55:0/2376597116 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c071f90 con 0x7ff28c06b660 2024-05-10T03:34:46.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.263+0000 7ff28b400640 1 --2- 172.21.15.55:0/2376597116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c06b660 0x7ff28c15dc10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d90 tx=0x7ff28402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b827065d3395f6a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.263+0000 7ff28a000640 1 -- 172.21.15.55:0/2376597116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284031600 con 0x7ff28c06b660 2024-05-10T03:34:46.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.263+0000 7ff28a000640 1 -- 172.21.15.55:0/2376597116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28402d050 con 0x7ff28c06b660 2024-05-10T03:34:46.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.263+0000 7ff28a000640 1 -- 172.21.15.55:0/2376597116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284031980 con 0x7ff28c06b660 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 -- 172.21.15.55:0/2376597116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c06b660 msgr2=0x7ff28c15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 --2- 172.21.15.55:0/2376597116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c06b660 0x7ff28c15dc10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff284009d90 tx=0x7ff28402a610 comp rx=0 tx=0).stop 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 -- 172.21.15.55:0/2376597116 shutdown_connections 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 --2- 172.21.15.55:0/2376597116 >> v2:172.21.15.162:3300/0 conn(0x7ff28c1546f0 0x7ff28c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 --2- 172.21.15.55:0/2376597116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c06b660 0x7ff28c15dc10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 -- 172.21.15.55:0/2376597116 >> 172.21.15.55:0/2376597116 conn(0x7ff28c06c750 msgr2=0x7ff28c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 -- 172.21.15.55:0/2376597116 shutdown_connections 2024-05-10T03:34:46.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.264+0000 7ff28be00640 1 -- 172.21.15.55:0/2376597116 wait complete. 2024-05-10T03:34:46.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.267+0000 7ff28be00640 1 Processor -- start 2024-05-10T03:34:46.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.267+0000 7ff28be00640 1 -- start start 2024-05-10T03:34:46.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 0x7ff28c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 0x7ff28c143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c1430d0 con 0x7ff28c161e70 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c158d10 con 0x7ff28c06b660 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.268+0000 7ff28be00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c15d540 con 0x7ff28c1546f0 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48386/0 (socket says 172.21.15.55:48386) 2024-05-10T03:34:46.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 -- 172.21.15.55:0/4158402087 learned_addr learned my addr 172.21.15.55:0/4158402087 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:46.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28aa00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 0x7ff28c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff283e00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 0x7ff28c143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 -- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 msgr2=0x7ff28c143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 0x7ff28c143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 -- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 msgr2=0x7ff28c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 0x7ff28c15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28b400640 1 -- 172.21.15.55:0/4158402087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284009a70 con 0x7ff28c161e70 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff28aa00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 0x7ff28c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.269+0000 7ff283e00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 0x7ff28c143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff28b400640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff284009eb0 tx=0x7ff284004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28403b820 con 0x7ff28c161e70 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28403b980 con 0x7ff28c161e70 2024-05-10T03:34:46.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28403bcf0 con 0x7ff28c161e70 2024-05-10T03:34:46.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c156600 con 0x7ff28c161e70 2024-05-10T03:34:46.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.270+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c156bc0 con 0x7ff28c161e70 2024-05-10T03:34:46.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.272+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c04e290 con 0x7ff28c161e70 2024-05-10T03:34:46.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.273+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff28403a3a0 con 0x7ff28c161e70 2024-05-10T03:34:46.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.274+0000 7ff282a00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 0x7ff26806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.274+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff2840aa5a0 con 0x7ff28c161e70 2024-05-10T03:34:46.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.275+0000 7ff28aa00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 0x7ff26806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.276+0000 7ff28aa00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 0x7ff26806d140 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000f60 tx=0x7ff27c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.279+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2840758a0 con 0x7ff28c161e70 2024-05-10T03:34:46.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.478+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff28c158ee0 con 0x7ff28c161e70 2024-05-10T03:34:46.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.479+0000 7ff282a00640 1 -- 172.21.15.55:0/4158402087 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff284078ec0 con 0x7ff28c161e70 2024-05-10T03:34:46.482 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:46.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.481+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 msgr2=0x7ff26806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.481+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 0x7ff26806d140 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff27c000f60 tx=0x7ff27c009040 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.481+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 msgr2=0x7ff28c144040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.481+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff284009eb0 tx=0x7ff284004a40 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 shutdown_connections 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff28c161e70 0x7ff28c144040 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff28c1546f0 0x7ff28c143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff26806ac30 0x7ff26806d140 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 --2- 172.21.15.55:0/4158402087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff28c06b660 0x7ff28c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 >> 172.21.15.55:0/4158402087 conn(0x7ff28c06c750 msgr2=0x7ff28c070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 shutdown_connections 2024-05-10T03:34:46.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.482+0000 7ff28be00640 1 -- 172.21.15.55:0/4158402087 wait complete. 2024-05-10T03:34:46.513 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-05-10T03:34:46.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.795+0000 7fd68fe00640 1 Processor -- start 2024-05-10T03:34:46.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.795+0000 7fd68fe00640 1 -- start start 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd69006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd6900b3190 0x7fd6900cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69006d350 con 0x7fd690144710 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd690144530 con 0x7fd6900b3190 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69006dd20 con 0x7fd690144120 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd690144710 0x7fd6900b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36608/0 (socket says 172.21.15.55:36608) 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68ea00640 1 -- 172.21.15.55:0/2643295464 learned_addr learned my addr 172.21.15.55:0/2643295464 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:46.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.798+0000 7fd68f400640 1 --2- 172.21.15.55:0/2643295464 >> v2:172.21.15.55:3300/0 conn(0x7fd6900b3190 0x7fd6900cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd686000640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd69006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68e000640 1 -- 172.21.15.55:0/2643295464 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61960872 0 0) 0x7fd69006d350 con 0x7fd690144710 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68e000640 1 -- 172.21.15.55:0/2643295464 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd674003520 con 0x7fd690144710 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68f400640 1 -- 172.21.15.55:0/2643295464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 msgr2=0x7fd69006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68f400640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd69006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68f400640 1 -- 172.21.15.55:0/2643295464 >> v1:172.21.15.55:6789/0 conn(0x7fd690144710 legacy=0x7fd6900b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.799+0000 7fd68f400640 1 -- 172.21.15.55:0/2643295464 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69006c740 con 0x7fd6900b3190 2024-05-10T03:34:46.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.800+0000 7fd68f400640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd6900cd9c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd688009d90 tx=0x7fd68802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427091bc4f1e9f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.800+0000 7fd68e000640 1 -- 172.21.15.55:0/2643295464 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688007740 con 0x7fd6900b3190 2024-05-10T03:34:46.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68e000640 1 -- 172.21.15.55:0/2643295464 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68802d050 con 0x7fd6900b3190 2024-05-10T03:34:46.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68e000640 1 -- 172.21.15.55:0/2643295464 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688007aa0 con 0x7fd6900b3190 2024-05-10T03:34:46.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 -- 172.21.15.55:0/2643295464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 msgr2=0x7fd6900cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd6900cd9c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd688009d90 tx=0x7fd68802a540 comp rx=0 tx=0).stop 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 -- 172.21.15.55:0/2643295464 shutdown_connections 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd69006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2643295464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd6900cd9c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.801+0000 7fd68fe00640 1 -- 172.21.15.55:0/2643295464 >> 172.21.15.55:0/2643295464 conn(0x7fd69006a110 msgr2=0x7fd69006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.802+0000 7fd68fe00640 1 -- 172.21.15.55:0/2643295464 shutdown_connections 2024-05-10T03:34:46.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.802+0000 7fd68fe00640 1 -- 172.21.15.55:0/2643295464 wait complete. 2024-05-10T03:34:46.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.804+0000 7fd68fe00640 1 Processor -- start 2024-05-10T03:34:46.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.804+0000 7fd68fe00640 1 -- start start 2024-05-10T03:34:46.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd6900acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd690144710 0x7fd6900cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900a9e10 con 0x7fd6900b3190 2024-05-10T03:34:46.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900adad0 con 0x7fd690144710 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900a8640 con 0x7fd690144120 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48402/0 (socket says 172.21.15.55:48402) 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.806+0000 7fd68ea00640 1 -- 172.21.15.55:0/2574313041 learned_addr learned my addr 172.21.15.55:0/2574313041 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68f400640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd690144710 0x7fd6900cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 -- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 msgr2=0x7fd6900acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd6900acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 -- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd690144710 msgr2=0x7fd6900cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd690144710 0x7fd6900cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:46.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 -- 172.21.15.55:0/2574313041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009a70 con 0x7fd6900b3190 2024-05-10T03:34:46.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68ea00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c390 tx=0x7fd68000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680005ce0 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680005e60 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6900add20 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680007310 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.807+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6900ae2e0 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.809+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd690066ce0 con 0x7fd6900b3190 2024-05-10T03:34:46.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.810+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd680007470 con 0x7fd6900b3190 2024-05-10T03:34:46.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.810+0000 7fd687400640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 0x7fd66c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:46.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.811+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd6800887f0 con 0x7fd6900b3190 2024-05-10T03:34:46.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.811+0000 7fd686000640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 0x7fd66c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:46.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.812+0000 7fd686000640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 0x7fd66c06d140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:46.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:46.815+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd680053b00 con 0x7fd6900b3190 2024-05-10T03:34:47.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.081+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) v1 -- 0x7fd69004e290 con 0x7fd6900b3190 2024-05-10T03:34:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.413+0000 7fd687400640 1 -- 172.21.15.55:0/2574313041 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v317) v1 ==== 148+0+0 (secure 0 0 0) 0x7fd680017070 con 0x7fd6900b3190 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.415+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 msgr2=0x7fd66c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.415+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 0x7fd66c06d140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd678006fd0 tx=0x7fd678008040 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 msgr2=0x7fd69015afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c390 tx=0x7fd68000c850 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 shutdown_connections 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd66c06ac30 0x7fd66c06d140 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd690144710 0x7fd6900cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd690144120 0x7fd6900acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 --2- 172.21.15.55:0/2574313041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6900b3190 0x7fd69015afd0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 >> 172.21.15.55:0/2574313041 conn(0x7fd69006a110 msgr2=0x7fd69013eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 shutdown_connections 2024-05-10T03:34:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.416+0000 7fd68fe00640 1 -- 172.21.15.55:0/2574313041 wait complete. 2024-05-10T03:34:47.448 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-10T03:34:47.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.734+0000 7fe469e00640 1 Processor -- start 2024-05-10T03:34:47.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.734+0000 7fe469e00640 1 -- start start 2024-05-10T03:34:47.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe464157c30 0x7fe46406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe46406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe46406df50 0x7fe46406a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46415e940 con 0x7fe464157c30 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46406cb80 con 0x7fe46406df50 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe469e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe464158040 con 0x7fe46406c770 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe468a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe464157c30 0x7fe46406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe468a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe464157c30 0x7fe46406c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48404/0 (socket says 172.21.15.55:48404) 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.737+0000 7fe468a00640 1 -- 172.21.15.55:0/4272879933 learned_addr learned my addr 172.21.15.55:0/4272879933 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe463e00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe46406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe469400640 1 --2- 172.21.15.55:0/4272879933 >> v2:172.21.15.162:3300/0 conn(0x7fe46406df50 0x7fe46406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe468a00640 1 -- 172.21.15.55:0/4272879933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 msgr2=0x7fe46406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:47.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe468a00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe46406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe468a00640 1 -- 172.21.15.55:0/4272879933 >> v2:172.21.15.162:3300/0 conn(0x7fe46406df50 msgr2=0x7fe46406a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:47.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe468a00640 1 --2- 172.21.15.55:0/4272879933 >> v2:172.21.15.162:3300/0 conn(0x7fe46406df50 0x7fe46406a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe468a00640 1 -- 172.21.15.55:0/4272879933 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46415dcc0 con 0x7fe464157c30 2024-05-10T03:34:47.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.738+0000 7fe463e00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe46406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:47.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.739+0000 7fe468a00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe464157c30 0x7fe46406c230 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe4640afa20 tx=0x7fe45402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5700b8f904de42d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:47.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.739+0000 7fe463400640 1 -- 172.21.15.55:0/4272879933 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454031600 con 0x7fe464157c30 2024-05-10T03:34:47.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe463400640 1 -- 172.21.15.55:0/4272879933 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45402d050 con 0x7fe464157c30 2024-05-10T03:34:47.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe463400640 1 -- 172.21.15.55:0/4272879933 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454031980 con 0x7fe464157c30 2024-05-10T03:34:47.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 -- 172.21.15.55:0/4272879933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe464157c30 msgr2=0x7fe46406c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe464157c30 0x7fe46406c230 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe4640afa20 tx=0x7fe45402a610 comp rx=0 tx=0).stop 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 -- 172.21.15.55:0/4272879933 shutdown_connections 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 --2- 172.21.15.55:0/4272879933 >> v2:172.21.15.162:3300/0 conn(0x7fe46406df50 0x7fe46406a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe46406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 --2- 172.21.15.55:0/4272879933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe464157c30 0x7fe46406c230 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 -- 172.21.15.55:0/4272879933 >> 172.21.15.55:0/4272879933 conn(0x7fe464070eb0 msgr2=0x7fe4640712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 -- 172.21.15.55:0/4272879933 shutdown_connections 2024-05-10T03:34:47.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.740+0000 7fe469e00640 1 -- 172.21.15.55:0/4272879933 wait complete. 2024-05-10T03:34:47.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.743+0000 7fe469e00640 1 Processor -- start 2024-05-10T03:34:47.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.743+0000 7fe469e00640 1 -- start start 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe464151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe464157c30 0x7fe464153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464152240 con 0x7fe46406df50 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46415ca20 con 0x7fe464157c30 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.744+0000 7fe469e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe464158560 con 0x7fe46406c770 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48412/0 (socket says 172.21.15.55:48412) 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 -- 172.21.15.55:0/875943824 learned_addr learned my addr 172.21.15.55:0/875943824 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe469400640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe464157c30 0x7fe464153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe468a00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe464151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 -- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 msgr2=0x7fe464151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe464151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 -- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe464157c30 msgr2=0x7fe464153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe464157c30 0x7fe464153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.745+0000 7fe463e00640 1 -- 172.21.15.55:0/875943824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454009b30 con 0x7fe46406df50 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe463e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c820 tx=0x7fe44c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c007a30 con 0x7fe46406df50 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c007b90 con 0x7fe46406df50 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00ce10 con 0x7fe46406df50 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4641587b0 con 0x7fe46406df50 2024-05-10T03:34:47.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.746+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe464158d70 con 0x7fe46406df50 2024-05-10T03:34:47.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.747+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46404e290 con 0x7fe46406df50 2024-05-10T03:34:47.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.749+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe44c0113f0 con 0x7fe46406df50 2024-05-10T03:34:47.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.749+0000 7fe460c00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 0x7fe44006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:47.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.749+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe44c089260 con 0x7fe46406df50 2024-05-10T03:34:47.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.750+0000 7fe468a00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 0x7fe44006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:47.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.752+0000 7fe468a00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 0x7fe44006d110 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe454033000 tx=0x7fe454009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.755+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe44c054580 con 0x7fe46406df50 2024-05-10T03:34:47.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.947+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe464154e70 con 0x7fe46406df50 2024-05-10T03:34:47.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.960+0000 7fe460c00640 1 -- 172.21.15.55:0/875943824 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe44c057ba0 con 0x7fe46406df50 2024-05-10T03:34:47.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 msgr2=0x7fe44006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 0x7fe44006d110 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe454033000 tx=0x7fe454009000 comp rx=0 tx=0).stop 2024-05-10T03:34:47.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 msgr2=0x7fe4640ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:47.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00c820 tx=0x7fe44c00cce0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 shutdown_connections 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe44006ac00 0x7fe44006d110 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe464157c30 0x7fe464153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe46406df50 0x7fe4640ceac0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 --2- 172.21.15.55:0/875943824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe46406c770 0x7fe464151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 >> 172.21.15.55:0/875943824 conn(0x7fe464070eb0 msgr2=0x7fe4640b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 shutdown_connections 2024-05-10T03:34:47.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:47.963+0000 7fe469e00640 1 -- 172.21.15.55:0/875943824 wait complete. 2024-05-10T03:34:47.991 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:34:47.991 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:48.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:34:48.040 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:34:48.085 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:34:48.183 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:34:48.183 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:48.183 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:48.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:48.241 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:48.241 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:48.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:48.299 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:34:48.299 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:48.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:34:48.361 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:34:48.361 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:34:48.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:34:48.420 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-10T03:34:48.420 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:48.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.710+0000 7fa20fe00640 1 Processor -- start 2024-05-10T03:34:48.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.710+0000 7fa20fe00640 1 -- start start 2024-05-10T03:34:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.712+0000 7fa20fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa21015ddd0 0x7fa210157e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.712+0000 7fa20fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 0x7fa2100b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.712+0000 7fa20fe00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21015edc0 con 0x7fa2101613c0 2024-05-10T03:34:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.712+0000 7fa20fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2101623d0 con 0x7fa21015ddd0 2024-05-10T03:34:48.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.712+0000 7fa20fe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21015e1e0 con 0x7fa2100ce5f0 2024-05-10T03:34:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa21015ddd0 0x7fa210157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa21015ddd0 0x7fa210157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48420/0 (socket says 172.21.15.55:48420) 2024-05-10T03:34:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 -- 172.21.15.55:0/2037357998 learned_addr learned my addr 172.21.15.55:0/2037357998 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20f400640 1 --2- 172.21.15.55:0/2037357998 >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 0x7fa2100b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 -- 172.21.15.55:0/2037357998 >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 msgr2=0x7fa2100b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:48.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 --2- 172.21.15.55:0/2037357998 >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 0x7fa2100b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 -- 172.21.15.55:0/2037357998 >> v1:172.21.15.162:6789/0 conn(0x7fa2101613c0 legacy=0x7fa2100cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:48.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20ea00640 1 -- 172.21.15.55:0/2037357998 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100cecd0 con 0x7fa21015ddd0 2024-05-10T03:34:48.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.713+0000 7fa20f400640 1 --2- 172.21.15.55:0/2037357998 >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 0x7fa2100b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:48.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.714+0000 7fa20ea00640 1 --2- 172.21.15.55:0/2037357998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa21015ddd0 0x7fa210157e80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa2100cf690 tx=0x7fa20002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f967d3303ca896 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:48.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.715+0000 7fa20e000640 1 -- 172.21.15.55:0/2037357998 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200032440 con 0x7fa21015ddd0 2024-05-10T03:34:48.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.715+0000 7fa20e000640 1 -- 172.21.15.55:0/2037357998 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2000325a0 con 0x7fa21015ddd0 2024-05-10T03:34:48.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.715+0000 7fa20e000640 1 -- 172.21.15.55:0/2037357998 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200032930 con 0x7fa21015ddd0 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 -- 172.21.15.55:0/2037357998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa21015ddd0 msgr2=0x7fa210157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 --2- 172.21.15.55:0/2037357998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa21015ddd0 0x7fa210157e80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa2100cf690 tx=0x7fa20002a230 comp rx=0 tx=0).stop 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 -- 172.21.15.55:0/2037357998 shutdown_connections 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 --2- 172.21.15.55:0/2037357998 >> v2:172.21.15.162:3300/0 conn(0x7fa2100ce5f0 0x7fa2100b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 --2- 172.21.15.55:0/2037357998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa21015ddd0 0x7fa210157e80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 -- 172.21.15.55:0/2037357998 >> 172.21.15.55:0/2037357998 conn(0x7fa2100d0370 msgr2=0x7fa2100d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 -- 172.21.15.55:0/2037357998 shutdown_connections 2024-05-10T03:34:48.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.716+0000 7fa20fe00640 1 -- 172.21.15.55:0/2037357998 wait complete. 2024-05-10T03:34:48.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.718+0000 7fa20fe00640 1 Processor -- start 2024-05-10T03:34:48.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.719+0000 7fa20fe00640 1 -- start start 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 0x7fa2101f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2100b5830 0x7fa2100b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21015a350 con 0x7fa2101613c0 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100c79b0 con 0x7fa21015ddd0 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.720+0000 7fa20fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21015af80 con 0x7fa2100b5830 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48430/0 (socket says 172.21.15.55:48430) 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 -- 172.21.15.55:0/464244327 learned_addr learned my addr 172.21.15.55:0/464244327 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa20ea00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 0x7fa2101f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa20f400640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2100b5830 0x7fa2100b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 -- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2100b5830 msgr2=0x7fa2100b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2100b5830 0x7fa2100b7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 -- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 msgr2=0x7fa2101f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 0x7fa2101f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 -- 172.21.15.55:0/464244327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000094d0 con 0x7fa2101613c0 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa20ea00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 0x7fa2101f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.721+0000 7fa207e00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b3e0 tx=0x7fa1f800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.722+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011830 con 0x7fa2101613c0 2024-05-10T03:34:48.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.722+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f80119b0 con 0x7fa2101613c0 2024-05-10T03:34:48.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.722+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011d60 con 0x7fa2101613c0 2024-05-10T03:34:48.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.722+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2101548b0 con 0x7fa2101613c0 2024-05-10T03:34:48.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.722+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21015b1d0 con 0x7fa2101613c0 2024-05-10T03:34:48.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.723+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2100667e0 con 0x7fa2101613c0 2024-05-10T03:34:48.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.724+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa1f801a400 con 0x7fa2101613c0 2024-05-10T03:34:48.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.725+0000 7fa206a00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 0x7fa1ec06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:48.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.725+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa1f80893e0 con 0x7fa2101613c0 2024-05-10T03:34:48.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.726+0000 7fa20ea00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 0x7fa1ec06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:48.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.727+0000 7fa20ea00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 0x7fa1ec06d110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa200034000 tx=0x7fa20002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:48.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.730+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa1f80546e0 con 0x7fa2101613c0 2024-05-10T03:34:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.966+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa210154a40 con 0x7fa2101613c0 2024-05-10T03:34:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.968+0000 7fa206a00640 1 -- 172.21.15.55:0/464244327 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fa1f8015de0 con 0x7fa2101613c0 2024-05-10T03:34:48.971 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:48.971 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1382644244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1382644244},{"type":"v1","addr":"172.21.15.162:6835","nonce":1382644244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/907564080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":907564080},{"type":"v1","addr":"172.21.15.162:6837","nonce":907564080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1319368578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1319368578},{"type":"v1","addr":"172.21.15.55:6833","nonce":1319368578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/969272675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":969272675},{"type":"v1","addr":"172.21.15.55:6835","nonce":969272675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[]} 2024-05-10T03:34:48.971 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 49 2024-05-10T03:34:48.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 msgr2=0x7fa1ec06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:48.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 0x7fa1ec06d110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa200034000 tx=0x7fa20002e000 comp rx=0 tx=0).stop 2024-05-10T03:34:48.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 msgr2=0x7fa2100b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:48.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b3e0 tx=0x7fa1f800b8a0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 shutdown_connections 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2100b5830 0x7fa2100b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2101613c0 0x7fa2100b7770 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa1ec06ac00 0x7fa1ec06d110 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 --2- 172.21.15.55:0/464244327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa21015ddd0 0x7fa2101f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.970+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 >> 172.21.15.55:0/464244327 conn(0x7fa2100d0370 msgr2=0x7fa210155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.971+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 shutdown_connections 2024-05-10T03:34:48.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:48.971+0000 7fa20fe00640 1 -- 172.21.15.55:0/464244327 wait complete. 2024-05-10T03:34:49.001 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:34:49.001 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-10T03:34:49.001 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:34:49.002 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:49.002 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:34:49.002 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-10T03:34:49.002 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:34:49.002 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:49.002 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:34:49.002 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-10T03:34:49.003 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:34:49.003 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:49.003 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:34:49.003 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-10T03:34:49.003 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:34:49.003 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:34:49.031 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:34:49.031 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:34:49.032 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:34:49.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:34:49.034 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:34:49.034 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:34:49.051 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:34:49.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:34:49.081 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:34:49.082 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:49.088 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:34:49.088 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:49.098 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:34:49.099 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:49.105 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:34:49.105 DEBUG:teuthology.parallel:result is None 2024-05-10T03:34:49.105 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-10T03:34:49.309 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:34:49.312 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:34:49.329 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:34:49.332 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:34:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.364+0000 7f6d46e00640 1 Processor -- start 2024-05-10T03:34:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.364+0000 7f6d46e00640 1 -- start start 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d46e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 0x7f6d4015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d46e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 0x7f6d400ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d46e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400d2b70 con 0x7f6d400cd9a0 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d46e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4006c270 con 0x7f6d400b6bf0 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d46e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4006bcb0 con 0x7f6d400cdef0 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d44e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6d400cd9a0 0x7f6d40161880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36618/0 (socket says 172.21.15.55:36618) 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.366+0000 7f6d44e00640 1 -- 172.21.15.55:0/3722588120 learned_addr learned my addr 172.21.15.55:0/3722588120 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d37e00640 1 --2- 172.21.15.55:0/3722588120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 0x7f6d400ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d3fe00640 1 --2- 172.21.15.55:0/3722588120 >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 0x7f6d4015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d3f400640 1 -- 172.21.15.55:0/3722588120 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005710565 0 0) 0x7f6d400d2b70 con 0x7f6d400cd9a0 2024-05-10T03:34:49.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d3f400640 1 -- 172.21.15.55:0/3722588120 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d24003520 con 0x7f6d400cd9a0 2024-05-10T03:34:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d37e00640 1 -- 172.21.15.55:0/3722588120 >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 msgr2=0x7f6d4015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d37e00640 1 --2- 172.21.15.55:0/3722588120 >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 0x7f6d4015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d37e00640 1 -- 172.21.15.55:0/3722588120 >> v1:172.21.15.55:6789/0 conn(0x7f6d400cd9a0 legacy=0x7f6d40161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d37e00640 1 -- 172.21.15.55:0/3722588120 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4006bf90 con 0x7f6d400cdef0 2024-05-10T03:34:49.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.367+0000 7f6d3fe00640 1 --2- 172.21.15.55:0/3722588120 >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 0x7f6d4015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.368+0000 7f6d37e00640 1 --2- 172.21.15.55:0/3722588120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 0x7f6d400ce440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cea223f0b3578b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d3f400640 1 -- 172.21.15.55:0/3722588120 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28032440 con 0x7f6d400cdef0 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d3f400640 1 -- 172.21.15.55:0/3722588120 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d280325a0 con 0x7f6d400cdef0 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d3f400640 1 -- 172.21.15.55:0/3722588120 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28032930 con 0x7f6d400cdef0 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 -- 172.21.15.55:0/3722588120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 msgr2=0x7f6d400ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3722588120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 0x7f6d400ce440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802a210 comp rx=0 tx=0).stop 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 -- 172.21.15.55:0/3722588120 shutdown_connections 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3722588120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400cdef0 0x7f6d400ce440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3722588120 >> v2:172.21.15.162:3300/0 conn(0x7f6d400b6bf0 0x7f6d4015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 -- 172.21.15.55:0/3722588120 >> 172.21.15.55:0/3722588120 conn(0x7f6d400d32c0 msgr2=0x7f6d400d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.369+0000 7f6d46e00640 1 -- 172.21.15.55:0/3722588120 shutdown_connections 2024-05-10T03:34:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.370+0000 7f6d46e00640 1 -- 172.21.15.55:0/3722588120 wait complete. 2024-05-10T03:34:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.372+0000 7f6d46e00640 1 Processor -- start 2024-05-10T03:34:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.372+0000 7f6d46e00640 1 -- start start 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.373+0000 7f6d46e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400b6bf0 0x7f6d40149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.373+0000 7f6d46e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 0x7f6d400b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.373+0000 7f6d46e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d46e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40149c00 con 0x7f6d400cdef0 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d46e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b38f0 con 0x7f6d400cd9a0 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d46e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b3b40 con 0x7f6d400b6bf0 2024-05-10T03:34:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48472/0 (socket says 172.21.15.55:48472) 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 -- 172.21.15.55:0/3746798804 learned_addr learned my addr 172.21.15.55:0/3746798804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d37e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 0x7f6d400b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d3fe00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400b6bf0 0x7f6d40149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 -- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400b6bf0 msgr2=0x7f6d40149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400b6bf0 0x7f6d40149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 -- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 msgr2=0x7f6d400b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 0x7f6d400b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d44e00640 1 -- 172.21.15.55:0/3746798804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009400 con 0x7f6d400cdef0 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.374+0000 7f6d37e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 0x7f6d400b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d44e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800caa0 tx=0x7f6d3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:49.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800aba0 con 0x7f6d400cdef0 2024-05-10T03:34:49.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3800ad00 con 0x7f6d400cdef0 2024-05-10T03:34:49.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38012640 con 0x7f6d400cdef0 2024-05-10T03:34:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d28009400 con 0x7f6d400cdef0 2024-05-10T03:34:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.375+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d400b4100 con 0x7f6d400cdef0 2024-05-10T03:34:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.377+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40067220 con 0x7f6d400cdef0 2024-05-10T03:34:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.378+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d38007070 con 0x7f6d400cdef0 2024-05-10T03:34:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.379+0000 7f6d3cc00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 0x7f6d1c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.379+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f6d38089100 con 0x7f6d400cdef0 2024-05-10T03:34:49.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.379+0000 7f6d3fe00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 0x7f6d1c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:49.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.380+0000 7f6d3fe00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 0x7f6d1c06d140 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009bc0 tx=0x7f6d30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:49.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.383+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6d38054400 con 0x7f6d400cdef0 2024-05-10T03:34:49.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:49.599+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6d4006ac90 con 0x7f6d400cdef0 2024-05-10T03:34:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.443+0000 7f6d3cc00640 1 -- 172.21.15.55:0/3746798804 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6d38057a20 con 0x7f6d400cdef0 2024-05-10T03:34:50.446 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.446+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 msgr2=0x7f6d1c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 0x7f6d1c06d140 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009bc0 tx=0x7f6d30008040 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 msgr2=0x7f6d400b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800caa0 tx=0x7f6d3800cf60 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 shutdown_connections 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6d400cdef0 0x7f6d400b5120 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d1c06ac30 0x7f6d1c06d140 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d400cd9a0 0x7f6d400b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 --2- 172.21.15.55:0/3746798804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d400b6bf0 0x7f6d40149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 >> 172.21.15.55:0/3746798804 conn(0x7f6d400d32c0 msgr2=0x7f6d4015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 shutdown_connections 2024-05-10T03:34:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.447+0000 7f6d46e00640 1 -- 172.21.15.55:0/3746798804 wait complete. 2024-05-10T03:34:50.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-10T03:34:50.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.752+0000 7f0a2e800640 1 Processor -- start 2024-05-10T03:34:50.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.752+0000 7f0a2e800640 1 -- start start 2024-05-10T03:34:50.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.755+0000 7f0a2e800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 0x7f0a280b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.755+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.755+0000 7f0a2e800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280b37a0 con 0x7f0a28161830 2024-05-10T03:34:50.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.755+0000 7f0a2e800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a28161c40 con 0x7f0a281549e0 2024-05-10T03:34:50.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.755+0000 7f0a2e800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280b39f0 con 0x7f0a280b3390 2024-05-10T03:34:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.756+0000 7f0a2ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.756+0000 7f0a2d400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 0x7f0a280b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.756+0000 7f0a2d400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 0x7f0a280b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:38308/0 (socket says 172.21.15.55:38308) 2024-05-10T03:34:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.756+0000 7f0a2ca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:35332/0 (socket says 172.21.15.55:35332) 2024-05-10T03:34:50.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.756+0000 7f0a2d400640 1 -- 172.21.15.55:0/750252367 learned_addr learned my addr 172.21.15.55:0/750252367 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.757+0000 7f0a2ca00640 1 -- 172.21.15.55:0/750252367 >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 msgr2=0x7f0a280b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.757+0000 7f0a2ca00640 1 --2- 172.21.15.55:0/750252367 >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 0x7f0a280b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.757+0000 7f0a2ca00640 1 -- 172.21.15.55:0/750252367 >> v1:172.21.15.162:6789/0 conn(0x7f0a28161830 legacy=0x7f0a28071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.757+0000 7f0a2ca00640 1 -- 172.21.15.55:0/750252367 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280b3cd0 con 0x7f0a280b3390 2024-05-10T03:34:50.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.758+0000 7f0a2ca00640 1 --2- 172.21.15.55:0/750252367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a140097a0 tx=0x7f0a1402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d644866b026278 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.759+0000 7f0a27e00640 1 -- 172.21.15.55:0/750252367 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1402ae50 con 0x7f0a280b3390 2024-05-10T03:34:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.759+0000 7f0a27e00640 1 -- 172.21.15.55:0/750252367 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14032440 con 0x7f0a280b3390 2024-05-10T03:34:50.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.759+0000 7f0a27e00640 1 -- 172.21.15.55:0/750252367 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140327d0 con 0x7f0a280b3390 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 -- 172.21.15.55:0/750252367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 msgr2=0x7f0a281612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 --2- 172.21.15.55:0/750252367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a140097a0 tx=0x7f0a1402a100 comp rx=0 tx=0).stop 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 -- 172.21.15.55:0/750252367 shutdown_connections 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 --2- 172.21.15.55:0/750252367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a281612f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 --2- 172.21.15.55:0/750252367 >> v2:172.21.15.162:3300/0 conn(0x7f0a281549e0 0x7f0a280b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 -- 172.21.15.55:0/750252367 >> 172.21.15.55:0/750252367 conn(0x7f0a2806a110 msgr2=0x7f0a2806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 -- 172.21.15.55:0/750252367 shutdown_connections 2024-05-10T03:34:50.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.760+0000 7f0a2e800640 1 -- 172.21.15.55:0/750252367 wait complete. 2024-05-10T03:34:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.763+0000 7f0a2e800640 1 Processor -- start 2024-05-10T03:34:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.764+0000 7f0a2e800640 1 -- start start 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a28152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a281549e0 0x7f0a280cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281f4e40 con 0x7f0a28161830 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280c4a00 con 0x7f0a281549e0 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2e800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280c4c50 con 0x7f0a280b3390 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48484/0 (socket says 172.21.15.55:48484) 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2de00640 1 -- 172.21.15.55:0/2715911255 learned_addr learned my addr 172.21.15.55:0/2715911255 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2d400640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a28152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.766+0000 7f0a2ca00640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a281549e0 0x7f0a280cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 -- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 msgr2=0x7f0a28152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a28152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 -- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a281549e0 msgr2=0x7f0a280cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a281549e0 0x7f0a280cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 -- 172.21.15.55:0/2715911255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14009400 con 0x7f0a28161830 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a2de00640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c680 tx=0x7f0a2000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.767+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20003b70 con 0x7f0a28161830 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.768+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20003cd0 con 0x7f0a28161830 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.768+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a280c4f30 con 0x7f0a28161830 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.768+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20005310 con 0x7f0a28161830 2024-05-10T03:34:50.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.768+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280c54f0 con 0x7f0a28161830 2024-05-10T03:34:50.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.769+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a28160950 con 0x7f0a28161830 2024-05-10T03:34:50.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.770+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0a20005470 con 0x7f0a28161830 2024-05-10T03:34:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.771+0000 7f0a25600640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 0x7f0a0806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.771+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0a20088f80 con 0x7f0a28161830 2024-05-10T03:34:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.772+0000 7f0a2d400640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 0x7f0a0806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:50.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.773+0000 7f0a2d400640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 0x7f0a0806d110 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a280cb890 tx=0x7f0a1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:50.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:50.777+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0a200bc900 con 0x7f0a28161830 2024-05-10T03:34:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.042+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0a2804e290 con 0x7f0a28161830 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.043+0000 7f0a25600640 1 -- 172.21.15.55:0/2715911255 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0a200542e0 con 0x7f0a28161830 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mgWm/IRAAIlcs6FoRnefVY6HLQZ9lGw==", 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:51.046 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mel02DBAAdZyRibShKjQ2Yb7YJRFC9Q==", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.047 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mGp2eNRAA4c2bfo7uGcMRtiSPmT279w==", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1mUsEWIBAAygDGfgUU7NuSj6txsdTHzA==", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:51.048 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBdlD1mm0UqIhAAieDvZywy60Ck8gO6bGy10w==", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2024-05-10T03:34:51.049 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBhlD1mVsaCBxAAoV+1YcZl/E1A/Ubuv14Z+A==", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBklD1mW9kBGhAA7QoWVYntozdc1QARVaOH8Q==", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.050 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBolD1mQ3y8ABAAZOLrQwTt9ZzyLzVanURZWg==", 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2024-05-10T03:34:51.051 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBulD1mZ41hLxAAbUAnjOW4mmrBMp6V50gU6Q==", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBylD1mXcKFFBAAbvDXBzqmf6GpmkGylNfTsA==", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.052 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB1lD1mmO8ENBAAgeGXOWrP/zd9+a29e2Psxg==", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQB5lD1mgOwIFhAAZKlBV6LJacgWv1sEO8n/FA==", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2024-05-10T03:34:51.053 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBVlD1m0Ak6MRAAq/fHjzAWq3P/2r236CBbuQ==", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2024-05-10T03:34:51.054 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mjaw7ExAAwMrXOnpcjQ2mzEI28zoXuw==", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mEDaqBBAAZmt3q0vdRWJMwPzGutTpHg==", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:51.055 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBWlD1mlb0XIhAANFNAu0rPAWzukB39npEJIg==", 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.056 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBTlD1mgQqFEhAADuMg4AIYrZqOiMzmY1PWGw==", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1mUEO0EhAAqdQFf6f1C00vV2OcsKbfGg==", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:34:51.057 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQBUlD1m57CgBBAAkvCgvf2HY+iIEo44iWuhsw==", 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:34:51.058 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:34:51.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.045+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 msgr2=0x7f0a0806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.045+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 0x7f0a0806d110 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0a280cb890 tx=0x7f0a1c008040 comp rx=0 tx=0).stop 2024-05-10T03:34:51.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.045+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 msgr2=0x7f0a280ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.045+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c680 tx=0x7f0a2000cb40 comp rx=0 tx=0).stop 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 shutdown_connections 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a28161830 0x7f0a280ce050 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0a0806ac00 0x7f0a0806d110 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a281549e0 0x7f0a280cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 --2- 172.21.15.55:0/2715911255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a280b3390 0x7f0a28152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 >> 172.21.15.55:0/2715911255 conn(0x7f0a2806a110 msgr2=0x7f0a28070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 shutdown_connections 2024-05-10T03:34:51.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.046+0000 7f0a2e800640 1 -- 172.21.15.55:0/2715911255 wait complete. 2024-05-10T03:34:51.074 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:51.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.343+0000 7f596f200640 1 Processor -- start 2024-05-10T03:34:51.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.343+0000 7f596f200640 1 -- start start 2024-05-10T03:34:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.345+0000 7f596f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f59700cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596f200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597015ad30 con 0x7f59700cb730 2024-05-10T03:34:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596f200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597015b010 con 0x7f59700b4880 2024-05-10T03:34:51.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597015b2f0 con 0x7f597015a5d0 2024-05-10T03:34:51.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596d400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f59700cb730 0x7f59700b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36626/0 (socket says 172.21.15.55:36626) 2024-05-10T03:34:51.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596d400640 1 -- 172.21.15.55:0/1727645193 learned_addr learned my addr 172.21.15.55:0/1727645193 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596de00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f59700cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.346+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058022 0 0) 0x7f597015ad30 con 0x7f59700cb730 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5954003520 con 0x7f59700cb730 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031818560 0 0) 0x7f5954003520 con 0x7f59700cb730 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 msgr2=0x7f59700cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f59700cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 >> v1:172.21.15.162:6789/0 conn(0x7f59700b4880 legacy=0x7f59700ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597015b5d0 con 0x7f59700cb730 2024-05-10T03:34:51.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f595c001690 con 0x7f59700cb730 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 >> v1:172.21.15.55:6789/0 conn(0x7f59700cb730 legacy=0x7f59700b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f5954003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59540047c0 0x7f5954004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f5954005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597015b2f0 con 0x7f5954003a10 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595400dd20 con 0x7f59540047c0 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.347+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595400dee0 con 0x7f597015a5d0 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f5954003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596e800640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f5954005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 -- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 msgr2=0x7f5954005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596d400640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59540047c0 0x7f5954004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f5954005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 -- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59540047c0 msgr2=0x7f5954004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59540047c0 0x7f5954004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596de00640 1 -- 172.21.15.55:0/1727645193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954003520 con 0x7f5954003a10 2024-05-10T03:34:51.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596e800640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f597015a5d0 0x7f5954005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:51.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.348+0000 7f596d400640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59540047c0 0x7f5954004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.349+0000 7f596de00640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f5954003e80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5960009a90 tx=0x7f59540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848a0e58f5750c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960031400 con 0x7f5954003a10 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596002d050 con 0x7f5954003a10 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596ca00640 1 -- 172.21.15.55:0/1727645193 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600317a0 con 0x7f5954003a10 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596f200640 1 -- 172.21.15.55:0/1727645193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 msgr2=0x7f5954003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596f200640 1 --2- 172.21.15.55:0/1727645193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f5954003e80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5960009a90 tx=0x7f59540057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596e800640 1 -- 172.21.15.55:0/1727645193 reap_dead start 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596f200640 1 -- 172.21.15.55:0/1727645193 shutdown_connections 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.350+0000 7f596f200640 1 -- 172.21.15.55:0/1727645193 >> 172.21.15.55:0/1727645193 conn(0x7f597006a110 msgr2=0x7f597006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.351+0000 7f596f200640 1 -- 172.21.15.55:0/1727645193 shutdown_connections 2024-05-10T03:34:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.351+0000 7f596f200640 1 -- 172.21.15.55:0/1727645193 wait complete. 2024-05-10T03:34:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.353+0000 7f596f200640 1 Processor -- start 2024-05-10T03:34:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.353+0000 7f596f200640 1 -- start start 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 0x7f59701ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 0x7f597015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59701efe60 con 0x7f5954003a10 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700ce7d0 con 0x7f59700ccb10 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596f200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597014b410 con 0x7f59701efa10 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48502/0 (socket says 172.21.15.55:48502) 2024-05-10T03:34:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.355+0000 7f596de00640 1 -- 172.21.15.55:0/2971702517 learned_addr learned my addr 172.21.15.55:0/2971702517 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596d400640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 0x7f59701ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596e800640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 0x7f597015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 -- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 msgr2=0x7f597015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 0x7f597015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 -- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 msgr2=0x7f59701ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 0x7f59701ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 -- 172.21.15.55:0/2971702517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700ced00 con 0x7f5954003a10 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596d400640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 0x7f59701ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596e800640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 0x7f597015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596de00640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5960034040 tx=0x7f5960004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596003b610 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596003b790 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596003bb40 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.356+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700cea20 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.357+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700b0700 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.358+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59700b08c0 con 0x7f5954003a10 2024-05-10T03:34:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.359+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5960038050 con 0x7f5954003a10 2024-05-10T03:34:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.360+0000 7f5966000640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 0x7f595006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.360+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f59600a9270 con 0x7f5954003a10 2024-05-10T03:34:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.360+0000 7f596d400640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 0x7f595006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.362+0000 7f596d400640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 0x7f595006d140 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c001b60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.365+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5960074650 con 0x7f5954003a10 2024-05-10T03:34:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.568+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59700cb1d0 con 0x7f5954003a10 2024-05-10T03:34:51.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.570+0000 7f5966000640 1 -- 172.21.15.55:0/2971702517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+12596 (secure 0 0 0) 0x7f59700cb1d0 con 0x7f5954003a10 2024-05-10T03:34:51.573 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:51.573 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":100,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:50.421226+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 msgr2=0x7f595006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 0x7f595006d140 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f595c000c00 tx=0x7f595c001b60 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 msgr2=0x7f597014aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5960034040 tx=0x7f5960004730 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 shutdown_connections 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59701efa10 0x7f597015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f595006ac30 0x7f595006d140 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59700ccb10 0x7f59701ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 --2- 172.21.15.55:0/2971702517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5954003a10 0x7f597014aed0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 >> 172.21.15.55:0/2971702517 conn(0x7f597006a110 msgr2=0x7f597006ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 shutdown_connections 2024-05-10T03:34:51.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.573+0000 7f596f200640 1 -- 172.21.15.55:0/2971702517 wait complete. 2024-05-10T03:34:51.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:34:51.605 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:34:51.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.886+0000 7eff2e400640 1 Processor -- start 2024-05-10T03:34:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.886+0000 7eff2e400640 1 -- start start 2024-05-10T03:34:51.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2e400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7eff280cb800 0x7eff280ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2e400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2e400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280cbc10 con 0x7eff280709f0 2024-05-10T03:34:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2e400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2815e640 con 0x7eff280cb800 2024-05-10T03:34:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2e400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff280b3e20 con 0x7eff2806b120 2024-05-10T03:34:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.888+0000 7eff2da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2d000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7eff280cb800 0x7eff280ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2da00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:35380/0 (socket says 172.21.15.55:35380) 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2da00640 1 -- 172.21.15.55:0/4034981005 learned_addr learned my addr 172.21.15.55:0/4034981005 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2d000640 1 -- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 msgr2=0x7eff2806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2d000640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2d000640 1 -- 172.21.15.55:0/4034981005 >> v1:172.21.15.162:6789/0 conn(0x7eff280709f0 legacy=0x7eff2815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2d000640 1 -- 172.21.15.55:0/4034981005 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2806ba70 con 0x7eff280cb800 2024-05-10T03:34:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.889+0000 7eff2da00640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:34:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.891+0000 7eff2d000640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff280cb800 0x7eff280ce7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0097a0 tx=0x7eff1c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5638a270e78026b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.891+0000 7eff27400640 1 -- 172.21.15.55:0/4034981005 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c032440 con 0x7eff280cb800 2024-05-10T03:34:51.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.891+0000 7eff27400640 1 -- 172.21.15.55:0/4034981005 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c0325a0 con 0x7eff280cb800 2024-05-10T03:34:51.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.891+0000 7eff27400640 1 -- 172.21.15.55:0/4034981005 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c032930 con 0x7eff280cb800 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 -- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff280cb800 msgr2=0x7eff280ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff280cb800 0x7eff280ce7c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0097a0 tx=0x7eff1c02a180 comp rx=0 tx=0).stop 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 -- 172.21.15.55:0/4034981005 shutdown_connections 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff2806b120 0x7eff2806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 --2- 172.21.15.55:0/4034981005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff280cb800 0x7eff280ce7c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.892+0000 7eff2e400640 1 -- 172.21.15.55:0/4034981005 >> 172.21.15.55:0/4034981005 conn(0x7eff2806c750 msgr2=0x7eff280b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.893+0000 7eff2e400640 1 -- 172.21.15.55:0/4034981005 shutdown_connections 2024-05-10T03:34:51.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.893+0000 7eff2e400640 1 -- 172.21.15.55:0/4034981005 wait complete. 2024-05-10T03:34:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.895+0000 7eff2e400640 1 Processor -- start 2024-05-10T03:34:51.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.895+0000 7eff2e400640 1 -- start start 2024-05-10T03:34:51.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2806b120 0x7eff2813a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff280709f0 0x7eff28149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff281446b0 con 0x7eff280cb800 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2813d760 con 0x7eff2806b120 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2e400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2813da40 con 0x7eff280709f0 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2da00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48508/0 (socket says 172.21.15.55:48508) 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.897+0000 7eff2da00640 1 -- 172.21.15.55:0/4294503751 learned_addr learned my addr 172.21.15.55:0/4294503751 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2d000640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2806b120 0x7eff2813a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff27e00640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff280709f0 0x7eff28149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 -- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff280709f0 msgr2=0x7eff28149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff280709f0 0x7eff28149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 -- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2806b120 msgr2=0x7eff2813a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2806b120 0x7eff2813a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:51.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 -- 172.21.15.55:0/4294503751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20009b30 con 0x7eff280cb800 2024-05-10T03:34:51.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff2da00640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7eff2000efc0 tx=0x7eff2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20018070 con 0x7eff280cb800 2024-05-10T03:34:51.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.898+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2000f040 con 0x7eff280cb800 2024-05-10T03:34:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.899+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000ce00 con 0x7eff280cb800 2024-05-10T03:34:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.899+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c009400 con 0x7eff280cb800 2024-05-10T03:34:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.899+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2813e000 con 0x7eff280cb800 2024-05-10T03:34:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.900+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2804e290 con 0x7eff280cb800 2024-05-10T03:34:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.901+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7eff20014480 con 0x7eff280cb800 2024-05-10T03:34:51.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.902+0000 7eff24c00640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 0x7eff0806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:51.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.902+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5177+0+0 (secure 0 0 0) 0x7eff200893e0 con 0x7eff280cb800 2024-05-10T03:34:51.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.903+0000 7eff2d000640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 0x7eff0806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:51.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.904+0000 7eff2d000640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 0x7eff0806d110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:51.907+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7eff20054860 con 0x7eff280cb800 2024-05-10T03:34:52.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.112+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff281484f0 con 0x7eff280cb800 2024-05-10T03:34:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.113+0000 7eff24c00640 1 -- 172.21.15.55:0/4294503751 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff20057e80 con 0x7eff280cb800 2024-05-10T03:34:52.115 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 msgr2=0x7eff0806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 0x7eff0806d110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 msgr2=0x7eff2814a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7eff2000efc0 tx=0x7eff2000c490 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 shutdown_connections 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff280cb800 0x7eff2814a390 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff280709f0 0x7eff28149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7eff0806ac00 0x7eff0806d110 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 --2- 172.21.15.55:0/4294503751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff2806b120 0x7eff2813a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 >> 172.21.15.55:0/4294503751 conn(0x7eff2806c750 msgr2=0x7eff280cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 shutdown_connections 2024-05-10T03:34:52.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.116+0000 7eff2e400640 1 -- 172.21.15.55:0/4294503751 wait complete. 2024-05-10T03:34:52.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T03:34:52.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.426+0000 7f2cc2c00640 1 Processor -- start 2024-05-10T03:34:52.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.426+0000 7f2cc2c00640 1 -- start start 2024-05-10T03:34:52.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.428+0000 7f2cc2c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cbc1435d0 0x7f2cbc143020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.428+0000 7f2cc2c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0cdbe0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.428+0000 7f2cc2c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc15e8b0 con 0x7f2cbc06b120 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.428+0000 7f2cc2c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ce140 con 0x7f2cbc1435d0 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc1800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2cbc0b2040 0x7f2cbc0ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36636/0 (socket says 172.21.15.55:36636) 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc1800640 1 -- 172.21.15.55:0/408028773 learned_addr learned my addr 172.21.15.55:0/408028773 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc0e00640 1 --2- 172.21.15.55:0/408028773 >> v2:172.21.15.55:3300/0 conn(0x7f2cbc1435d0 0x7f2cbc143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cbbe00640 1 -- 172.21.15.55:0/408028773 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719662567 0 0) 0x7f2cbc0cdbe0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cbbe00640 1 -- 172.21.15.55:0/408028773 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca0003520 con 0x7f2cbc0b2040 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc0e00640 1 -- 172.21.15.55:0/408028773 >> v1:172.21.15.162:6789/0 conn(0x7f2cbc06b120 legacy=0x7f2cbc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc0e00640 1 -- 172.21.15.55:0/408028773 >> v1:172.21.15.55:6789/0 conn(0x7f2cbc0b2040 legacy=0x7f2cbc0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.429+0000 7f2cc0e00640 1 -- 172.21.15.55:0/408028773 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15ede0 con 0x7f2cbc1435d0 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.431+0000 7f2cc0e00640 1 --2- 172.21.15.55:0/408028773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc1435d0 0x7f2cbc143020 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009e10 tx=0x7f2ca802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e797d50e23ed7fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.431+0000 7f2cbbe00640 1 -- 172.21.15.55:0/408028773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8007740 con 0x7f2cbc1435d0 2024-05-10T03:34:52.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.431+0000 7f2cbbe00640 1 -- 172.21.15.55:0/408028773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca802d050 con 0x7f2cbc1435d0 2024-05-10T03:34:52.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.431+0000 7f2cbbe00640 1 -- 172.21.15.55:0/408028773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8007ac0 con 0x7f2cbc1435d0 2024-05-10T03:34:52.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 -- 172.21.15.55:0/408028773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc1435d0 msgr2=0x7f2cbc143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/408028773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc1435d0 0x7f2cbc143020 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009e10 tx=0x7f2ca802a6a0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 -- 172.21.15.55:0/408028773 shutdown_connections 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/408028773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc1435d0 0x7f2cbc143020 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 -- 172.21.15.55:0/408028773 >> 172.21.15.55:0/408028773 conn(0x7f2cbc06c730 msgr2=0x7f2cbc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 -- 172.21.15.55:0/408028773 shutdown_connections 2024-05-10T03:34:52.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.432+0000 7f2cc2c00640 1 -- 172.21.15.55:0/408028773 wait complete. 2024-05-10T03:34:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.434+0000 7f2cc2c00640 1 Processor -- start 2024-05-10T03:34:52.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.434+0000 7f2cc2c00640 1 -- start start 2024-05-10T03:34:52.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc06b120 0x7f2cbc15cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc1435d0 0x7f2cbc1e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15b270 con 0x7f2cbc0b2040 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15d260 con 0x7f2cbc06b120 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc2c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc1e4140 con 0x7f2cbc1435d0 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc0e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48526/0 (socket says 172.21.15.55:48526) 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.436+0000 7f2cc0e00640 1 -- 172.21.15.55:0/3969010913 learned_addr learned my addr 172.21.15.55:0/3969010913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc1800640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc06b120 0x7f2cbc15cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc2200640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc1435d0 0x7f2cbc1e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 -- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc1435d0 msgr2=0x7f2cbc1e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc1435d0 0x7f2cbc1e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 -- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc06b120 msgr2=0x7f2cbc15cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc06b120 0x7f2cbc15cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 -- 172.21.15.55:0/3969010913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca8009a70 con 0x7f2cbc0b2040 2024-05-10T03:34:52.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.437+0000 7f2cc0e00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009f30 tx=0x7f2ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.438+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8003b90 con 0x7f2cbc0b2040 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.438+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8003cf0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.438+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8004220 con 0x7f2cbc0b2040 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.438+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc15da70 con 0x7f2cbc0b2040 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.438+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15d4b0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.439+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc0b1680 con 0x7f2cbc0b2040 2024-05-10T03:34:52.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.440+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2ca8004380 con 0x7f2cbc0b2040 2024-05-10T03:34:52.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.441+0000 7f2cb9600640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 0x7f2c9c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.441+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2ca80aa2d0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.442+0000 7f2cc1800640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 0x7f2c9c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.443+0000 7f2cc1800640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 0x7f2c9c06d140 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005880 tx=0x7f2cac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.446+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2ca8075620 con 0x7f2cbc0b2040 2024-05-10T03:34:52.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.654+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2cbc0cd1c0 con 0x7f2cbc0b2040 2024-05-10T03:34:52.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.655+0000 7f2cb9600640 1 -- 172.21.15.55:0/3969010913 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2ca8078c40 con 0x7f2cbc0b2040 2024-05-10T03:34:52.658 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:52.658 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-05-10T03:34:52.659 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:34:52.659 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-05-10T03:34:52.659 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-05-10T03:34:52.659 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:34:52.659 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 msgr2=0x7f2c9c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 0x7f2c9c06d140 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2cac005880 tx=0x7f2cac009040 comp rx=0 tx=0).stop 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 msgr2=0x7f2cbc13e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009f30 tx=0x7f2ca8004a40 comp rx=0 tx=0).stop 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 shutdown_connections 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cbc1435d0 0x7f2cbc1e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cbc0b2040 0x7f2cbc13e750 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2c9c06ac30 0x7f2c9c06d140 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 --2- 172.21.15.55:0/3969010913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cbc06b120 0x7f2cbc15cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 >> 172.21.15.55:0/3969010913 conn(0x7f2cbc06c730 msgr2=0x7f2cbc15dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 shutdown_connections 2024-05-10T03:34:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.658+0000 7f2cc2c00640 1 -- 172.21.15.55:0/3969010913 wait complete. 2024-05-10T03:34:52.690 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T03:34:52.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T03:34:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.972+0000 7f2ccda00640 1 Processor -- start 2024-05-10T03:34:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.973+0000 7f2ccda00640 1 -- start start 2024-05-10T03:34:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cc8071e70 0x7f2cc815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cc806b840 0x7f2cc80cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc8072280 con 0x7f2cc806b840 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc815bc20 con 0x7f2cc8071e70 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.975+0000 7f2ccda00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8071b80 con 0x7f2cc815c350 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cc806b840 0x7f2cc80cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2cc806b840 0x7f2cc80cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41080/0 (socket says 172.21.15.55:41080) 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 -- 172.21.15.55:0/3116398708 learned_addr learned my addr 172.21.15.55:0/3116398708 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2cbfe00640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2cc7e00640 1 --2- 172.21.15.55:0/3116398708 >> v2:172.21.15.162:3300/0 conn(0x7f2cc8071e70 0x7f2cc815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 -- 172.21.15.55:0/3116398708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 msgr2=0x7f2cc806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:52.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 -- 172.21.15.55:0/3116398708 >> v2:172.21.15.162:3300/0 conn(0x7f2cc8071e70 msgr2=0x7f2cc815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 --2- 172.21.15.55:0/3116398708 >> v2:172.21.15.162:3300/0 conn(0x7f2cc8071e70 0x7f2cc815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.976+0000 7f2ccd000640 1 -- 172.21.15.55:0/3116398708 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc815f910 con 0x7f2cc806b840 2024-05-10T03:34:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.977+0000 7f2ccd000640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b840 0x7f2cc80cb900 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2cc806f740 tx=0x7f2cc002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a37b328b4ed60e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.978+0000 7f2cc7400640 1 -- 172.21.15.55:0/3116398708 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0031600 con 0x7f2cc806b840 2024-05-10T03:34:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.978+0000 7f2cc7400640 1 -- 172.21.15.55:0/3116398708 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc002d050 con 0x7f2cc806b840 2024-05-10T03:34:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.978+0000 7f2cc7400640 1 -- 172.21.15.55:0/3116398708 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0031980 con 0x7f2cc806b840 2024-05-10T03:34:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 -- 172.21.15.55:0/3116398708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b840 msgr2=0x7f2cc80cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b840 0x7f2cc80cb900 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2cc806f740 tx=0x7f2cc002a610 comp rx=0 tx=0).stop 2024-05-10T03:34:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 -- 172.21.15.55:0/3116398708 shutdown_connections 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc806b840 0x7f2cc80cb900 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3116398708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc806b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3116398708 >> v2:172.21.15.162:3300/0 conn(0x7f2cc8071e70 0x7f2cc815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 -- 172.21.15.55:0/3116398708 >> 172.21.15.55:0/3116398708 conn(0x7f2cc806c070 msgr2=0x7f2cc806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 -- 172.21.15.55:0/3116398708 shutdown_connections 2024-05-10T03:34:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.979+0000 7f2ccda00640 1 -- 172.21.15.55:0/3116398708 wait complete. 2024-05-10T03:34:52.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.983+0000 7f2ccda00640 1 Processor -- start 2024-05-10T03:34:52.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.983+0000 7f2ccda00640 1 -- start start 2024-05-10T03:34:52.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc806b840 0x7f2cc8144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc8156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc813c1d0 con 0x7f2cc8071e70 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc81430a0 con 0x7f2cc806b840 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2ccda00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc81451b0 con 0x7f2cc815c350 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2cbfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2cbfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41092/0 (socket says 172.21.15.55:41092) 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.985+0000 7f2cbfe00640 1 -- 172.21.15.55:0/3917233488 learned_addr learned my addr 172.21.15.55:0/3917233488 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cc7e00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc806b840 0x7f2cc8144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2ccd000640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cbfe00640 1 -- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 msgr2=0x7f2cc8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cbfe00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cbfe00640 1 -- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc806b840 msgr2=0x7f2cc8144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cbfe00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc806b840 0x7f2cc8144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.986+0000 7f2cbfe00640 1 -- 172.21.15.55:0/3917233488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0009b30 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2cbfe00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000d5f0 tx=0x7f2cb000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00125f0 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0012750 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0012b00 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc815a420 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.987+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc815a9c0 con 0x7f2cc8071e70 2024-05-10T03:34:52.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.988+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8067220 con 0x7f2cc8071e70 2024-05-10T03:34:52.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.990+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2cb0012c60 con 0x7f2cc8071e70 2024-05-10T03:34:52.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.990+0000 7f2cc4c00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 0x7f2ca406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:52.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.991+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2cb0089570 con 0x7f2cc8071e70 2024-05-10T03:34:52.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.991+0000 7f2cc7e00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 0x7f2ca406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:52.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.992+0000 7f2cc7e00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 0x7f2ca406d110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009a20 tx=0x7f2cb8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:52.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:52.996+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2cb0054980 con 0x7f2cc8071e70 2024-05-10T03:34:53.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.197+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2cc8070e80 con 0x7f2cc8071e70 2024-05-10T03:34:53.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.475+0000 7f2cc4c00640 1 -- 172.21.15.55:0/3917233488 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2cb0057fa0 con 0x7f2cc8071e70 2024-05-10T03:34:53.478 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-05-10T03:34:53.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.480+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 msgr2=0x7f2ca406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:53.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.480+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 0x7f2ca406d110 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009a20 tx=0x7f2cb8009190 comp rx=0 tx=0).stop 2024-05-10T03:34:53.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.480+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 msgr2=0x7f2cc813bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:53.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.480+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000d5f0 tx=0x7f2cb000dab0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 shutdown_connections 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cc815c350 0x7f2cc8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cc8071e70 0x7f2cc813bc90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2ca406ac00 0x7f2ca406d110 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 --2- 172.21.15.55:0/3917233488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cc806b840 0x7f2cc8144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 >> 172.21.15.55:0/3917233488 conn(0x7f2cc806c070 msgr2=0x7f2cc815d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:53.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 shutdown_connections 2024-05-10T03:34:53.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.481+0000 7f2ccda00640 1 -- 172.21.15.55:0/3917233488 wait complete. 2024-05-10T03:34:53.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T03:34:53.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.804+0000 7f0399200640 1 Processor -- start 2024-05-10T03:34:53.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.804+0000 7f0399200640 1 -- start start 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0399200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f039415e260 0x7f03940ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0399200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f039406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0399200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039415ad80 con 0x7f03940cc620 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0399200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940cee60 con 0x7f039415e260 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0399200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0394072250 con 0x7f039406b120 2024-05-10T03:34:53.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0391e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f039415e260 0x7f03940ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0391e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f039415e260 0x7f03940ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41106/0 (socket says 172.21.15.55:41106) 2024-05-10T03:34:53.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.807+0000 7f0391e00640 1 -- 172.21.15.55:0/1546573854 learned_addr learned my addr 172.21.15.55:0/1546573854 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.808+0000 7f0392800640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f039406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.808+0000 7f0391e00640 1 -- 172.21.15.55:0/1546573854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 msgr2=0x7f039406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.808+0000 7f0391e00640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f039406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.808+0000 7f0391e00640 1 -- 172.21.15.55:0/1546573854 >> v1:172.21.15.162:6789/0 conn(0x7f03940cc620 legacy=0x7f03940706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.808+0000 7f0391e00640 1 -- 172.21.15.55:0/1546573854 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039406ba70 con 0x7f039415e260 2024-05-10T03:34:53.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.809+0000 7f0391e00640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03940ce920 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f039406f540 tx=0x7f038402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fa5c84afb7d613 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:53.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.810+0000 7f0391400640 1 -- 172.21.15.55:0/1546573854 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402ae70 con 0x7f039415e260 2024-05-10T03:34:53.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.810+0000 7f0391400640 1 -- 172.21.15.55:0/1546573854 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0384032440 con 0x7f039415e260 2024-05-10T03:34:53.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.810+0000 7f0391400640 1 -- 172.21.15.55:0/1546573854 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03840327d0 con 0x7f039415e260 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 -- 172.21.15.55:0/1546573854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 msgr2=0x7f03940ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03940ce920 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f039406f540 tx=0x7f038402a160 comp rx=0 tx=0).stop 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 -- 172.21.15.55:0/1546573854 shutdown_connections 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f039406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 --2- 172.21.15.55:0/1546573854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03940ce920 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 -- 172.21.15.55:0/1546573854 >> 172.21.15.55:0/1546573854 conn(0x7f039406c120 msgr2=0x7f039406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 -- 172.21.15.55:0/1546573854 shutdown_connections 2024-05-10T03:34:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.811+0000 7f0399200640 1 -- 172.21.15.55:0/1546573854 wait complete. 2024-05-10T03:34:53.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.813+0000 7f0399200640 1 Processor -- start 2024-05-10T03:34:53.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.814+0000 7f0399200640 1 -- start start 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f03940c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940cc620 0x7f0394151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03941e5980 con 0x7f039415e260 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c9a00 con 0x7f03940cc620 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.815+0000 7f0399200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03941e7630 con 0x7f039406b120 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41112/0 (socket says 172.21.15.55:41112) 2024-05-10T03:34:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 -- 172.21.15.55:0/2063403270 learned_addr learned my addr 172.21.15.55:0/2063403270 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0391e00640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f03940c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0389400640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940cc620 0x7f0394151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 -- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 msgr2=0x7f03940c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:53.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f03940c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 -- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940cc620 msgr2=0x7f0394151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940cc620 0x7f0394151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0392800640 1 -- 172.21.15.55:0/2063403270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03840094d0 con 0x7f039415e260 2024-05-10T03:34:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.816+0000 7f0391e00640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f03940c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:53.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f0392800640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f038c00cc30 tx=0x7f038c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c00ce40 con 0x7f039415e260 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c003990 con 0x7f039415e260 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0394151b90 con 0x7f039415e260 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c003d20 con 0x7f039415e260 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.817+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03941e52c0 con 0x7f039415e260 2024-05-10T03:34:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.818+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0394066f60 con 0x7f039415e260 2024-05-10T03:34:53.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.820+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f038c003e80 con 0x7f039415e260 2024-05-10T03:34:53.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.821+0000 7f038aa00640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 0x7f037006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:53.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.821+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f038c089b00 con 0x7f039415e260 2024-05-10T03:34:53.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.822+0000 7f0391e00640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 0x7f037006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.823+0000 7f0391e00640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 0x7f037006d110 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0384034000 tx=0x7f038402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:53.825+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f038c054d60 con 0x7f039415e260 2024-05-10T03:34:54.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.025+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f03940ce560 con 0x7f039415e260 2024-05-10T03:34:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.486+0000 7f038aa00640 1 -- 172.21.15.55:0/2063403270 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) v1 ==== 170+0+0 (secure 0 0 0) 0x7f038c01d070 con 0x7f039415e260 2024-05-10T03:34:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-05-10T03:34:54.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.497+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 msgr2=0x7f037006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.497+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 0x7f037006d110 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0384034000 tx=0x7f038402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:54.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.497+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 msgr2=0x7f03941e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.497+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f038c00cc30 tx=0x7f038c00a5c0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 shutdown_connections 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f039415e260 0x7f03941e6e00 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f037006ac00 0x7f037006d110 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03940cc620 0x7f0394151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 --2- 172.21.15.55:0/2063403270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039406b120 0x7f03940c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.498+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 >> 172.21.15.55:0/2063403270 conn(0x7f039406c120 msgr2=0x7f0394155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.499+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 shutdown_connections 2024-05-10T03:34:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.500+0000 7f0399200640 1 -- 172.21.15.55:0/2063403270 wait complete. 2024-05-10T03:34:54.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T03:34:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.797+0000 7f1bb6800640 1 Processor -- start 2024-05-10T03:34:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.798+0000 7f1bb6800640 1 -- start start 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb6800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bb0071e00 0x7f1bb0166210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb6800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb015fdf0 con 0x7f1bb00719f0 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb6800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb015f1e0 con 0x7f1bb00710f0 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb6800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb0141d10 con 0x7f1bb0071e00 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1bb00719f0 0x7f1bb006b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39848/0 (socket says 172.21.15.55:39848) 2024-05-10T03:34:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.800+0000 7f1bb5400640 1 -- 172.21.15.55:0/2063929821 learned_addr learned my addr 172.21.15.55:0/2063929821 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/2063929821 >> v2:172.21.15.162:3300/0 conn(0x7f1bb0071e00 0x7f1bb0166210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491988134 0 0) 0x7f1bb015fdf0 con 0x7f1bb00719f0 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b98003520 con 0x7f1bb00719f0 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100617982 0 0) 0x7f1b98003520 con 0x7f1bb00719f0 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 >> v2:172.21.15.162:3300/0 conn(0x7f1bb0071e00 msgr2=0x7f1bb0166210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 --2- 172.21.15.55:0/2063929821 >> v2:172.21.15.162:3300/0 conn(0x7f1bb0071e00 0x7f1bb0166210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 >> v1:172.21.15.162:6789/0 conn(0x7f1bb00710f0 legacy=0x7f1bb01405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb015f4c0 con 0x7f1bb00719f0 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f1ba4001670 con 0x7f1bb00719f0 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 >> v1:172.21.15.55:6789/0 conn(0x7f1bb00719f0 legacy=0x7f1bb006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.801+0000 7f1bafe00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980039e0 0x7f1b98003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bafe00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980047c0 0x7f1b98004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bafe00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 0x7f1b9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0141d10 con 0x7f1b98005310 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9800c210 con 0x7f1b980039e0 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9800c3d0 con 0x7f1b980047c0 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 0x7f1b9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5400640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980039e0 0x7f1b98003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb4a00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980047c0 0x7f1b98004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 -- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980047c0 msgr2=0x7f1b98004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b980047c0 0x7f1b98004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 -- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980039e0 msgr2=0x7f1b98003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b980039e0 0x7f1b98003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.802+0000 7f1bb5e00640 1 -- 172.21.15.55:0/2063929821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98003520 con 0x7f1b98005310 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.803+0000 7f1bb5e00640 1 -- 172.21.15.55:0/2063929821 reap_dead start 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.804+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 0x7f1b9800b980 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009f90 tx=0x7f1b980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916271fb66cb05df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.804+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8003b90 con 0x7f1b98005310 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.804+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba802d070 con 0x7f1b98005310 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.804+0000 7f1bafe00640 1 -- 172.21.15.55:0/2063929821 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004020 con 0x7f1b98005310 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 -- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 msgr2=0x7f1b9800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 0x7f1b9800b980 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009f90 tx=0x7f1b980057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 -- 172.21.15.55:0/2063929821 shutdown_connections 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 --2- 172.21.15.55:0/2063929821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b98005310 0x7f1b9800b980 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 -- 172.21.15.55:0/2063929821 >> 172.21.15.55:0/2063929821 conn(0x7f1bb006c750 msgr2=0x7f1bb00b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 -- 172.21.15.55:0/2063929821 shutdown_connections 2024-05-10T03:34:54.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.805+0000 7f1bb6800640 1 -- 172.21.15.55:0/2063929821 wait complete. 2024-05-10T03:34:54.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.807+0000 7f1bb6800640 1 Processor -- start 2024-05-10T03:34:54.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.808+0000 7f1bb6800640 1 -- start start 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b98005310 0x7f1bb0155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb013d850 0x7f1bb00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0154660 con 0x7f1bb013d440 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb013dd40 con 0x7f1bb013d850 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb6800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb013df90 con 0x7f1b98005310 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb4a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41126/0 (socket says 172.21.15.55:41126) 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.809+0000 7f1bb4a00640 1 -- 172.21.15.55:0/3326940695 learned_addr learned my addr 172.21.15.55:0/3326940695 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb5e00640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb013d850 0x7f1bb00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb5400640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b98005310 0x7f1bb0155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 -- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b98005310 msgr2=0x7f1bb0155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b98005310 0x7f1bb0155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 -- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb013d850 msgr2=0x7f1bb00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb013d850 0x7f1bb00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:54.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 -- 172.21.15.55:0/3326940695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb013e270 con 0x7f1bb013d440 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.810+0000 7f1bb4a00640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e410 tx=0x7f1b9c00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.811+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0052e0 con 0x7f1bb013d440 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.811+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c005440 con 0x7f1bb013d440 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.811+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003810 con 0x7f1bb013d440 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.811+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb013e550 con 0x7f1bb013d440 2024-05-10T03:34:54.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.811+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb013eb10 con 0x7f1bb013d440 2024-05-10T03:34:54.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.812+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb013ecd0 con 0x7f1bb013d440 2024-05-10T03:34:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.814+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1b9c003970 con 0x7f1bb013d440 2024-05-10T03:34:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.816+0000 7f1bad600640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 0x7f1b9006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.816+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f1b9c089370 con 0x7f1bb013d440 2024-05-10T03:34:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.816+0000 7f1bb5400640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 0x7f1b9006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.818+0000 7f1bb5400640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 0x7f1b9006d140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4001050 tx=0x7f1ba4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:54.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:54.820+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1b9c054600 con 0x7f1bb013d440 2024-05-10T03:34:55.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.057+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1bb004e290 con 0x7f1bb013d440 2024-05-10T03:34:55.072 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 15 and data pool 16 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.066+0000 7f1bad600640 1 -- 172.21.15.55:0/3326940695 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1b9c057c20 con 0x7f1bb013d440 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 msgr2=0x7f1b9006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 0x7f1b9006d140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4001050 tx=0x7f1ba4009040 comp rx=0 tx=0).stop 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 msgr2=0x7f1bb00b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e410 tx=0x7f1b9c00e8d0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 shutdown_connections 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1b9006ac30 0x7f1b9006d140 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb013d850 0x7f1bb00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1bb013d440 0x7f1bb00b2b70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 --2- 172.21.15.55:0/3326940695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b98005310 0x7f1bb0155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.070+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 >> 172.21.15.55:0/3326940695 conn(0x7f1bb006c750 msgr2=0x7f1bb00ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.081+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 shutdown_connections 2024-05-10T03:34:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.081+0000 7f1bb6800640 1 -- 172.21.15.55:0/3326940695 wait complete. 2024-05-10T03:34:55.127 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.367+0000 7f5f81e00640 1 Processor -- start 2024-05-10T03:34:55.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.368+0000 7f5f81e00640 1 -- start start 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f81e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5f7c144710 0x7f5f7c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c0b3190 0x7f5f7c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f81e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c06d350 con 0x7f5f7c06d7e0 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f81e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c06c740 con 0x7f5f7c144710 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f81e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c06ca20 con 0x7f5f7c0b3190 2024-05-10T03:34:55.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.370+0000 7f5f7be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5f7c06d7e0 0x7f5f7c0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39850/0 (socket says 172.21.15.55:39850) 2024-05-10T03:34:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f7be00640 1 -- 172.21.15.55:0/784366449 learned_addr learned my addr 172.21.15.55:0/784366449 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:55.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f80a00640 1 --2- 172.21.15.55:0/784366449 >> v2:172.21.15.55:3300/0 conn(0x7f5f7c144710 0x7f5f7c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f81400640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c0b3190 0x7f5f7c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f7b400640 1 -- 172.21.15.55:0/784366449 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100199068 0 0) 0x7f5f7c06d350 con 0x7f5f7c06d7e0 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f7b400640 1 -- 172.21.15.55:0/784366449 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f60003520 con 0x7f5f7c06d7e0 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f80a00640 1 -- 172.21.15.55:0/784366449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c0b3190 msgr2=0x7f5f7c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f80a00640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c0b3190 0x7f5f7c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f80a00640 1 -- 172.21.15.55:0/784366449 >> v1:172.21.15.55:6789/0 conn(0x7f5f7c06d7e0 legacy=0x7f5f7c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.371+0000 7f5f80a00640 1 -- 172.21.15.55:0/784366449 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c06cd00 con 0x7f5f7c144710 2024-05-10T03:34:55.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.372+0000 7f5f80a00640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c144710 0x7f5f7c144120 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097a0 tx=0x7f5f6c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5725bfaf822cedd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.373+0000 7f5f7b400640 1 -- 172.21.15.55:0/784366449 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c02adb0 con 0x7f5f7c144710 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.373+0000 7f5f7b400640 1 -- 172.21.15.55:0/784366449 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c032440 con 0x7f5f7c144710 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.373+0000 7f5f7b400640 1 -- 172.21.15.55:0/784366449 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0327d0 con 0x7f5f7c144710 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.373+0000 7f5f81e00640 1 -- 172.21.15.55:0/784366449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c144710 msgr2=0x7f5f7c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.373+0000 7f5f81e00640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c144710 0x7f5f7c144120 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097a0 tx=0x7f5f6c02a060 comp rx=0 tx=0).stop 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 -- 172.21.15.55:0/784366449 shutdown_connections 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c0b3190 0x7f5f7c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 --2- 172.21.15.55:0/784366449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c144710 0x7f5f7c144120 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 -- 172.21.15.55:0/784366449 >> 172.21.15.55:0/784366449 conn(0x7f5f7c06a110 msgr2=0x7f5f7c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 -- 172.21.15.55:0/784366449 shutdown_connections 2024-05-10T03:34:55.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.374+0000 7f5f81e00640 1 -- 172.21.15.55:0/784366449 wait complete. 2024-05-10T03:34:55.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.377+0000 7f5f81e00640 1 Processor -- start 2024-05-10T03:34:55.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.377+0000 7f5f81e00640 1 -- start start 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7c06d7e0 0x7f5f7c13e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c144710 0x7f5f7c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c138970 con 0x7f5f7c0b3190 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c137c10 con 0x7f5f7c06d7e0 2024-05-10T03:34:55.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f81e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c070cf0 con 0x7f5f7c144710 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f7be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f7be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41148/0 (socket says 172.21.15.55:41148) 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.379+0000 7f5f7be00640 1 -- 172.21.15.55:0/908712988 learned_addr learned my addr 172.21.15.55:0/908712988 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 -- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c144710 msgr2=0x7f5f7c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c144710 0x7f5f7c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 -- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7c06d7e0 msgr2=0x7f5f7c13e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f80a00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7c06d7e0 0x7f5f7c13e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7c06d7e0 0x7f5f7c13e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 -- 172.21.15.55:0/908712988 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c009400 con 0x7f5f7c0b3190 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f7be00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c410 tx=0x7f5f6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68005ce0 con 0x7f5f7c0b3190 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f68005e40 con 0x7f5f7c0b3190 2024-05-10T03:34:55.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c13e880 con 0x7f5f7c0b3190 2024-05-10T03:34:55.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f680072f0 con 0x7f5f7c0b3190 2024-05-10T03:34:55.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.380+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c071220 con 0x7f5f7c0b3190 2024-05-10T03:34:55.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.382+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c066ce0 con 0x7f5f7c0b3190 2024-05-10T03:34:55.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.383+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5f68007510 con 0x7f5f7c0b3190 2024-05-10T03:34:55.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.384+0000 7f5f78c00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 0x7f5f5c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.384+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5f68088ba0 con 0x7f5f7c0b3190 2024-05-10T03:34:55.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.385+0000 7f5f80a00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 0x7f5f5c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.386+0000 7f5f80a00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 0x7f5f5c06d110 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c034000 tx=0x7f5f6c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.389+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5f68053c60 con 0x7f5f7c0b3190 2024-05-10T03:34:55.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.585+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f7c04e290 con 0x7f5f7c0b3190 2024-05-10T03:34:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.587+0000 7f5f78c00640 1 -- 172.21.15.55:0/908712988 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16261 (secure 0 0 0) 0x7f5f68017090 con 0x7f5f7c0b3190 2024-05-10T03:34:55.590 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:55.590 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":103,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:55.058934+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 msgr2=0x7f5f5c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 0x7f5f5c06d110 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c034000 tx=0x7f5f6c02e000 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 msgr2=0x7f5f7c1f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c410 tx=0x7f5f6800c8d0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 shutdown_connections 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f7c144710 0x7f5f7c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f7c0b3190 0x7f5f7c1f30d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5f5c06ac00 0x7f5f5c06d110 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 --2- 172.21.15.55:0/908712988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f7c06d7e0 0x7f5f7c13e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 >> 172.21.15.55:0/908712988 conn(0x7f5f7c06a110 msgr2=0x7f5f7c066310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 shutdown_connections 2024-05-10T03:34:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.590+0000 7f5f81e00640 1 -- 172.21.15.55:0/908712988 wait complete. 2024-05-10T03:34:55.616 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:55.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.892+0000 7f34e1e00640 1 Processor -- start 2024-05-10T03:34:55.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.892+0000 7f34e1e00640 1 -- start start 2024-05-10T03:34:55.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e1e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34dc144710 0x7f34dc144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e1e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f34dc0b3190 0x7f34dc0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e1e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc06d350 con 0x7f34dc06d7e0 2024-05-10T03:34:55.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e1e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc06c740 con 0x7f34dc144710 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e1e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc06ca20 con 0x7f34dc0b3190 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34dc144710 0x7f34dc144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e0a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34dc144710 0x7f34dc144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41162/0 (socket says 172.21.15.55:41162) 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.895+0000 7f34e0a00640 1 -- 172.21.15.55:0/1696950759 learned_addr learned my addr 172.21.15.55:0/1696950759 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.896+0000 7f34e1400640 1 --2- 172.21.15.55:0/1696950759 >> v2:172.21.15.162:3300/0 conn(0x7f34dc0b3190 0x7f34dc0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.896+0000 7f34e0a00640 1 -- 172.21.15.55:0/1696950759 >> v2:172.21.15.162:3300/0 conn(0x7f34dc0b3190 msgr2=0x7f34dc0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.896+0000 7f34e0a00640 1 --2- 172.21.15.55:0/1696950759 >> v2:172.21.15.162:3300/0 conn(0x7f34dc0b3190 0x7f34dc0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.896+0000 7f34e0a00640 1 -- 172.21.15.55:0/1696950759 >> v1:172.21.15.162:6789/0 conn(0x7f34dc06d7e0 legacy=0x7f34dc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:55.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.896+0000 7f34e0a00640 1 -- 172.21.15.55:0/1696950759 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc06cd00 con 0x7f34dc144710 2024-05-10T03:34:55.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.897+0000 7f34e0a00640 1 --2- 172.21.15.55:0/1696950759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc144710 0x7f34dc144120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b5200 tx=0x7f34cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b92bdecafa727 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.897+0000 7f34db400640 1 -- 172.21.15.55:0/1696950759 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc02ae20 con 0x7f34dc144710 2024-05-10T03:34:55.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.897+0000 7f34db400640 1 -- 172.21.15.55:0/1696950759 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc032440 con 0x7f34dc144710 2024-05-10T03:34:55.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.897+0000 7f34db400640 1 -- 172.21.15.55:0/1696950759 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0327d0 con 0x7f34dc144710 2024-05-10T03:34:55.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 -- 172.21.15.55:0/1696950759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc144710 msgr2=0x7f34dc144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 --2- 172.21.15.55:0/1696950759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc144710 0x7f34dc144120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0b5200 tx=0x7f34cc02a110 comp rx=0 tx=0).stop 2024-05-10T03:34:55.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 -- 172.21.15.55:0/1696950759 shutdown_connections 2024-05-10T03:34:55.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 --2- 172.21.15.55:0/1696950759 >> v2:172.21.15.162:3300/0 conn(0x7f34dc0b3190 0x7f34dc0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 --2- 172.21.15.55:0/1696950759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc144710 0x7f34dc144120 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.898+0000 7f34e1e00640 1 -- 172.21.15.55:0/1696950759 >> 172.21.15.55:0/1696950759 conn(0x7f34dc06a110 msgr2=0x7f34dc06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:55.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.899+0000 7f34e1e00640 1 -- 172.21.15.55:0/1696950759 shutdown_connections 2024-05-10T03:34:55.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.899+0000 7f34e1e00640 1 -- 172.21.15.55:0/1696950759 wait complete. 2024-05-10T03:34:55.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.901+0000 7f34e1e00640 1 Processor -- start 2024-05-10T03:34:55.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.902+0000 7f34e1e00640 1 -- start start 2024-05-10T03:34:55.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34dc0b3190 0x7f34dc1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34dc144710 0x7f34dc1f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc131740 con 0x7f34dc06d7e0 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0adce0 con 0x7f34dc0b3190 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e1e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc1f5870 con 0x7f34dc144710 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.903+0000 7f34e0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41172/0 (socket says 172.21.15.55:41172) 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 -- 172.21.15.55:0/3156090682 learned_addr learned my addr 172.21.15.55:0/3156090682 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 -- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34dc144710 msgr2=0x7f34dc1f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34dc144710 0x7f34dc1f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 -- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34dc0b3190 msgr2=0x7f34dc1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34dc0b3190 0x7f34dc1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:55.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 -- 172.21.15.55:0/3156090682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0094d0 con 0x7f34dc06d7e0 2024-05-10T03:34:55.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.904+0000 7f34e0a00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02a150 tx=0x7f34cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.905+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc030850 con 0x7f34dc06d7e0 2024-05-10T03:34:55.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.905+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc0309b0 con 0x7f34dc06d7e0 2024-05-10T03:34:55.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.905+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc1f5ac0 con 0x7f34dc06d7e0 2024-05-10T03:34:55.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.905+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc030d40 con 0x7f34dc06d7e0 2024-05-10T03:34:55.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.905+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc130da0 con 0x7f34dc06d7e0 2024-05-10T03:34:55.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.906+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc131880 con 0x7f34dc06d7e0 2024-05-10T03:34:55.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.907+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f34cc039040 con 0x7f34dc06d7e0 2024-05-10T03:34:55.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.908+0000 7f34d8c00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 0x7f34bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:55.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.908+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f34cc0aa0e0 con 0x7f34dc06d7e0 2024-05-10T03:34:55.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.909+0000 7f34dbe00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 0x7f34bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:55.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.910+0000 7f34dbe00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 0x7f34bc06d140 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f34c8006fd0 tx=0x7f34c8006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:55.913+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f34dc131880 con 0x7f34dc06d7e0 2024-05-10T03:34:56.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.118+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34dc131880 con 0x7f34dc06d7e0 2024-05-10T03:34:56.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.121+0000 7f34d8c00640 1 -- 172.21.15.55:0/3156090682 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f34dc131880 con 0x7f34dc06d7e0 2024-05-10T03:34:56.124 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:56.124 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":104,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:56.064514+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 msgr2=0x7f34bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 0x7f34bc06d140 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f34c8006fd0 tx=0x7f34c8006bc0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 msgr2=0x7f34dc1422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02a150 tx=0x7f34cc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 shutdown_connections 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34dc144710 0x7f34dc1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f34bc06ac30 0x7f34bc06d140 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34dc0b3190 0x7f34dc1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 --2- 172.21.15.55:0/3156090682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34dc06d7e0 0x7f34dc1422c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 >> 172.21.15.55:0/3156090682 conn(0x7f34dc06a110 msgr2=0x7f34dc0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 shutdown_connections 2024-05-10T03:34:56.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.124+0000 7f34e1e00640 1 -- 172.21.15.55:0/3156090682 wait complete. 2024-05-10T03:34:56.154 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T03:34:56.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.423+0000 7fc20d800640 1 Processor -- start 2024-05-10T03:34:56.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.423+0000 7fc20d800640 1 -- start start 2024-05-10T03:34:56.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20d800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc20815e920 0x7fc2080cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 0x7fc20806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20d800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080b2a30 con 0x7fc2080710f0 2024-05-10T03:34:56.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20d800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20815a900 con 0x7fc20815e920 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815eec0 con 0x7fc20806b120 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20ce00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc2080710f0 0x7fc208071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39860/0 (socket says 172.21.15.55:39860) 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.425+0000 7fc20ce00640 1 -- 172.21.15.55:0/3932155315 learned_addr learned my addr 172.21.15.55:0/3932155315 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc207e00640 1 --2- 172.21.15.55:0/3932155315 >> v2:172.21.15.162:3300/0 conn(0x7fc20815e920 0x7fc2080cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc207400640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 0x7fc20806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576928857 0 0) 0x7fc2080b2a30 con 0x7fc2080710f0 2024-05-10T03:34:56.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ec003520 con 0x7fc2080710f0 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699662129 0 0) 0x7fc1ec003520 con 0x7fc2080710f0 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 msgr2=0x7fc20806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 0x7fc20806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 >> v2:172.21.15.162:3300/0 conn(0x7fc20815e920 msgr2=0x7fc2080cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 --2- 172.21.15.55:0/3932155315 >> v2:172.21.15.162:3300/0 conn(0x7fc20815e920 0x7fc2080cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20806ba70 con 0x7fc2080710f0 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fc200001a10 con 0x7fc2080710f0 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 >> v1:172.21.15.55:6789/0 conn(0x7fc2080710f0 legacy=0x7fc208071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ec0039e0 0x7fc1ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc1ec004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 0x7fc1ec005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815eec0 con 0x7fc1ec0047c0 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec00dc40 con 0x7fc1ec0039e0 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.426+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec00ddd0 con 0x7fc20806b120 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc1ec004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207e00640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ec0039e0 0x7fc1ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 -- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 msgr2=0x7fc1ec005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20806b120 0x7fc1ec005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 -- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ec0039e0 msgr2=0x7fc1ec003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1ec0039e0 0x7fc1ec003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.427+0000 7fc207400640 1 -- 172.21.15.55:0/3932155315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec003520 con 0x7fc1ec0047c0 2024-05-10T03:34:56.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.428+0000 7fc207400640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc1ec004bd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009ac0 tx=0x7fc1ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b674fd84a1b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:56.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.428+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0031400 con 0x7fc1ec0047c0 2024-05-10T03:34:56.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.428+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f002d050 con 0x7fc1ec0047c0 2024-05-10T03:34:56.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.428+0000 7fc206a00640 1 -- 172.21.15.55:0/3932155315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0031780 con 0x7fc1ec0047c0 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 -- 172.21.15.55:0/3932155315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 msgr2=0x7fc1ec004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 --2- 172.21.15.55:0/3932155315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc1ec004bd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009ac0 tx=0x7fc1ec0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20ce00640 1 -- 172.21.15.55:0/3932155315 reap_dead start 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 -- 172.21.15.55:0/3932155315 shutdown_connections 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 -- 172.21.15.55:0/3932155315 >> 172.21.15.55:0/3932155315 conn(0x7fc20806c750 msgr2=0x7fc2080b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 -- 172.21.15.55:0/3932155315 shutdown_connections 2024-05-10T03:34:56.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.429+0000 7fc20d800640 1 -- 172.21.15.55:0/3932155315 wait complete. 2024-05-10T03:34:56.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.432+0000 7fc20d800640 1 Processor -- start 2024-05-10T03:34:56.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.432+0000 7fc20d800640 1 -- start start 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20813d9f0 0x7fc208149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20813de00 0x7fc208149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20813fce0 con 0x7fc1ec0047c0 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20814a730 con 0x7fc20813de00 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc20d800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20813e3b0 con 0x7fc20813d9f0 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc207e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc207e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41190/0 (socket says 172.21.15.55:41190) 2024-05-10T03:34:56.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.433+0000 7fc207e00640 1 -- 172.21.15.55:0/858625874 learned_addr learned my addr 172.21.15.55:0/858625874 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc20ce00640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20813de00 0x7fc208149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207400640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20813d9f0 0x7fc208149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 -- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20813d9f0 msgr2=0x7fc208149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20813d9f0 0x7fc208149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 -- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20813de00 msgr2=0x7fc208149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20813de00 0x7fc208149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 -- 172.21.15.55:0/858625874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20813f210 con 0x7fc1ec0047c0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.434+0000 7fc207e00640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e810 tx=0x7fc1f800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.435+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800c6d0 con 0x7fc1ec0047c0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.435+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800c830 con 0x7fc1ec0047c0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.435+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800cba0 con 0x7fc1ec0047c0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.435+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20814c670 con 0x7fc1ec0047c0 2024-05-10T03:34:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.435+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20813e600 con 0x7fc1ec0047c0 2024-05-10T03:34:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.436+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20804e290 con 0x7fc1ec0047c0 2024-05-10T03:34:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.437+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1f800cd00 con 0x7fc1ec0047c0 2024-05-10T03:34:56.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.438+0000 7fc1ffe00640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 0x7fc1e406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.438+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc1f8089370 con 0x7fc1ec0047c0 2024-05-10T03:34:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.439+0000 7fc207400640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 0x7fc1e406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:56.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.440+0000 7fc207400640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 0x7fc1e406d140 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033000 tx=0x7fc1f0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.442+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc1f8054370 con 0x7fc1ec0047c0 2024-05-10T03:34:56.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:56.684+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc20815b380 con 0x7fc1ec0047c0 2024-05-10T03:34:57.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.525+0000 7fc1ffe00640 1 -- 172.21.15.55:0/858625874 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1f8057990 con 0x7fc1ec0047c0 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.528+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 msgr2=0x7fc1e406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.528+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 0x7fc1e406d140 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0033000 tx=0x7fc1f0008030 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.528+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 msgr2=0x7fc20814c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.528+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e810 tx=0x7fc1f800ecd0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 shutdown_connections 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc1e406ac30 0x7fc1e406d140 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc20813de00 0x7fc208149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc20813d9f0 0x7fc208149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 --2- 172.21.15.55:0/858625874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1ec0047c0 0x7fc20814c130 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 >> 172.21.15.55:0/858625874 conn(0x7fc20806c750 msgr2=0x7fc2080b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 shutdown_connections 2024-05-10T03:34:57.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.529+0000 7fc20d800640 1 -- 172.21.15.55:0/858625874 wait complete. 2024-05-10T03:34:57.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-10T03:34:57.564 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-10T03:34:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.857+0000 7f56b9000640 1 Processor -- start 2024-05-10T03:34:57.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.857+0000 7f56b9000640 1 -- start start 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.859+0000 7f56b9000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f56b40cc3a0 0x7f56b40cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.859+0000 7f56b9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f56b406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.859+0000 7f56b9000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b415b5e0 con 0x7f56b406b660 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.859+0000 7f56b9000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b415f1e0 con 0x7f56b40cc3a0 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.859+0000 7f56b9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40b4380 con 0x7f56b4140900 2024-05-10T03:34:57.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56b2800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f56b406b660 0x7f56b4071160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39868/0 (socket says 172.21.15.55:39868) 2024-05-10T03:34:57.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56b2800640 1 -- 172.21.15.55:0/2605883385 learned_addr learned my addr 172.21.15.55:0/2605883385 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2605883385 >> v2:172.21.15.162:3300/0 conn(0x7f56b40cc3a0 0x7f56b40cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56a9400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f56b406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027904400 0 0) 0x7f56b415b5e0 con 0x7f56b406b660 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.860+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5698005730 con 0x7f56b406b660 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830642925 0 0) 0x7f5698005730 con 0x7f56b406b660 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 msgr2=0x7f56b406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f56b406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 >> v2:172.21.15.162:3300/0 conn(0x7f56b40cc3a0 msgr2=0x7f56b40cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:57.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 --2- 172.21.15.55:0/2605883385 >> v2:172.21.15.162:3300/0 conn(0x7f56b40cc3a0 0x7f56b40cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b406ba70 con 0x7f56b406b660 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f56ac0019f0 con 0x7f56b406b660 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 >> v1:172.21.15.55:6789/0 conn(0x7f56b406b660 legacy=0x7f56b4071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f5698005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5698006100 0x7f56980069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006f10 0x7f5698007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b415f1e0 con 0x7f5698006100 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40b4380 con 0x7f5698006f10 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569800dbe0 con 0x7f56b4140900 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.861+0000 7f56a9400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5698006100 0x7f56980069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56b2800640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006f10 0x7f5698007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f5698005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56a9400640 1 -- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 msgr2=0x7f5698005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:57.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56a9400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f5698005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56a9400640 1 -- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006f10 msgr2=0x7f5698007320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:57.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56a9400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006f10 0x7f5698007320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56a9400640 1 -- 172.21.15.55:0/2605883385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5698005730 con 0x7f5698006100 2024-05-10T03:34:57.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.862+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4140900 0x7f5698005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:34:57.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.863+0000 7f56a9400640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5698006100 0x7f56980069d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a90 tx=0x7f5698007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319c5050d449609a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:57.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.863+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c031400 con 0x7f5698006100 2024-05-10T03:34:57.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.863+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c02d050 con 0x7f5698006100 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.863+0000 7f56b1400640 1 -- 172.21.15.55:0/2605883385 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c031780 con 0x7f5698006100 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.864+0000 7f56b9000640 1 -- 172.21.15.55:0/2605883385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5698006100 msgr2=0x7f56980069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.864+0000 7f56b9000640 1 --2- 172.21.15.55:0/2605883385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5698006100 0x7f56980069d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a90 tx=0x7f5698007a00 comp rx=0 tx=0).stop 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.864+0000 7f56b2800640 1 -- 172.21.15.55:0/2605883385 reap_dead start 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.864+0000 7f56b9000640 1 -- 172.21.15.55:0/2605883385 shutdown_connections 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.864+0000 7f56b9000640 1 -- 172.21.15.55:0/2605883385 >> 172.21.15.55:0/2605883385 conn(0x7f56b406c120 msgr2=0x7f56b406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.865+0000 7f56b9000640 1 -- 172.21.15.55:0/2605883385 shutdown_connections 2024-05-10T03:34:57.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.865+0000 7f56b9000640 1 -- 172.21.15.55:0/2605883385 wait complete. 2024-05-10T03:34:57.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.867+0000 7f56b9000640 1 Processor -- start 2024-05-10T03:34:57.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.867+0000 7f56b9000640 1 -- start start 2024-05-10T03:34:57.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006100 0x7f56b4132130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4132cf0 0x7f56b4133100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4156e80 con 0x7f56b4138e00 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40c9eb0 con 0x7f5698006100 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56b9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40ca750 con 0x7f56b4132cf0 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56a9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56a9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41210/0 (socket says 172.21.15.55:41210) 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.869+0000 7f56a9400640 1 -- 172.21.15.55:0/2988285750 learned_addr learned my addr 172.21.15.55:0/2988285750 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006100 0x7f56b4132130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 -- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4132cf0 msgr2=0x7f56b4133100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56b2800640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4132cf0 0x7f56b4133100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4132cf0 0x7f56b4133100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 -- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006100 msgr2=0x7f56b4132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:57.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006100 0x7f56b4132130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:57.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 -- 172.21.15.55:0/2988285750 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b41570d0 con 0x7f56b4138e00 2024-05-10T03:34:57.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56a9400640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f569c009bb0 tx=0x7f569c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c003e20 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c03c420 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c03c7b0 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.870+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4135c90 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.871+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b40ca3e0 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.872+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4066f60 con 0x7f56b4138e00 2024-05-10T03:34:57.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.873+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f569c03c910 con 0x7f56b4138e00 2024-05-10T03:34:57.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.874+0000 7f56aaa00640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 0x7f569006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:57.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.874+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f569c0a9b10 con 0x7f56b4138e00 2024-05-10T03:34:57.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.874+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 0x7f569006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:57.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.876+0000 7f56b1e00640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 0x7f569006d110 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009a40 tx=0x7f56a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:57.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:57.879+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f569c074a80 con 0x7f56b4138e00 2024-05-10T03:34:58.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.114+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f56b4067220 con 0x7f56b4138e00 2024-05-10T03:34:58.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.531+0000 7f56aaa00640 1 -- 172.21.15.55:0/2988285750 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f569c0400c0 con 0x7f56b4138e00 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.534+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 msgr2=0x7f569006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.534+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 0x7f569006d110 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009a40 tx=0x7f56a4008040 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.534+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 msgr2=0x7f56b4132670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.534+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f569c009bb0 tx=0x7f569c004730 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 shutdown_connections 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56b4132cf0 0x7f56b4133100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56b4138e00 0x7f56b4132670 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f569006ac00 0x7f569006d110 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 --2- 172.21.15.55:0/2988285750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5698006100 0x7f56b4132130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 >> 172.21.15.55:0/2988285750 conn(0x7f56b406c120 msgr2=0x7f56b415bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:58.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 shutdown_connections 2024-05-10T03:34:58.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.535+0000 7f56b9000640 1 -- 172.21.15.55:0/2988285750 wait complete. 2024-05-10T03:34:58.570 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-10T03:34:58.571 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:34:58.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.849+0000 7fe40e600640 1 Processor -- start 2024-05-10T03:34:58.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.850+0000 7fe40e600640 1 -- start start 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.852+0000 7fe40e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe408144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.852+0000 7fe40e600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40806d350 con 0x7fe40806d7e0 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.852+0000 7fe40e600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40806c740 con 0x7fe4080b3190 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.852+0000 7fe40e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40806ca20 con 0x7fe408144710 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.853+0000 7fe407e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe40806d7e0 0x7fe4080b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39882/0 (socket says 172.21.15.55:39882) 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.853+0000 7fe407e00640 1 -- 172.21.15.55:0/803137529 learned_addr learned my addr 172.21.15.55:0/803137529 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.853+0000 7fe40d200640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe408144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.853+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415160258 0 0) 0x7fe40806d350 con 0x7fe40806d7e0 2024-05-10T03:34:58.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.853+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3ec003520 con 0x7fe40806d7e0 2024-05-10T03:34:58.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3831761841 0 0) 0x7fe3ec003520 con 0x7fe40806d7e0 2024-05-10T03:34:58.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 msgr2=0x7fe408144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe408144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 >> v1:172.21.15.162:6789/0 conn(0x7fe4080b3190 legacy=0x7fe4080cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40806cd00 con 0x7fe40806d7e0 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe3f80019f0 con 0x7fe40806d7e0 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 >> v1:172.21.15.55:6789/0 conn(0x7fe40806d7e0 legacy=0x7fe4080b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3ec003a10 0x7fe3ec003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe3ec004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe3ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40806ca20 con 0x7fe3ec0047c0 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec005570 con 0x7fe3ec003a10 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec00bec0 con 0x7fe408144710 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.854+0000 7fe407e00640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe3ec004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe40d200640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3ec003a10 0x7fe3ec003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe40dc00640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe3ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe407e00640 1 -- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 msgr2=0x7fe3ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe407e00640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe408144710 0x7fe3ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe407e00640 1 -- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3ec003a10 msgr2=0x7fe3ec003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe407e00640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3ec003a10 0x7fe3ec003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.855+0000 7fe407e00640 1 -- 172.21.15.55:0/803137529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec003520 con 0x7fe3ec0047c0 2024-05-10T03:34:58.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.856+0000 7fe407e00640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe3ec004bd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe40806f690 tx=0x7fe3ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8df90e337c6c507 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.856+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f802b9c0 con 0x7fe3ec0047c0 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.856+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f802bb20 con 0x7fe3ec0047c0 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.856+0000 7fe407400640 1 -- 172.21.15.55:0/803137529 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8034470 con 0x7fe3ec0047c0 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 -- 172.21.15.55:0/803137529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 msgr2=0x7fe3ec004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 --2- 172.21.15.55:0/803137529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe3ec004bd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fe40806f690 tx=0x7fe3ec0057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40dc00640 1 -- 172.21.15.55:0/803137529 reap_dead start 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 -- 172.21.15.55:0/803137529 shutdown_connections 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 -- 172.21.15.55:0/803137529 >> 172.21.15.55:0/803137529 conn(0x7fe40806a110 msgr2=0x7fe40806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 -- 172.21.15.55:0/803137529 shutdown_connections 2024-05-10T03:34:58.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.857+0000 7fe40e600640 1 -- 172.21.15.55:0/803137529 wait complete. 2024-05-10T03:34:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.860+0000 7fe40e600640 1 Processor -- start 2024-05-10T03:34:58.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.860+0000 7fe40e600640 1 -- start start 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.861+0000 7fe40e600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4080a84d0 0x7fe4081502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4080a88e0 0x7fe4080abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40e600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080ac8b0 con 0x7fe3ec0047c0 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080a8e60 con 0x7fe4080a84d0 2024-05-10T03:34:58.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40815a9b0 con 0x7fe4080a88e0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41226/0 (socket says 172.21.15.55:41226) 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 -- 172.21.15.55:0/3937058891 learned_addr learned my addr 172.21.15.55:0/3937058891 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe407e00640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4080a84d0 0x7fe4081502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40dc00640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4080a88e0 0x7fe4080abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 -- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4080a88e0 msgr2=0x7fe4080abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4080a88e0 0x7fe4080abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 -- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4080a84d0 msgr2=0x7fe4081502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4080a84d0 0x7fe4081502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.862+0000 7fe40d200640 1 -- 172.21.15.55:0/3937058891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080a90b0 con 0x7fe3ec0047c0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe40d200640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00115d0 tx=0x7fe3f0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0015870 con 0x7fe3ec0047c0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00159d0 con 0x7fe3ec0047c0 2024-05-10T03:34:58.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4080a9390 con 0x7fe3ec0047c0 2024-05-10T03:34:58.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0015d80 con 0x7fe3ec0047c0 2024-05-10T03:34:58.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.863+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080a9950 con 0x7fe3ec0047c0 2024-05-10T03:34:58.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.865+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe408066ce0 con 0x7fe3ec0047c0 2024-05-10T03:34:58.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.866+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe3f000f480 con 0x7fe3ec0047c0 2024-05-10T03:34:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.866+0000 7fe404c00640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 0x7fe3e406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:58.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.867+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe3f0089d80 con 0x7fe3ec0047c0 2024-05-10T03:34:58.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.867+0000 7fe407e00640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 0x7fe3e406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:58.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.869+0000 7fe407e00640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 0x7fe3e406d110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8033000 tx=0x7fe3f8036040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:58.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:58.871+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe3f0054d90 con 0x7fe3ec0047c0 2024-05-10T03:34:59.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.109+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe40804e290 con 0x7fe3ec0047c0 2024-05-10T03:34:59.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.111+0000 7fe404c00640 1 -- 172.21.15.55:0/3937058891 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4730 (secure 0 0 0) 0x7fe3f0022020 con 0x7fe3ec0047c0 2024-05-10T03:34:59.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:59.114 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/49138572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":49138572},{"type":"v1","addr":"172.21.15.162:6835","nonce":49138572}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/476611421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":476611421},{"type":"v1","addr":"172.21.15.162:6837","nonce":476611421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7356,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2254900234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2254900234},{"type":"v1","addr":"172.21.15.55:6833","nonce":2254900234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:34:58.520498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/315649334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":315649334},{"type":"v1","addr":"172.21.15.55:6835","nonce":315649334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:34:59.114 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 55 2024-05-10T03:34:59.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 msgr2=0x7fe3e406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 0x7fe3e406d110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8033000 tx=0x7fe3f8036040 comp rx=0 tx=0).stop 2024-05-10T03:34:59.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 msgr2=0x7fe4080aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00115d0 tx=0x7fe3f0011a90 comp rx=0 tx=0).stop 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 shutdown_connections 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe4080a88e0 0x7fe4080abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe3e406ac00 0x7fe3e406d110 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.114+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4080a84d0 0x7fe4081502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.115+0000 7fe40e600640 1 --2- 172.21.15.55:0/3937058891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3ec0047c0 0x7fe4080aa150 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.115+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 >> 172.21.15.55:0/3937058891 conn(0x7fe40806a110 msgr2=0x7fe4080aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.115+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 shutdown_connections 2024-05-10T03:34:59.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.115+0000 7fe40e600640 1 -- 172.21.15.55:0/3937058891 wait complete. 2024-05-10T03:34:59.145 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:34:59.424 INFO:tasks.ceph.mds.c.smithi055.stderr:2024-05-10T03:34:59.420+0000 7ff885c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:34:59.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.424+0000 7f5abca00640 1 Processor -- start 2024-05-10T03:34:59.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.425+0000 7f5abca00640 1 -- start start 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.426+0000 7f5abca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 0x7f5ab8144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5abca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ab806d7e0 0x7f5ab80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5abca00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab806d350 con 0x7f5ab80b2fc0 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5abca00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab806c740 con 0x7f5ab806d7e0 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5abca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab806ca20 con 0x7f5ab8144710 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5ab5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ab806d7e0 0x7f5ab80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5ab5e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5ab806d7e0 0x7f5ab80b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41236/0 (socket says 172.21.15.55:41236) 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5ab5e00640 1 -- 172.21.15.55:0/1147106918 learned_addr learned my addr 172.21.15.55:0/1147106918 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:59.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.427+0000 7f5ab6800640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 0x7f5ab8144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.428+0000 7f5ab5e00640 1 -- 172.21.15.55:0/1147106918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 msgr2=0x7f5ab8144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.428+0000 7f5ab5e00640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 0x7f5ab8144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.428+0000 7f5ab5e00640 1 -- 172.21.15.55:0/1147106918 >> v1:172.21.15.162:6789/0 conn(0x7f5ab80b2fc0 legacy=0x7f5ab815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.428+0000 7f5ab5e00640 1 -- 172.21.15.55:0/1147106918 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab806cd00 con 0x7f5ab806d7e0 2024-05-10T03:34:59.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.428+0000 7f5ab6800640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 0x7f5ab8144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:34:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.429+0000 7f5ab5e00640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab80b2a80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009870 tx=0x7f5aa402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862e05fed6614c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.429+0000 7f5ab5400640 1 -- 172.21.15.55:0/1147106918 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4032440 con 0x7f5ab806d7e0 2024-05-10T03:34:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.429+0000 7f5ab5400640 1 -- 172.21.15.55:0/1147106918 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa40325a0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.429+0000 7f5ab5400640 1 -- 172.21.15.55:0/1147106918 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4032930 con 0x7f5ab806d7e0 2024-05-10T03:34:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 -- 172.21.15.55:0/1147106918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 msgr2=0x7f5ab80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab80b2a80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009870 tx=0x7f5aa402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 -- 172.21.15.55:0/1147106918 shutdown_connections 2024-05-10T03:34:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab80b2a80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 --2- 172.21.15.55:0/1147106918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab8144710 0x7f5ab8144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 -- 172.21.15.55:0/1147106918 >> 172.21.15.55:0/1147106918 conn(0x7f5ab806a110 msgr2=0x7f5ab806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 -- 172.21.15.55:0/1147106918 shutdown_connections 2024-05-10T03:34:59.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.430+0000 7f5abca00640 1 -- 172.21.15.55:0/1147106918 wait complete. 2024-05-10T03:34:59.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.433+0000 7f5abca00640 1 Processor -- start 2024-05-10T03:34:59.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.433+0000 7f5abca00640 1 -- start start 2024-05-10T03:34:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab80b2fc0 0x7f5ab813bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab8144710 0x7f5ab806b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab815ca20 con 0x7f5ab806d7e0 2024-05-10T03:34:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8139260 con 0x7f5ab8144710 2024-05-10T03:34:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.434+0000 7f5abca00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab815c4f0 con 0x7f5ab80b2fc0 2024-05-10T03:34:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41244/0 (socket says 172.21.15.55:41244) 2024-05-10T03:34:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 -- 172.21.15.55:0/3461606727 learned_addr learned my addr 172.21.15.55:0/3461606727 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab5e00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab80b2fc0 0x7f5ab813bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 -- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab80b2fc0 msgr2=0x7f5ab813bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab80b2fc0 0x7f5ab813bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 -- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab8144710 msgr2=0x7f5ab806b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab7200640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab8144710 0x7f5ab806b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab8144710 0x7f5ab806b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.435+0000 7f5ab6800640 1 -- 172.21.15.55:0/3461606727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40094d0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5ab6800640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d800 tx=0x7f5aa800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800b6c0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa800b820 con 0x7f5ab806d7e0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800bbd0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81416d0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.436+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab8138dd0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.437+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8066ce0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.438+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5aa800bd30 con 0x7f5ab806d7e0 2024-05-10T03:34:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.439+0000 7f5aaea00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 0x7f5a9806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.439+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5aa8089a30 con 0x7f5ab806d7e0 2024-05-10T03:34:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.440+0000 7f5ab5e00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 0x7f5a9806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.441+0000 7f5ab5e00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 0x7f5a9806d110 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4034000 tx=0x7f5aa402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.445+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5aa80549a0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.646+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ab804e290 con 0x7f5ab806d7e0 2024-05-10T03:34:59.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.648+0000 7f5aaea00640 1 -- 172.21.15.55:0/3461606727 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16248 (secure 0 0 0) 0x7f5aa8057fc0 con 0x7f5ab806d7e0 2024-05-10T03:34:59.652 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:34:59.652 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":104,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:56.064514+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 msgr2=0x7f5a9806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 0x7f5a9806d110 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4034000 tx=0x7f5aa402e000 comp rx=0 tx=0).stop 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 msgr2=0x7f5ab815bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d800 tx=0x7f5aa800dcc0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 shutdown_connections 2024-05-10T03:34:59.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5a9806ac00 0x7f5a9806d110 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab8144710 0x7f5ab806b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab80b2fc0 0x7f5ab813bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 --2- 172.21.15.55:0/3461606727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ab806d7e0 0x7f5ab815bef0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 >> 172.21.15.55:0/3461606727 conn(0x7f5ab806a110 msgr2=0x7f5ab80af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 shutdown_connections 2024-05-10T03:34:59.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.651+0000 7f5abca00640 1 -- 172.21.15.55:0/3461606727 wait complete. 2024-05-10T03:34:59.682 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T03:34:59.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:34:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.961+0000 7f914ee00640 1 Processor -- start 2024-05-10T03:34:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.961+0000 7f914ee00640 1 -- start start 2024-05-10T03:34:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.963+0000 7f914ee00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 0x7f91480cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.963+0000 7f914ee00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91480cb730 0x7f91480b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914ee00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480b4ac0 con 0x7f91480b46b0 2024-05-10T03:34:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914ee00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91480cdc90 con 0x7f91480cb730 2024-05-10T03:34:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914ee00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815ad30 con 0x7f914815a5d0 2024-05-10T03:34:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914e400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f91480b46b0 0x7f914815df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39898/0 (socket says 172.21.15.55:39898) 2024-05-10T03:34:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914e400640 1 -- 172.21.15.55:0/1836043599 learned_addr learned my addr 172.21.15.55:0/1836043599 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914da00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 0x7f91480cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f914d000640 1 --2- 172.21.15.55:0/1836043599 >> v2:172.21.15.162:3300/0 conn(0x7f91480cb730 0x7f91480b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.964+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245116374 0 0) 0x7f91480b4ac0 con 0x7f91480b46b0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9130003520 con 0x7f91480b46b0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007101407 0 0) 0x7f9130003520 con 0x7f91480b46b0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 msgr2=0x7f91480cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 0x7f91480cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 >> v2:172.21.15.162:3300/0 conn(0x7f91480cb730 msgr2=0x7f91480b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 --2- 172.21.15.55:0/1836043599 >> v2:172.21.15.162:3300/0 conn(0x7f91480cb730 0x7f91480b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815b010 con 0x7f91480b46b0 2024-05-10T03:34:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f9140001a10 con 0x7f91480b46b0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 >> v1:172.21.15.55:6789/0 conn(0x7f91480b46b0 legacy=0x7f914815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91300039e0 0x7f9130003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91300047c0 0x7f9130004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 0x7f9130005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480cdc90 con 0x7f91300039e0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914815ad30 con 0x7f91300047c0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913000db90 con 0x7f914815a5d0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.965+0000 7f914da00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91300039e0 0x7f9130003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914d000640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91300047c0 0x7f9130004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914da00640 1 -- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 msgr2=0x7f9130005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914da00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914815a5d0 0x7f9130005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914da00640 1 -- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91300047c0 msgr2=0x7f9130004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914da00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91300047c0 0x7f9130004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.966+0000 7f914da00640 1 -- 172.21.15.55:0/1836043599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815b010 con 0x7f91300039e0 2024-05-10T03:34:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.967+0000 7f914da00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91300039e0 0x7f9130003e50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f913c009a90 tx=0x7f91300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ecd9e4d3f7845d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.967+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c031400 con 0x7f91300039e0 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.967+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c02d050 con 0x7f91300039e0 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.967+0000 7f9147e00640 1 -- 172.21.15.55:0/1836043599 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0317a0 con 0x7f91300039e0 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 -- 172.21.15.55:0/1836043599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91300039e0 msgr2=0x7f9130003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 --2- 172.21.15.55:0/1836043599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91300039e0 0x7f9130003e50 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f913c009a90 tx=0x7f91300057f0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914e400640 1 -- 172.21.15.55:0/1836043599 reap_dead start 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 -- 172.21.15.55:0/1836043599 shutdown_connections 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 -- 172.21.15.55:0/1836043599 >> 172.21.15.55:0/1836043599 conn(0x7f914806a110 msgr2=0x7f914806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 -- 172.21.15.55:0/1836043599 shutdown_connections 2024-05-10T03:34:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.968+0000 7f914ee00640 1 -- 172.21.15.55:0/1836043599 wait complete. 2024-05-10T03:34:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.971+0000 7f914ee00640 1 Processor -- start 2024-05-10T03:34:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.971+0000 7f914ee00640 1 -- start start 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91300039e0 0x7f914815d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9148153010 0x7f9148142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148153420 con 0x7f91481403c0 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481411c0 con 0x7f9148153010 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.972+0000 7f914ee00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815bd80 con 0x7f91300039e0 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41252/0 (socket says 172.21.15.55:41252) 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 -- 172.21.15.55:0/4170396394 learned_addr learned my addr 172.21.15.55:0/4170396394 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914da00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91300039e0 0x7f914815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914d000640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9148153010 0x7f9148142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 -- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91300039e0 msgr2=0x7f914815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91300039e0 0x7f914815d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 -- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9148153010 msgr2=0x7f9148142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:34:59.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9148153010 0x7f9148142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:34:59.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.973+0000 7f914e400640 1 -- 172.21.15.55:0/4170396394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148140d10 con 0x7f91481403c0 2024-05-10T03:34:59.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f914e400640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f91480b22e0 tx=0x7f914000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914000a640 con 0x7f91481403c0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914000a7a0 con 0x7f91481403c0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914000ab50 con 0x7f91481403c0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914815bfd0 con 0x7f91481403c0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.974+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914815c590 con 0x7f91481403c0 2024-05-10T03:34:59.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.976+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9148066ce0 con 0x7f91481403c0 2024-05-10T03:34:59.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.977+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f914000acb0 con 0x7f91481403c0 2024-05-10T03:34:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.978+0000 7f9145600640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 0x7f912806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:34:59.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.978+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f91400885c0 con 0x7f91481403c0 2024-05-10T03:34:59.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.978+0000 7f914da00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 0x7f912806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:34:59.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.980+0000 7f914da00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 0x7f912806d110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f913c033000 tx=0x7f913c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:34:59.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:34:59.983+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f91400535d0 con 0x7f91481403c0 2024-05-10T03:35:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.184+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f914815b5d0 con 0x7f912806ac00 2024-05-10T03:35:00.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.208+0000 7f9145600640 1 -- 172.21.15.55:0/4170396394 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287468 (secure 0 0 0) 0x7f913c0315a0 con 0x7f912806ac00 2024-05-10T03:35:00.217 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:00.218 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 msgr2=0x7f912806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 0x7f912806d110 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f913c033000 tx=0x7f913c008030 comp rx=0 tx=0).stop 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 msgr2=0x7f91481407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f91480b22e0 tx=0x7f914000ccd0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 shutdown_connections 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91481403c0 0x7f91481407d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f912806ac00 0x7f912806d110 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9148153010 0x7f9148142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 --2- 172.21.15.55:0/4170396394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91300039e0 0x7f914815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 >> 172.21.15.55:0/4170396394 conn(0x7f914806a110 msgr2=0x7f9148070410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:00.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 shutdown_connections 2024-05-10T03:35:00.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.216+0000 7f914ee00640 1 -- 172.21.15.55:0/4170396394 wait complete. 2024-05-10T03:35:00.249 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-05-10T03:34:59.073115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260236,"kb_used_data":11268,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714484,"statfs":{"total":773094113280,"available":772827631616,"internally_reserved":0,"allocated":11538432,"data_stored":5860232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.602065"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088547+0000","last_change":"2024-05-10T03:34:55.077649+0000","last_active":"2024-05-10T03:34:56.088547+0000","last_peered":"2024-05-10T03:34:56.088547+0000","last_clean":"2024-05-10T03:34:56.088547+0000","last_became_active":"2024-05-10T03:34:55.077340+0000","last_became_peered":"2024-05-10T03:34:55.077340+0000","last_unstale":"2024-05-10T03:34:56.088547+0000","last_undegraded":"2024-05-10T03:34:56.088547+0000","last_fullsized":"2024-05-10T03:34:56.088547+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:06.604455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094711+0000","last_change":"2024-05-10T03:34:55.109920+0000","last_active":"2024-05-10T03:34:56.094711+0000","last_peered":"2024-05-10T03:34:56.094711+0000","last_clean":"2024-05-10T03:34:56.094711+0000","last_became_active":"2024-05-10T03:34:55.109571+0000","last_became_peered":"2024-05-10T03:34:55.109571+0000","last_unstale":"2024-05-10T03:34:56.094711+0000","last_undegraded":"2024-05-10T03:34:56.094711+0000","last_fullsized":"2024-05-10T03:34:56.094711+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:08.980669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088174+0000","last_change":"2024-05-10T03:34:55.105594+0000","last_active":"2024-05-10T03:34:56.088174+0000","last_peered":"2024-05-10T03:34:56.088174+0000","last_clean":"2024-05-10T03:34:56.088174+0000","last_became_active":"2024-05-10T03:34:55.105311+0000","last_became_peered":"2024-05-10T03:34:55.105311+0000","last_unstale":"2024-05-10T03:34:56.088174+0000","last_undegraded":"2024-05-10T03:34:56.088174+0000","last_fullsized":"2024-05-10T03:34:56.088174+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:27.539672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065085+0000","last_change":"2024-05-10T03:34:55.115698+0000","last_active":"2024-05-10T03:34:57.065085+0000","last_peered":"2024-05-10T03:34:57.065085+0000","last_clean":"2024-05-10T03:34:57.065085+0000","last_became_active":"2024-05-10T03:34:55.115364+0000","last_became_peered":"2024-05-10T03:34:55.115364+0000","last_unstale":"2024-05-10T03:34:57.065085+0000","last_undegraded":"2024-05-10T03:34:57.065085+0000","last_fullsized":"2024-05-10T03:34:57.065085+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:18.215176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.112336+0000","last_change":"2024-05-10T03:34:55.111718+0000","last_active":"2024-05-10T03:34:55.112336+0000","last_peered":"2024-05-10T03:34:55.112336+0000","last_clean":"2024-05-10T03:34:55.112336+0000","last_became_active":"2024-05-10T03:34:55.111397+0000","last_became_peered":"2024-05-10T03:34:55.111397+0000","last_unstale":"2024-05-10T03:34:55.112336+0000","last_undegraded":"2024-05-10T03:34:55.112336+0000","last_fullsized":"2024-05-10T03:34:55.112336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:09.157760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095633+0000","last_change":"2024-05-10T03:34:55.095876+0000","last_active":"2024-05-10T03:34:56.095633+0000","last_peered":"2024-05-10T03:34:56.095633+0000","last_clean":"2024-05-10T03:34:56.095633+0000","last_became_active":"2024-05-10T03:34:55.095414+0000","last_became_peered":"2024-05-10T03:34:55.095414+0000","last_unstale":"2024-05-10T03:34:56.095633+0000","last_undegraded":"2024-05-10T03:34:56.095633+0000","last_fullsized":"2024-05-10T03:34:56.095633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.839548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093709+0000","last_change":"2024-05-10T03:34:55.095724+0000","last_active":"2024-05-10T03:34:56.093709+0000","last_peered":"2024-05-10T03:34:56.093709+0000","last_clean":"2024-05-10T03:34:56.093709+0000","last_became_active":"2024-05-10T03:34:55.095020+0000","last_became_peered":"2024-05-10T03:34:55.095020+0000","last_unstale":"2024-05-10T03:34:56.093709+0000","last_undegraded":"2024-05-10T03:34:56.093709+0000","last_fullsized":"2024-05-10T03:34:56.093709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:04.192750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.103211+0000","last_change":"2024-05-10T03:34:55.094909+0000","last_active":"2024-05-10T03:34:55.103211+0000","last_peered":"2024-05-10T03:34:55.103211+0000","last_clean":"2024-05-10T03:34:55.103211+0000","last_became_active":"2024-05-10T03:34:55.092853+0000","last_became_peered":"2024-05-10T03:34:55.092853+0000","last_unstale":"2024-05-10T03:34:55.103211+0000","last_undegraded":"2024-05-10T03:34:55.103211+0000","last_fullsized":"2024-05-10T03:34:55.103211+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:41.983368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094661+0000","last_change":"2024-05-10T03:34:55.109194+0000","last_active":"2024-05-10T03:34:56.094661+0000","last_peered":"2024-05-10T03:34:56.094661+0000","last_clean":"2024-05-10T03:34:56.094661+0000","last_became_active":"2024-05-10T03:34:55.108820+0000","last_became_peered":"2024-05-10T03:34:55.108820+0000","last_unstale":"2024-05-10T03:34:56.094661+0000","last_undegraded":"2024-05-10T03:34:56.094661+0000","last_fullsized":"2024-05-10T03:34:56.094661+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:12.843880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101413+0000","last_change":"2024-05-10T03:34:55.095814+0000","last_active":"2024-05-10T03:34:55.101413+0000","last_peered":"2024-05-10T03:34:55.101413+0000","last_clean":"2024-05-10T03:34:55.101413+0000","last_became_active":"2024-05-10T03:34:55.095466+0000","last_became_peered":"2024-05-10T03:34:55.095466+0000","last_unstale":"2024-05-10T03:34:55.101413+0000","last_undegraded":"2024-05-10T03:34:55.101413+0000","last_fullsized":"2024-05-10T03:34:55.101413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:01.591436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089055+0000","last_change":"2024-05-10T03:34:55.104405+0000","last_active":"2024-05-10T03:34:56.089055+0000","last_peered":"2024-05-10T03:34:56.089055+0000","last_clean":"2024-05-10T03:34:56.089055+0000","last_became_active":"2024-05-10T03:34:55.104031+0000","last_became_peered":"2024-05-10T03:34:55.104031+0000","last_unstale":"2024-05-10T03:34:56.089055+0000","last_undegraded":"2024-05-10T03:34:56.089055+0000","last_fullsized":"2024-05-10T03:34:56.089055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:08.392940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089816+0000","last_change":"2024-05-10T03:34:55.112050+0000","last_active":"2024-05-10T03:34:56.089816+0000","last_peered":"2024-05-10T03:34:56.089816+0000","last_clean":"2024-05-10T03:34:56.089816+0000","last_became_active":"2024-05-10T03:34:55.111738+0000","last_became_peered":"2024-05-10T03:34:55.111738+0000","last_unstale":"2024-05-10T03:34:56.089816+0000","last_undegraded":"2024-05-10T03:34:56.089816+0000","last_fullsized":"2024-05-10T03:34:56.089816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:29.132456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064730+0000","last_change":"2024-05-10T03:34:55.100903+0000","last_active":"2024-05-10T03:34:57.064730+0000","last_peered":"2024-05-10T03:34:57.064730+0000","last_clean":"2024-05-10T03:34:57.064730+0000","last_became_active":"2024-05-10T03:34:55.100632+0000","last_became_peered":"2024-05-10T03:34:55.100632+0000","last_unstale":"2024-05-10T03:34:57.064730+0000","last_undegraded":"2024-05-10T03:34:57.064730+0000","last_fullsized":"2024-05-10T03:34:57.064730+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:32.167414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096101+0000","last_change":"2024-05-10T03:34:55.113735+0000","last_active":"2024-05-10T03:34:56.096101+0000","last_peered":"2024-05-10T03:34:56.096101+0000","last_clean":"2024-05-10T03:34:56.096101+0000","last_became_active":"2024-05-10T03:34:55.111309+0000","last_became_peered":"2024-05-10T03:34:55.111309+0000","last_unstale":"2024-05-10T03:34:56.096101+0000","last_undegraded":"2024-05-10T03:34:56.096101+0000","last_fullsized":"2024-05-10T03:34:56.096101+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:43.452496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095066+0000","last_change":"2024-05-10T03:34:55.105226+0000","last_active":"2024-05-10T03:34:56.095066+0000","last_peered":"2024-05-10T03:34:56.095066+0000","last_clean":"2024-05-10T03:34:56.095066+0000","last_became_active":"2024-05-10T03:34:55.104915+0000","last_became_peered":"2024-05-10T03:34:55.104915+0000","last_unstale":"2024-05-10T03:34:56.095066+0000","last_undegraded":"2024-05-10T03:34:56.095066+0000","last_fullsized":"2024-05-10T03:34:56.095066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:08.392940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095046+0000","last_change":"2024-05-10T03:34:55.107749+0000","last_active":"2024-05-10T03:34:56.095046+0000","last_peered":"2024-05-10T03:34:56.095046+0000","last_clean":"2024-05-10T03:34:56.095046+0000","last_became_active":"2024-05-10T03:34:55.107434+0000","last_became_peered":"2024-05-10T03:34:55.107434+0000","last_unstale":"2024-05-10T03:34:56.095046+0000","last_undegraded":"2024-05-10T03:34:56.095046+0000","last_fullsized":"2024-05-10T03:34:56.095046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:26:57.303197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088744+0000","last_change":"2024-05-10T03:34:55.102087+0000","last_active":"2024-05-10T03:34:56.088744+0000","last_peered":"2024-05-10T03:34:56.088744+0000","last_clean":"2024-05-10T03:34:56.088744+0000","last_became_active":"2024-05-10T03:34:55.101605+0000","last_became_peered":"2024-05-10T03:34:55.101605+0000","last_unstale":"2024-05-10T03:34:56.088744+0000","last_undegraded":"2024-05-10T03:34:56.088744+0000","last_fullsized":"2024-05-10T03:34:56.088744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:46:02.980543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094863+0000","last_change":"2024-05-10T03:34:55.095622+0000","last_active":"2024-05-10T03:34:56.094863+0000","last_peered":"2024-05-10T03:34:56.094863+0000","last_clean":"2024-05-10T03:34:56.094863+0000","last_became_active":"2024-05-10T03:34:55.095170+0000","last_became_peered":"2024-05-10T03:34:55.095170+0000","last_unstale":"2024-05-10T03:34:56.094863+0000","last_undegraded":"2024-05-10T03:34:56.094863+0000","last_fullsized":"2024-05-10T03:34:56.094863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:09.157760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088334+0000","last_change":"2024-05-10T03:34:55.100803+0000","last_active":"2024-05-10T03:34:56.088334+0000","last_peered":"2024-05-10T03:34:56.088334+0000","last_clean":"2024-05-10T03:34:56.088334+0000","last_became_active":"2024-05-10T03:34:55.100497+0000","last_became_peered":"2024-05-10T03:34:55.100497+0000","last_unstale":"2024-05-10T03:34:56.088334+0000","last_undegraded":"2024-05-10T03:34:56.088334+0000","last_fullsized":"2024-05-10T03:34:56.088334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:56:36.190853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.090078+0000","last_change":"2024-05-10T03:34:55.114691+0000","last_active":"2024-05-10T03:34:56.090078+0000","last_peered":"2024-05-10T03:34:56.090078+0000","last_clean":"2024-05-10T03:34:56.090078+0000","last_became_active":"2024-05-10T03:34:55.114357+0000","last_became_peered":"2024-05-10T03:34:55.114357+0000","last_unstale":"2024-05-10T03:34:56.090078+0000","last_undegraded":"2024-05-10T03:34:56.090078+0000","last_fullsized":"2024-05-10T03:34:56.090078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:03.172489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109346+0000","last_change":"2024-05-10T03:34:55.105353+0000","last_active":"2024-05-10T03:34:55.109346+0000","last_peered":"2024-05-10T03:34:55.109346+0000","last_clean":"2024-05-10T03:34:55.109346+0000","last_became_active":"2024-05-10T03:34:55.105081+0000","last_became_peered":"2024-05-10T03:34:55.105081+0000","last_unstale":"2024-05-10T03:34:55.109346+0000","last_undegraded":"2024-05-10T03:34:55.109346+0000","last_fullsized":"2024-05-10T03:34:55.109346+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:49:41.797240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.105565+0000","last_change":"2024-05-10T03:34:55.100864+0000","last_active":"2024-05-10T03:34:55.105565+0000","last_peered":"2024-05-10T03:34:55.105565+0000","last_clean":"2024-05-10T03:34:55.105565+0000","last_became_active":"2024-05-10T03:34:55.100543+0000","last_became_peered":"2024-05-10T03:34:55.100543+0000","last_unstale":"2024-05-10T03:34:55.105565+0000","last_undegraded":"2024-05-10T03:34:55.105565+0000","last_fullsized":"2024-05-10T03:34:55.105565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:43.865361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088226+0000","last_change":"2024-05-10T03:34:55.087411+0000","last_active":"2024-05-10T03:34:56.088226+0000","last_peered":"2024-05-10T03:34:56.088226+0000","last_clean":"2024-05-10T03:34:56.088226+0000","last_became_active":"2024-05-10T03:34:55.087123+0000","last_became_peered":"2024-05-10T03:34:55.087123+0000","last_unstale":"2024-05-10T03:34:56.088226+0000","last_undegraded":"2024-05-10T03:34:56.088226+0000","last_fullsized":"2024-05-10T03:34:56.088226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:13.676468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.090058+0000","last_change":"2024-05-10T03:34:55.080587+0000","last_active":"2024-05-10T03:34:55.090058+0000","last_peered":"2024-05-10T03:34:55.090058+0000","last_clean":"2024-05-10T03:34:55.090058+0000","last_became_active":"2024-05-10T03:34:55.080271+0000","last_became_peered":"2024-05-10T03:34:55.080271+0000","last_unstale":"2024-05-10T03:34:55.090058+0000","last_undegraded":"2024-05-10T03:34:55.090058+0000","last_fullsized":"2024-05-10T03:34:55.090058+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:04.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089088+0000","last_change":"2024-05-10T03:34:55.104086+0000","last_active":"2024-05-10T03:34:56.089088+0000","last_peered":"2024-05-10T03:34:56.089088+0000","last_clean":"2024-05-10T03:34:56.089088+0000","last_became_active":"2024-05-10T03:34:55.103765+0000","last_became_peered":"2024-05-10T03:34:55.103765+0000","last_unstale":"2024-05-10T03:34:56.089088+0000","last_undegraded":"2024-05-10T03:34:56.089088+0000","last_fullsized":"2024-05-10T03:34:56.089088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:30.486847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088109+0000","last_change":"2024-05-10T03:34:55.116121+0000","last_active":"2024-05-10T03:34:56.088109+0000","last_peered":"2024-05-10T03:34:56.088109+0000","last_clean":"2024-05-10T03:34:56.088109+0000","last_became_active":"2024-05-10T03:34:55.115759+0000","last_became_peered":"2024-05-10T03:34:55.115759+0000","last_unstale":"2024-05-10T03:34:56.088109+0000","last_undegraded":"2024-05-10T03:34:56.088109+0000","last_fullsized":"2024-05-10T03:34:56.088109+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:50.553885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089473+0000","last_change":"2024-05-10T03:34:55.111309+0000","last_active":"2024-05-10T03:34:56.089473+0000","last_peered":"2024-05-10T03:34:56.089473+0000","last_clean":"2024-05-10T03:34:56.089473+0000","last_became_active":"2024-05-10T03:34:55.110766+0000","last_became_peered":"2024-05-10T03:34:55.110766+0000","last_unstale":"2024-05-10T03:34:56.089473+0000","last_undegraded":"2024-05-10T03:34:56.089473+0000","last_fullsized":"2024-05-10T03:34:56.089473+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:52.993476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094038+0000","last_change":"2024-05-10T03:34:55.101911+0000","last_active":"2024-05-10T03:34:56.094038+0000","last_peered":"2024-05-10T03:34:56.094038+0000","last_clean":"2024-05-10T03:34:56.094038+0000","last_became_active":"2024-05-10T03:34:55.101597+0000","last_became_peered":"2024-05-10T03:34:55.101597+0000","last_unstale":"2024-05-10T03:34:56.094038+0000","last_undegraded":"2024-05-10T03:34:56.094038+0000","last_fullsized":"2024-05-10T03:34:56.094038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:01.591436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109554+0000","last_change":"2024-05-10T03:34:55.107224+0000","last_active":"2024-05-10T03:34:55.109554+0000","last_peered":"2024-05-10T03:34:55.109554+0000","last_clean":"2024-05-10T03:34:55.109554+0000","last_became_active":"2024-05-10T03:34:55.106951+0000","last_became_peered":"2024-05-10T03:34:55.106951+0000","last_unstale":"2024-05-10T03:34:55.109554+0000","last_undegraded":"2024-05-10T03:34:55.109554+0000","last_fullsized":"2024-05-10T03:34:55.109554+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:37:42.690277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088315+0000","last_change":"2024-05-10T03:34:55.110671+0000","last_active":"2024-05-10T03:34:56.088315+0000","last_peered":"2024-05-10T03:34:56.088315+0000","last_clean":"2024-05-10T03:34:56.088315+0000","last_became_active":"2024-05-10T03:34:55.110353+0000","last_became_peered":"2024-05-10T03:34:55.110353+0000","last_unstale":"2024-05-10T03:34:56.088315+0000","last_undegraded":"2024-05-10T03:34:56.088315+0000","last_fullsized":"2024-05-10T03:34:56.088315+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:59:38.486380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093759+0000","last_change":"2024-05-10T03:34:55.088540+0000","last_active":"2024-05-10T03:34:56.093759+0000","last_peered":"2024-05-10T03:34:56.093759+0000","last_clean":"2024-05-10T03:34:56.093759+0000","last_became_active":"2024-05-10T03:34:55.087418+0000","last_became_peered":"2024-05-10T03:34:55.087418+0000","last_unstale":"2024-05-10T03:34:56.093759+0000","last_undegraded":"2024-05-10T03:34:56.093759+0000","last_fullsized":"2024-05-10T03:34:56.093759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.114004+0000","last_change":"2024-05-10T03:34:55.113390+0000","last_active":"2024-05-10T03:34:55.114004+0000","last_peered":"2024-05-10T03:34:55.114004+0000","last_clean":"2024-05-10T03:34:55.114004+0000","last_became_active":"2024-05-10T03:34:55.112738+0000","last_became_peered":"2024-05-10T03:34:55.112738+0000","last_unstale":"2024-05-10T03:34:55.114004+0000","last_undegraded":"2024-05-10T03:34:55.114004+0000","last_fullsized":"2024-05-10T03:34:55.114004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:14.973937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088798+0000","last_change":"2024-05-10T03:34:55.096482+0000","last_active":"2024-05-10T03:34:56.088798+0000","last_peered":"2024-05-10T03:34:56.088798+0000","last_clean":"2024-05-10T03:34:56.088798+0000","last_became_active":"2024-05-10T03:34:55.094241+0000","last_became_peered":"2024-05-10T03:34:55.094241+0000","last_unstale":"2024-05-10T03:34:56.088798+0000","last_undegraded":"2024-05-10T03:34:56.088798+0000","last_fullsized":"2024-05-10T03:34:56.088798+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:46.679850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065314+0000","last_change":"2024-05-10T03:34:54.521590+0000","last_active":"2024-05-10T03:34:57.065314+0000","last_peered":"2024-05-10T03:34:57.065314+0000","last_clean":"2024-05-10T03:34:57.065314+0000","last_became_active":"2024-05-10T03:34:54.520440+0000","last_became_peered":"2024-05-10T03:34:54.520440+0000","last_unstale":"2024-05-10T03:34:57.065314+0000","last_undegraded":"2024-05-10T03:34:57.065314+0000","last_fullsized":"2024-05-10T03:34:57.065314+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:01.729350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101072+0000","last_change":"2024-05-10T03:34:55.093051+0000","last_active":"2024-05-10T03:34:55.101072+0000","last_peered":"2024-05-10T03:34:55.101072+0000","last_clean":"2024-05-10T03:34:55.101072+0000","last_became_active":"2024-05-10T03:34:55.091314+0000","last_became_peered":"2024-05-10T03:34:55.091314+0000","last_unstale":"2024-05-10T03:34:55.101072+0000","last_undegraded":"2024-05-10T03:34:55.101072+0000","last_fullsized":"2024-05-10T03:34:55.101072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:57.131341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.116965+0000","last_change":"2024-05-10T03:34:54.514610+0000","last_active":"2024-05-10T03:34:55.116965+0000","last_peered":"2024-05-10T03:34:55.116965+0000","last_clean":"2024-05-10T03:34:55.116965+0000","last_became_active":"2024-05-10T03:34:54.514274+0000","last_became_peered":"2024-05-10T03:34:54.514274+0000","last_unstale":"2024-05-10T03:34:55.116965+0000","last_undegraded":"2024-05-10T03:34:55.116965+0000","last_fullsized":"2024-05-10T03:34:55.116965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:11.673874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088605+0000","last_change":"2024-05-10T03:34:55.113011+0000","last_active":"2024-05-10T03:34:56.088605+0000","last_peered":"2024-05-10T03:34:56.088605+0000","last_clean":"2024-05-10T03:34:56.088605+0000","last_became_active":"2024-05-10T03:34:55.110922+0000","last_became_peered":"2024-05-10T03:34:55.110922+0000","last_unstale":"2024-05-10T03:34:56.088605+0000","last_undegraded":"2024-05-10T03:34:56.088605+0000","last_fullsized":"2024-05-10T03:34:56.088605+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:17.592050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089792+0000","last_change":"2024-05-10T03:34:54.512678+0000","last_active":"2024-05-10T03:34:56.089792+0000","last_peered":"2024-05-10T03:34:56.089792+0000","last_clean":"2024-05-10T03:34:56.089792+0000","last_became_active":"2024-05-10T03:34:54.512278+0000","last_became_peered":"2024-05-10T03:34:54.512278+0000","last_unstale":"2024-05-10T03:34:56.089792+0000","last_undegraded":"2024-05-10T03:34:56.089792+0000","last_fullsized":"2024-05-10T03:34:56.089792+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:07.965562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089412+0000","last_change":"2024-05-10T03:34:55.117653+0000","last_active":"2024-05-10T03:34:56.089412+0000","last_peered":"2024-05-10T03:34:56.089412+0000","last_clean":"2024-05-10T03:34:56.089412+0000","last_became_active":"2024-05-10T03:34:55.117320+0000","last_became_peered":"2024-05-10T03:34:55.117320+0000","last_unstale":"2024-05-10T03:34:56.089412+0000","last_undegraded":"2024-05-10T03:34:56.089412+0000","last_fullsized":"2024-05-10T03:34:56.089412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:12.843880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109926+0000","last_change":"2024-05-10T03:34:54.518091+0000","last_active":"2024-05-10T03:34:55.109926+0000","last_peered":"2024-05-10T03:34:55.109926+0000","last_clean":"2024-05-10T03:34:55.109926+0000","last_became_active":"2024-05-10T03:34:54.517380+0000","last_became_peered":"2024-05-10T03:34:54.517380+0000","last_unstale":"2024-05-10T03:34:55.109926+0000","last_undegraded":"2024-05-10T03:34:55.109926+0000","last_fullsized":"2024-05-10T03:34:55.109926+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:18.138740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065247+0000","last_change":"2024-05-10T03:34:55.089287+0000","last_active":"2024-05-10T03:34:57.065247+0000","last_peered":"2024-05-10T03:34:57.065247+0000","last_clean":"2024-05-10T03:34:57.065247+0000","last_became_active":"2024-05-10T03:34:55.088901+0000","last_became_peered":"2024-05-10T03:34:55.088901+0000","last_unstale":"2024-05-10T03:34:57.065247+0000","last_undegraded":"2024-05-10T03:34:57.065247+0000","last_fullsized":"2024-05-10T03:34:57.065247+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:30.486847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096269+0000","last_change":"2024-05-10T03:34:54.520586+0000","last_active":"2024-05-10T03:34:56.096269+0000","last_peered":"2024-05-10T03:34:56.096269+0000","last_clean":"2024-05-10T03:34:56.096269+0000","last_became_active":"2024-05-10T03:34:54.520290+0000","last_became_peered":"2024-05-10T03:34:54.520290+0000","last_unstale":"2024-05-10T03:34:56.096269+0000","last_undegraded":"2024-05-10T03:34:56.096269+0000","last_fullsized":"2024-05-10T03:34:56.096269+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:07.965562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088645+0000","last_change":"2024-05-10T03:34:55.099462+0000","last_active":"2024-05-10T03:34:56.088645+0000","last_peered":"2024-05-10T03:34:56.088645+0000","last_clean":"2024-05-10T03:34:56.088645+0000","last_became_active":"2024-05-10T03:34:55.099150+0000","last_became_peered":"2024-05-10T03:34:55.099150+0000","last_unstale":"2024-05-10T03:34:56.088645+0000","last_undegraded":"2024-05-10T03:34:56.088645+0000","last_fullsized":"2024-05-10T03:34:56.088645+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:30:55.384602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088202+0000","last_change":"2024-05-10T03:34:54.507498+0000","last_active":"2024-05-10T03:34:56.088202+0000","last_peered":"2024-05-10T03:34:56.088202+0000","last_clean":"2024-05-10T03:34:56.088202+0000","last_became_active":"2024-05-10T03:34:54.507088+0000","last_became_peered":"2024-05-10T03:34:54.507088+0000","last_unstale":"2024-05-10T03:34:56.088202+0000","last_undegraded":"2024-05-10T03:34:56.088202+0000","last_fullsized":"2024-05-10T03:34:56.088202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:42.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094992+0000","last_change":"2024-05-10T03:34:55.104750+0000","last_active":"2024-05-10T03:34:56.094992+0000","last_peered":"2024-05-10T03:34:56.094992+0000","last_clean":"2024-05-10T03:34:56.094992+0000","last_became_active":"2024-05-10T03:34:55.104384+0000","last_became_peered":"2024-05-10T03:34:55.104384+0000","last_unstale":"2024-05-10T03:34:56.094992+0000","last_undegraded":"2024-05-10T03:34:56.094992+0000","last_fullsized":"2024-05-10T03:34:56.094992+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:22.811888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089087+0000","last_change":"2024-05-10T03:34:54.514671+0000","last_active":"2024-05-10T03:34:56.089087+0000","last_peered":"2024-05-10T03:34:56.089087+0000","last_clean":"2024-05-10T03:34:56.089087+0000","last_became_active":"2024-05-10T03:34:54.514302+0000","last_became_peered":"2024-05-10T03:34:54.514302+0000","last_unstale":"2024-05-10T03:34:56.089087+0000","last_undegraded":"2024-05-10T03:34:56.089087+0000","last_fullsized":"2024-05-10T03:34:56.089087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:00.576330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065310+0000","last_change":"2024-05-10T03:34:55.079396+0000","last_active":"2024-05-10T03:34:57.065310+0000","last_peered":"2024-05-10T03:34:57.065310+0000","last_clean":"2024-05-10T03:34:57.065310+0000","last_became_active":"2024-05-10T03:34:55.079115+0000","last_became_peered":"2024-05-10T03:34:55.079115+0000","last_unstale":"2024-05-10T03:34:57.065310+0000","last_undegraded":"2024-05-10T03:34:57.065310+0000","last_fullsized":"2024-05-10T03:34:57.065310+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:29.132456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064715+0000","last_change":"2024-05-10T03:34:54.511889+0000","last_active":"2024-05-10T03:34:57.064715+0000","last_peered":"2024-05-10T03:34:57.064715+0000","last_clean":"2024-05-10T03:34:57.064715+0000","last_became_active":"2024-05-10T03:34:54.511592+0000","last_became_peered":"2024-05-10T03:34:54.511592+0000","last_unstale":"2024-05-10T03:34:57.064715+0000","last_undegraded":"2024-05-10T03:34:57.064715+0000","last_fullsized":"2024-05-10T03:34:57.064715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:07.377833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088216+0000","last_change":"2024-05-10T03:34:55.094506+0000","last_active":"2024-05-10T03:34:56.088216+0000","last_peered":"2024-05-10T03:34:56.088216+0000","last_clean":"2024-05-10T03:34:56.088216+0000","last_became_active":"2024-05-10T03:34:55.093280+0000","last_became_peered":"2024-05-10T03:34:55.093280+0000","last_unstale":"2024-05-10T03:34:56.088216+0000","last_undegraded":"2024-05-10T03:34:56.088216+0000","last_fullsized":"2024-05-10T03:34:56.088216+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:26:01.526134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089354+0000","last_change":"2024-05-10T03:34:54.516666+0000","last_active":"2024-05-10T03:34:56.089354+0000","last_peered":"2024-05-10T03:34:56.089354+0000","last_clean":"2024-05-10T03:34:56.089354+0000","last_became_active":"2024-05-10T03:34:54.514815+0000","last_became_peered":"2024-05-10T03:34:54.514815+0000","last_unstale":"2024-05-10T03:34:56.089354+0000","last_undegraded":"2024-05-10T03:34:56.089354+0000","last_fullsized":"2024-05-10T03:34:56.089354+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:48.537060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.093485+0000","last_change":"2024-05-10T03:34:55.090264+0000","last_active":"2024-05-10T03:34:55.093485+0000","last_peered":"2024-05-10T03:34:55.093485+0000","last_clean":"2024-05-10T03:34:55.093485+0000","last_became_active":"2024-05-10T03:34:55.088684+0000","last_became_peered":"2024-05-10T03:34:55.088684+0000","last_unstale":"2024-05-10T03:34:55.093485+0000","last_undegraded":"2024-05-10T03:34:55.093485+0000","last_fullsized":"2024-05-10T03:34:55.093485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088399+0000","last_change":"2024-05-10T03:34:54.511475+0000","last_active":"2024-05-10T03:34:56.088399+0000","last_peered":"2024-05-10T03:34:56.088399+0000","last_clean":"2024-05-10T03:34:56.088399+0000","last_became_active":"2024-05-10T03:34:54.511115+0000","last_became_peered":"2024-05-10T03:34:54.511115+0000","last_unstale":"2024-05-10T03:34:56.088399+0000","last_undegraded":"2024-05-10T03:34:56.088399+0000","last_fullsized":"2024-05-10T03:34:56.088399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:17.200070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.098291+0000","last_change":"2024-05-10T03:34:55.092183+0000","last_active":"2024-05-10T03:34:55.098291+0000","last_peered":"2024-05-10T03:34:55.098291+0000","last_clean":"2024-05-10T03:34:55.098291+0000","last_became_active":"2024-05-10T03:34:55.089705+0000","last_became_peered":"2024-05-10T03:34:55.089705+0000","last_unstale":"2024-05-10T03:34:55.098291+0000","last_undegraded":"2024-05-10T03:34:55.098291+0000","last_fullsized":"2024-05-10T03:34:55.098291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:48.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095628+0000","last_change":"2024-05-10T03:34:54.511250+0000","last_active":"2024-05-10T03:34:56.095628+0000","last_peered":"2024-05-10T03:34:56.095628+0000","last_clean":"2024-05-10T03:34:56.095628+0000","last_became_active":"2024-05-10T03:34:54.507953+0000","last_became_peered":"2024-05-10T03:34:54.507953+0000","last_unstale":"2024-05-10T03:34:56.095628+0000","last_undegraded":"2024-05-10T03:34:56.095628+0000","last_fullsized":"2024-05-10T03:34:56.095628+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:08.142653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.096305+0000","last_change":"2024-05-10T03:34:55.091651+0000","last_active":"2024-05-10T03:34:55.096305+0000","last_peered":"2024-05-10T03:34:55.096305+0000","last_clean":"2024-05-10T03:34:55.096305+0000","last_became_active":"2024-05-10T03:34:55.090792+0000","last_became_peered":"2024-05-10T03:34:55.090792+0000","last_unstale":"2024-05-10T03:34:55.096305+0000","last_undegraded":"2024-05-10T03:34:55.096305+0000","last_fullsized":"2024-05-10T03:34:55.096305+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:08.980669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094711+0000","last_change":"2024-05-10T03:34:54.523896+0000","last_active":"2024-05-10T03:34:56.094711+0000","last_peered":"2024-05-10T03:34:56.094711+0000","last_clean":"2024-05-10T03:34:56.094711+0000","last_became_active":"2024-05-10T03:34:54.523578+0000","last_became_peered":"2024-05-10T03:34:54.523578+0000","last_unstale":"2024-05-10T03:34:56.094711+0000","last_undegraded":"2024-05-10T03:34:56.094711+0000","last_fullsized":"2024-05-10T03:34:56.094711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:10:06.945240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087958+0000","last_change":"2024-05-10T03:34:55.084506+0000","last_active":"2024-05-10T03:34:56.087958+0000","last_peered":"2024-05-10T03:34:56.087958+0000","last_clean":"2024-05-10T03:34:56.087958+0000","last_became_active":"2024-05-10T03:34:55.083882+0000","last_became_peered":"2024-05-10T03:34:55.083882+0000","last_unstale":"2024-05-10T03:34:56.087958+0000","last_undegraded":"2024-05-10T03:34:56.087958+0000","last_fullsized":"2024-05-10T03:34:56.087958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.106807+0000","last_change":"2024-05-10T03:34:54.517352+0000","last_active":"2024-05-10T03:34:55.106807+0000","last_peered":"2024-05-10T03:34:55.106807+0000","last_clean":"2024-05-10T03:34:55.106807+0000","last_became_active":"2024-05-10T03:34:54.516835+0000","last_became_peered":"2024-05-10T03:34:54.516835+0000","last_unstale":"2024-05-10T03:34:55.106807+0000","last_undegraded":"2024-05-10T03:34:55.106807+0000","last_fullsized":"2024-05-10T03:34:55.106807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:46.268806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095462+0000","last_change":"2024-05-10T03:34:55.092718+0000","last_active":"2024-05-10T03:34:56.095462+0000","last_peered":"2024-05-10T03:34:56.095462+0000","last_clean":"2024-05-10T03:34:56.095462+0000","last_became_active":"2024-05-10T03:34:55.091154+0000","last_became_peered":"2024-05-10T03:34:55.091154+0000","last_unstale":"2024-05-10T03:34:56.095462+0000","last_undegraded":"2024-05-10T03:34:56.095462+0000","last_fullsized":"2024-05-10T03:34:56.095462+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:52.993476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065042+0000","last_change":"2024-05-10T03:34:54.512402+0000","last_active":"2024-05-10T03:34:57.065042+0000","last_peered":"2024-05-10T03:34:57.065042+0000","last_clean":"2024-05-10T03:34:57.065042+0000","last_became_active":"2024-05-10T03:34:54.512118+0000","last_became_peered":"2024-05-10T03:34:54.512118+0000","last_unstale":"2024-05-10T03:34:57.065042+0000","last_undegraded":"2024-05-10T03:34:57.065042+0000","last_fullsized":"2024-05-10T03:34:57.065042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:51.978369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.110134+0000","last_change":"2024-05-10T03:34:55.108115+0000","last_active":"2024-05-10T03:34:55.110134+0000","last_peered":"2024-05-10T03:34:55.110134+0000","last_clean":"2024-05-10T03:34:55.110134+0000","last_became_active":"2024-05-10T03:34:55.105381+0000","last_became_peered":"2024-05-10T03:34:55.105381+0000","last_unstale":"2024-05-10T03:34:55.110134+0000","last_undegraded":"2024-05-10T03:34:55.110134+0000","last_fullsized":"2024-05-10T03:34:55.110134+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:02.986812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080355+0000","last_change":"2024-05-10T03:34:54.520988+0000","last_active":"2024-05-10T03:34:55.080355+0000","last_peered":"2024-05-10T03:34:55.080355+0000","last_clean":"2024-05-10T03:34:55.080355+0000","last_became_active":"2024-05-10T03:34:54.520413+0000","last_became_peered":"2024-05-10T03:34:54.520413+0000","last_unstale":"2024-05-10T03:34:55.080355+0000","last_undegraded":"2024-05-10T03:34:55.080355+0000","last_fullsized":"2024-05-10T03:34:55.080355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:03.620248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088681+0000","last_change":"2024-05-10T03:34:55.101871+0000","last_active":"2024-05-10T03:34:56.088681+0000","last_peered":"2024-05-10T03:34:56.088681+0000","last_clean":"2024-05-10T03:34:56.088681+0000","last_became_active":"2024-05-10T03:34:55.101237+0000","last_became_peered":"2024-05-10T03:34:55.101237+0000","last_unstale":"2024-05-10T03:34:56.088681+0000","last_undegraded":"2024-05-10T03:34:56.088681+0000","last_fullsized":"2024-05-10T03:34:56.088681+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:35:24.704232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088527+0000","last_change":"2024-05-10T03:34:54.512737+0000","last_active":"2024-05-10T03:34:56.088527+0000","last_peered":"2024-05-10T03:34:56.088527+0000","last_clean":"2024-05-10T03:34:56.088527+0000","last_became_active":"2024-05-10T03:34:54.512342+0000","last_became_peered":"2024-05-10T03:34:54.512342+0000","last_unstale":"2024-05-10T03:34:56.088527+0000","last_undegraded":"2024-05-10T03:34:56.088527+0000","last_fullsized":"2024-05-10T03:34:56.088527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:08.142653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.086401+0000","last_change":"2024-05-10T03:34:55.084825+0000","last_active":"2024-05-10T03:34:55.086401+0000","last_peered":"2024-05-10T03:34:55.086401+0000","last_clean":"2024-05-10T03:34:55.086401+0000","last_became_active":"2024-05-10T03:34:55.084275+0000","last_became_peered":"2024-05-10T03:34:55.084275+0000","last_unstale":"2024-05-10T03:34:55.086401+0000","last_undegraded":"2024-05-10T03:34:55.086401+0000","last_fullsized":"2024-05-10T03:34:55.086401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:58.857065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094953+0000","last_change":"2024-05-10T03:34:54.524379+0000","last_active":"2024-05-10T03:34:56.094953+0000","last_peered":"2024-05-10T03:34:56.094953+0000","last_clean":"2024-05-10T03:34:56.094953+0000","last_became_active":"2024-05-10T03:34:54.524027+0000","last_became_peered":"2024-05-10T03:34:54.524027+0000","last_unstale":"2024-05-10T03:34:56.094953+0000","last_undegraded":"2024-05-10T03:34:56.094953+0000","last_fullsized":"2024-05-10T03:34:56.094953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:18.138740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095377+0000","last_change":"2024-05-10T03:34:55.093363+0000","last_active":"2024-05-10T03:34:56.095377+0000","last_peered":"2024-05-10T03:34:56.095377+0000","last_clean":"2024-05-10T03:34:56.095377+0000","last_became_active":"2024-05-10T03:34:55.090952+0000","last_became_peered":"2024-05-10T03:34:55.090952+0000","last_unstale":"2024-05-10T03:34:56.095377+0000","last_undegraded":"2024-05-10T03:34:56.095377+0000","last_fullsized":"2024-05-10T03:34:56.095377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:07.919776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095906+0000","last_change":"2024-05-10T03:34:54.518488+0000","last_active":"2024-05-10T03:34:56.095906+0000","last_peered":"2024-05-10T03:34:56.095906+0000","last_clean":"2024-05-10T03:34:56.095906+0000","last_became_active":"2024-05-10T03:34:54.518087+0000","last_became_peered":"2024-05-10T03:34:54.518087+0000","last_unstale":"2024-05-10T03:34:56.095906+0000","last_undegraded":"2024-05-10T03:34:56.095906+0000","last_fullsized":"2024-05-10T03:34:56.095906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:48.537060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109891+0000","last_change":"2024-05-10T03:34:55.107994+0000","last_active":"2024-05-10T03:34:55.109891+0000","last_peered":"2024-05-10T03:34:55.109891+0000","last_clean":"2024-05-10T03:34:55.109891+0000","last_became_active":"2024-05-10T03:34:55.106882+0000","last_became_peered":"2024-05-10T03:34:55.106882+0000","last_unstale":"2024-05-10T03:34:55.109891+0000","last_undegraded":"2024-05-10T03:34:55.109891+0000","last_fullsized":"2024-05-10T03:34:55.109891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:22.811888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088257+0000","last_change":"2024-05-10T03:34:54.521020+0000","last_active":"2024-05-10T03:34:56.088257+0000","last_peered":"2024-05-10T03:34:56.088257+0000","last_clean":"2024-05-10T03:34:56.088257+0000","last_became_active":"2024-05-10T03:34:54.520389+0000","last_became_peered":"2024-05-10T03:34:54.520389+0000","last_unstale":"2024-05-10T03:34:56.088257+0000","last_undegraded":"2024-05-10T03:34:56.088257+0000","last_fullsized":"2024-05-10T03:34:56.088257+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:03:25.055186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088091+0000","last_change":"2024-05-10T03:34:55.079860+0000","last_active":"2024-05-10T03:34:56.088091+0000","last_peered":"2024-05-10T03:34:56.088091+0000","last_clean":"2024-05-10T03:34:56.088091+0000","last_became_active":"2024-05-10T03:34:55.079558+0000","last_became_peered":"2024-05-10T03:34:55.079558+0000","last_unstale":"2024-05-10T03:34:56.088091+0000","last_undegraded":"2024-05-10T03:34:56.088091+0000","last_fullsized":"2024-05-10T03:34:56.088091+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:50.663986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109413+0000","last_change":"2024-05-10T03:34:54.515985+0000","last_active":"2024-05-10T03:34:55.109413+0000","last_peered":"2024-05-10T03:34:55.109413+0000","last_clean":"2024-05-10T03:34:55.109413+0000","last_became_active":"2024-05-10T03:34:54.515618+0000","last_became_peered":"2024-05-10T03:34:54.515618+0000","last_unstale":"2024-05-10T03:34:55.109413+0000","last_undegraded":"2024-05-10T03:34:55.109413+0000","last_fullsized":"2024-05-10T03:34:55.109413+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:37.732608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088761+0000","last_change":"2024-05-10T03:34:55.087774+0000","last_active":"2024-05-10T03:34:56.088761+0000","last_peered":"2024-05-10T03:34:56.088761+0000","last_clean":"2024-05-10T03:34:56.088761+0000","last_became_active":"2024-05-10T03:34:55.087445+0000","last_became_peered":"2024-05-10T03:34:55.087445+0000","last_unstale":"2024-05-10T03:34:56.088761+0000","last_undegraded":"2024-05-10T03:34:56.088761+0000","last_fullsized":"2024-05-10T03:34:56.088761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:57:02.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080048+0000","last_change":"2024-05-10T03:34:54.519580+0000","last_active":"2024-05-10T03:34:55.080048+0000","last_peered":"2024-05-10T03:34:55.080048+0000","last_clean":"2024-05-10T03:34:55.080048+0000","last_became_active":"2024-05-10T03:34:54.519247+0000","last_became_peered":"2024-05-10T03:34:54.519247+0000","last_unstale":"2024-05-10T03:34:55.080048+0000","last_undegraded":"2024-05-10T03:34:55.080048+0000","last_fullsized":"2024-05-10T03:34:55.080048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:41.100149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065141+0000","last_change":"2024-05-10T03:34:55.094082+0000","last_active":"2024-05-10T03:34:57.065141+0000","last_peered":"2024-05-10T03:34:57.065141+0000","last_clean":"2024-05-10T03:34:57.065141+0000","last_became_active":"2024-05-10T03:34:55.093750+0000","last_became_peered":"2024-05-10T03:34:55.093750+0000","last_unstale":"2024-05-10T03:34:57.065141+0000","last_undegraded":"2024-05-10T03:34:57.065141+0000","last_fullsized":"2024-05-10T03:34:57.065141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:44.635470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093981+0000","last_change":"2024-05-10T03:34:54.517823+0000","last_active":"2024-05-10T03:34:56.093981+0000","last_peered":"2024-05-10T03:34:56.093981+0000","last_clean":"2024-05-10T03:34:56.093981+0000","last_became_active":"2024-05-10T03:34:54.517155+0000","last_became_peered":"2024-05-10T03:34:54.517155+0000","last_unstale":"2024-05-10T03:34:56.093981+0000","last_undegraded":"2024-05-10T03:34:56.093981+0000","last_fullsized":"2024-05-10T03:34:56.093981+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:46.268806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101265+0000","last_change":"2024-05-10T03:34:55.094321+0000","last_active":"2024-05-10T03:34:55.101265+0000","last_peered":"2024-05-10T03:34:55.101265+0000","last_clean":"2024-05-10T03:34:55.101265+0000","last_became_active":"2024-05-10T03:34:55.094027+0000","last_became_peered":"2024-05-10T03:34:55.094027+0000","last_unstale":"2024-05-10T03:34:55.101265+0000","last_undegraded":"2024-05-10T03:34:55.101265+0000","last_fullsized":"2024-05-10T03:34:55.101265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:31:07.759872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.081118+0000","last_change":"2024-05-10T03:34:54.515195+0000","last_active":"2024-05-10T03:34:55.081118+0000","last_peered":"2024-05-10T03:34:55.081118+0000","last_clean":"2024-05-10T03:34:55.081118+0000","last_became_active":"2024-05-10T03:34:54.514874+0000","last_became_peered":"2024-05-10T03:34:54.514874+0000","last_unstale":"2024-05-10T03:34:55.081118+0000","last_undegraded":"2024-05-10T03:34:55.081118+0000","last_fullsized":"2024-05-10T03:34:55.081118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:47:13.749165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087941+0000","last_change":"2024-05-10T03:34:55.078719+0000","last_active":"2024-05-10T03:34:56.087941+0000","last_peered":"2024-05-10T03:34:56.087941+0000","last_clean":"2024-05-10T03:34:56.087941+0000","last_became_active":"2024-05-10T03:34:55.077693+0000","last_became_peered":"2024-05-10T03:34:55.077693+0000","last_unstale":"2024-05-10T03:34:56.087941+0000","last_undegraded":"2024-05-10T03:34:56.087941+0000","last_fullsized":"2024-05-10T03:34:56.087941+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:25.507044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.106495+0000","last_change":"2024-05-10T03:34:54.519707+0000","last_active":"2024-05-10T03:34:55.106495+0000","last_peered":"2024-05-10T03:34:55.106495+0000","last_clean":"2024-05-10T03:34:55.106495+0000","last_became_active":"2024-05-10T03:34:54.516925+0000","last_became_peered":"2024-05-10T03:34:54.516925+0000","last_unstale":"2024-05-10T03:34:55.106495+0000","last_undegraded":"2024-05-10T03:34:55.106495+0000","last_fullsized":"2024-05-10T03:34:55.106495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:10:06.945240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088824+0000","last_change":"2024-05-10T03:34:55.106047+0000","last_active":"2024-05-10T03:34:56.088824+0000","last_peered":"2024-05-10T03:34:56.088824+0000","last_clean":"2024-05-10T03:34:56.088824+0000","last_became_active":"2024-05-10T03:34:55.105565+0000","last_became_peered":"2024-05-10T03:34:55.105565+0000","last_unstale":"2024-05-10T03:34:56.088824+0000","last_undegraded":"2024-05-10T03:34:56.088824+0000","last_fullsized":"2024-05-10T03:34:56.088824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:10.408226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089033+0000","last_change":"2024-05-10T03:34:54.507611+0000","last_active":"2024-05-10T03:34:56.089033+0000","last_peered":"2024-05-10T03:34:56.089033+0000","last_clean":"2024-05-10T03:34:56.089033+0000","last_became_active":"2024-05-10T03:34:54.507259+0000","last_became_peered":"2024-05-10T03:34:54.507259+0000","last_unstale":"2024-05-10T03:34:56.089033+0000","last_undegraded":"2024-05-10T03:34:56.089033+0000","last_fullsized":"2024-05-10T03:34:56.089033+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:21:27.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064742+0000","last_change":"2024-05-10T03:34:55.093587+0000","last_active":"2024-05-10T03:34:57.064742+0000","last_peered":"2024-05-10T03:34:57.064742+0000","last_clean":"2024-05-10T03:34:57.064742+0000","last_became_active":"2024-05-10T03:34:55.093276+0000","last_became_peered":"2024-05-10T03:34:55.093276+0000","last_unstale":"2024-05-10T03:34:57.064742+0000","last_undegraded":"2024-05-10T03:34:57.064742+0000","last_fullsized":"2024-05-10T03:34:57.064742+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:50.553885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095716+0000","last_change":"2024-05-10T03:34:54.521688+0000","last_active":"2024-05-10T03:34:56.095716+0000","last_peered":"2024-05-10T03:34:56.095716+0000","last_clean":"2024-05-10T03:34:56.095716+0000","last_became_active":"2024-05-10T03:34:54.521319+0000","last_became_peered":"2024-05-10T03:34:54.521319+0000","last_unstale":"2024-05-10T03:34:56.095716+0000","last_undegraded":"2024-05-10T03:34:56.095716+0000","last_fullsized":"2024-05-10T03:34:56.095716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:58:13.741483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089086+0000","last_change":"2024-05-10T03:34:55.096300+0000","last_active":"2024-05-10T03:34:56.089086+0000","last_peered":"2024-05-10T03:34:56.089086+0000","last_clean":"2024-05-10T03:34:56.089086+0000","last_became_active":"2024-05-10T03:34:55.095944+0000","last_became_peered":"2024-05-10T03:34:55.095944+0000","last_unstale":"2024-05-10T03:34:56.089086+0000","last_undegraded":"2024-05-10T03:34:56.089086+0000","last_fullsized":"2024-05-10T03:34:56.089086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:04.192750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.085655+0000","last_change":"2024-05-10T03:34:54.518313+0000","last_active":"2024-05-10T03:34:55.085655+0000","last_peered":"2024-05-10T03:34:55.085655+0000","last_clean":"2024-05-10T03:34:55.085655+0000","last_became_active":"2024-05-10T03:34:54.517783+0000","last_became_peered":"2024-05-10T03:34:54.517783+0000","last_unstale":"2024-05-10T03:34:55.085655+0000","last_undegraded":"2024-05-10T03:34:55.085655+0000","last_fullsized":"2024-05-10T03:34:55.085655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:48.486514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.100776+0000","last_change":"2024-05-10T03:34:55.093645+0000","last_active":"2024-05-10T03:34:55.100776+0000","last_peered":"2024-05-10T03:34:55.100776+0000","last_clean":"2024-05-10T03:34:55.100776+0000","last_became_active":"2024-05-10T03:34:55.092181+0000","last_became_peered":"2024-05-10T03:34:55.092181+0000","last_unstale":"2024-05-10T03:34:55.100776+0000","last_undegraded":"2024-05-10T03:34:55.100776+0000","last_fullsized":"2024-05-10T03:34:55.100776+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:51:16.204050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089056+0000","last_change":"2024-05-10T03:34:54.521008+0000","last_active":"2024-05-10T03:34:56.089056+0000","last_peered":"2024-05-10T03:34:56.089056+0000","last_clean":"2024-05-10T03:34:56.089056+0000","last_became_active":"2024-05-10T03:34:54.520611+0000","last_became_peered":"2024-05-10T03:34:54.520611+0000","last_unstale":"2024-05-10T03:34:56.089056+0000","last_undegraded":"2024-05-10T03:34:56.089056+0000","last_fullsized":"2024-05-10T03:34:56.089056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:06.904669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.104771+0000","last_change":"2024-05-10T03:34:55.094383+0000","last_active":"2024-05-10T03:34:55.104771+0000","last_peered":"2024-05-10T03:34:55.104771+0000","last_clean":"2024-05-10T03:34:55.104771+0000","last_became_active":"2024-05-10T03:34:55.094081+0000","last_became_peered":"2024-05-10T03:34:55.094081+0000","last_unstale":"2024-05-10T03:34:55.104771+0000","last_undegraded":"2024-05-10T03:34:55.104771+0000","last_fullsized":"2024-05-10T03:34:55.104771+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:23.035355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080173+0000","last_change":"2024-05-10T03:34:54.523075+0000","last_active":"2024-05-10T03:34:55.080173+0000","last_peered":"2024-05-10T03:34:55.080173+0000","last_clean":"2024-05-10T03:34:55.080173+0000","last_became_active":"2024-05-10T03:34:54.522591+0000","last_became_peered":"2024-05-10T03:34:54.522591+0000","last_unstale":"2024-05-10T03:34:55.080173+0000","last_undegraded":"2024-05-10T03:34:55.080173+0000","last_fullsized":"2024-05-10T03:34:55.080173+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:47:50.004046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096646+0000","last_change":"2024-05-10T03:34:55.111983+0000","last_active":"2024-05-10T03:34:56.096646+0000","last_peered":"2024-05-10T03:34:56.096646+0000","last_clean":"2024-05-10T03:34:56.096646+0000","last_became_active":"2024-05-10T03:34:55.111652+0000","last_became_peered":"2024-05-10T03:34:55.111652+0000","last_unstale":"2024-05-10T03:34:56.096646+0000","last_undegraded":"2024-05-10T03:34:56.096646+0000","last_fullsized":"2024-05-10T03:34:56.096646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:02.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094046+0000","last_change":"2024-05-10T03:34:54.520030+0000","last_active":"2024-05-10T03:34:56.094046+0000","last_peered":"2024-05-10T03:34:56.094046+0000","last_clean":"2024-05-10T03:34:56.094046+0000","last_became_active":"2024-05-10T03:34:54.519457+0000","last_became_peered":"2024-05-10T03:34:54.519457+0000","last_unstale":"2024-05-10T03:34:56.094046+0000","last_undegraded":"2024-05-10T03:34:56.094046+0000","last_fullsized":"2024-05-10T03:34:56.094046+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:48.486514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089404+0000","last_change":"2024-05-10T03:34:55.111533+0000","last_active":"2024-05-10T03:34:56.089404+0000","last_peered":"2024-05-10T03:34:56.089404+0000","last_clean":"2024-05-10T03:34:56.089404+0000","last_became_active":"2024-05-10T03:34:55.111210+0000","last_became_peered":"2024-05-10T03:34:55.111210+0000","last_unstale":"2024-05-10T03:34:56.089404+0000","last_undegraded":"2024-05-10T03:34:56.089404+0000","last_fullsized":"2024-05-10T03:34:56.089404+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:02.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.081224+0000","last_change":"2024-05-10T03:34:54.522396+0000","last_active":"2024-05-10T03:34:55.081224+0000","last_peered":"2024-05-10T03:34:55.081224+0000","last_clean":"2024-05-10T03:34:55.081224+0000","last_became_active":"2024-05-10T03:34:54.520868+0000","last_became_peered":"2024-05-10T03:34:54.520868+0000","last_unstale":"2024-05-10T03:34:55.081224+0000","last_undegraded":"2024-05-10T03:34:55.081224+0000","last_fullsized":"2024-05-10T03:34:55.081224+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:29.250596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087887+0000","last_change":"2024-05-10T03:34:55.088334+0000","last_active":"2024-05-10T03:34:56.087887+0000","last_peered":"2024-05-10T03:34:56.087887+0000","last_clean":"2024-05-10T03:34:56.087887+0000","last_became_active":"2024-05-10T03:34:55.087593+0000","last_became_peered":"2024-05-10T03:34:55.087593+0000","last_unstale":"2024-05-10T03:34:56.087887+0000","last_undegraded":"2024-05-10T03:34:56.087887+0000","last_fullsized":"2024-05-10T03:34:56.087887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:07.919776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109477+0000","last_change":"2024-05-10T03:34:54.522959+0000","last_active":"2024-05-10T03:34:55.109477+0000","last_peered":"2024-05-10T03:34:55.109477+0000","last_clean":"2024-05-10T03:34:55.109477+0000","last_became_active":"2024-05-10T03:34:54.521219+0000","last_became_peered":"2024-05-10T03:34:54.521219+0000","last_unstale":"2024-05-10T03:34:55.109477+0000","last_undegraded":"2024-05-10T03:34:55.109477+0000","last_fullsized":"2024-05-10T03:34:55.109477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:02:03.301656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":235,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.091825+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:34:55.091825+0000","last_peered":"2024-05-10T03:34:55.091825+0000","last_clean":"2024-05-10T03:34:55.091825+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:34:55.091825+0000","last_undegraded":"2024-05-10T03:34:55.091825+0000","last_fullsized":"2024-05-10T03:34:55.091825+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1892352,"data_stored":1177357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2470000000000001,"15min":1.2350000000000001},"min":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.77,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3420000000000001,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.151,"15min":1.145},"min":{"1min":0.71399999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.671,"5min":1.853,"15min":1.853},"last":1.26},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2470000000000001,"15min":1.2669999999999999},"min":{"1min":0.86299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.2030000000000001}]},{"osd":2,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2909999999999999,"15min":1.359},"min":{"1min":0.80400000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8560000000000001,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3440000000000001,"15min":1.417},"min":{"1min":0.88400000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.4649999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.383},"min":{"1min":0.93500000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.9750000000000001,"5min":2.222,"15min":2.222},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.341},"min":{"1min":0.95299999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.649,"5min":2.367,"15min":2.367},"last":1.587}]},{"osd":4,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.419,"15min":1.46},"min":{"1min":0.89300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.079},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.504,"15min":1.502},"min":{"1min":0.70499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.851,"5min":2.323,"15min":2.323},"last":1.3380000000000001}]},{"osd":5,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5580000000000001,"15min":1.52},"min":{"1min":0.94299999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.429},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.677,"15min":1.6919999999999999},"min":{"1min":1.0129999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5109999999999999}]},{"osd":6,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.6839999999999999,"15min":1.7150000000000001},"min":{"1min":1.0660000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.403},{"interface":"front","average":{"1min":1.599,"5min":1.6539999999999999,"15min":1.641},"min":{"1min":1.236,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1000000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.48}]}]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1310720,"data_stored":588997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.145,"15min":1.095},"min":{"1min":0.75900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6859999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.321},{"interface":"front","average":{"1min":1.075,"5min":1.228,"15min":1.3180000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.5760000000000001}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.276,"15min":1.306},"min":{"1min":0.874,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3620000000000001,"5min":3.113,"15min":3.113},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.341,"15min":1.4530000000000001},"min":{"1min":0.90600000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.821,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.91500000000000004}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.353,"15min":1.268},"min":{"1min":0.92000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7509999999999999,"5min":2.02,"15min":2.02},"last":1.181},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.371,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.8999999999999999,"5min":3.569,"15min":3.569},"last":0.995}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.446,"15min":1.5800000000000001},"min":{"1min":1.0580000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.4260000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.363,"15min":1.323},"min":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2130000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.4299999999999999},"min":{"1min":1.0529999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.073,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.548},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5229999999999999,"15min":1.571},"min":{"1min":1.127,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.704}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.482,"15min":1.4179999999999999},"min":{"1min":1.1359999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.5179999999999998,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.111,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.6499999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.6659999999999999,"15min":1.6719999999999999},"min":{"1min":1.2170000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.867},{"interface":"front","average":{"1min":1.7470000000000001,"5min":1.681,"15min":1.665},"min":{"1min":1.262,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.2649999999999997,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":1.502}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1290240,"data_stored":586829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.109,"15min":1.077},"min":{"1min":0.73699999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.173,"15min":1.1930000000000001},"min":{"1min":0.75700000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.25}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.69699999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5580000000000001,"5min":1.855,"15min":1.855},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.2090000000000001,"15min":1.2210000000000001},"min":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.76,"5min":1.954,"15min":1.954},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.80100000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.706,"5min":2.359,"15min":2.359},"last":1.127},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3149999999999999,"15min":1.333},"min":{"1min":0.90100000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.875,"5min":1.913,"15min":1.913},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3500000000000001,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7869999999999999,"5min":1.881,"15min":1.881},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.345,"5min":1.3700000000000001,"15min":1.3089999999999999},"min":{"1min":0.92800000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8260000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.379,"15min":1.407},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8240000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.488},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.569,"15min":1.595},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9430000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.93899999999999995}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.53,"15min":1.5169999999999999},"min":{"1min":0.78500000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.931,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.421},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.548,"15min":1.5509999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.98,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.7509999999999999,"15min":1.796},"min":{"1min":0.94799999999999995,"5min":0.753,"15min":0.753},"max":{"1min":2.3420000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.383},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.617},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.54}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1298432,"data_stored":586921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.141,"15min":1.167},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5109999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.131,"15min":1.137},"min":{"1min":0.747,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3859999999999999,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.97399999999999998}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1930000000000001,"15min":1.165},"min":{"1min":0.86699999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6910000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.679},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.92500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6060000000000001,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2390000000000001,"15min":1.27},"min":{"1min":0.81399999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3620000000000001,"15min":1.395},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.764,"5min":2.29,"15min":2.29},"last":1.54}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3580000000000001,"15min":1.4430000000000001},"min":{"1min":1.028,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7829999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.534,"15min":1.5700000000000001},"min":{"1min":1.054,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.893,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.496}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4830000000000001,"15min":1.4990000000000001},"min":{"1min":1.131,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.944,"5min":2.512,"15min":2.512},"last":1.417},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4359999999999999,"15min":1.5289999999999999},"min":{"1min":0.83099999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.958,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.452}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.542,"15min":1.5780000000000001},"min":{"1min":1.0409999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0459999999999998,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.266},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.661,"15min":1.756},"min":{"1min":1.266,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9570000000000001,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6100000000000001,"15min":1.6759999999999999},"min":{"1min":1.071,"5min":0.495,"15min":0.495},"max":{"1min":2.1349999999999998,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.71,"15min":1.734},"min":{"1min":1.1839999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1499999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1282048,"data_stored":580791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92100000000000004,"15min":0.94899999999999995},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.4199999999999999,"5min":1.754,"15min":1.754},"last":2.0129999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3520000000000001,"5min":1.708,"15min":1.708},"last":1.05}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.94499999999999995,"15min":0.96599999999999997},"min":{"1min":0.51500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5620000000000001,"5min":1.603,"15min":1.603},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.1639999999999999,"15min":1.2290000000000001},"min":{"1min":0.57699999999999996,"5min":0.377,"15min":0.377},"max":{"1min":1.5429999999999999,"5min":1.964,"15min":1.964},"last":2.149}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.151,"15min":1.175},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.651,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2450000000000001,"15min":1.337},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7050000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.649}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3520000000000001,"15min":1.3700000000000001},"min":{"1min":0.82599999999999996,"5min":0.626,"15min":0.626},"max":{"1min":2.0910000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4359999999999999,"15min":1.3939999999999999},"min":{"1min":0.94699999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0590000000000002,"5min":2.464,"15min":2.464},"last":1.369}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.546,"15min":1.6519999999999999},"min":{"1min":0.93100000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.2200000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.45},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.454,"15min":1.4810000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0470000000000002,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.8149999999999999}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.355,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":2.1190000000000002},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6879999999999999,"15min":1.7749999999999999},"min":{"1min":1.0509999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.1339999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.8759999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.5229999999999999,"15min":1.577},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.913,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.625,"5min":1.5980000000000001,"15min":1.6439999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.29,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.952}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1282048,"data_stored":580243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89600000000000002,"15min":0.96999999999999997},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4259999999999999,"5min":1.772,"15min":1.772},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.28,"5min":1.746,"15min":1.746},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.1259999999999999,"15min":1.1719999999999999},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5549999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.93200000000000005}]},{"osd":3,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0549999999999999,"15min":1.107},"min":{"1min":0.58799999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.375,"5min":2.169,"15min":2.169},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2050000000000001,"15min":1.248},"min":{"1min":0.66700000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9910000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4319999999999999,"15min":1.516},"min":{"1min":0.97399999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9159999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.577},{"interface":"front","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.5089999999999999},"min":{"1min":0.79400000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.0489999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.9490000000000001}]},{"osd":5,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4199999999999999,"15min":1.4690000000000001},"min":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.74,"5min":2.508,"15min":2.508},"last":1.605},{"interface":"front","average":{"1min":1.52,"5min":1.5680000000000001,"15min":1.6100000000000001},"min":{"1min":0.91500000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.9550000000000001,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.448}]},{"osd":6,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5,"15min":1.5409999999999999},"min":{"1min":1.0589999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1989999999999998,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.827},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5129999999999999,"15min":1.575},"min":{"1min":1.042,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8500000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.544}]},{"osd":7,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.579,"15min":1.639},"min":{"1min":1.0620000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.3149999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.585,"15min":1.6539999999999999},"min":{"1min":1.119,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.829,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.7470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1282048,"data_stored":580243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.499,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.343,"5min":1.607,"15min":1.607},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.995,"15min":1.0680000000000001},"min":{"1min":0.71899999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.996,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.284}]},{"osd":2,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0029999999999999},"min":{"1min":0.58199999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.4419999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.228},{"interface":"front","average":{"1min":1.161,"5min":1.135,"15min":1.179},"min":{"1min":0.752,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5800000000000001,"5min":2.444,"15min":2.444},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.242,"15min":1.3080000000000001},"min":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8280000000000001,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.5},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.278},"min":{"1min":0.92800000000000005,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.5349999999999999}]},{"osd":4,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.4690000000000001,"15min":1.512},"min":{"1min":1.2110000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.153,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.4710000000000001,"15min":1.5649999999999999},"min":{"1min":1,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.736,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4339999999999999,"15min":1.556},"min":{"1min":1.032,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.728,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.522,"15min":1.6140000000000001},"min":{"1min":1.0660000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.3290000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.095}]},{"osd":6,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5680000000000001,"15min":1.6930000000000001},"min":{"1min":1.292,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.9730000000000001,"5min":3.48,"15min":3.48},"last":1.677},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5760000000000001,"15min":1.702},"min":{"1min":1.157,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.9179999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.028}]},{"osd":7,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6419999999999999,"15min":1.7190000000000001},"min":{"1min":1.0029999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.202,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.631,"15min":1.7529999999999999},"min":{"1min":1.2030000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.2269999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.607}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1900544,"data_stored":1178851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.89500000000000002,"15min":1.0009999999999999},"min":{"1min":0.45600000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7270000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97599999999999998,"15min":1.1499999999999999},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.405,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.3799999999999999}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.075,"15min":1.1950000000000001},"min":{"1min":0.58499999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.6379999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.0609999999999999,"15min":1.1870000000000001},"min":{"1min":0.61099999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.8400000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.333}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0840000000000001},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4339999999999999,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.2210000000000001,"15min":1.3049999999999999},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.891,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.1759999999999999}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3859999999999999,"15min":1.5820000000000001},"min":{"1min":0.877,"5min":0.745,"15min":0.745},"max":{"1min":1.8620000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":2.2799999999999998},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3500000000000001,"15min":1.419},"min":{"1min":0.84199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6519999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.611}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3700000000000001,"15min":1.4550000000000001},"min":{"1min":0.90000000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7929999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4670000000000001,"15min":1.6000000000000001},"min":{"1min":1.1180000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.2120000000000002,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.772}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5469999999999999,"15min":1.627},"min":{"1min":1.04,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.3239999999999998,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":2.6339999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5349999999999999,"15min":1.7030000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.028,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.877}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5620000000000001,"15min":1.75},"min":{"1min":0.97799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.1070000000000002,"5min":3.149,"15min":3.149},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.51,"15min":1.5640000000000001},"min":{"1min":0.86899999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1509999999999998,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.4059999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:35:00.255 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:35:00.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.576+0000 7f1563200640 1 Processor -- start 2024-05-10T03:35:00.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.576+0000 7f1563200640 1 -- start start 2024-05-10T03:35:00.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.578+0000 7f1563200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.578+0000 7f1563200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15641618b0 0x7f156406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1563200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15640d25a0 con 0x7f15640b6680 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1563200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15640d2b20 con 0x7f15641618b0 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1563200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156415db20 con 0x7f15640b2170 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1562800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f15641618b0 0x7f156406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1561e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f15640b6680 0x7f15641611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39910/0 (socket says 172.21.15.55:39910) 2024-05-10T03:35:00.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1561e00640 1 -- 172.21.15.55:0/1946842446 learned_addr learned my addr 172.21.15.55:0/1946842446 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.579+0000 7f1561400640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.580+0000 7f1562800640 1 -- 172.21.15.55:0/1946842446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 msgr2=0x7f156415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.580+0000 7f1562800640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.580+0000 7f1562800640 1 -- 172.21.15.55:0/1946842446 >> v1:172.21.15.55:6789/0 conn(0x7f15640b6680 legacy=0x7f15641611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.580+0000 7f1560a00640 1 -- 172.21.15.55:0/1946842446 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670748718 0 0) 0x7f15640d25a0 con 0x7f15640b6680 2024-05-10T03:35:00.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.580+0000 7f1562800640 1 -- 172.21.15.55:0/1946842446 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640b28d0 con 0x7f15641618b0 2024-05-10T03:35:00.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.581+0000 7f1562800640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15641618b0 0x7f156406aab0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f155c0097a0 tx=0x7f155c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1b3b647b6cfcc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:00.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.581+0000 7f1560a00640 1 -- 172.21.15.55:0/1946842446 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c032440 con 0x7f15641618b0 2024-05-10T03:35:00.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.581+0000 7f1560a00640 1 -- 172.21.15.55:0/1946842446 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0325a0 con 0x7f15641618b0 2024-05-10T03:35:00.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.581+0000 7f1560a00640 1 -- 172.21.15.55:0/1946842446 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c032930 con 0x7f15641618b0 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 -- 172.21.15.55:0/1946842446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15641618b0 msgr2=0x7f156406aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15641618b0 0x7f156406aab0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f155c0097a0 tx=0x7f155c02a180 comp rx=0 tx=0).stop 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 -- 172.21.15.55:0/1946842446 shutdown_connections 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 --2- 172.21.15.55:0/1946842446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15641618b0 0x7f156406aab0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 -- 172.21.15.55:0/1946842446 >> 172.21.15.55:0/1946842446 conn(0x7f15640b5730 msgr2=0x7f15640d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 -- 172.21.15.55:0/1946842446 shutdown_connections 2024-05-10T03:35:00.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.582+0000 7f1563200640 1 -- 172.21.15.55:0/1946842446 wait complete. 2024-05-10T03:35:00.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.584+0000 7f1563200640 1 Processor -- start 2024-05-10T03:35:00.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.584+0000 7f1563200640 1 -- start start 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15641618b0 0x7f15640c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156415dec0 con 0x7f15640b6680 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156414eae0 con 0x7f15641618b0 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1563200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156415f620 con 0x7f15640b2170 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1561400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:00.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1561400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41282/0 (socket says 172.21.15.55:41282) 2024-05-10T03:35:00.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.586+0000 7f1561400640 1 -- 172.21.15.55:0/871836725 learned_addr learned my addr 172.21.15.55:0/871836725 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:00.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 -- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 msgr2=0x7f156414e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561e00640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 -- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15641618b0 msgr2=0x7f15640c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15641618b0 0x7f15640c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 -- 172.21.15.55:0/871836725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009400 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1561400640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f154c007e70 tx=0x7f154c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c014620 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c014780 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.587+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15640c7950 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.588+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c014b30 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.588+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15640c7ba0 con 0x7f15640b6680 2024-05-10T03:35:00.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.589+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1564067740 con 0x7f15640b6680 2024-05-10T03:35:00.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.590+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f154c014c90 con 0x7f15640b6680 2024-05-10T03:35:00.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.591+0000 7f155a000640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 0x7f154006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:00.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.591+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f154c089e80 con 0x7f15640b6680 2024-05-10T03:35:00.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.592+0000 7f1561e00640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 0x7f154006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:00.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.593+0000 7f1561e00640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 0x7f154006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1554000f60 tx=0x7f1554009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.596+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f154c054e70 con 0x7f15640b6680 2024-05-10T03:35:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.798+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f156406a590 con 0x7f154006ac80 2024-05-10T03:35:00.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.823+0000 7f155a000640 1 -- 172.21.15.55:0/871836725 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287468 (secure 0 0 0) 0x7f15540070d0 con 0x7f154006ac80 2024-05-10T03:35:00.831 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.829+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 msgr2=0x7f154006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.829+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 0x7f154006d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1554000f60 tx=0x7f1554009040 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.829+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 msgr2=0x7f1564160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.829+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f154c007e70 tx=0x7f154c00e4b0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 shutdown_connections 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f154006ac80 0x7f154006d190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15641618b0 0x7f15640c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15640b6680 0x7f1564160680 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 --2- 172.21.15.55:0/871836725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15640b2170 0x7f156414e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 >> 172.21.15.55:0/871836725 conn(0x7f15640b5730 msgr2=0x7f15640b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 shutdown_connections 2024-05-10T03:35:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:00.830+0000 7f1563200640 1 -- 172.21.15.55:0/871836725 wait complete. 2024-05-10T03:35:00.863 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-05-10T03:34:59.073115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260236,"kb_used_data":11268,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714484,"statfs":{"total":773094113280,"available":772827631616,"internally_reserved":0,"allocated":11538432,"data_stored":5860232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.602065"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088547+0000","last_change":"2024-05-10T03:34:55.077649+0000","last_active":"2024-05-10T03:34:56.088547+0000","last_peered":"2024-05-10T03:34:56.088547+0000","last_clean":"2024-05-10T03:34:56.088547+0000","last_became_active":"2024-05-10T03:34:55.077340+0000","last_became_peered":"2024-05-10T03:34:55.077340+0000","last_unstale":"2024-05-10T03:34:56.088547+0000","last_undegraded":"2024-05-10T03:34:56.088547+0000","last_fullsized":"2024-05-10T03:34:56.088547+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:06.604455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094711+0000","last_change":"2024-05-10T03:34:55.109920+0000","last_active":"2024-05-10T03:34:56.094711+0000","last_peered":"2024-05-10T03:34:56.094711+0000","last_clean":"2024-05-10T03:34:56.094711+0000","last_became_active":"2024-05-10T03:34:55.109571+0000","last_became_peered":"2024-05-10T03:34:55.109571+0000","last_unstale":"2024-05-10T03:34:56.094711+0000","last_undegraded":"2024-05-10T03:34:56.094711+0000","last_fullsized":"2024-05-10T03:34:56.094711+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:08.980669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088174+0000","last_change":"2024-05-10T03:34:55.105594+0000","last_active":"2024-05-10T03:34:56.088174+0000","last_peered":"2024-05-10T03:34:56.088174+0000","last_clean":"2024-05-10T03:34:56.088174+0000","last_became_active":"2024-05-10T03:34:55.105311+0000","last_became_peered":"2024-05-10T03:34:55.105311+0000","last_unstale":"2024-05-10T03:34:56.088174+0000","last_undegraded":"2024-05-10T03:34:56.088174+0000","last_fullsized":"2024-05-10T03:34:56.088174+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:27.539672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065085+0000","last_change":"2024-05-10T03:34:55.115698+0000","last_active":"2024-05-10T03:34:57.065085+0000","last_peered":"2024-05-10T03:34:57.065085+0000","last_clean":"2024-05-10T03:34:57.065085+0000","last_became_active":"2024-05-10T03:34:55.115364+0000","last_became_peered":"2024-05-10T03:34:55.115364+0000","last_unstale":"2024-05-10T03:34:57.065085+0000","last_undegraded":"2024-05-10T03:34:57.065085+0000","last_fullsized":"2024-05-10T03:34:57.065085+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:18.215176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.112336+0000","last_change":"2024-05-10T03:34:55.111718+0000","last_active":"2024-05-10T03:34:55.112336+0000","last_peered":"2024-05-10T03:34:55.112336+0000","last_clean":"2024-05-10T03:34:55.112336+0000","last_became_active":"2024-05-10T03:34:55.111397+0000","last_became_peered":"2024-05-10T03:34:55.111397+0000","last_unstale":"2024-05-10T03:34:55.112336+0000","last_undegraded":"2024-05-10T03:34:55.112336+0000","last_fullsized":"2024-05-10T03:34:55.112336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:09.157760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095633+0000","last_change":"2024-05-10T03:34:55.095876+0000","last_active":"2024-05-10T03:34:56.095633+0000","last_peered":"2024-05-10T03:34:56.095633+0000","last_clean":"2024-05-10T03:34:56.095633+0000","last_became_active":"2024-05-10T03:34:55.095414+0000","last_became_peered":"2024-05-10T03:34:55.095414+0000","last_unstale":"2024-05-10T03:34:56.095633+0000","last_undegraded":"2024-05-10T03:34:56.095633+0000","last_fullsized":"2024-05-10T03:34:56.095633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:26:45.839548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093709+0000","last_change":"2024-05-10T03:34:55.095724+0000","last_active":"2024-05-10T03:34:56.093709+0000","last_peered":"2024-05-10T03:34:56.093709+0000","last_clean":"2024-05-10T03:34:56.093709+0000","last_became_active":"2024-05-10T03:34:55.095020+0000","last_became_peered":"2024-05-10T03:34:55.095020+0000","last_unstale":"2024-05-10T03:34:56.093709+0000","last_undegraded":"2024-05-10T03:34:56.093709+0000","last_fullsized":"2024-05-10T03:34:56.093709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:04.192750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.103211+0000","last_change":"2024-05-10T03:34:55.094909+0000","last_active":"2024-05-10T03:34:55.103211+0000","last_peered":"2024-05-10T03:34:55.103211+0000","last_clean":"2024-05-10T03:34:55.103211+0000","last_became_active":"2024-05-10T03:34:55.092853+0000","last_became_peered":"2024-05-10T03:34:55.092853+0000","last_unstale":"2024-05-10T03:34:55.103211+0000","last_undegraded":"2024-05-10T03:34:55.103211+0000","last_fullsized":"2024-05-10T03:34:55.103211+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:33:41.983368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094661+0000","last_change":"2024-05-10T03:34:55.109194+0000","last_active":"2024-05-10T03:34:56.094661+0000","last_peered":"2024-05-10T03:34:56.094661+0000","last_clean":"2024-05-10T03:34:56.094661+0000","last_became_active":"2024-05-10T03:34:55.108820+0000","last_became_peered":"2024-05-10T03:34:55.108820+0000","last_unstale":"2024-05-10T03:34:56.094661+0000","last_undegraded":"2024-05-10T03:34:56.094661+0000","last_fullsized":"2024-05-10T03:34:56.094661+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:12.843880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101413+0000","last_change":"2024-05-10T03:34:55.095814+0000","last_active":"2024-05-10T03:34:55.101413+0000","last_peered":"2024-05-10T03:34:55.101413+0000","last_clean":"2024-05-10T03:34:55.101413+0000","last_became_active":"2024-05-10T03:34:55.095466+0000","last_became_peered":"2024-05-10T03:34:55.095466+0000","last_unstale":"2024-05-10T03:34:55.101413+0000","last_undegraded":"2024-05-10T03:34:55.101413+0000","last_fullsized":"2024-05-10T03:34:55.101413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:01.591436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089055+0000","last_change":"2024-05-10T03:34:55.104405+0000","last_active":"2024-05-10T03:34:56.089055+0000","last_peered":"2024-05-10T03:34:56.089055+0000","last_clean":"2024-05-10T03:34:56.089055+0000","last_became_active":"2024-05-10T03:34:55.104031+0000","last_became_peered":"2024-05-10T03:34:55.104031+0000","last_unstale":"2024-05-10T03:34:56.089055+0000","last_undegraded":"2024-05-10T03:34:56.089055+0000","last_fullsized":"2024-05-10T03:34:56.089055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:08.392940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089816+0000","last_change":"2024-05-10T03:34:55.112050+0000","last_active":"2024-05-10T03:34:56.089816+0000","last_peered":"2024-05-10T03:34:56.089816+0000","last_clean":"2024-05-10T03:34:56.089816+0000","last_became_active":"2024-05-10T03:34:55.111738+0000","last_became_peered":"2024-05-10T03:34:55.111738+0000","last_unstale":"2024-05-10T03:34:56.089816+0000","last_undegraded":"2024-05-10T03:34:56.089816+0000","last_fullsized":"2024-05-10T03:34:56.089816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:29.132456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064730+0000","last_change":"2024-05-10T03:34:55.100903+0000","last_active":"2024-05-10T03:34:57.064730+0000","last_peered":"2024-05-10T03:34:57.064730+0000","last_clean":"2024-05-10T03:34:57.064730+0000","last_became_active":"2024-05-10T03:34:55.100632+0000","last_became_peered":"2024-05-10T03:34:55.100632+0000","last_unstale":"2024-05-10T03:34:57.064730+0000","last_undegraded":"2024-05-10T03:34:57.064730+0000","last_fullsized":"2024-05-10T03:34:57.064730+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:07:32.167414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096101+0000","last_change":"2024-05-10T03:34:55.113735+0000","last_active":"2024-05-10T03:34:56.096101+0000","last_peered":"2024-05-10T03:34:56.096101+0000","last_clean":"2024-05-10T03:34:56.096101+0000","last_became_active":"2024-05-10T03:34:55.111309+0000","last_became_peered":"2024-05-10T03:34:55.111309+0000","last_unstale":"2024-05-10T03:34:56.096101+0000","last_undegraded":"2024-05-10T03:34:56.096101+0000","last_fullsized":"2024-05-10T03:34:56.096101+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:01:43.452496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095066+0000","last_change":"2024-05-10T03:34:55.105226+0000","last_active":"2024-05-10T03:34:56.095066+0000","last_peered":"2024-05-10T03:34:56.095066+0000","last_clean":"2024-05-10T03:34:56.095066+0000","last_became_active":"2024-05-10T03:34:55.104915+0000","last_became_peered":"2024-05-10T03:34:55.104915+0000","last_unstale":"2024-05-10T03:34:56.095066+0000","last_undegraded":"2024-05-10T03:34:56.095066+0000","last_fullsized":"2024-05-10T03:34:56.095066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:08.392940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095046+0000","last_change":"2024-05-10T03:34:55.107749+0000","last_active":"2024-05-10T03:34:56.095046+0000","last_peered":"2024-05-10T03:34:56.095046+0000","last_clean":"2024-05-10T03:34:56.095046+0000","last_became_active":"2024-05-10T03:34:55.107434+0000","last_became_peered":"2024-05-10T03:34:55.107434+0000","last_unstale":"2024-05-10T03:34:56.095046+0000","last_undegraded":"2024-05-10T03:34:56.095046+0000","last_fullsized":"2024-05-10T03:34:56.095046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:26:57.303197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088744+0000","last_change":"2024-05-10T03:34:55.102087+0000","last_active":"2024-05-10T03:34:56.088744+0000","last_peered":"2024-05-10T03:34:56.088744+0000","last_clean":"2024-05-10T03:34:56.088744+0000","last_became_active":"2024-05-10T03:34:55.101605+0000","last_became_peered":"2024-05-10T03:34:55.101605+0000","last_unstale":"2024-05-10T03:34:56.088744+0000","last_undegraded":"2024-05-10T03:34:56.088744+0000","last_fullsized":"2024-05-10T03:34:56.088744+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:46:02.980543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094863+0000","last_change":"2024-05-10T03:34:55.095622+0000","last_active":"2024-05-10T03:34:56.094863+0000","last_peered":"2024-05-10T03:34:56.094863+0000","last_clean":"2024-05-10T03:34:56.094863+0000","last_became_active":"2024-05-10T03:34:55.095170+0000","last_became_peered":"2024-05-10T03:34:55.095170+0000","last_unstale":"2024-05-10T03:34:56.094863+0000","last_undegraded":"2024-05-10T03:34:56.094863+0000","last_fullsized":"2024-05-10T03:34:56.094863+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:09.157760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088334+0000","last_change":"2024-05-10T03:34:55.100803+0000","last_active":"2024-05-10T03:34:56.088334+0000","last_peered":"2024-05-10T03:34:56.088334+0000","last_clean":"2024-05-10T03:34:56.088334+0000","last_became_active":"2024-05-10T03:34:55.100497+0000","last_became_peered":"2024-05-10T03:34:55.100497+0000","last_unstale":"2024-05-10T03:34:56.088334+0000","last_undegraded":"2024-05-10T03:34:56.088334+0000","last_fullsized":"2024-05-10T03:34:56.088334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:56:36.190853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.090078+0000","last_change":"2024-05-10T03:34:55.114691+0000","last_active":"2024-05-10T03:34:56.090078+0000","last_peered":"2024-05-10T03:34:56.090078+0000","last_clean":"2024-05-10T03:34:56.090078+0000","last_became_active":"2024-05-10T03:34:55.114357+0000","last_became_peered":"2024-05-10T03:34:55.114357+0000","last_unstale":"2024-05-10T03:34:56.090078+0000","last_undegraded":"2024-05-10T03:34:56.090078+0000","last_fullsized":"2024-05-10T03:34:56.090078+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:03.172489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109346+0000","last_change":"2024-05-10T03:34:55.105353+0000","last_active":"2024-05-10T03:34:55.109346+0000","last_peered":"2024-05-10T03:34:55.109346+0000","last_clean":"2024-05-10T03:34:55.109346+0000","last_became_active":"2024-05-10T03:34:55.105081+0000","last_became_peered":"2024-05-10T03:34:55.105081+0000","last_unstale":"2024-05-10T03:34:55.109346+0000","last_undegraded":"2024-05-10T03:34:55.109346+0000","last_fullsized":"2024-05-10T03:34:55.109346+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:49:41.797240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.105565+0000","last_change":"2024-05-10T03:34:55.100864+0000","last_active":"2024-05-10T03:34:55.105565+0000","last_peered":"2024-05-10T03:34:55.105565+0000","last_clean":"2024-05-10T03:34:55.105565+0000","last_became_active":"2024-05-10T03:34:55.100543+0000","last_became_peered":"2024-05-10T03:34:55.100543+0000","last_unstale":"2024-05-10T03:34:55.105565+0000","last_undegraded":"2024-05-10T03:34:55.105565+0000","last_fullsized":"2024-05-10T03:34:55.105565+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:43.865361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088226+0000","last_change":"2024-05-10T03:34:55.087411+0000","last_active":"2024-05-10T03:34:56.088226+0000","last_peered":"2024-05-10T03:34:56.088226+0000","last_clean":"2024-05-10T03:34:56.088226+0000","last_became_active":"2024-05-10T03:34:55.087123+0000","last_became_peered":"2024-05-10T03:34:55.087123+0000","last_unstale":"2024-05-10T03:34:56.088226+0000","last_undegraded":"2024-05-10T03:34:56.088226+0000","last_fullsized":"2024-05-10T03:34:56.088226+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:30:13.676468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.090058+0000","last_change":"2024-05-10T03:34:55.080587+0000","last_active":"2024-05-10T03:34:55.090058+0000","last_peered":"2024-05-10T03:34:55.090058+0000","last_clean":"2024-05-10T03:34:55.090058+0000","last_became_active":"2024-05-10T03:34:55.080271+0000","last_became_peered":"2024-05-10T03:34:55.080271+0000","last_unstale":"2024-05-10T03:34:55.090058+0000","last_undegraded":"2024-05-10T03:34:55.090058+0000","last_fullsized":"2024-05-10T03:34:55.090058+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:14:04.371561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089088+0000","last_change":"2024-05-10T03:34:55.104086+0000","last_active":"2024-05-10T03:34:56.089088+0000","last_peered":"2024-05-10T03:34:56.089088+0000","last_clean":"2024-05-10T03:34:56.089088+0000","last_became_active":"2024-05-10T03:34:55.103765+0000","last_became_peered":"2024-05-10T03:34:55.103765+0000","last_unstale":"2024-05-10T03:34:56.089088+0000","last_undegraded":"2024-05-10T03:34:56.089088+0000","last_fullsized":"2024-05-10T03:34:56.089088+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:30.486847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088109+0000","last_change":"2024-05-10T03:34:55.116121+0000","last_active":"2024-05-10T03:34:56.088109+0000","last_peered":"2024-05-10T03:34:56.088109+0000","last_clean":"2024-05-10T03:34:56.088109+0000","last_became_active":"2024-05-10T03:34:55.115759+0000","last_became_peered":"2024-05-10T03:34:55.115759+0000","last_unstale":"2024-05-10T03:34:56.088109+0000","last_undegraded":"2024-05-10T03:34:56.088109+0000","last_fullsized":"2024-05-10T03:34:56.088109+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:50.553885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089473+0000","last_change":"2024-05-10T03:34:55.111309+0000","last_active":"2024-05-10T03:34:56.089473+0000","last_peered":"2024-05-10T03:34:56.089473+0000","last_clean":"2024-05-10T03:34:56.089473+0000","last_became_active":"2024-05-10T03:34:55.110766+0000","last_became_peered":"2024-05-10T03:34:55.110766+0000","last_unstale":"2024-05-10T03:34:56.089473+0000","last_undegraded":"2024-05-10T03:34:56.089473+0000","last_fullsized":"2024-05-10T03:34:56.089473+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:52.993476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094038+0000","last_change":"2024-05-10T03:34:55.101911+0000","last_active":"2024-05-10T03:34:56.094038+0000","last_peered":"2024-05-10T03:34:56.094038+0000","last_clean":"2024-05-10T03:34:56.094038+0000","last_became_active":"2024-05-10T03:34:55.101597+0000","last_became_peered":"2024-05-10T03:34:55.101597+0000","last_unstale":"2024-05-10T03:34:56.094038+0000","last_undegraded":"2024-05-10T03:34:56.094038+0000","last_fullsized":"2024-05-10T03:34:56.094038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:01.591436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109554+0000","last_change":"2024-05-10T03:34:55.107224+0000","last_active":"2024-05-10T03:34:55.109554+0000","last_peered":"2024-05-10T03:34:55.109554+0000","last_clean":"2024-05-10T03:34:55.109554+0000","last_became_active":"2024-05-10T03:34:55.106951+0000","last_became_peered":"2024-05-10T03:34:55.106951+0000","last_unstale":"2024-05-10T03:34:55.109554+0000","last_undegraded":"2024-05-10T03:34:55.109554+0000","last_fullsized":"2024-05-10T03:34:55.109554+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:37:42.690277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088315+0000","last_change":"2024-05-10T03:34:55.110671+0000","last_active":"2024-05-10T03:34:56.088315+0000","last_peered":"2024-05-10T03:34:56.088315+0000","last_clean":"2024-05-10T03:34:56.088315+0000","last_became_active":"2024-05-10T03:34:55.110353+0000","last_became_peered":"2024-05-10T03:34:55.110353+0000","last_unstale":"2024-05-10T03:34:56.088315+0000","last_undegraded":"2024-05-10T03:34:56.088315+0000","last_fullsized":"2024-05-10T03:34:56.088315+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:59:38.486380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093759+0000","last_change":"2024-05-10T03:34:55.088540+0000","last_active":"2024-05-10T03:34:56.093759+0000","last_peered":"2024-05-10T03:34:56.093759+0000","last_clean":"2024-05-10T03:34:56.093759+0000","last_became_active":"2024-05-10T03:34:55.087418+0000","last_became_peered":"2024-05-10T03:34:55.087418+0000","last_unstale":"2024-05-10T03:34:56.093759+0000","last_undegraded":"2024-05-10T03:34:56.093759+0000","last_fullsized":"2024-05-10T03:34:56.093759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.114004+0000","last_change":"2024-05-10T03:34:55.113390+0000","last_active":"2024-05-10T03:34:55.114004+0000","last_peered":"2024-05-10T03:34:55.114004+0000","last_clean":"2024-05-10T03:34:55.114004+0000","last_became_active":"2024-05-10T03:34:55.112738+0000","last_became_peered":"2024-05-10T03:34:55.112738+0000","last_unstale":"2024-05-10T03:34:55.114004+0000","last_undegraded":"2024-05-10T03:34:55.114004+0000","last_fullsized":"2024-05-10T03:34:55.114004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:45:14.973937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088798+0000","last_change":"2024-05-10T03:34:55.096482+0000","last_active":"2024-05-10T03:34:56.088798+0000","last_peered":"2024-05-10T03:34:56.088798+0000","last_clean":"2024-05-10T03:34:56.088798+0000","last_became_active":"2024-05-10T03:34:55.094241+0000","last_became_peered":"2024-05-10T03:34:55.094241+0000","last_unstale":"2024-05-10T03:34:56.088798+0000","last_undegraded":"2024-05-10T03:34:56.088798+0000","last_fullsized":"2024-05-10T03:34:56.088798+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:21:46.679850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065314+0000","last_change":"2024-05-10T03:34:54.521590+0000","last_active":"2024-05-10T03:34:57.065314+0000","last_peered":"2024-05-10T03:34:57.065314+0000","last_clean":"2024-05-10T03:34:57.065314+0000","last_became_active":"2024-05-10T03:34:54.520440+0000","last_became_peered":"2024-05-10T03:34:54.520440+0000","last_unstale":"2024-05-10T03:34:57.065314+0000","last_undegraded":"2024-05-10T03:34:57.065314+0000","last_fullsized":"2024-05-10T03:34:57.065314+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:01.729350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101072+0000","last_change":"2024-05-10T03:34:55.093051+0000","last_active":"2024-05-10T03:34:55.101072+0000","last_peered":"2024-05-10T03:34:55.101072+0000","last_clean":"2024-05-10T03:34:55.101072+0000","last_became_active":"2024-05-10T03:34:55.091314+0000","last_became_peered":"2024-05-10T03:34:55.091314+0000","last_unstale":"2024-05-10T03:34:55.101072+0000","last_undegraded":"2024-05-10T03:34:55.101072+0000","last_fullsized":"2024-05-10T03:34:55.101072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:08:57.131341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.116965+0000","last_change":"2024-05-10T03:34:54.514610+0000","last_active":"2024-05-10T03:34:55.116965+0000","last_peered":"2024-05-10T03:34:55.116965+0000","last_clean":"2024-05-10T03:34:55.116965+0000","last_became_active":"2024-05-10T03:34:54.514274+0000","last_became_peered":"2024-05-10T03:34:54.514274+0000","last_unstale":"2024-05-10T03:34:55.116965+0000","last_undegraded":"2024-05-10T03:34:55.116965+0000","last_fullsized":"2024-05-10T03:34:55.116965+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:26:11.673874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088605+0000","last_change":"2024-05-10T03:34:55.113011+0000","last_active":"2024-05-10T03:34:56.088605+0000","last_peered":"2024-05-10T03:34:56.088605+0000","last_clean":"2024-05-10T03:34:56.088605+0000","last_became_active":"2024-05-10T03:34:55.110922+0000","last_became_peered":"2024-05-10T03:34:55.110922+0000","last_unstale":"2024-05-10T03:34:56.088605+0000","last_undegraded":"2024-05-10T03:34:56.088605+0000","last_fullsized":"2024-05-10T03:34:56.088605+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:17.592050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089792+0000","last_change":"2024-05-10T03:34:54.512678+0000","last_active":"2024-05-10T03:34:56.089792+0000","last_peered":"2024-05-10T03:34:56.089792+0000","last_clean":"2024-05-10T03:34:56.089792+0000","last_became_active":"2024-05-10T03:34:54.512278+0000","last_became_peered":"2024-05-10T03:34:54.512278+0000","last_unstale":"2024-05-10T03:34:56.089792+0000","last_undegraded":"2024-05-10T03:34:56.089792+0000","last_fullsized":"2024-05-10T03:34:56.089792+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:07.965562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089412+0000","last_change":"2024-05-10T03:34:55.117653+0000","last_active":"2024-05-10T03:34:56.089412+0000","last_peered":"2024-05-10T03:34:56.089412+0000","last_clean":"2024-05-10T03:34:56.089412+0000","last_became_active":"2024-05-10T03:34:55.117320+0000","last_became_peered":"2024-05-10T03:34:55.117320+0000","last_unstale":"2024-05-10T03:34:56.089412+0000","last_undegraded":"2024-05-10T03:34:56.089412+0000","last_fullsized":"2024-05-10T03:34:56.089412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:47:12.843880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109926+0000","last_change":"2024-05-10T03:34:54.518091+0000","last_active":"2024-05-10T03:34:55.109926+0000","last_peered":"2024-05-10T03:34:55.109926+0000","last_clean":"2024-05-10T03:34:55.109926+0000","last_became_active":"2024-05-10T03:34:54.517380+0000","last_became_peered":"2024-05-10T03:34:54.517380+0000","last_unstale":"2024-05-10T03:34:55.109926+0000","last_undegraded":"2024-05-10T03:34:55.109926+0000","last_fullsized":"2024-05-10T03:34:55.109926+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:18.138740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065247+0000","last_change":"2024-05-10T03:34:55.089287+0000","last_active":"2024-05-10T03:34:57.065247+0000","last_peered":"2024-05-10T03:34:57.065247+0000","last_clean":"2024-05-10T03:34:57.065247+0000","last_became_active":"2024-05-10T03:34:55.088901+0000","last_became_peered":"2024-05-10T03:34:55.088901+0000","last_unstale":"2024-05-10T03:34:57.065247+0000","last_undegraded":"2024-05-10T03:34:57.065247+0000","last_fullsized":"2024-05-10T03:34:57.065247+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:30.486847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096269+0000","last_change":"2024-05-10T03:34:54.520586+0000","last_active":"2024-05-10T03:34:56.096269+0000","last_peered":"2024-05-10T03:34:56.096269+0000","last_clean":"2024-05-10T03:34:56.096269+0000","last_became_active":"2024-05-10T03:34:54.520290+0000","last_became_peered":"2024-05-10T03:34:54.520290+0000","last_unstale":"2024-05-10T03:34:56.096269+0000","last_undegraded":"2024-05-10T03:34:56.096269+0000","last_fullsized":"2024-05-10T03:34:56.096269+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:07.965562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088645+0000","last_change":"2024-05-10T03:34:55.099462+0000","last_active":"2024-05-10T03:34:56.088645+0000","last_peered":"2024-05-10T03:34:56.088645+0000","last_clean":"2024-05-10T03:34:56.088645+0000","last_became_active":"2024-05-10T03:34:55.099150+0000","last_became_peered":"2024-05-10T03:34:55.099150+0000","last_unstale":"2024-05-10T03:34:56.088645+0000","last_undegraded":"2024-05-10T03:34:56.088645+0000","last_fullsized":"2024-05-10T03:34:56.088645+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:30:55.384602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088202+0000","last_change":"2024-05-10T03:34:54.507498+0000","last_active":"2024-05-10T03:34:56.088202+0000","last_peered":"2024-05-10T03:34:56.088202+0000","last_clean":"2024-05-10T03:34:56.088202+0000","last_became_active":"2024-05-10T03:34:54.507088+0000","last_became_peered":"2024-05-10T03:34:54.507088+0000","last_unstale":"2024-05-10T03:34:56.088202+0000","last_undegraded":"2024-05-10T03:34:56.088202+0000","last_fullsized":"2024-05-10T03:34:56.088202+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:03:42.850255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094992+0000","last_change":"2024-05-10T03:34:55.104750+0000","last_active":"2024-05-10T03:34:56.094992+0000","last_peered":"2024-05-10T03:34:56.094992+0000","last_clean":"2024-05-10T03:34:56.094992+0000","last_became_active":"2024-05-10T03:34:55.104384+0000","last_became_peered":"2024-05-10T03:34:55.104384+0000","last_unstale":"2024-05-10T03:34:56.094992+0000","last_undegraded":"2024-05-10T03:34:56.094992+0000","last_fullsized":"2024-05-10T03:34:56.094992+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:22.811888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089087+0000","last_change":"2024-05-10T03:34:54.514671+0000","last_active":"2024-05-10T03:34:56.089087+0000","last_peered":"2024-05-10T03:34:56.089087+0000","last_clean":"2024-05-10T03:34:56.089087+0000","last_became_active":"2024-05-10T03:34:54.514302+0000","last_became_peered":"2024-05-10T03:34:54.514302+0000","last_unstale":"2024-05-10T03:34:56.089087+0000","last_undegraded":"2024-05-10T03:34:56.089087+0000","last_fullsized":"2024-05-10T03:34:56.089087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:34:00.576330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065310+0000","last_change":"2024-05-10T03:34:55.079396+0000","last_active":"2024-05-10T03:34:57.065310+0000","last_peered":"2024-05-10T03:34:57.065310+0000","last_clean":"2024-05-10T03:34:57.065310+0000","last_became_active":"2024-05-10T03:34:55.079115+0000","last_became_peered":"2024-05-10T03:34:55.079115+0000","last_unstale":"2024-05-10T03:34:57.065310+0000","last_undegraded":"2024-05-10T03:34:57.065310+0000","last_fullsized":"2024-05-10T03:34:57.065310+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:29.132456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064715+0000","last_change":"2024-05-10T03:34:54.511889+0000","last_active":"2024-05-10T03:34:57.064715+0000","last_peered":"2024-05-10T03:34:57.064715+0000","last_clean":"2024-05-10T03:34:57.064715+0000","last_became_active":"2024-05-10T03:34:54.511592+0000","last_became_peered":"2024-05-10T03:34:54.511592+0000","last_unstale":"2024-05-10T03:34:57.064715+0000","last_undegraded":"2024-05-10T03:34:57.064715+0000","last_fullsized":"2024-05-10T03:34:57.064715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:34:07.377833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088216+0000","last_change":"2024-05-10T03:34:55.094506+0000","last_active":"2024-05-10T03:34:56.088216+0000","last_peered":"2024-05-10T03:34:56.088216+0000","last_clean":"2024-05-10T03:34:56.088216+0000","last_became_active":"2024-05-10T03:34:55.093280+0000","last_became_peered":"2024-05-10T03:34:55.093280+0000","last_unstale":"2024-05-10T03:34:56.088216+0000","last_undegraded":"2024-05-10T03:34:56.088216+0000","last_fullsized":"2024-05-10T03:34:56.088216+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:26:01.526134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089354+0000","last_change":"2024-05-10T03:34:54.516666+0000","last_active":"2024-05-10T03:34:56.089354+0000","last_peered":"2024-05-10T03:34:56.089354+0000","last_clean":"2024-05-10T03:34:56.089354+0000","last_became_active":"2024-05-10T03:34:54.514815+0000","last_became_peered":"2024-05-10T03:34:54.514815+0000","last_unstale":"2024-05-10T03:34:56.089354+0000","last_undegraded":"2024-05-10T03:34:56.089354+0000","last_fullsized":"2024-05-10T03:34:56.089354+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:48.537060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.093485+0000","last_change":"2024-05-10T03:34:55.090264+0000","last_active":"2024-05-10T03:34:55.093485+0000","last_peered":"2024-05-10T03:34:55.093485+0000","last_clean":"2024-05-10T03:34:55.093485+0000","last_became_active":"2024-05-10T03:34:55.088684+0000","last_became_peered":"2024-05-10T03:34:55.088684+0000","last_unstale":"2024-05-10T03:34:55.093485+0000","last_undegraded":"2024-05-10T03:34:55.093485+0000","last_fullsized":"2024-05-10T03:34:55.093485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088399+0000","last_change":"2024-05-10T03:34:54.511475+0000","last_active":"2024-05-10T03:34:56.088399+0000","last_peered":"2024-05-10T03:34:56.088399+0000","last_clean":"2024-05-10T03:34:56.088399+0000","last_became_active":"2024-05-10T03:34:54.511115+0000","last_became_peered":"2024-05-10T03:34:54.511115+0000","last_unstale":"2024-05-10T03:34:56.088399+0000","last_undegraded":"2024-05-10T03:34:56.088399+0000","last_fullsized":"2024-05-10T03:34:56.088399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:15:17.200070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.098291+0000","last_change":"2024-05-10T03:34:55.092183+0000","last_active":"2024-05-10T03:34:55.098291+0000","last_peered":"2024-05-10T03:34:55.098291+0000","last_clean":"2024-05-10T03:34:55.098291+0000","last_became_active":"2024-05-10T03:34:55.089705+0000","last_became_peered":"2024-05-10T03:34:55.089705+0000","last_unstale":"2024-05-10T03:34:55.098291+0000","last_undegraded":"2024-05-10T03:34:55.098291+0000","last_fullsized":"2024-05-10T03:34:55.098291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:24:48.420848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095628+0000","last_change":"2024-05-10T03:34:54.511250+0000","last_active":"2024-05-10T03:34:56.095628+0000","last_peered":"2024-05-10T03:34:56.095628+0000","last_clean":"2024-05-10T03:34:56.095628+0000","last_became_active":"2024-05-10T03:34:54.507953+0000","last_became_peered":"2024-05-10T03:34:54.507953+0000","last_unstale":"2024-05-10T03:34:56.095628+0000","last_undegraded":"2024-05-10T03:34:56.095628+0000","last_fullsized":"2024-05-10T03:34:56.095628+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:08.142653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.096305+0000","last_change":"2024-05-10T03:34:55.091651+0000","last_active":"2024-05-10T03:34:55.096305+0000","last_peered":"2024-05-10T03:34:55.096305+0000","last_clean":"2024-05-10T03:34:55.096305+0000","last_became_active":"2024-05-10T03:34:55.090792+0000","last_became_peered":"2024-05-10T03:34:55.090792+0000","last_unstale":"2024-05-10T03:34:55.096305+0000","last_undegraded":"2024-05-10T03:34:55.096305+0000","last_fullsized":"2024-05-10T03:34:55.096305+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:08.980669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094711+0000","last_change":"2024-05-10T03:34:54.523896+0000","last_active":"2024-05-10T03:34:56.094711+0000","last_peered":"2024-05-10T03:34:56.094711+0000","last_clean":"2024-05-10T03:34:56.094711+0000","last_became_active":"2024-05-10T03:34:54.523578+0000","last_became_peered":"2024-05-10T03:34:54.523578+0000","last_unstale":"2024-05-10T03:34:56.094711+0000","last_undegraded":"2024-05-10T03:34:56.094711+0000","last_fullsized":"2024-05-10T03:34:56.094711+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:10:06.945240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087958+0000","last_change":"2024-05-10T03:34:55.084506+0000","last_active":"2024-05-10T03:34:56.087958+0000","last_peered":"2024-05-10T03:34:56.087958+0000","last_clean":"2024-05-10T03:34:56.087958+0000","last_became_active":"2024-05-10T03:34:55.083882+0000","last_became_peered":"2024-05-10T03:34:55.083882+0000","last_unstale":"2024-05-10T03:34:56.087958+0000","last_undegraded":"2024-05-10T03:34:56.087958+0000","last_fullsized":"2024-05-10T03:34:56.087958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:49.552166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.106807+0000","last_change":"2024-05-10T03:34:54.517352+0000","last_active":"2024-05-10T03:34:55.106807+0000","last_peered":"2024-05-10T03:34:55.106807+0000","last_clean":"2024-05-10T03:34:55.106807+0000","last_became_active":"2024-05-10T03:34:54.516835+0000","last_became_peered":"2024-05-10T03:34:54.516835+0000","last_unstale":"2024-05-10T03:34:55.106807+0000","last_undegraded":"2024-05-10T03:34:55.106807+0000","last_fullsized":"2024-05-10T03:34:55.106807+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:46.268806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095462+0000","last_change":"2024-05-10T03:34:55.092718+0000","last_active":"2024-05-10T03:34:56.095462+0000","last_peered":"2024-05-10T03:34:56.095462+0000","last_clean":"2024-05-10T03:34:56.095462+0000","last_became_active":"2024-05-10T03:34:55.091154+0000","last_became_peered":"2024-05-10T03:34:55.091154+0000","last_unstale":"2024-05-10T03:34:56.095462+0000","last_undegraded":"2024-05-10T03:34:56.095462+0000","last_fullsized":"2024-05-10T03:34:56.095462+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:52.993476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065042+0000","last_change":"2024-05-10T03:34:54.512402+0000","last_active":"2024-05-10T03:34:57.065042+0000","last_peered":"2024-05-10T03:34:57.065042+0000","last_clean":"2024-05-10T03:34:57.065042+0000","last_became_active":"2024-05-10T03:34:54.512118+0000","last_became_peered":"2024-05-10T03:34:54.512118+0000","last_unstale":"2024-05-10T03:34:57.065042+0000","last_undegraded":"2024-05-10T03:34:57.065042+0000","last_fullsized":"2024-05-10T03:34:57.065042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:51.978369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.110134+0000","last_change":"2024-05-10T03:34:55.108115+0000","last_active":"2024-05-10T03:34:55.110134+0000","last_peered":"2024-05-10T03:34:55.110134+0000","last_clean":"2024-05-10T03:34:55.110134+0000","last_became_active":"2024-05-10T03:34:55.105381+0000","last_became_peered":"2024-05-10T03:34:55.105381+0000","last_unstale":"2024-05-10T03:34:55.110134+0000","last_undegraded":"2024-05-10T03:34:55.110134+0000","last_fullsized":"2024-05-10T03:34:55.110134+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:02.986812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080355+0000","last_change":"2024-05-10T03:34:54.520988+0000","last_active":"2024-05-10T03:34:55.080355+0000","last_peered":"2024-05-10T03:34:55.080355+0000","last_clean":"2024-05-10T03:34:55.080355+0000","last_became_active":"2024-05-10T03:34:54.520413+0000","last_became_peered":"2024-05-10T03:34:54.520413+0000","last_unstale":"2024-05-10T03:34:55.080355+0000","last_undegraded":"2024-05-10T03:34:55.080355+0000","last_fullsized":"2024-05-10T03:34:55.080355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:03.620248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088681+0000","last_change":"2024-05-10T03:34:55.101871+0000","last_active":"2024-05-10T03:34:56.088681+0000","last_peered":"2024-05-10T03:34:56.088681+0000","last_clean":"2024-05-10T03:34:56.088681+0000","last_became_active":"2024-05-10T03:34:55.101237+0000","last_became_peered":"2024-05-10T03:34:55.101237+0000","last_unstale":"2024-05-10T03:34:56.088681+0000","last_undegraded":"2024-05-10T03:34:56.088681+0000","last_fullsized":"2024-05-10T03:34:56.088681+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:35:24.704232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088527+0000","last_change":"2024-05-10T03:34:54.512737+0000","last_active":"2024-05-10T03:34:56.088527+0000","last_peered":"2024-05-10T03:34:56.088527+0000","last_clean":"2024-05-10T03:34:56.088527+0000","last_became_active":"2024-05-10T03:34:54.512342+0000","last_became_peered":"2024-05-10T03:34:54.512342+0000","last_unstale":"2024-05-10T03:34:56.088527+0000","last_undegraded":"2024-05-10T03:34:56.088527+0000","last_fullsized":"2024-05-10T03:34:56.088527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:08.142653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.086401+0000","last_change":"2024-05-10T03:34:55.084825+0000","last_active":"2024-05-10T03:34:55.086401+0000","last_peered":"2024-05-10T03:34:55.086401+0000","last_clean":"2024-05-10T03:34:55.086401+0000","last_became_active":"2024-05-10T03:34:55.084275+0000","last_became_peered":"2024-05-10T03:34:55.084275+0000","last_unstale":"2024-05-10T03:34:55.086401+0000","last_undegraded":"2024-05-10T03:34:55.086401+0000","last_fullsized":"2024-05-10T03:34:55.086401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:13:58.857065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094953+0000","last_change":"2024-05-10T03:34:54.524379+0000","last_active":"2024-05-10T03:34:56.094953+0000","last_peered":"2024-05-10T03:34:56.094953+0000","last_clean":"2024-05-10T03:34:56.094953+0000","last_became_active":"2024-05-10T03:34:54.524027+0000","last_became_peered":"2024-05-10T03:34:54.524027+0000","last_unstale":"2024-05-10T03:34:56.094953+0000","last_undegraded":"2024-05-10T03:34:56.094953+0000","last_fullsized":"2024-05-10T03:34:56.094953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:19:18.138740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095377+0000","last_change":"2024-05-10T03:34:55.093363+0000","last_active":"2024-05-10T03:34:56.095377+0000","last_peered":"2024-05-10T03:34:56.095377+0000","last_clean":"2024-05-10T03:34:56.095377+0000","last_became_active":"2024-05-10T03:34:55.090952+0000","last_became_peered":"2024-05-10T03:34:55.090952+0000","last_unstale":"2024-05-10T03:34:56.095377+0000","last_undegraded":"2024-05-10T03:34:56.095377+0000","last_fullsized":"2024-05-10T03:34:56.095377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:07.919776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095906+0000","last_change":"2024-05-10T03:34:54.518488+0000","last_active":"2024-05-10T03:34:56.095906+0000","last_peered":"2024-05-10T03:34:56.095906+0000","last_clean":"2024-05-10T03:34:56.095906+0000","last_became_active":"2024-05-10T03:34:54.518087+0000","last_became_peered":"2024-05-10T03:34:54.518087+0000","last_unstale":"2024-05-10T03:34:56.095906+0000","last_undegraded":"2024-05-10T03:34:56.095906+0000","last_fullsized":"2024-05-10T03:34:56.095906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:27:48.537060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109891+0000","last_change":"2024-05-10T03:34:55.107994+0000","last_active":"2024-05-10T03:34:55.109891+0000","last_peered":"2024-05-10T03:34:55.109891+0000","last_clean":"2024-05-10T03:34:55.109891+0000","last_became_active":"2024-05-10T03:34:55.106882+0000","last_became_peered":"2024-05-10T03:34:55.106882+0000","last_unstale":"2024-05-10T03:34:55.109891+0000","last_undegraded":"2024-05-10T03:34:55.109891+0000","last_fullsized":"2024-05-10T03:34:55.109891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:22.811888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088257+0000","last_change":"2024-05-10T03:34:54.521020+0000","last_active":"2024-05-10T03:34:56.088257+0000","last_peered":"2024-05-10T03:34:56.088257+0000","last_clean":"2024-05-10T03:34:56.088257+0000","last_became_active":"2024-05-10T03:34:54.520389+0000","last_became_peered":"2024-05-10T03:34:54.520389+0000","last_unstale":"2024-05-10T03:34:56.088257+0000","last_undegraded":"2024-05-10T03:34:56.088257+0000","last_fullsized":"2024-05-10T03:34:56.088257+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:03:25.055186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088091+0000","last_change":"2024-05-10T03:34:55.079860+0000","last_active":"2024-05-10T03:34:56.088091+0000","last_peered":"2024-05-10T03:34:56.088091+0000","last_clean":"2024-05-10T03:34:56.088091+0000","last_became_active":"2024-05-10T03:34:55.079558+0000","last_became_peered":"2024-05-10T03:34:55.079558+0000","last_unstale":"2024-05-10T03:34:56.088091+0000","last_undegraded":"2024-05-10T03:34:56.088091+0000","last_fullsized":"2024-05-10T03:34:56.088091+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:42:50.663986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109413+0000","last_change":"2024-05-10T03:34:54.515985+0000","last_active":"2024-05-10T03:34:55.109413+0000","last_peered":"2024-05-10T03:34:55.109413+0000","last_clean":"2024-05-10T03:34:55.109413+0000","last_became_active":"2024-05-10T03:34:54.515618+0000","last_became_peered":"2024-05-10T03:34:54.515618+0000","last_unstale":"2024-05-10T03:34:55.109413+0000","last_undegraded":"2024-05-10T03:34:55.109413+0000","last_fullsized":"2024-05-10T03:34:55.109413+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:11:37.732608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088761+0000","last_change":"2024-05-10T03:34:55.087774+0000","last_active":"2024-05-10T03:34:56.088761+0000","last_peered":"2024-05-10T03:34:56.088761+0000","last_clean":"2024-05-10T03:34:56.088761+0000","last_became_active":"2024-05-10T03:34:55.087445+0000","last_became_peered":"2024-05-10T03:34:55.087445+0000","last_unstale":"2024-05-10T03:34:56.088761+0000","last_undegraded":"2024-05-10T03:34:56.088761+0000","last_fullsized":"2024-05-10T03:34:56.088761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:57:02.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080048+0000","last_change":"2024-05-10T03:34:54.519580+0000","last_active":"2024-05-10T03:34:55.080048+0000","last_peered":"2024-05-10T03:34:55.080048+0000","last_clean":"2024-05-10T03:34:55.080048+0000","last_became_active":"2024-05-10T03:34:54.519247+0000","last_became_peered":"2024-05-10T03:34:54.519247+0000","last_unstale":"2024-05-10T03:34:55.080048+0000","last_undegraded":"2024-05-10T03:34:55.080048+0000","last_fullsized":"2024-05-10T03:34:55.080048+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:41.100149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.065141+0000","last_change":"2024-05-10T03:34:55.094082+0000","last_active":"2024-05-10T03:34:57.065141+0000","last_peered":"2024-05-10T03:34:57.065141+0000","last_clean":"2024-05-10T03:34:57.065141+0000","last_became_active":"2024-05-10T03:34:55.093750+0000","last_became_peered":"2024-05-10T03:34:55.093750+0000","last_unstale":"2024-05-10T03:34:57.065141+0000","last_undegraded":"2024-05-10T03:34:57.065141+0000","last_fullsized":"2024-05-10T03:34:57.065141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:15:44.635470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.093981+0000","last_change":"2024-05-10T03:34:54.517823+0000","last_active":"2024-05-10T03:34:56.093981+0000","last_peered":"2024-05-10T03:34:56.093981+0000","last_clean":"2024-05-10T03:34:56.093981+0000","last_became_active":"2024-05-10T03:34:54.517155+0000","last_became_peered":"2024-05-10T03:34:54.517155+0000","last_unstale":"2024-05-10T03:34:56.093981+0000","last_undegraded":"2024-05-10T03:34:56.093981+0000","last_fullsized":"2024-05-10T03:34:56.093981+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:33:46.268806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.101265+0000","last_change":"2024-05-10T03:34:55.094321+0000","last_active":"2024-05-10T03:34:55.101265+0000","last_peered":"2024-05-10T03:34:55.101265+0000","last_clean":"2024-05-10T03:34:55.101265+0000","last_became_active":"2024-05-10T03:34:55.094027+0000","last_became_peered":"2024-05-10T03:34:55.094027+0000","last_unstale":"2024-05-10T03:34:55.101265+0000","last_undegraded":"2024-05-10T03:34:55.101265+0000","last_fullsized":"2024-05-10T03:34:55.101265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:31:07.759872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.081118+0000","last_change":"2024-05-10T03:34:54.515195+0000","last_active":"2024-05-10T03:34:55.081118+0000","last_peered":"2024-05-10T03:34:55.081118+0000","last_clean":"2024-05-10T03:34:55.081118+0000","last_became_active":"2024-05-10T03:34:54.514874+0000","last_became_peered":"2024-05-10T03:34:54.514874+0000","last_unstale":"2024-05-10T03:34:55.081118+0000","last_undegraded":"2024-05-10T03:34:55.081118+0000","last_fullsized":"2024-05-10T03:34:55.081118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:47:13.749165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087941+0000","last_change":"2024-05-10T03:34:55.078719+0000","last_active":"2024-05-10T03:34:56.087941+0000","last_peered":"2024-05-10T03:34:56.087941+0000","last_clean":"2024-05-10T03:34:56.087941+0000","last_became_active":"2024-05-10T03:34:55.077693+0000","last_became_peered":"2024-05-10T03:34:55.077693+0000","last_unstale":"2024-05-10T03:34:56.087941+0000","last_undegraded":"2024-05-10T03:34:56.087941+0000","last_fullsized":"2024-05-10T03:34:56.087941+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:18:25.507044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.106495+0000","last_change":"2024-05-10T03:34:54.519707+0000","last_active":"2024-05-10T03:34:55.106495+0000","last_peered":"2024-05-10T03:34:55.106495+0000","last_clean":"2024-05-10T03:34:55.106495+0000","last_became_active":"2024-05-10T03:34:54.516925+0000","last_became_peered":"2024-05-10T03:34:54.516925+0000","last_unstale":"2024-05-10T03:34:55.106495+0000","last_undegraded":"2024-05-10T03:34:55.106495+0000","last_fullsized":"2024-05-10T03:34:55.106495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:10:06.945240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.088824+0000","last_change":"2024-05-10T03:34:55.106047+0000","last_active":"2024-05-10T03:34:56.088824+0000","last_peered":"2024-05-10T03:34:56.088824+0000","last_clean":"2024-05-10T03:34:56.088824+0000","last_became_active":"2024-05-10T03:34:55.105565+0000","last_became_peered":"2024-05-10T03:34:55.105565+0000","last_unstale":"2024-05-10T03:34:56.088824+0000","last_undegraded":"2024-05-10T03:34:56.088824+0000","last_fullsized":"2024-05-10T03:34:56.088824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:49:10.408226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089033+0000","last_change":"2024-05-10T03:34:54.507611+0000","last_active":"2024-05-10T03:34:56.089033+0000","last_peered":"2024-05-10T03:34:56.089033+0000","last_clean":"2024-05-10T03:34:56.089033+0000","last_became_active":"2024-05-10T03:34:54.507259+0000","last_became_peered":"2024-05-10T03:34:54.507259+0000","last_unstale":"2024-05-10T03:34:56.089033+0000","last_undegraded":"2024-05-10T03:34:56.089033+0000","last_fullsized":"2024-05-10T03:34:56.089033+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:21:27.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:57.064742+0000","last_change":"2024-05-10T03:34:55.093587+0000","last_active":"2024-05-10T03:34:57.064742+0000","last_peered":"2024-05-10T03:34:57.064742+0000","last_clean":"2024-05-10T03:34:57.064742+0000","last_became_active":"2024-05-10T03:34:55.093276+0000","last_became_peered":"2024-05-10T03:34:55.093276+0000","last_unstale":"2024-05-10T03:34:57.064742+0000","last_undegraded":"2024-05-10T03:34:57.064742+0000","last_fullsized":"2024-05-10T03:34:57.064742+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:02:50.553885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.095716+0000","last_change":"2024-05-10T03:34:54.521688+0000","last_active":"2024-05-10T03:34:56.095716+0000","last_peered":"2024-05-10T03:34:56.095716+0000","last_clean":"2024-05-10T03:34:56.095716+0000","last_became_active":"2024-05-10T03:34:54.521319+0000","last_became_peered":"2024-05-10T03:34:54.521319+0000","last_unstale":"2024-05-10T03:34:56.095716+0000","last_undegraded":"2024-05-10T03:34:56.095716+0000","last_fullsized":"2024-05-10T03:34:56.095716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:58:13.741483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089086+0000","last_change":"2024-05-10T03:34:55.096300+0000","last_active":"2024-05-10T03:34:56.089086+0000","last_peered":"2024-05-10T03:34:56.089086+0000","last_clean":"2024-05-10T03:34:56.089086+0000","last_became_active":"2024-05-10T03:34:55.095944+0000","last_became_peered":"2024-05-10T03:34:55.095944+0000","last_unstale":"2024-05-10T03:34:56.089086+0000","last_undegraded":"2024-05-10T03:34:56.089086+0000","last_fullsized":"2024-05-10T03:34:56.089086+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:33:04.192750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.085655+0000","last_change":"2024-05-10T03:34:54.518313+0000","last_active":"2024-05-10T03:34:55.085655+0000","last_peered":"2024-05-10T03:34:55.085655+0000","last_clean":"2024-05-10T03:34:55.085655+0000","last_became_active":"2024-05-10T03:34:54.517783+0000","last_became_peered":"2024-05-10T03:34:54.517783+0000","last_unstale":"2024-05-10T03:34:55.085655+0000","last_undegraded":"2024-05-10T03:34:55.085655+0000","last_fullsized":"2024-05-10T03:34:55.085655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:48.486514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.100776+0000","last_change":"2024-05-10T03:34:55.093645+0000","last_active":"2024-05-10T03:34:55.100776+0000","last_peered":"2024-05-10T03:34:55.100776+0000","last_clean":"2024-05-10T03:34:55.100776+0000","last_became_active":"2024-05-10T03:34:55.092181+0000","last_became_peered":"2024-05-10T03:34:55.092181+0000","last_unstale":"2024-05-10T03:34:55.100776+0000","last_undegraded":"2024-05-10T03:34:55.100776+0000","last_fullsized":"2024-05-10T03:34:55.100776+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:51:16.204050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089056+0000","last_change":"2024-05-10T03:34:54.521008+0000","last_active":"2024-05-10T03:34:56.089056+0000","last_peered":"2024-05-10T03:34:56.089056+0000","last_clean":"2024-05-10T03:34:56.089056+0000","last_became_active":"2024-05-10T03:34:54.520611+0000","last_became_peered":"2024-05-10T03:34:54.520611+0000","last_unstale":"2024-05-10T03:34:56.089056+0000","last_undegraded":"2024-05-10T03:34:56.089056+0000","last_fullsized":"2024-05-10T03:34:56.089056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:06.904669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.104771+0000","last_change":"2024-05-10T03:34:55.094383+0000","last_active":"2024-05-10T03:34:55.104771+0000","last_peered":"2024-05-10T03:34:55.104771+0000","last_clean":"2024-05-10T03:34:55.104771+0000","last_became_active":"2024-05-10T03:34:55.094081+0000","last_became_peered":"2024-05-10T03:34:55.094081+0000","last_unstale":"2024-05-10T03:34:55.104771+0000","last_undegraded":"2024-05-10T03:34:55.104771+0000","last_fullsized":"2024-05-10T03:34:55.104771+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:01:23.035355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.080173+0000","last_change":"2024-05-10T03:34:54.523075+0000","last_active":"2024-05-10T03:34:55.080173+0000","last_peered":"2024-05-10T03:34:55.080173+0000","last_clean":"2024-05-10T03:34:55.080173+0000","last_became_active":"2024-05-10T03:34:54.522591+0000","last_became_peered":"2024-05-10T03:34:54.522591+0000","last_unstale":"2024-05-10T03:34:55.080173+0000","last_undegraded":"2024-05-10T03:34:55.080173+0000","last_fullsized":"2024-05-10T03:34:55.080173+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:47:50.004046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.096646+0000","last_change":"2024-05-10T03:34:55.111983+0000","last_active":"2024-05-10T03:34:56.096646+0000","last_peered":"2024-05-10T03:34:56.096646+0000","last_clean":"2024-05-10T03:34:56.096646+0000","last_became_active":"2024-05-10T03:34:55.111652+0000","last_became_peered":"2024-05-10T03:34:55.111652+0000","last_unstale":"2024-05-10T03:34:56.096646+0000","last_undegraded":"2024-05-10T03:34:56.096646+0000","last_fullsized":"2024-05-10T03:34:56.096646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:02.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.094046+0000","last_change":"2024-05-10T03:34:54.520030+0000","last_active":"2024-05-10T03:34:56.094046+0000","last_peered":"2024-05-10T03:34:56.094046+0000","last_clean":"2024-05-10T03:34:56.094046+0000","last_became_active":"2024-05-10T03:34:54.519457+0000","last_became_peered":"2024-05-10T03:34:54.519457+0000","last_unstale":"2024-05-10T03:34:56.094046+0000","last_undegraded":"2024-05-10T03:34:56.094046+0000","last_fullsized":"2024-05-10T03:34:56.094046+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:32:48.486514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.089404+0000","last_change":"2024-05-10T03:34:55.111533+0000","last_active":"2024-05-10T03:34:56.089404+0000","last_peered":"2024-05-10T03:34:56.089404+0000","last_clean":"2024-05-10T03:34:56.089404+0000","last_became_active":"2024-05-10T03:34:55.111210+0000","last_became_peered":"2024-05-10T03:34:55.111210+0000","last_unstale":"2024-05-10T03:34:56.089404+0000","last_undegraded":"2024-05-10T03:34:56.089404+0000","last_fullsized":"2024-05-10T03:34:56.089404+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:02.744456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.081224+0000","last_change":"2024-05-10T03:34:54.522396+0000","last_active":"2024-05-10T03:34:55.081224+0000","last_peered":"2024-05-10T03:34:55.081224+0000","last_clean":"2024-05-10T03:34:55.081224+0000","last_became_active":"2024-05-10T03:34:54.520868+0000","last_became_peered":"2024-05-10T03:34:54.520868+0000","last_unstale":"2024-05-10T03:34:55.081224+0000","last_undegraded":"2024-05-10T03:34:55.081224+0000","last_fullsized":"2024-05-10T03:34:55.081224+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:33:29.250596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-10T03:34:56.087887+0000","last_change":"2024-05-10T03:34:55.088334+0000","last_active":"2024-05-10T03:34:56.087887+0000","last_peered":"2024-05-10T03:34:56.087887+0000","last_clean":"2024-05-10T03:34:56.087887+0000","last_became_active":"2024-05-10T03:34:55.087593+0000","last_became_peered":"2024-05-10T03:34:55.087593+0000","last_unstale":"2024-05-10T03:34:56.087887+0000","last_undegraded":"2024-05-10T03:34:56.087887+0000","last_fullsized":"2024-05-10T03:34:56.087887+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:54.467737+0000","last_clean_scrub_stamp":"2024-05-10T03:34:54.467737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:12:07.919776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.109477+0000","last_change":"2024-05-10T03:34:54.522959+0000","last_active":"2024-05-10T03:34:55.109477+0000","last_peered":"2024-05-10T03:34:55.109477+0000","last_clean":"2024-05-10T03:34:55.109477+0000","last_became_active":"2024-05-10T03:34:54.521219+0000","last_became_peered":"2024-05-10T03:34:54.521219+0000","last_unstale":"2024-05-10T03:34:55.109477+0000","last_undegraded":"2024-05-10T03:34:55.109477+0000","last_fullsized":"2024-05-10T03:34:55.109477+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:34:53.452630+0000","last_clean_scrub_stamp":"2024-05-10T03:34:53.452630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:02:03.301656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":235,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-10T03:34:55.091825+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:34:55.091825+0000","last_peered":"2024-05-10T03:34:55.091825+0000","last_clean":"2024-05-10T03:34:55.091825+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:34:55.091825+0000","last_undegraded":"2024-05-10T03:34:55.091825+0000","last_fullsized":"2024-05-10T03:34:55.091825+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1892352,"data_stored":1177357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2470000000000001,"15min":1.2350000000000001},"min":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.77,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3420000000000001,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.151,"15min":1.145},"min":{"1min":0.71399999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.671,"5min":1.853,"15min":1.853},"last":1.26},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2470000000000001,"15min":1.2669999999999999},"min":{"1min":0.86299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.2030000000000001}]},{"osd":2,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2909999999999999,"15min":1.359},"min":{"1min":0.80400000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8560000000000001,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3440000000000001,"15min":1.417},"min":{"1min":0.88400000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.4649999999999999,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.383},"min":{"1min":0.93500000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.9750000000000001,"5min":2.222,"15min":2.222},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.341},"min":{"1min":0.95299999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.649,"5min":2.367,"15min":2.367},"last":1.587}]},{"osd":4,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.419,"15min":1.46},"min":{"1min":0.89300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.079},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.504,"15min":1.502},"min":{"1min":0.70499999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.851,"5min":2.323,"15min":2.323},"last":1.3380000000000001}]},{"osd":5,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5580000000000001,"15min":1.52},"min":{"1min":0.94299999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.429},{"interface":"front","average":{"1min":1.7549999999999999,"5min":1.677,"15min":1.6919999999999999},"min":{"1min":1.0129999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5109999999999999}]},{"osd":6,"last update":"Fri May 10 03:34:42 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.6839999999999999,"15min":1.7150000000000001},"min":{"1min":1.0660000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.403},{"interface":"front","average":{"1min":1.599,"5min":1.6539999999999999,"15min":1.641},"min":{"1min":1.236,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1000000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.48}]}]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1310720,"data_stored":588997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.145,"15min":1.095},"min":{"1min":0.75900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.6859999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.321},{"interface":"front","average":{"1min":1.075,"5min":1.228,"15min":1.3180000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.5760000000000001}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.276,"15min":1.306},"min":{"1min":0.874,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3620000000000001,"5min":3.113,"15min":3.113},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.341,"15min":1.4530000000000001},"min":{"1min":0.90600000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.821,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.91500000000000004}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.353,"15min":1.268},"min":{"1min":0.92000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.7509999999999999,"5min":2.02,"15min":2.02},"last":1.181},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.371,"15min":1.484},"min":{"1min":0.88600000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.8999999999999999,"5min":3.569,"15min":3.569},"last":0.995}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.446,"15min":1.5800000000000001},"min":{"1min":1.0580000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.4260000000000002,"5min":4.0869999999999997,"15min":4.0869999999999997},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.363,"15min":1.323},"min":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2130000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.4299999999999999},"min":{"1min":1.0529999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.073,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.548},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5229999999999999,"15min":1.571},"min":{"1min":1.127,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.704}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.482,"15min":1.4179999999999999},"min":{"1min":1.1359999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.5179999999999998,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.6259999999999999,"15min":1.6240000000000001},"min":{"1min":1.111,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.6499999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.6659999999999999,"15min":1.6719999999999999},"min":{"1min":1.2170000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8410000000000002,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.867},{"interface":"front","average":{"1min":1.7470000000000001,"5min":1.681,"15min":1.665},"min":{"1min":1.262,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.2649999999999997,"5min":4.2649999999999997,"15min":4.2649999999999997},"last":1.502}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1290240,"data_stored":586829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.109,"15min":1.077},"min":{"1min":0.73699999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.173,"15min":1.1930000000000001},"min":{"1min":0.75700000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.25}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.69699999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5580000000000001,"5min":1.855,"15min":1.855},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.226,"5min":1.2090000000000001,"15min":1.2210000000000001},"min":{"1min":0.77700000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.76,"5min":1.954,"15min":1.954},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.80100000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.706,"5min":2.359,"15min":2.359},"last":1.127},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3149999999999999,"15min":1.333},"min":{"1min":0.90100000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.875,"5min":1.913,"15min":1.913},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3500000000000001,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7869999999999999,"5min":1.881,"15min":1.881},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.345,"5min":1.3700000000000001,"15min":1.3089999999999999},"min":{"1min":0.92800000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8260000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.379,"15min":1.407},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8240000000000001,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.488},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.569,"15min":1.595},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9430000000000001,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.93899999999999995}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.53,"15min":1.5169999999999999},"min":{"1min":0.78500000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.931,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.421},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.548,"15min":1.5509999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.98,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.7509999999999999,"15min":1.796},"min":{"1min":0.94799999999999995,"5min":0.753,"15min":0.753},"max":{"1min":2.3420000000000001,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":1.383},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.617},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.54}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1298432,"data_stored":586921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.141,"15min":1.167},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5109999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.131,"15min":1.137},"min":{"1min":0.747,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3859999999999999,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.97399999999999998}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1930000000000001,"15min":1.165},"min":{"1min":0.86699999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6910000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.679},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.234,"15min":1.268},"min":{"1min":0.92500000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6060000000000001,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2390000000000001,"15min":1.27},"min":{"1min":0.81399999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3620000000000001,"15min":1.395},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.764,"5min":2.29,"15min":2.29},"last":1.54}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3580000000000001,"15min":1.4430000000000001},"min":{"1min":1.028,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.7829999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.534,"15min":1.5700000000000001},"min":{"1min":1.054,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.893,"5min":2.7709999999999999,"15min":2.7709999999999999},"last":1.496}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4830000000000001,"15min":1.4990000000000001},"min":{"1min":1.131,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.944,"5min":2.512,"15min":2.512},"last":1.417},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4359999999999999,"15min":1.5289999999999999},"min":{"1min":0.83099999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.958,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.452}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.542,"15min":1.5780000000000001},"min":{"1min":1.0409999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0459999999999998,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.266},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.661,"15min":1.756},"min":{"1min":1.266,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9570000000000001,"5min":5.3529999999999998,"15min":5.3529999999999998},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6100000000000001,"15min":1.6759999999999999},"min":{"1min":1.071,"5min":0.495,"15min":0.495},"max":{"1min":2.1349999999999998,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.71,"15min":1.734},"min":{"1min":1.1839999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1499999999999999,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1282048,"data_stored":580791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92100000000000004,"15min":0.94899999999999995},"min":{"1min":0.34200000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.4199999999999999,"5min":1.754,"15min":1.754},"last":2.0129999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3520000000000001,"5min":1.708,"15min":1.708},"last":1.05}]},{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.94499999999999995,"15min":0.96599999999999997},"min":{"1min":0.51500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5620000000000001,"5min":1.603,"15min":1.603},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.1639999999999999,"15min":1.2290000000000001},"min":{"1min":0.57699999999999996,"5min":0.377,"15min":0.377},"max":{"1min":1.5429999999999999,"5min":1.964,"15min":1.964},"last":2.149}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.151,"15min":1.175},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.651,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2450000000000001,"15min":1.337},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7050000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.649}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3520000000000001,"15min":1.3700000000000001},"min":{"1min":0.82599999999999996,"5min":0.626,"15min":0.626},"max":{"1min":2.0910000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4359999999999999,"15min":1.3939999999999999},"min":{"1min":0.94699999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0590000000000002,"5min":2.464,"15min":2.464},"last":1.369}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.546,"15min":1.6519999999999999},"min":{"1min":0.93100000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":2.2200000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.45},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.454,"15min":1.4810000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.0470000000000002,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.8149999999999999}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.548,"15min":1.5700000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.355,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":2.1190000000000002},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6879999999999999,"15min":1.7749999999999999},"min":{"1min":1.0509999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.1339999999999999,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":1.8759999999999999}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.5229999999999999,"15min":1.577},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.913,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.625,"5min":1.5980000000000001,"15min":1.6439999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.29,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.952}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1282048,"data_stored":580243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89600000000000002,"15min":0.96999999999999997},"min":{"1min":0.35599999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4259999999999999,"5min":1.772,"15min":1.772},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.381,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.28,"5min":1.746,"15min":1.746},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.1259999999999999,"15min":1.1719999999999999},"min":{"1min":0.50700000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5549999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.93200000000000005}]},{"osd":3,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0549999999999999,"15min":1.107},"min":{"1min":0.58799999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.375,"5min":2.169,"15min":2.169},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2050000000000001,"15min":1.248},"min":{"1min":0.66700000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.9910000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.97899999999999998}]},{"osd":4,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4319999999999999,"15min":1.516},"min":{"1min":0.97399999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9159999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.577},{"interface":"front","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.5089999999999999},"min":{"1min":0.79400000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.0489999999999999,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.9490000000000001}]},{"osd":5,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.4199999999999999,"15min":1.4690000000000001},"min":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.74,"5min":2.508,"15min":2.508},"last":1.605},{"interface":"front","average":{"1min":1.52,"5min":1.5680000000000001,"15min":1.6100000000000001},"min":{"1min":0.91500000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.9550000000000001,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.448}]},{"osd":6,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5,"15min":1.5409999999999999},"min":{"1min":1.0589999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1989999999999998,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.827},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.5129999999999999,"15min":1.575},"min":{"1min":1.042,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8500000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.544}]},{"osd":7,"last update":"Fri May 10 03:34:35 2024","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.579,"15min":1.639},"min":{"1min":1.0620000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.3149999999999999,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.585,"15min":1.6539999999999999},"min":{"1min":1.119,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.829,"5min":2.7690000000000001,"15min":2.7690000000000001},"last":1.7470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1282048,"data_stored":580243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.499,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.343,"5min":1.607,"15min":1.607},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.995,"15min":1.0680000000000001},"min":{"1min":0.71899999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.996,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.284}]},{"osd":2,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0029999999999999},"min":{"1min":0.58199999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.4419999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.228},{"interface":"front","average":{"1min":1.161,"5min":1.135,"15min":1.179},"min":{"1min":0.752,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5800000000000001,"5min":2.444,"15min":2.444},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.242,"15min":1.3080000000000001},"min":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8280000000000001,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.5},{"interface":"front","average":{"1min":1.167,"5min":1.2050000000000001,"15min":1.278},"min":{"1min":0.92800000000000005,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.5349999999999999}]},{"osd":4,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.4690000000000001,"15min":1.512},"min":{"1min":1.2110000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.153,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.4710000000000001,"15min":1.5649999999999999},"min":{"1min":1,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.736,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.4339999999999999,"15min":1.556},"min":{"1min":1.032,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.728,"5min":3.8029999999999999,"15min":3.8029999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.522,"15min":1.6140000000000001},"min":{"1min":1.0660000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.3290000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.095}]},{"osd":6,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5680000000000001,"15min":1.6930000000000001},"min":{"1min":1.292,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.9730000000000001,"5min":3.48,"15min":3.48},"last":1.677},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5760000000000001,"15min":1.702},"min":{"1min":1.157,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.9179999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.028}]},{"osd":7,"last update":"Fri May 10 03:34:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.6419999999999999,"15min":1.7190000000000001},"min":{"1min":1.0029999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.202,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.631,"15min":1.7529999999999999},"min":{"1min":1.2030000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.2269999999999999,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.607}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1900544,"data_stored":1178851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.89500000000000002,"15min":1.0009999999999999},"min":{"1min":0.45600000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7270000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97599999999999998,"15min":1.1499999999999999},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.405,"5min":2.9390000000000001,"15min":2.9390000000000001},"last":1.3799999999999999}]},{"osd":2,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.075,"15min":1.1950000000000001},"min":{"1min":0.58499999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.6379999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.0609999999999999,"15min":1.1870000000000001},"min":{"1min":0.61099999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.8400000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.333}]},{"osd":3,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0840000000000001},"min":{"1min":0.56999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4339999999999999,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.276},{"interface":"front","average":{"1min":1.131,"5min":1.2210000000000001,"15min":1.3049999999999999},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.891,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.1759999999999999}]},{"osd":4,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3859999999999999,"15min":1.5820000000000001},"min":{"1min":0.877,"5min":0.745,"15min":0.745},"max":{"1min":1.8620000000000001,"5min":3.1040000000000001,"15min":3.1040000000000001},"last":2.2799999999999998},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3500000000000001,"15min":1.419},"min":{"1min":0.84199999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6519999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.611}]},{"osd":5,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3700000000000001,"15min":1.4550000000000001},"min":{"1min":0.90000000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7929999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4670000000000001,"15min":1.6000000000000001},"min":{"1min":1.1180000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.2120000000000002,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":1.772}]},{"osd":6,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5469999999999999,"15min":1.627},"min":{"1min":1.04,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.3239999999999998,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":2.6339999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5349999999999999,"15min":1.7030000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.028,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.877}]},{"osd":7,"last update":"Fri May 10 03:34:39 2024","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5620000000000001,"15min":1.75},"min":{"1min":0.97799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.1070000000000002,"5min":3.149,"15min":3.149},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.51,"15min":1.5640000000000001},"min":{"1min":0.86899999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1509999999999998,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.4059999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:35:00.868 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T03:35:00.868 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-10T03:35:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.203+0000 7f474c400640 1 Processor -- start 2024-05-10T03:35:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.203+0000 7f474c400640 1 -- start start 2024-05-10T03:35:01.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.205+0000 7f474c400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47440cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474c400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474406e490 0x7f474406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474c400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47440ce790 con 0x7f474415a670 2024-05-10T03:35:01.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474c400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474415aa80 con 0x7f474406e490 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474c400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440ccb50 con 0x7f47440cc740 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474a400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474406e490 0x7f474406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474a400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f474406e490 0x7f474406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41298/0 (socket says 172.21.15.55:41298) 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f474a400640 1 -- 172.21.15.55:0/2060027461 learned_addr learned my addr 172.21.15.55:0/2060027461 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.206+0000 7f4749000640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47440cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.207+0000 7f474a400640 1 -- 172.21.15.55:0/2060027461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 msgr2=0x7f47440cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.207+0000 7f474a400640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47440cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.207+0000 7f474a400640 1 -- 172.21.15.55:0/2060027461 >> v1:172.21.15.162:6789/0 conn(0x7f474415a670 legacy=0x7f47440cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:01.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.207+0000 7f474a400640 1 -- 172.21.15.55:0/2060027461 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474406ede0 con 0x7f474406e490 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.208+0000 7f474a400640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f474406e8a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f473c009e50 tx=0x7f473c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397b325a28619bec server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.208+0000 7f4743e00640 1 -- 172.21.15.55:0/2060027461 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c02aea0 con 0x7f474406e490 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.208+0000 7f4743e00640 1 -- 172.21.15.55:0/2060027461 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c02d050 con 0x7f474406e490 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.208+0000 7f4743e00640 1 -- 172.21.15.55:0/2060027461 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c031820 con 0x7f474406e490 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 -- 172.21.15.55:0/2060027461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 msgr2=0x7f474406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f474406e8a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f473c009e50 tx=0x7f473c02a6c0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 -- 172.21.15.55:0/2060027461 shutdown_connections 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f474406e8a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 --2- 172.21.15.55:0/2060027461 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47440cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 -- 172.21.15.55:0/2060027461 >> 172.21.15.55:0/2060027461 conn(0x7f47440b1c70 msgr2=0x7f47440b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 -- 172.21.15.55:0/2060027461 shutdown_connections 2024-05-10T03:35:01.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.209+0000 7f474c400640 1 -- 172.21.15.55:0/2060027461 wait complete. 2024-05-10T03:35:01.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.212+0000 7f474c400640 1 Processor -- start 2024-05-10T03:35:01.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.212+0000 7f474c400640 1 -- start start 2024-05-10T03:35:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47441419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474415a670 0x7f47441f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744142b20 con 0x7f474406e490 2024-05-10T03:35:01.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47441f79f0 con 0x7f474415a670 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.213+0000 7f474c400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474406b7b0 con 0x7f47440cc740 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41304/0 (socket says 172.21.15.55:41304) 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 -- 172.21.15.55:0/140534374 learned_addr learned my addr 172.21.15.55:0/140534374 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749000640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47441419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 -- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 msgr2=0x7f47441419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47441419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f474a400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474415a670 0x7f47441f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 -- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474415a670 msgr2=0x7f47441f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474415a670 0x7f47441f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.214+0000 7f4749a00640 1 -- 172.21.15.55:0/140534374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c009b30 con 0x7f474406e490 2024-05-10T03:35:01.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f4749a00640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f473800b3f0 tx=0x7f473800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738011830 con 0x7f474406e490 2024-05-10T03:35:01.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738011990 con 0x7f474406e490 2024-05-10T03:35:01.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4744141f20 con 0x7f474406e490 2024-05-10T03:35:01.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738011d40 con 0x7f474406e490 2024-05-10T03:35:01.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.215+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47441f7f20 con 0x7f474406e490 2024-05-10T03:35:01.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.216+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4744067930 con 0x7f474406e490 2024-05-10T03:35:01.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.218+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f473801a400 con 0x7f474406e490 2024-05-10T03:35:01.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.219+0000 7f4741600640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 0x7f472806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.219+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4738089460 con 0x7f474406e490 2024-05-10T03:35:01.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.219+0000 7f4749000640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 0x7f472806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.221+0000 7f4749000640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 0x7f472806d110 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.223+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f47380543d0 con 0x7f474406e490 2024-05-10T03:35:01.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.500+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f474406b8f0 con 0x7f474406e490 2024-05-10T03:35:01.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.514+0000 7f4741600640 1 -- 172.21.15.55:0/140534374 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f47380579f0 con 0x7f474406e490 2024-05-10T03:35:01.518 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2024-05-10T03:35:01.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 msgr2=0x7f472806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 0x7f472806d110 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).stop 2024-05-10T03:35:01.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 msgr2=0x7f4744142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f473800b3f0 tx=0x7f473800b8b0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 shutdown_connections 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f472806ac00 0x7f472806d110 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f474415a670 0x7f47441f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47440cc740 0x7f47441419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 --2- 172.21.15.55:0/140534374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f474406e490 0x7f4744142220 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.517+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 >> 172.21.15.55:0/140534374 conn(0x7f47440b1c70 msgr2=0x7f47440afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.518+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 shutdown_connections 2024-05-10T03:35:01.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.518+0000 7f474c400640 1 -- 172.21.15.55:0/140534374 wait complete. 2024-05-10T03:35:01.547 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:01.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.821+0000 7f49e7200640 1 Processor -- start 2024-05-10T03:35:01.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.821+0000 7f49e7200640 1 -- start start 2024-05-10T03:35:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49e8168510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e806bac0 con 0x7f49e80b2170 2024-05-10T03:35:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e7200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e8161fc0 con 0x7f49e80b6680 2024-05-10T03:35:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80d2b20 con 0x7f49e815d5e0 2024-05-10T03:35:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e5400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f49e80b2170 0x7f49e81618b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39926/0 (socket says 172.21.15.55:39926) 2024-05-10T03:35:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e5400640 1 -- 172.21.15.55:0/2800592435 learned_addr learned my addr 172.21.15.55:0/2800592435 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.824+0000 7f49e6800640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49e8168510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327566747 0 0) 0x7f49e806bac0 con 0x7f49e80b2170 2024-05-10T03:35:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc003520 con 0x7f49e80b2170 2024-05-10T03:35:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153633267 0 0) 0x7f49cc003520 con 0x7f49e80b2170 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 msgr2=0x7f49e8168510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49e8168510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 >> v1:172.21.15.162:6789/0 conn(0x7f49e80b6680 legacy=0x7f49e81611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80d25a0 con 0x7f49e80b2170 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f49d4001670 con 0x7f49e80b2170 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.825+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 >> v1:172.21.15.55:6789/0 conn(0x7f49e80b2170 legacy=0x7f49e81618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49cc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49cc003f70 0x7f49cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49cc004d00 0x7f49cc005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80d2b20 con 0x7f49cc003f70 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00bb10 con 0x7f49cc004d00 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00bcd0 con 0x7f49e815d5e0 2024-05-10T03:35:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e5400640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49cc003f70 0x7f49cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e5e00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49cc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.826+0000 7f49e6800640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49cc004d00 0x7f49cc005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5400640 1 -- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 msgr2=0x7f49cc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5400640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49cc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5400640 1 -- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49cc004d00 msgr2=0x7f49cc005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5400640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49cc004d00 0x7f49cc005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5400640 1 -- 172.21.15.55:0/2800592435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc003520 con 0x7f49cc003f70 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e5e00640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e815d5e0 0x7f49cc003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.827+0000 7f49e6800640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49cc004d00 0x7f49cc005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.828+0000 7f49e5400640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49cc003f70 0x7f49cc0047c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c240 tx=0x7f49cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e22929c273e27c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.828+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4005ad0 con 0x7f49cc003f70 2024-05-10T03:35:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.828+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d402d070 con 0x7f49cc003f70 2024-05-10T03:35:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e4a00640 1 -- 172.21.15.55:0/2800592435 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4005e70 con 0x7f49cc003f70 2024-05-10T03:35:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 -- 172.21.15.55:0/2800592435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49cc003f70 msgr2=0x7f49cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 --2- 172.21.15.55:0/2800592435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49cc003f70 0x7f49cc0047c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c240 tx=0x7f49cc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e6800640 1 -- 172.21.15.55:0/2800592435 reap_dead start 2024-05-10T03:35:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 -- 172.21.15.55:0/2800592435 shutdown_connections 2024-05-10T03:35:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 -- 172.21.15.55:0/2800592435 >> 172.21.15.55:0/2800592435 conn(0x7f49e80b5730 msgr2=0x7f49e80d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 -- 172.21.15.55:0/2800592435 shutdown_connections 2024-05-10T03:35:01.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.829+0000 7f49e7200640 1 -- 172.21.15.55:0/2800592435 wait complete. 2024-05-10T03:35:01.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.832+0000 7f49e7200640 1 Processor -- start 2024-05-10T03:35:01.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.832+0000 7f49e7200640 1 -- start start 2024-05-10T03:35:01.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49cc003f70 0x7f49e80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e8150510 0x7f49e815f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80c8300 con 0x7f49e815f660 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80c9a90 con 0x7f49e8150510 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e81f6a10 con 0x7f49cc003f70 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41320/0 (socket says 172.21.15.55:41320) 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e6800640 1 -- 172.21.15.55:0/2448786512 learned_addr learned my addr 172.21.15.55:0/2448786512 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.834+0000 7f49e5e00640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49cc003f70 0x7f49e80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e5400640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e8150510 0x7f49e815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 -- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49cc003f70 msgr2=0x7f49e80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49cc003f70 0x7f49e80cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 -- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e8150510 msgr2=0x7f49e815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e8150510 0x7f49e815f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 -- 172.21.15.55:0/2448786512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e815fa70 con 0x7f49e815f660 2024-05-10T03:35:01.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e6800640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f49e806c220 tx=0x7f49d8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004500 con 0x7f49e815f660 2024-05-10T03:35:01.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8003610 con 0x7f49e815f660 2024-05-10T03:35:01.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80039a0 con 0x7f49e815f660 2024-05-10T03:35:01.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e81f6760 con 0x7f49e815f660 2024-05-10T03:35:01.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.835+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e81f6f40 con 0x7f49e815f660 2024-05-10T03:35:01.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.837+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e804e290 con 0x7f49e815f660 2024-05-10T03:35:01.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.838+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f49d8003b00 con 0x7f49e815f660 2024-05-10T03:35:01.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.839+0000 7f49de000640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 0x7f49c806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:01.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.839+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f49d80898e0 con 0x7f49e815f660 2024-05-10T03:35:01.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.840+0000 7f49e5e00640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 0x7f49c806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:01.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.841+0000 7f49e5e00640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 0x7f49c806d110 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f49e0001e60 tx=0x7f49e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:01.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:01.844+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49d8054a20 con 0x7f49e815f660 2024-05-10T03:35:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.085+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49e80d1510 con 0x7f49e815f660 2024-05-10T03:35:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.086+0000 7f49de000640 1 -- 172.21.15.55:0/2448786512 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4730 (secure 0 0 0) 0x7f49d8019090 con 0x7f49e815f660 2024-05-10T03:35:02.090 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:02.090 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/49138572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":49138572},{"type":"v1","addr":"172.21.15.162:6835","nonce":49138572}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/476611421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":476611421},{"type":"v1","addr":"172.21.15.162:6837","nonce":476611421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":7356,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2254900234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2254900234},{"type":"v1","addr":"172.21.15.55:6833","nonce":2254900234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:34:58.520498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/315649334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":315649334},{"type":"v1","addr":"172.21.15.55:6835","nonce":315649334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:02.090 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 55 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 msgr2=0x7f49c806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 0x7f49c806d110 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f49e0001e60 tx=0x7f49e0009040 comp rx=0 tx=0).stop 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 msgr2=0x7f49e80c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f49e806c220 tx=0x7f49d8011920 comp rx=0 tx=0).stop 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 shutdown_connections 2024-05-10T03:35:02.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49e815f660 0x7f49e80c6bd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49c806ac00 0x7f49c806d110 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e8150510 0x7f49e815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 --2- 172.21.15.55:0/2448786512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49cc003f70 0x7f49e80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 >> 172.21.15.55:0/2448786512 conn(0x7f49e80b5730 msgr2=0x7f49e80cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 shutdown_connections 2024-05-10T03:35:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.089+0000 7f49e7200640 1 -- 172.21.15.55:0/2448786512 wait complete. 2024-05-10T03:35:02.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:34:58.520498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7359}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7359': {'gid': 7359, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/315649334', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 315649334}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 315649334}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:02.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:35:02.524 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T03:35:02.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.814+0000 7fef81e00640 1 Processor -- start 2024-05-10T03:35:02.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.814+0000 7fef81e00640 1 -- start start 2024-05-10T03:35:02.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06d230 0x7fef7c0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fef7c06ebc0 0x7fef7c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c06d640 con 0x7fef7c06a110 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c0b40d0 con 0x7fef7c06ebc0 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c06e160 con 0x7fef7c06d230 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef81400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fef7c06ebc0 0x7fef7c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef7be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fef7c06a110 0x7fef7c06a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39928/0 (socket says 172.21.15.55:39928) 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.816+0000 7fef7be00640 1 -- 172.21.15.55:0/3383339394 learned_addr learned my addr 172.21.15.55:0/3383339394 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.817+0000 7fef80a00640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06d230 0x7fef7c0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.817+0000 7fef81400640 1 -- 172.21.15.55:0/3383339394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06d230 msgr2=0x7fef7c0b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.817+0000 7fef81400640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06d230 0x7fef7c0b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.817+0000 7fef81400640 1 -- 172.21.15.55:0/3383339394 >> v1:172.21.15.55:6789/0 conn(0x7fef7c06a110 legacy=0x7fef7c06a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.817+0000 7fef81400640 1 -- 172.21.15.55:0/3383339394 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c06d8c0 con 0x7fef7c06ebc0 2024-05-10T03:35:02.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.818+0000 7fef81400640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06ebc0 0x7fef7c06efd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d1f7210c289eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.818+0000 7fef7b400640 1 -- 172.21.15.55:0/3383339394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7402adb0 con 0x7fef7c06ebc0 2024-05-10T03:35:02.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.818+0000 7fef7b400640 1 -- 172.21.15.55:0/3383339394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74032440 con 0x7fef7c06ebc0 2024-05-10T03:35:02.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.818+0000 7fef7b400640 1 -- 172.21.15.55:0/3383339394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef740327d0 con 0x7fef7c06ebc0 2024-05-10T03:35:02.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 -- 172.21.15.55:0/3383339394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06ebc0 msgr2=0x7fef7c06efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06ebc0 0x7fef7c06efd0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a060 comp rx=0 tx=0).stop 2024-05-10T03:35:02.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 -- 172.21.15.55:0/3383339394 shutdown_connections 2024-05-10T03:35:02.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06ebc0 0x7fef7c06efd0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 --2- 172.21.15.55:0/3383339394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06d230 0x7fef7c0b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 -- 172.21.15.55:0/3383339394 >> 172.21.15.55:0/3383339394 conn(0x7fef7c071d60 msgr2=0x7fef7c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 -- 172.21.15.55:0/3383339394 shutdown_connections 2024-05-10T03:35:02.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.819+0000 7fef81e00640 1 -- 172.21.15.55:0/3383339394 wait complete. 2024-05-10T03:35:02.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.822+0000 7fef81e00640 1 Processor -- start 2024-05-10T03:35:02.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.822+0000 7fef81e00640 1 -- start start 2024-05-10T03:35:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef7c06a110 0x7fef7c13f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06ebc0 0x7fef7c140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c13fd50 con 0x7fef7c06d230 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c142480 con 0x7fef7c06a110 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.823+0000 7fef81e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c150af0 con 0x7fef7c06ebc0 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41346/0 (socket says 172.21.15.55:41346) 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 -- 172.21.15.55:0/2882922167 learned_addr learned my addr 172.21.15.55:0/2882922167 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef81400640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06ebc0 0x7fef7c140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 -- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06ebc0 msgr2=0x7fef7c140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06ebc0 0x7fef7c140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 -- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef7c06a110 msgr2=0x7fef7c13f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef80a00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef7c06a110 0x7fef7c13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef7c06a110 0x7fef7c13f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.824+0000 7fef7be00640 1 -- 172.21.15.55:0/2882922167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009400 con 0x7fef7c06d230 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef7be00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fef6400caa0 tx=0x7fef6400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef78c00640 1 -- 172.21.15.55:0/2882922167 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400ac00 con 0x7fef7c06d230 2024-05-10T03:35:02.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef78c00640 1 -- 172.21.15.55:0/2882922167 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400ad60 con 0x7fef7c06d230 2024-05-10T03:35:02.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef78c00640 1 -- 172.21.15.55:0/2882922167 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64012620 con 0x7fef7c06d230 2024-05-10T03:35:02.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c150d40 con 0x7fef7c06d230 2024-05-10T03:35:02.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.825+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1536c0 con 0x7fef7c06d230 2024-05-10T03:35:02.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.828+0000 7fef78c00640 1 -- 172.21.15.55:0/2882922167 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fef64007070 con 0x7fef7c06d230 2024-05-10T03:35:02.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.829+0000 7fef78c00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 0x7fef5806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.829+0000 7fef78c00640 1 -- 172.21.15.55:0/2882922167 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef64089470 con 0x7fef7c06d230 2024-05-10T03:35:02.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.829+0000 7fef80a00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 0x7fef5806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.829+0000 7fef8a372740 1 Processor -- start 2024-05-10T03:35:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.830+0000 7fef8a372740 1 -- start start 2024-05-10T03:35:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef80a00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 0x7fef5806d110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b7c0 tx=0x7fef6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa3f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa31ba70 con 0x5622fa013270 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa319200 con 0x5622f9fea3b0 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622fa314af0 con 0x5622f9ff2590 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.831+0000 7fef80a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41350/0 (socket says 172.21.15.55:41350) 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 -- 172.21.15.55:0/1620855958 learned_addr learned my addr 172.21.15.55:0/1620855958 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef7be00640 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa3f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef81400640 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 -- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 msgr2=0x5622fa3f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 -- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 msgr2=0x5622fa3f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa3f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 -- 172.21.15.55:0/1620855958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622fa2dfb90 con 0x5622fa013270 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.832+0000 7fef80a00640 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00db90 tx=0x7fef6c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c90eaa449f7148a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.833+0000 7fef71200640 1 -- 172.21.15.55:0/1620855958 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00b050 con 0x5622fa013270 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.833+0000 7fef71200640 1 -- 172.21.15.55:0/1620855958 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c017070 con 0x5622fa013270 2024-05-10T03:35:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.833+0000 7fef71200640 1 -- 172.21.15.55:0/1620855958 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c012e20 con 0x5622fa013270 2024-05-10T03:35:02.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.833+0000 7fef8a372740 1 -- 172.21.15.55:0/1620855958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 msgr2=0x5622fa3dccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.833+0000 7fef8a372740 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00db90 tx=0x7fef6c010000 comp rx=0 tx=0).stop 2024-05-10T03:35:02.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 -- 172.21.15.55:0/1620855958 shutdown_connections 2024-05-10T03:35:02.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa3f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 --2- 172.21.15.55:0/1620855958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3dccd0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 -- 172.21.15.55:0/1620855958 >> 172.21.15.55:0/1620855958 conn(0x5622fa0073c0 msgr2=0x5622fa3b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 -- 172.21.15.55:0/1620855958 shutdown_connections 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.834+0000 7fef8a372740 1 -- 172.21.15.55:0/1620855958 wait complete. 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.835+0000 7fef8a372740 1 -- start start 2024-05-10T03:35:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.835+0000 7fef8a372740 1 Processor -- start 2024-05-10T03:35:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa40a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa2b6220 con 0x5622fa013270 2024-05-10T03:35:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa2b5ec0 con 0x5622f9ff2590 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622fa07bf20 con 0x5622f9fea3b0 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef8a372740 10 client.0 _ll_register_callbacks cb 0x5622fa30e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41356/0 (socket says 172.21.15.55:41356) 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 -- 172.21.15.55:0/3957162349 learned_addr learned my addr 172.21.15.55:0/3957162349 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef80a00640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 -- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 msgr2=0x5622fa3df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 -- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 msgr2=0x5622fa40a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa40a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef81400640 1 -- 172.21.15.55:0/3957162349 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c00d840 con 0x5622fa013270 2024-05-10T03:35:02.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.837+0000 7fef80a00640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef81400640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fef74009890 tx=0x7fef74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5cdcb1ebc89ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef69600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7403c440 con 0x5622fa013270 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622fa33ec90 con 0x5622fa013270 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef7403c5a0 con 0x5622fa013270 2024-05-10T03:35:02.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.838+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7403c760 con 0x5622fa013270 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 10 client.7491 fetch_fsmap learned FSMap version 55 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5622fa313170 con 0x5622fa013270 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5622fa33ec90 con 0x5622fa013270 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 55) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fef7403c8c0 con 0x5622fa013270 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 10 client.7491 fetch_fsmap finished waiting for FSMap version 55 2024-05-10T03:35:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 10 client.7491 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 20 client.7491 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622f9fd1e40 0x5622fa3ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 4 client.7491 mds_command: new command op to 7359 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 --> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622fa0083c0 con 0x5622f9fd1e40 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.839+0000 7fef80a00640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622f9fd1e40 0x5622fa3ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.840+0000 7fef80a00640 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622f9fd1e40 0x5622fa3ef440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e48c3253cec491 server_cookie=b8ce4c4bfcafbcb9 in_seq=0 out_seq=0 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.840+0000 7fef69600640 10 client.7491 ms_handle_connect on v2:172.21.15.55:6834/315649334 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef69600640 1 -- 172.21.15.55:0/3957162349 <== mds.0 v2:172.21.15.55:6834/315649334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622fa0083c0 con 0x5622f9fd1e40 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef69600640 10 client.7491 handle_command_reply: tid=0 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 client.7491 shutdown 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 msgr2=0x5622fa3b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fef74009890 tx=0x7fef74004730 comp rx=0 tx=0).stop 2024-05-10T03:35:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 shutdown_connections 2024-05-10T03:35:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622f9fd1e40 0x5622fa3ef440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b0fc0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa40a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 --2- 172.21.15.55:0/3957162349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef69600640 0 client.7491 ms_handle_reset on v2:172.21.15.55:6834/315649334 2024-05-10T03:35:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 >> 172.21.15.55:0/3957162349 conn(0x5622fa0073c0 msgr2=0x5622fa3b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 shutdown_connections 2024-05-10T03:35:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.841+0000 7fef8a372740 1 -- 172.21.15.55:0/3957162349 wait complete. 2024-05-10T03:35:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.842+0000 7fef8a372740 20 client.7491 trim_cache size 0 max 16384 2024-05-10T03:35:02.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.859+0000 7fef8a372740 1 Processor -- start 2024-05-10T03:35:02.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.859+0000 7fef8a372740 1 -- start start 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa409c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa3ee690 con 0x5622fa013270 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa40a900 con 0x5622f9fea3b0 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622fa400d20 con 0x5622f9ff2590 2024-05-10T03:35:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38204/0 (socket says 172.21.15.55:38204) 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.861+0000 7fef81400640 1 -- 172.21.15.55:0/3017133493 learned_addr learned my addr 172.21.15.55:0/3017133493 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef7be00640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef80a00640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa409c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 -- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 msgr2=0x5622fa3df7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3df7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 -- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 msgr2=0x5622fa409c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa409c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 -- 172.21.15.55:0/3017133493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622fa348130 con 0x5622fa013270 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef80a00640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa409c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef81400640 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fef74016030 tx=0x7fef7403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee35b46256e8bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef71c00640 1 -- 172.21.15.55:0/3017133493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74006e10 con 0x5622fa013270 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef71c00640 1 -- 172.21.15.55:0/3017133493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7403b3a0 con 0x5622fa013270 2024-05-10T03:35:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.862+0000 7fef71c00640 1 -- 172.21.15.55:0/3017133493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74004280 con 0x5622fa013270 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 -- 172.21.15.55:0/3017133493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 msgr2=0x5622fa3b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fef74016030 tx=0x7fef7403b000 comp rx=0 tx=0).stop 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 -- 172.21.15.55:0/3017133493 shutdown_connections 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3b3be0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9ff2590 0x5622fa3df7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 --2- 172.21.15.55:0/3017133493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9fea3b0 0x5622fa409c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 -- 172.21.15.55:0/3017133493 >> 172.21.15.55:0/3017133493 conn(0x5622fa0073c0 msgr2=0x5622fa3f0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 -- 172.21.15.55:0/3017133493 shutdown_connections 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.863+0000 7fef8a372740 1 -- 172.21.15.55:0/3017133493 wait complete. 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.864+0000 7fef8a372740 1 -- start start 2024-05-10T03:35:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.865+0000 7fef8a372740 1 Processor -- start 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa3ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa4025d0 con 0x5622fa013270 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622fa3f94c0 con 0x5622f9ff2590 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622fa3f9950 con 0x5622f9fea3b0 2024-05-10T03:35:02.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.866+0000 7fef8a372740 10 client.0 _ll_register_callbacks cb 0x5622fa3fa580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38208/0 (socket says 172.21.15.55:38208) 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 -- 172.21.15.55:0/3376875916 learned_addr learned my addr 172.21.15.55:0/3376875916 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef7be00640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa3ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef80a00640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 -- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 msgr2=0x5622fa3de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 -- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 msgr2=0x5622fa3ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa3ede80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.867+0000 7fef81400640 1 -- 172.21.15.55:0/3376875916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622fa313170 con 0x5622fa013270 2024-05-10T03:35:02.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef81400640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fef74050040 tx=0x7fef74009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525095b407e7bd10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef69600640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74004510 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fef74000e40 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef6c017070 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7403c440 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef8a372740 10 client.7497 fetch_fsmap learned FSMap version 55 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.868+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5622fa2b5430 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c017070 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 55) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fef74054030 con 0x5622fa013270 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 10 client.7497 fetch_fsmap finished waiting for FSMap version 55 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 10 client.7497 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 20 client.7497 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622fa409c70 0x5622fa3e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 4 client.7497 mds_command: new command op to 7359 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:35:02.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 --> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5622fa0196c0 con 0x5622fa409c70 2024-05-10T03:35:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.869+0000 7fef80a00640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622fa409c70 0x5622fa3e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef80a00640 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622fa409c70 0x5622fa3e07f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2a02516155b7dc server_cookie=9af85987e19ce3d1 in_seq=0 out_seq=0 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef69600640 10 client.7497 ms_handle_connect on v2:172.21.15.55:6834/315649334 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef69600640 1 -- 172.21.15.55:0/3376875916 <== mds.0 v2:172.21.15.55:6834/315649334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5622fa0196c0 con 0x5622fa409c70 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef69600640 10 client.7497 handle_command_reply: tid=0 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 client.7497 shutdown 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 msgr2=0x5622fa3e02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fef74050040 tx=0x7fef74009190 comp rx=0 tx=0).stop 2024-05-10T03:35:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 shutdown_connections 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:6834/315649334,v1:172.21.15.55:6835/315649334] conn(0x5622fa409c70 0x5622fa3e07f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5622fa013270 0x5622fa3e02b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5622f9ff2590 0x5622fa3ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 --2- 172.21.15.55:0/3376875916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5622f9fea3b0 0x5622fa3de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef69600640 0 client.7497 ms_handle_reset on v2:172.21.15.55:6834/315649334 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 >> 172.21.15.55:0/3376875916 conn(0x5622fa0073c0 msgr2=0x5622fa3f0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.870+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 shutdown_connections 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef8a372740 1 -- 172.21.15.55:0/3376875916 wait complete. 2024-05-10T03:35:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef8a372740 20 client.7497 trim_cache size 0 max 16384 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7359, 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.8062254089999996, 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 55, 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 104, 2024-05-10T03:35:02.879 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 104, 2024-05-10T03:35:02.880 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.556584258999999 2024-05-10T03:35:02.880 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 msgr2=0x7fef5806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 0x7fef5806d110 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b7c0 tx=0x7fef6c008040 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 msgr2=0x7fef7c13ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fef6400caa0 tx=0x7fef6400cf60 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 shutdown_connections 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c06ebc0 0x7fef7c140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef7c06d230 0x7fef7c13ffc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef5806ac00 0x7fef5806d110 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 --2- 172.21.15.55:0/2882922167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef7c06a110 0x7fef7c13f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.871+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 >> 172.21.15.55:0/2882922167 conn(0x7fef7c071d60 msgr2=0x7fef7c1564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.872+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 shutdown_connections 2024-05-10T03:35:02.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:02.872+0000 7fef81e00640 1 -- 172.21.15.55:0/2882922167 wait complete. 2024-05-10T03:35:02.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:35:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.188+0000 7fe751a00640 1 Processor -- start 2024-05-10T03:35:03.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.189+0000 7fe751a00640 1 -- start start 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe751a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 0x7fe74c0715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe751a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe74c15af40 0x7fe74c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe751a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c0cd710 con 0x7fe74c0cdba0 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe751a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c0cdfb0 con 0x7fe74c15af40 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe751a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c06b4d0 con 0x7fe74c15ab30 2024-05-10T03:35:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.191+0000 7fe74aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe74c0cdba0 0x7fe74c06a9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51674/0 (socket says 172.21.15.55:51674) 2024-05-10T03:35:03.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74aa00640 1 -- 172.21.15.55:0/1786790126 learned_addr learned my addr 172.21.15.55:0/1786790126 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:03.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74b400640 1 --2- 172.21.15.55:0/1786790126 >> v2:172.21.15.55:3300/0 conn(0x7fe74c15af40 0x7fe74c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe743e00640 1 --2- 172.21.15.55:0/1786790126 >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 0x7fe74c0715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74a000640 1 -- 172.21.15.55:0/1786790126 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852963273 0 0) 0x7fe74c0cd710 con 0x7fe74c0cdba0 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74a000640 1 -- 172.21.15.55:0/1786790126 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe730003520 con 0x7fe74c0cdba0 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74b400640 1 -- 172.21.15.55:0/1786790126 >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 msgr2=0x7fe74c0715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74b400640 1 --2- 172.21.15.55:0/1786790126 >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 0x7fe74c0715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74b400640 1 -- 172.21.15.55:0/1786790126 >> v1:172.21.15.55:6789/0 conn(0x7fe74c0cdba0 legacy=0x7fe74c06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe74b400640 1 -- 172.21.15.55:0/1786790126 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c06b1f0 con 0x7fe74c15af40 2024-05-10T03:35:03.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.192+0000 7fe743e00640 1 --2- 172.21.15.55:0/1786790126 >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 0x7fe74c0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.194+0000 7fe74b400640 1 --2- 172.21.15.55:0/1786790126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c071b20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0cb800 tx=0x7fe74402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def996da2c5b219 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.194+0000 7fe74a000640 1 -- 172.21.15.55:0/1786790126 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744007740 con 0x7fe74c15af40 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.194+0000 7fe74a000640 1 -- 172.21.15.55:0/1786790126 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74402d050 con 0x7fe74c15af40 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.194+0000 7fe74a000640 1 -- 172.21.15.55:0/1786790126 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744007aa0 con 0x7fe74c15af40 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 -- 172.21.15.55:0/1786790126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 msgr2=0x7fe74c071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 --2- 172.21.15.55:0/1786790126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c071b20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0cb800 tx=0x7fe74402a560 comp rx=0 tx=0).stop 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 -- 172.21.15.55:0/1786790126 shutdown_connections 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 --2- 172.21.15.55:0/1786790126 >> v2:172.21.15.162:3300/0 conn(0x7fe74c15ab30 0x7fe74c0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 --2- 172.21.15.55:0/1786790126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c071b20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 -- 172.21.15.55:0/1786790126 >> 172.21.15.55:0/1786790126 conn(0x7fe74c0cbb90 msgr2=0x7fe74c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 -- 172.21.15.55:0/1786790126 shutdown_connections 2024-05-10T03:35:03.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.195+0000 7fe751a00640 1 -- 172.21.15.55:0/1786790126 wait complete. 2024-05-10T03:35:03.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.197+0000 7fe751a00640 1 Processor -- start 2024-05-10T03:35:03.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.198+0000 7fe751a00640 1 -- start start 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe74c0cdba0 0x7fe74c146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe74c15ab30 0x7fe74c13e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c145360 con 0x7fe74c15af40 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c151ee0 con 0x7fe74c0cdba0 2024-05-10T03:35:03.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe751a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c143b90 con 0x7fe74c15ab30 2024-05-10T03:35:03.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe74b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.199+0000 7fe74b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38230/0 (socket says 172.21.15.55:38230) 2024-05-10T03:35:03.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 -- 172.21.15.55:0/585531844 learned_addr learned my addr 172.21.15.55:0/585531844 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:03.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe743e00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe74c15ab30 0x7fe74c13e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74aa00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe74c0cdba0 0x7fe74c146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 -- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe74c15ab30 msgr2=0x7fe74c13e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe74c15ab30 0x7fe74c13e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 -- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe74c0cdba0 msgr2=0x7fe74c146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe74c0cdba0 0x7fe74c146f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 -- 172.21.15.55:0/585531844 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009a70 con 0x7fe74c15af40 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.200+0000 7fe74b400640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe74402ab50 tx=0x7fe744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.201+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74403b820 con 0x7fe74c15af40 2024-05-10T03:35:03.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.201+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74403b980 con 0x7fe74c15af40 2024-05-10T03:35:03.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.201+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74403bd10 con 0x7fe74c15af40 2024-05-10T03:35:03.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.201+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c152130 con 0x7fe74c15af40 2024-05-10T03:35:03.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.201+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c1526f0 con 0x7fe74c15af40 2024-05-10T03:35:03.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.202+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c066f60 con 0x7fe74c15af40 2024-05-10T03:35:03.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.203+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe744031400 con 0x7fe74c15af40 2024-05-10T03:35:03.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.204+0000 7fe742a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 0x7fe72806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:03.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.204+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe7440aada0 con 0x7fe74c15af40 2024-05-10T03:35:03.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.205+0000 7fe74aa00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 0x7fe72806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:03.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.206+0000 7fe74aa00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 0x7fe72806d110 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe73c005880 tx=0x7fe73c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:03.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.209+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe744075ec0 con 0x7fe74c15af40 2024-05-10T03:35:03.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.456+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe74c067220 con 0x7fe74c15af40 2024-05-10T03:35:03.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.457+0000 7fe742a00640 1 -- 172.21.15.55:0/585531844 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe744049060 con 0x7fe74c15af40 2024-05-10T03:35:03.460 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.459+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 msgr2=0x7fe72806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 0x7fe72806d110 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe73c005880 tx=0x7fe73c009040 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 msgr2=0x7fe74c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe74402ab50 tx=0x7fe744004a40 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 shutdown_connections 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe74c15af40 0x7fe74c158d30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe74c15ab30 0x7fe74c13e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe72806ac00 0x7fe72806d110 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 --2- 172.21.15.55:0/585531844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe74c0cdba0 0x7fe74c146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 >> 172.21.15.55:0/585531844 conn(0x7fe74c0cbb90 msgr2=0x7fe74c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 shutdown_connections 2024-05-10T03:35:03.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:03.460+0000 7fe751a00640 1 -- 172.21.15.55:0/585531844 wait complete. 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:35:03.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:03.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.494 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:35:03.549 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:35:03.549 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:35:03.549 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:35:03.550 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.550 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:35:03.550 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.550 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:03.550 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:03.551 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42148sec preferred_lft 42148sec 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:03.552 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:03.553 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:35:03.553 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:35:03.553 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:03.553 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:35:03.553 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.554 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:35:03.554 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.554 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:03.554 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:35:03.554 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:35:03.555 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.555 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:03.555 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:35:03.556 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:35:03.556 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:03.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.556 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:35:03.642 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:35:03.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.647 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:35:03.702 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:35:03.702 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:35:03.702 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:35:03.702 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:35:03.702 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:35:03.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:35:03.703 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:35:03.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.864 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:35:03.916 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:35:03.916 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:35:03.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:03.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:03.918 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:35:04.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:04.007 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:04.059 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:04.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:04.059 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:04.115 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:35:04.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:04.115 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:04.329 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:04.329 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:04.329 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:04.358 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:04.358 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:04.358 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:04.358 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:04.358 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:35:04.401 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:35:04.404 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:35:04.404 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:35:04.404 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:35:04.404 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:35:04.404 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:35:04.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:35:04.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:04.408 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:04.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:04.483 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:35:04.548 INFO:teuthology.orchestra.run.smithi055.stdout:client5705 2024-05-10T03:35:04.548 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:04.548 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:04.598 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:35:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.926+0000 7f71c7200640 1 Processor -- start 2024-05-10T03:35:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.926+0000 7f71c7200640 1 -- start start 2024-05-10T03:35:04.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.928+0000 7f71c7200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71c015d5e0 0x7f71c0168510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:04.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.928+0000 7f71c7200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c006bac0 con 0x7f71c00b6680 2024-05-10T03:35:04.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.928+0000 7f71c7200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c0161fc0 con 0x7f71c00b2170 2024-05-10T03:35:04.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.928+0000 7f71c7200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c00d2b20 con 0x7f71c015d5e0 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c6800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f71c015d5e0 0x7f71c0168510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c5e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f71c00b6680 0x7f71c01611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51678/0 (socket says 172.21.15.55:51678) 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c5e00640 1 -- 172.21.15.55:0/2566474049 learned_addr learned my addr 172.21.15.55:0/2566474049 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c6800640 1 -- 172.21.15.55:0/2566474049 >> v1:172.21.15.162:6789/0 conn(0x7f71c00b2170 legacy=0x7f71c01618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c6800640 1 -- 172.21.15.55:0/2566474049 >> v1:172.21.15.55:6789/0 conn(0x7f71c00b6680 legacy=0x7f71c01611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:04.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.929+0000 7f71c6800640 1 -- 172.21.15.55:0/2566474049 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00d25a0 con 0x7f71c015d5e0 2024-05-10T03:35:04.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.931+0000 7f71c6800640 1 --2- 172.21.15.55:0/2566474049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0168510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009d90 tx=0x7f71bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedc0ba025920365 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:04.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.931+0000 7f71c4a00640 1 -- 172.21.15.55:0/2566474049 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007740 con 0x7f71c015d5e0 2024-05-10T03:35:04.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.931+0000 7f71c4a00640 1 -- 172.21.15.55:0/2566474049 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc02d050 con 0x7f71c015d5e0 2024-05-10T03:35:04.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.931+0000 7f71c4a00640 1 -- 172.21.15.55:0/2566474049 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007ac0 con 0x7f71c015d5e0 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 -- 172.21.15.55:0/2566474049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 msgr2=0x7f71c0168510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 --2- 172.21.15.55:0/2566474049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0168510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009d90 tx=0x7f71bc02a6f0 comp rx=0 tx=0).stop 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 -- 172.21.15.55:0/2566474049 shutdown_connections 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 --2- 172.21.15.55:0/2566474049 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0168510 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 -- 172.21.15.55:0/2566474049 >> 172.21.15.55:0/2566474049 conn(0x7f71c00b5730 msgr2=0x7f71c00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 -- 172.21.15.55:0/2566474049 shutdown_connections 2024-05-10T03:35:04.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.932+0000 7f71c7200640 1 -- 172.21.15.55:0/2566474049 wait complete. 2024-05-10T03:35:04.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.935+0000 7f71c7200640 1 Processor -- start 2024-05-10T03:35:04.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.936+0000 7f71c7200640 1 -- start start 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c00b2170 0x7f71c01523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71c00b6680 0x7f71c01553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0142a30 con 0x7f71c015d5e0 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0152940 con 0x7f71c00b2170 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c7200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c015ff00 con 0x7f71c00b6680 2024-05-10T03:35:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c6800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38254/0 (socket says 172.21.15.55:38254) 2024-05-10T03:35:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.938+0000 7f71c6800640 1 -- 172.21.15.55:0/356921586 learned_addr learned my addr 172.21.15.55:0/356921586 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c5e00640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c00b2170 0x7f71c01523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 -- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71c00b6680 msgr2=0x7f71c01553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71c00b6680 0x7f71c01553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 -- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c00b2170 msgr2=0x7f71c01523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c00b2170 0x7f71c01523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 -- 172.21.15.55:0/356921586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009a70 con 0x7f71c015d5e0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71c6800640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009eb0 tx=0x7f71bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.939+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc031910 con 0x7f71c015d5e0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.940+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc031a70 con 0x7f71c015d5e0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.940+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc031de0 con 0x7f71c015d5e0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.940+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c0152cf0 con 0x7f71c015d5e0 2024-05-10T03:35:04.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.940+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c015fb90 con 0x7f71c015d5e0 2024-05-10T03:35:04.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.941+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c0153210 con 0x7f71c015d5e0 2024-05-10T03:35:04.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.942+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f71bc044460 con 0x7f71c015d5e0 2024-05-10T03:35:04.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.943+0000 7f71ba000640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 0x7f71a406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:04.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.943+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f71bc0aac40 con 0x7f71c015d5e0 2024-05-10T03:35:04.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.944+0000 7f71c5e00640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 0x7f71a406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:04.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.945+0000 7f71c5e00640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 0x7f71a406d110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005880 tx=0x7f71b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:04.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:04.948+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f71c0153210 con 0x7f71c015d5e0 2024-05-10T03:35:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.187+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f71c015f9f0 con 0x7f71c015d5e0 2024-05-10T03:35:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.188+0000 7f71ba000640 1 -- 172.21.15.55:0/356921586 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f71c015f9f0 con 0x7f71c015d5e0 2024-05-10T03:35:05.192 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.190+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 msgr2=0x7f71a406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.190+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 0x7f71a406d110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005880 tx=0x7f71b4009040 comp rx=0 tx=0).stop 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.190+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 msgr2=0x7f71c0160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.190+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009eb0 tx=0x7f71bc004a40 comp rx=0 tx=0).stop 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 shutdown_connections 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71c015d5e0 0x7f71c0160390 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f71c00b6680 0x7f71c01553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f71a406ac00 0x7f71a406d110 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 --2- 172.21.15.55:0/356921586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71c00b2170 0x7f71c01523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:05.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 >> 172.21.15.55:0/356921586 conn(0x7f71c00b5730 msgr2=0x7f71c006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:05.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 shutdown_connections 2024-05-10T03:35:05.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:05.191+0000 7f71c7200640 1 -- 172.21.15.55:0/356921586 wait complete. 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:35:05.225 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:05.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.226 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:35:05.286 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:35:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:35:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:35:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42171sec preferred_lft 42171sec 2024-05-10T03:35:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:35:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.288 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:35:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:05.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.291 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:35:05.374 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:35:05.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.378 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:35:05.430 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:35:05.430 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:35:05.430 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:35:05.430 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:35:05.430 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:35:05.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:35:05.431 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:35:05.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.577 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:35:05.627 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:35:05.627 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:35:05.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:05.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:35:05.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:05.705 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:05.758 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:05.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:05.758 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:05.806 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:35:05.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:05.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:06.003 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:06.003 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:06.003 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:06.032 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:06.032 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:06.032 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:06.032 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:06.032 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:35:06.075 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:35:06.078 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:35:06.078 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:35:06.078 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:35:06.078 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:35:06.078 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:35:06.083 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:35:06.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:06.083 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:06.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:06.164 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:35:06.219 INFO:teuthology.orchestra.run.smithi162.stdout:client7518 2024-05-10T03:35:06.219 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:06.219 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:06.269 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:06.269 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:06.331 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:06.331 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:06.391 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:06.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.679+0000 7fd2d5200640 1 Processor -- start 2024-05-10T03:35:06.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.679+0000 7fd2d5200640 1 -- start start 2024-05-10T03:35:06.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2d5200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2d0144710 0x7fd2d0144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2d5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2d5200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d006d350 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2d5200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d006c740 con 0x7fd2d0144710 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2d5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d006ca20 con 0x7fd2d006d7e0 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2d0144710 0x7fd2d0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2d0144710 0x7fd2d0144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38270/0 (socket says 172.21.15.55:38270) 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2cfe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd2d00b2fc0 0x7fd2d015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51680/0 (socket says 172.21.15.55:51680) 2024-05-10T03:35:06.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.681+0000 7fd2cf400640 1 -- 172.21.15.55:0/3725650804 learned_addr learned my addr 172.21.15.55:0/3725650804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cea00640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cf400640 1 -- 172.21.15.55:0/3725650804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 msgr2=0x7fd2d00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cf400640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cf400640 1 -- 172.21.15.55:0/3725650804 >> v1:172.21.15.55:6789/0 conn(0x7fd2d00b2fc0 legacy=0x7fd2d015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cf400640 1 -- 172.21.15.55:0/3725650804 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d006cd00 con 0x7fd2d0144710 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2ce000640 1 -- 172.21.15.55:0/3725650804 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258041977 0 0) 0x7fd2d006d350 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.682+0000 7fd2cea00640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.683+0000 7fd2cf400640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d0144710 0x7fd2d0144120 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009870 tx=0x7fd2c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5f3a2c584c4834 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.683+0000 7fd2ce000640 1 -- 172.21.15.55:0/3725650804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2d0144710 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.683+0000 7fd2ce000640 1 -- 172.21.15.55:0/3725650804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00325a0 con 0x7fd2d0144710 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.683+0000 7fd2ce000640 1 -- 172.21.15.55:0/3725650804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032930 con 0x7fd2d0144710 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 -- 172.21.15.55:0/3725650804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d0144710 msgr2=0x7fd2d0144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d0144710 0x7fd2d0144120 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009870 tx=0x7fd2c002a1c0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 -- 172.21.15.55:0/3725650804 shutdown_connections 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 --2- 172.21.15.55:0/3725650804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d0144710 0x7fd2d0144120 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 -- 172.21.15.55:0/3725650804 >> 172.21.15.55:0/3725650804 conn(0x7fd2d006a110 msgr2=0x7fd2d006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 -- 172.21.15.55:0/3725650804 shutdown_connections 2024-05-10T03:35:06.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.684+0000 7fd2d5200640 1 -- 172.21.15.55:0/3725650804 wait complete. 2024-05-10T03:35:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.686+0000 7fd2d5200640 1 Processor -- start 2024-05-10T03:35:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.687+0000 7fd2d5200640 1 -- start start 2024-05-10T03:35:06.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2d0144710 0x7fd2d0071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00ad420 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d015b2f0 con 0x7fd2d0144710 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2d5200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01410c0 con 0x7fd2d006d7e0 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.688+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38286/0 (socket says 172.21.15.55:38286) 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 -- 172.21.15.55:0/1880741756 learned_addr learned my addr 172.21.15.55:0/1880741756 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cfe00640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2d0144710 0x7fd2d0071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cf400640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 -- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 msgr2=0x7fd2d015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 -- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2d0144710 msgr2=0x7fd2d0071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2d0144710 0x7fd2d0071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 -- 172.21.15.55:0/1880741756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00094d0 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cf400640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.689+0000 7fd2cea00640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c820 tx=0x7fd2bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.690+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007a30 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.690+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc007b90 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.690+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00ce10 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.690+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d015dc70 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.690+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d015dec0 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.691+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d00ad690 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.692+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd2bc0113f0 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.693+0000 7fd2c7400640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 0x7fd2b006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.693+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd2bc089580 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.694+0000 7fd2cf400640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 0x7fd2b006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:06.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.695+0000 7fd2cf400640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 0x7fd2b006d140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:06.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.698+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd2bc054610 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.898+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2d0066ce0 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.902+0000 7fd2c7400640 1 -- 172.21.15.55:0/1880741756 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16248 (secure 0 0 0) 0x7fd2bc057c30 con 0x7fd2d00b2fc0 2024-05-10T03:35:06.906 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:06.906 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":104,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:56.064514+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:06.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 msgr2=0x7fd2b006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 0x7fd2b006d140 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).stop 2024-05-10T03:35:06.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 msgr2=0x7fd2d015d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c820 tx=0x7fd2bc00cce0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 shutdown_connections 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd2b006ac30 0x7fd2b006d140 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2d0144710 0x7fd2d0071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2d00b2fc0 0x7fd2d015d400 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 --2- 172.21.15.55:0/1880741756 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2d006d7e0 0x7fd2d015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 >> 172.21.15.55:0/1880741756 conn(0x7fd2d006a110 msgr2=0x7fd2d013e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 shutdown_connections 2024-05-10T03:35:06.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:06.905+0000 7fd2d5200640 1 -- 172.21.15.55:0/1880741756 wait complete. 2024-05-10T03:35:06.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:53.261121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:54.084182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:35:06.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:35:07.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.229+0000 7f6e33e00640 1 Processor -- start 2024-05-10T03:35:07.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.229+0000 7f6e33e00640 1 -- start start 2024-05-10T03:35:07.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.231+0000 7f6e33e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e3406b770 0x7f6e3415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e340ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3406b2e0 con 0x7f6e3406a110 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340b62e0 con 0x7f6e3406b770 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415dbc0 con 0x7f6e34161310 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e32a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e3406b770 0x7f6e3415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6e3406a110 0x7f6e340d1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51694/0 (socket says 172.21.15.55:51694) 2024-05-10T03:35:07.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e33400640 1 -- 172.21.15.55:0/2834210981 learned_addr learned my addr 172.21.15.55:0/2834210981 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.232+0000 7f6e2a000640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e340ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e32a00640 1 -- 172.21.15.55:0/2834210981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 msgr2=0x7f6e340ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e32a00640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e340ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e32a00640 1 -- 172.21.15.55:0/2834210981 >> v1:172.21.15.55:6789/0 conn(0x7f6e3406a110 legacy=0x7f6e340d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e32a00640 1 -- 172.21.15.55:0/2834210981 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3406a6d0 con 0x7f6e3406b770 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e32000640 1 -- 172.21.15.55:0/2834210981 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043071556 0 0) 0x7f6e3406b2e0 con 0x7f6e3406a110 2024-05-10T03:35:07.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.233+0000 7f6e2a000640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e340ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:07.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.234+0000 7f6e32a00640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406b770 0x7f6e3415d680 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6e240097a0 tx=0x7f6e2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d3ce4331c995d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.234+0000 7f6e32000640 1 -- 172.21.15.55:0/2834210981 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24032440 con 0x7f6e3406b770 2024-05-10T03:35:07.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.234+0000 7f6e32000640 1 -- 172.21.15.55:0/2834210981 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e240325a0 con 0x7f6e3406b770 2024-05-10T03:35:07.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.234+0000 7f6e32000640 1 -- 172.21.15.55:0/2834210981 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24032930 con 0x7f6e3406b770 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 -- 172.21.15.55:0/2834210981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406b770 msgr2=0x7f6e3415d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406b770 0x7f6e3415d680 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6e240097a0 tx=0x7f6e2402a180 comp rx=0 tx=0).stop 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 -- 172.21.15.55:0/2834210981 shutdown_connections 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e340ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 --2- 172.21.15.55:0/2834210981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406b770 0x7f6e3415d680 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 -- 172.21.15.55:0/2834210981 >> 172.21.15.55:0/2834210981 conn(0x7f6e340d3140 msgr2=0x7f6e340d3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 -- 172.21.15.55:0/2834210981 shutdown_connections 2024-05-10T03:35:07.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.235+0000 7f6e33e00640 1 -- 172.21.15.55:0/2834210981 wait complete. 2024-05-10T03:35:07.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.238+0000 7f6e33e00640 1 Processor -- start 2024-05-10T03:35:07.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.238+0000 7f6e33e00640 1 -- start start 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 0x7f6e34153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e341515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e341f4420 con 0x7f6e3406a110 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34154550 con 0x7f6e3406b770 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.239+0000 7f6e33e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cbc50 con 0x7f6e34161310 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38300/0 (socket says 172.21.15.55:38300) 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 -- 172.21.15.55:0/382185392 learned_addr learned my addr 172.21.15.55:0/382185392 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e2a000640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 0x7f6e34153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e33400640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e341515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 -- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 msgr2=0x7f6e341515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e341515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 -- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 msgr2=0x7f6e34153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 0x7f6e34153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e32a00640 1 -- 172.21.15.55:0/382185392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009400 con 0x7f6e3406a110 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.240+0000 7f6e2a000640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 0x7f6e34153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e32a00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24030b80 con 0x7f6e3406a110 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24030ce0 con 0x7f6e3406a110 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2403d680 con 0x7f6e3406a110 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e341547a0 con 0x7f6e3406a110 2024-05-10T03:35:07.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.241+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e340cb600 con 0x7f6e3406a110 2024-05-10T03:35:07.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.242+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340ced50 con 0x7f6e3406a110 2024-05-10T03:35:07.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.243+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6e24039040 con 0x7f6e3406a110 2024-05-10T03:35:07.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.244+0000 7f6e2b400640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 0x7f6e1006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.244+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6e240aa150 con 0x7f6e3406a110 2024-05-10T03:35:07.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.245+0000 7f6e2a000640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 0x7f6e1006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.246+0000 7f6e2a000640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 0x7f6e1006d140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c006fd0 tx=0x7f6e1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.249+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6e340ced50 con 0x7f6e3406a110 2024-05-10T03:35:07.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.451+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e340ced50 con 0x7f6e3406a110 2024-05-10T03:35:07.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.452+0000 7f6e2b400640 1 -- 172.21.15.55:0/382185392 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e340ced50 con 0x7f6e3406a110 2024-05-10T03:35:07.456 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 msgr2=0x7f6e1006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 0x7f6e1006d140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c006fd0 tx=0x7f6e1c008040 comp rx=0 tx=0).stop 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 msgr2=0x7f6e341f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c00 tx=0x7f6e240046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 shutdown_connections 2024-05-10T03:35:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e34161310 0x7f6e341515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6e1006ac30 0x7f6e1006d140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e3406b770 0x7f6e34153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 --2- 172.21.15.55:0/382185392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e3406a110 0x7f6e341f3bf0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 >> 172.21.15.55:0/382185392 conn(0x7f6e340d3140 msgr2=0x7f6e3415eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.455+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 shutdown_connections 2024-05-10T03:35:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.456+0000 7f6e33e00640 1 -- 172.21.15.55:0/382185392 wait complete. 2024-05-10T03:35:07.489 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:35:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.768+0000 7f49e0400640 1 Processor -- start 2024-05-10T03:35:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.768+0000 7f49e0400640 1 -- start start 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49e0400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f49d806b830 0x7f49d8165560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49e0400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d815b5e0 con 0x7f49d80cbbd0 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49e0400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80b4380 con 0x7f49d80cc3a0 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49e0400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d815f1e0 con 0x7f49d806b830 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49dd000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f49d80cbbd0 0x7f49d806b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51706/0 (socket says 172.21.15.55:51706) 2024-05-10T03:35:07.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.771+0000 7f49dd000640 1 -- 172.21.15.55:0/3370602141 learned_addr learned my addr 172.21.15.55:0/3370602141 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:07.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49de400640 1 --2- 172.21.15.55:0/3370602141 >> v2:172.21.15.162:3300/0 conn(0x7f49d806b830 0x7f49d8165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593177180 0 0) 0x7f49d815b5e0 con 0x7f49d80cbbd0 2024-05-10T03:35:07.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c0005730 con 0x7f49d80cbbd0 2024-05-10T03:35:07.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 354762972 0 0) 0x7f49c0005730 con 0x7f49d80cbbd0 2024-05-10T03:35:07.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 >> v2:172.21.15.162:3300/0 conn(0x7f49d806b830 msgr2=0x7f49d8165560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 --2- 172.21.15.55:0/3370602141 >> v2:172.21.15.162:3300/0 conn(0x7f49d806b830 0x7f49d8165560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 >> v1:172.21.15.162:6789/0 conn(0x7f49d80cc3a0 legacy=0x7f49d8140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d815acf0 con 0x7f49d80cbbd0 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f49c8001670 con 0x7f49d80cbbd0 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 >> v1:172.21.15.55:6789/0 conn(0x7f49d80cbbd0 legacy=0x7f49d806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49c0005b80 0x7f49c0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00069d0 0x7f49c0006de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.772+0000 7f49d7e00640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49c000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d815f1e0 con 0x7f49c0007390 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c000e330 con 0x7f49c00069d0 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c000e4f0 con 0x7f49c0005b80 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49c000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49dda00640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49c0005b80 0x7f49c0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49dd000640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00069d0 0x7f49c0006de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 -- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49c0005b80 msgr2=0x7f49c0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49c0005b80 0x7f49c0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 -- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00069d0 msgr2=0x7f49c0006de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c00069d0 0x7f49c0006de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.773+0000 7f49de400640 1 -- 172.21.15.55:0/3370602141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c0005730 con 0x7f49c0007390 2024-05-10T03:35:07.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.774+0000 7f49de400640 1 -- 172.21.15.55:0/3370602141 reap_dead start 2024-05-10T03:35:07.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.774+0000 7f49de400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49c000db90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009fc0 tx=0x7f49c0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b22b8d45cbc26d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.775+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0003b90 con 0x7f49c0007390 2024-05-10T03:35:07.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.775+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d000f040 con 0x7f49c0007390 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.775+0000 7f49d7e00640 1 -- 172.21.15.55:0/3370602141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004020 con 0x7f49c0007390 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.775+0000 7f49e0400640 1 -- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 msgr2=0x7f49c000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.775+0000 7f49e0400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49c000db90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009fc0 tx=0x7f49c0007a00 comp rx=0 tx=0).stop 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.776+0000 7f49e0400640 1 -- 172.21.15.55:0/3370602141 shutdown_connections 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.776+0000 7f49e0400640 1 --2- 172.21.15.55:0/3370602141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49c000db90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.776+0000 7f49e0400640 1 -- 172.21.15.55:0/3370602141 >> 172.21.15.55:0/3370602141 conn(0x7f49d806c120 msgr2=0x7f49d806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.776+0000 7f49e0400640 1 -- 172.21.15.55:0/3370602141 shutdown_connections 2024-05-10T03:35:07.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.776+0000 7f49e0400640 1 -- 172.21.15.55:0/3370602141 wait complete. 2024-05-10T03:35:07.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.778+0000 7f49e0400640 1 Processor -- start 2024-05-10T03:35:07.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.778+0000 7f49e0400640 1 -- start start 2024-05-10T03:35:07.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49d8159e00 0x7f49d8071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49d8155ed0 0x7f49d8071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81f0280 con 0x7f49c0007390 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8158b10 con 0x7f49d8159e00 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49e0400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8158d60 con 0x7f49d8155ed0 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49dda00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49dda00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38328/0 (socket says 172.21.15.55:38328) 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.780+0000 7f49dda00640 1 -- 172.21.15.55:0/1496218948 learned_addr learned my addr 172.21.15.55:0/1496218948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49de400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49d8155ed0 0x7f49d8071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dd000640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49d8159e00 0x7f49d8071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 -- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49d8155ed0 msgr2=0x7f49d8071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49d8155ed0 0x7f49d8071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 -- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49d8159e00 msgr2=0x7f49d8071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49d8159e00 0x7f49d8071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 -- 172.21.15.55:0/1496218948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8159040 con 0x7f49c0007390 2024-05-10T03:35:07.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49dda00640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e4d0 tx=0x7f49cc00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.781+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0052e0 con 0x7f49c0007390 2024-05-10T03:35:07.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.782+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc005440 con 0x7f49c0007390 2024-05-10T03:35:07.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.782+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc003810 con 0x7f49c0007390 2024-05-10T03:35:07.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.782+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8159320 con 0x7f49c0007390 2024-05-10T03:35:07.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.782+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81598e0 con 0x7f49c0007390 2024-05-10T03:35:07.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.783+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8159a70 con 0x7f49c0007390 2024-05-10T03:35:07.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.784+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f49cc003970 con 0x7f49c0007390 2024-05-10T03:35:07.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.785+0000 7f49d5600640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 0x7f49bc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:07.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.785+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f49cc089320 con 0x7f49c0007390 2024-05-10T03:35:07.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.786+0000 7f49dd000640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 0x7f49bc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:07.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.787+0000 7f49dd000640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 0x7f49bc06d110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f49c8001eb0 tx=0x7f49c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:07.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.791+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f49cc054290 con 0x7f49c0007390 2024-05-10T03:35:07.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.989+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49d8066f60 con 0x7f49c0007390 2024-05-10T03:35:07.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.991+0000 7f49d5600640 1 -- 172.21.15.55:0/1496218948 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49cc0578b0 con 0x7f49c0007390 2024-05-10T03:35:07.995 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:35:07.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 msgr2=0x7f49bc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 0x7f49bc06d110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f49c8001eb0 tx=0x7f49c8009040 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 msgr2=0x7f49d81582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e4d0 tx=0x7f49cc00e990 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 shutdown_connections 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49d8155ed0 0x7f49d8071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f49bc06ac00 0x7f49bc06d110 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49d8159e00 0x7f49d8071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 --2- 172.21.15.55:0/1496218948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49c0007390 0x7f49d81582a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.993+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 >> 172.21.15.55:0/1496218948 conn(0x7f49d806c120 msgr2=0x7f49d815bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.994+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 shutdown_connections 2024-05-10T03:35:07.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:07.994+0000 7f49e0400640 1 -- 172.21.15.55:0/1496218948 wait complete. 2024-05-10T03:35:08.023 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:35:08.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.310+0000 7f7ec7600640 1 Processor -- start 2024-05-10T03:35:08.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.311+0000 7f7ec7600640 1 -- start start 2024-05-10T03:35:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.313+0000 7f7ec7600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ec00719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.313+0000 7f7ec7600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00cd710 con 0x7f7ec00cdba0 2024-05-10T03:35:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ec7600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00cdfb0 con 0x7f7ec00b2fa0 2024-05-10T03:35:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ec7600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015b240 con 0x7f7ec00715e0 2024-05-10T03:35:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ec4c00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7ec00cdba0 0x7f7ec015ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51716/0 (socket says 172.21.15.55:51716) 2024-05-10T03:35:08.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ec4c00640 1 -- 172.21.15.55:0/1019022949 learned_addr learned my addr 172.21.15.55:0/1019022949 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:08.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ebfe00640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ec00719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228796602 0 0) 0x7f7ec00cd710 con 0x7f7ec00cdba0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.314+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea4005730 con 0x7f7ec00cdba0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 85023515 0 0) 0x7f7ea4005730 con 0x7f7ec00cdba0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 msgr2=0x7f7ec00719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ec00719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 >> v1:172.21.15.162:6789/0 conn(0x7f7ec00b2fa0 legacy=0x7f7ec00b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0071f30 con 0x7f7ec00cdba0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7eb00019f0 con 0x7f7ec00cdba0 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 >> v1:172.21.15.55:6789/0 conn(0x7f7ec00cdba0 legacy=0x7f7ec015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ea4005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ea4006130 0x7f7ea40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea4006f10 0x7f7ea40073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015b240 con 0x7f7ea4006f10 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea400de50 con 0x7f7ea4006130 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.315+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea400e010 con 0x7f7ec00715e0 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea4006f10 0x7f7ea40073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ebfe00640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ea4006130 0x7f7ea40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec4c00640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ea4005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 -- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 msgr2=0x7f7ea4005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec00715e0 0x7f7ea4005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 -- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ea4006130 msgr2=0x7f7ea40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ea4006130 0x7f7ea40069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.316+0000 7f7ec5600640 1 -- 172.21.15.55:0/1019022949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4005730 con 0x7f7ea4006f10 2024-05-10T03:35:08.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.317+0000 7f7ec5600640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea4006f10 0x7f7ea40073c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00ce570 tx=0x7f7ea4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45cc157f9c3a7d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.317+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80052d0 con 0x7f7ea4006f10 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.317+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb8005430 con 0x7f7ea4006f10 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.317+0000 7f7ebf400640 1 -- 172.21.15.55:0/1019022949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb802ab90 con 0x7f7ea4006f10 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec7600640 1 -- 172.21.15.55:0/1019022949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea4006f10 msgr2=0x7f7ea40073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec7600640 1 --2- 172.21.15.55:0/1019022949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ea4006f10 0x7f7ea40073c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00ce570 tx=0x7f7ea4007a00 comp rx=0 tx=0).stop 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec5600640 1 -- 172.21.15.55:0/1019022949 reap_dead start 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec7600640 1 -- 172.21.15.55:0/1019022949 shutdown_connections 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec7600640 1 -- 172.21.15.55:0/1019022949 >> 172.21.15.55:0/1019022949 conn(0x7f7ec00cbb90 msgr2=0x7f7ec00cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.318+0000 7f7ec7600640 1 -- 172.21.15.55:0/1019022949 shutdown_connections 2024-05-10T03:35:08.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.319+0000 7f7ec7600640 1 -- 172.21.15.55:0/1019022949 wait complete. 2024-05-10T03:35:08.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.321+0000 7f7ec7600640 1 Processor -- start 2024-05-10T03:35:08.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.321+0000 7f7ec7600640 1 -- start start 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 0x7f7ec01581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 0x7f7ec014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01593e0 con 0x7f7ec01e3120 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01e3a70 con 0x7f7ec0159870 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec7600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01587d0 con 0x7f7ec014e630 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38346/0 (socket says 172.21.15.55:38346) 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.323+0000 7f7ec5600640 1 -- 172.21.15.55:0/2415496913 learned_addr learned my addr 172.21.15.55:0/2415496913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ebfe00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 0x7f7ec014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec4c00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 0x7f7ec01581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 -- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 msgr2=0x7f7ec01581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 0x7f7ec01581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 -- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 msgr2=0x7f7ec014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 0x7f7ec014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 -- 172.21.15.55:0/2415496913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0158ab0 con 0x7f7ec01e3120 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec4c00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 0x7f7ec01581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ebfe00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 0x7f7ec014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ec5600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001810 tx=0x7f7eb8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.324+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8007230 con 0x7f7ec01e3120 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.325+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb8007390 con 0x7f7ec01e3120 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.325+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8007a30 con 0x7f7ec01e3120 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.325+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0158d90 con 0x7f7ec01e3120 2024-05-10T03:35:08.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.325+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec014e3d0 con 0x7f7ec01e3120 2024-05-10T03:35:08.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.326+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec00ae7a0 con 0x7f7ec01e3120 2024-05-10T03:35:08.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.327+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7eb80067d0 con 0x7f7ec01e3120 2024-05-10T03:35:08.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.328+0000 7f7ebcc00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 0x7f7e9c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.328+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7eb80a8770 con 0x7f7ec01e3120 2024-05-10T03:35:08.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.329+0000 7f7ec4c00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 0x7f7e9c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.330+0000 7f7ec4c00640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 0x7f7e9c06d140 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0004bc0 tx=0x7f7eb0001250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.333+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7eb80736e0 con 0x7f7ec01e3120 2024-05-10T03:35:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.529+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ec00ae7a0 con 0x7f7ec01e3120 2024-05-10T03:35:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.530+0000 7f7ebcc00640 1 -- 172.21.15.55:0/2415496913 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7eb8001370 con 0x7f7ec01e3120 2024-05-10T03:35:08.534 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 msgr2=0x7f7e9c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 0x7f7e9c06d140 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0004bc0 tx=0x7f7eb0001250 comp rx=0 tx=0).stop 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 msgr2=0x7f7ec01e3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001810 tx=0x7f7eb8001120 comp rx=0 tx=0).stop 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 shutdown_connections 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ec01e3120 0x7f7ec01e3530 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7e9c06ac30 0x7f7e9c06d140 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ec0159870 0x7f7ec014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 --2- 172.21.15.55:0/2415496913 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ec014e630 0x7f7ec01581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 >> 172.21.15.55:0/2415496913 conn(0x7f7ec00cbb90 msgr2=0x7f7ec0150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 shutdown_connections 2024-05-10T03:35:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.533+0000 7f7ec7600640 1 -- 172.21.15.55:0/2415496913 wait complete. 2024-05-10T03:35:08.562 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:35:08.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.842+0000 7ffad6a00640 1 Processor -- start 2024-05-10T03:35:08.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.842+0000 7ffad6a00640 1 -- start start 2024-05-10T03:35:08.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.845+0000 7ffad6a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffad013e390 0x7ffad015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.845+0000 7ffad6a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffad01414c0 0x7ffad006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.845+0000 7ffad6a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad013eae0 con 0x7ffad006b660 2024-05-10T03:35:08.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.845+0000 7ffad6a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad013e7a0 con 0x7ffad01414c0 2024-05-10T03:35:08.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.845+0000 7ffad6a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad006ba70 con 0x7ffad013e390 2024-05-10T03:35:08.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffad01414c0 0x7ffad006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffad01414c0 0x7ffad006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38352/0 (socket says 172.21.15.55:38352) 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 -- 172.21.15.55:0/556249879 learned_addr learned my addr 172.21.15.55:0/556249879 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad5600640 1 --2- 172.21.15.55:0/556249879 >> v2:172.21.15.162:3300/0 conn(0x7ffad013e390 0x7ffad015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 -- 172.21.15.55:0/556249879 >> v2:172.21.15.162:3300/0 conn(0x7ffad013e390 msgr2=0x7ffad015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 --2- 172.21.15.55:0/556249879 >> v2:172.21.15.162:3300/0 conn(0x7ffad013e390 0x7ffad015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 -- 172.21.15.55:0/556249879 >> v1:172.21.15.162:6789/0 conn(0x7ffad006b660 legacy=0x7ffad015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.846+0000 7ffad4c00640 1 -- 172.21.15.55:0/556249879 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad015c020 con 0x7ffad01414c0 2024-05-10T03:35:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.848+0000 7ffad4c00640 1 --2- 172.21.15.55:0/556249879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad01414c0 0x7ffad006b120 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b44c0 tx=0x7ffabc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1945ef7617c5d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.848+0000 7ffacfe00640 1 -- 172.21.15.55:0/556249879 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc032440 con 0x7ffad01414c0 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.848+0000 7ffacfe00640 1 -- 172.21.15.55:0/556249879 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc0325a0 con 0x7ffad01414c0 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.848+0000 7ffacfe00640 1 -- 172.21.15.55:0/556249879 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc032930 con 0x7ffad01414c0 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 -- 172.21.15.55:0/556249879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad01414c0 msgr2=0x7ffad006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 --2- 172.21.15.55:0/556249879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad01414c0 0x7ffad006b120 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b44c0 tx=0x7ffabc02a270 comp rx=0 tx=0).stop 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 -- 172.21.15.55:0/556249879 shutdown_connections 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 --2- 172.21.15.55:0/556249879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad01414c0 0x7ffad006b120 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 --2- 172.21.15.55:0/556249879 >> v2:172.21.15.162:3300/0 conn(0x7ffad013e390 0x7ffad015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 -- 172.21.15.55:0/556249879 >> 172.21.15.55:0/556249879 conn(0x7ffad006c730 msgr2=0x7ffad00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 -- 172.21.15.55:0/556249879 shutdown_connections 2024-05-10T03:35:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.849+0000 7ffad6a00640 1 -- 172.21.15.55:0/556249879 wait complete. 2024-05-10T03:35:08.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.851+0000 7ffad6a00640 1 Processor -- start 2024-05-10T03:35:08.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.851+0000 7ffad6a00640 1 -- start start 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffad013e390 0x7ffad0139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 0x7ffad00b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0155ba0 con 0x7ffad006b660 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01353f0 con 0x7ffad01414c0 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad6a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00b2440 con 0x7ffad013e390 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad5600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38368/0 (socket says 172.21.15.55:38368) 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.853+0000 7ffad5600640 1 -- 172.21.15.55:0/997009525 learned_addr learned my addr 172.21.15.55:0/997009525 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad6000640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 0x7ffad00b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad4c00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffad013e390 0x7ffad0139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 -- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffad013e390 msgr2=0x7ffad0139760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffad013e390 0x7ffad0139760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 -- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 msgr2=0x7ffad00b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:08.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 0x7ffad00b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 -- 172.21.15.55:0/997009525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009470 con 0x7ffad006b660 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad6000640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 0x7ffad00b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffad5600640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e840 tx=0x7ffac400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.854+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac40094f0 con 0x7ffad006b660 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.855+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac400c940 con 0x7ffad006b660 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.855+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400ccd0 con 0x7ffad006b660 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.855+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00b2690 con 0x7ffad006b660 2024-05-10T03:35:08.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.855+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00b2c50 con 0x7ffad006b660 2024-05-10T03:35:08.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.856+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad004e290 con 0x7ffad006b660 2024-05-10T03:35:08.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.857+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffac4013470 con 0x7ffad006b660 2024-05-10T03:35:08.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.858+0000 7ffacd600640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 0x7ffab006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:08.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.859+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffac4089690 con 0x7ffad006b660 2024-05-10T03:35:08.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.859+0000 7ffad4c00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 0x7ffab006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:08.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.861+0000 7ffad4c00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 0x7ffab006d140 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:08.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:08.863+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffac4054600 con 0x7ffad006b660 2024-05-10T03:35:09.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.068+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffad00cc360 con 0x7ffad006b660 2024-05-10T03:35:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.069+0000 7ffacd600640 1 -- 172.21.15.55:0/997009525 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffac4057c20 con 0x7ffad006b660 2024-05-10T03:35:09.073 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 msgr2=0x7ffab006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 0x7ffab006d140 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 msgr2=0x7ffad0155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e840 tx=0x7ffac400ed00 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 shutdown_connections 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffab006ac30 0x7ffab006d140 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffad01414c0 0x7ffad00b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffad013e390 0x7ffad0139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 --2- 172.21.15.55:0/997009525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffad006b660 0x7ffad0155660 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 >> 172.21.15.55:0/997009525 conn(0x7ffad006c730 msgr2=0x7ffad013fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 shutdown_connections 2024-05-10T03:35:09.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.072+0000 7ffad6a00640 1 -- 172.21.15.55:0/997009525 wait complete. 2024-05-10T03:35:09.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:35:09.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:09.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:09.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:35:09.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:09.199 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:35:09.253 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:09.254 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:09.311 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:09.311 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:09.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.643+0000 7ff4ae800640 1 Processor -- start 2024-05-10T03:35:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.643+0000 7ff4ae800640 1 -- start start 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ae800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4a8144120 0x7ff4a806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ae800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff4a80b3190 0x7ff4a80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ae800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a806d350 con 0x7ff4a8144710 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ae800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a8144530 con 0x7ff4a8144120 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ae800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a806dd20 con 0x7ff4a80b3190 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4aca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4a8144120 0x7ff4a806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4aca00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4a8144120 0x7ff4a806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38372/0 (socket says 172.21.15.55:38372) 2024-05-10T03:35:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4aca00640 1 -- 172.21.15.55:0/1558571166 learned_addr learned my addr 172.21.15.55:0/1558571166 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.645+0000 7ff4ade00640 1 --2- 172.21.15.55:0/1558571166 >> v2:172.21.15.162:3300/0 conn(0x7ff4a80b3190 0x7ff4a80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.646+0000 7ff4aca00640 1 -- 172.21.15.55:0/1558571166 >> v2:172.21.15.162:3300/0 conn(0x7ff4a80b3190 msgr2=0x7ff4a80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.646+0000 7ff4aca00640 1 --2- 172.21.15.55:0/1558571166 >> v2:172.21.15.162:3300/0 conn(0x7ff4a80b3190 0x7ff4a80cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.646+0000 7ff4aca00640 1 -- 172.21.15.55:0/1558571166 >> v1:172.21.15.55:6789/0 conn(0x7ff4a8144710 legacy=0x7ff4a80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.646+0000 7ff4aca00640 1 -- 172.21.15.55:0/1558571166 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a806c740 con 0x7ff4a8144120 2024-05-10T03:35:09.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.647+0000 7ff4aca00640 1 --2- 172.21.15.55:0/1558571166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144120 0x7ff4a806d7e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff4a806e040 tx=0x7ff49402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160e76a2c6e40c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:09.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.649+0000 7ff4a7e00640 1 -- 172.21.15.55:0/1558571166 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49402aea0 con 0x7ff4a8144120 2024-05-10T03:35:09.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.649+0000 7ff4a7e00640 1 -- 172.21.15.55:0/1558571166 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494032440 con 0x7ff4a8144120 2024-05-10T03:35:09.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.649+0000 7ff4a7e00640 1 -- 172.21.15.55:0/1558571166 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940327d0 con 0x7ff4a8144120 2024-05-10T03:35:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.649+0000 7ff4ae800640 1 -- 172.21.15.55:0/1558571166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144120 msgr2=0x7ff4a806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.649+0000 7ff4ae800640 1 --2- 172.21.15.55:0/1558571166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144120 0x7ff4a806d7e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff4a806e040 tx=0x7ff49402a150 comp rx=0 tx=0).stop 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 -- 172.21.15.55:0/1558571166 shutdown_connections 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 --2- 172.21.15.55:0/1558571166 >> v2:172.21.15.162:3300/0 conn(0x7ff4a80b3190 0x7ff4a80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 --2- 172.21.15.55:0/1558571166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144120 0x7ff4a806d7e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 -- 172.21.15.55:0/1558571166 >> 172.21.15.55:0/1558571166 conn(0x7ff4a806a110 msgr2=0x7ff4a806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 -- 172.21.15.55:0/1558571166 shutdown_connections 2024-05-10T03:35:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.650+0000 7ff4ae800640 1 -- 172.21.15.55:0/1558571166 wait complete. 2024-05-10T03:35:09.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.653+0000 7ff4ae800640 1 Processor -- start 2024-05-10T03:35:09.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.653+0000 7ff4ae800640 1 -- start start 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a80b3190 0x7ff4a815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a8144120 0x7ff4a81357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8134090 con 0x7ff4a8144710 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8142550 con 0x7ff4a80b3190 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ae800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a815c210 con 0x7ff4a8144120 2024-05-10T03:35:09.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38388/0 (socket says 172.21.15.55:38388) 2024-05-10T03:35:09.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.655+0000 7ff4ade00640 1 -- 172.21.15.55:0/3779745952 learned_addr learned my addr 172.21.15.55:0/3779745952 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ad400640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a80b3190 0x7ff4a815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4aca00640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a8144120 0x7ff4a81357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 -- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a8144120 msgr2=0x7ff4a81357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a8144120 0x7ff4a81357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 -- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a80b3190 msgr2=0x7ff4a815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a80b3190 0x7ff4a815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 -- 172.21.15.55:0/3779745952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940094d0 con 0x7ff4a8144710 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.656+0000 7ff4ade00640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007e60 tx=0x7ff4a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.657+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0014420 con 0x7ff4a8144710 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.657+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a00145a0 con 0x7ff4a8144710 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.657+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0014950 con 0x7ff4a8144710 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.657+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8135d10 con 0x7ff4a8144710 2024-05-10T03:35:09.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.657+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a815bbc0 con 0x7ff4a8144710 2024-05-10T03:35:09.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.658+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a815cbe0 con 0x7ff4a8144710 2024-05-10T03:35:09.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.659+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4a0014ab0 con 0x7ff4a8144710 2024-05-10T03:35:09.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.660+0000 7ff4a5600640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 0x7ff48806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:09.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.660+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff4a0089e40 con 0x7ff4a8144710 2024-05-10T03:35:09.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.661+0000 7ff4ad400640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 0x7ff48806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:09.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.662+0000 7ff4ad400640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 0x7ff48806d110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff49c000f60 tx=0x7ff49c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:09.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.664+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff4a0054e40 con 0x7ff4a8144710 2024-05-10T03:35:09.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.870+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4a815ba20 con 0x7ff4a8144710 2024-05-10T03:35:09.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.872+0000 7ff4a5600640 1 -- 172.21.15.55:0/3779745952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16248 (secure 0 0 0) 0x7ff4a815ba20 con 0x7ff4a8144710 2024-05-10T03:35:09.876 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:09.876 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":104,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:34:56.064514+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 msgr2=0x7ff48806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 0x7ff48806d110 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff49c000f60 tx=0x7ff49c009040 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 msgr2=0x7ff4a8133410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007e60 tx=0x7ff4a000e4b0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 shutdown_connections 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4a8144710 0x7ff4a8133410 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4a8144120 0x7ff4a81357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff48806ac00 0x7ff48806d110 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 --2- 172.21.15.55:0/3779745952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4a80b3190 0x7ff4a815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:09.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.875+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 >> 172.21.15.55:0/3779745952 conn(0x7ff4a806a110 msgr2=0x7ff4a813e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:09.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.876+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 shutdown_connections 2024-05-10T03:35:09.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:09.876+0000 7ff4ae800640 1 -- 172.21.15.55:0/3779745952 wait complete. 2024-05-10T03:35:09.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:53.261121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:54.084182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:35:09.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:35:10.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.196+0000 7fa433e00640 1 Processor -- start 2024-05-10T03:35:10.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.196+0000 7fa433e00640 1 -- start start 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa433e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa4340cc240 0x7fa4340ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa433e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340b2a30 con 0x7fa43415e920 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa433e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43415eec0 con 0x7fa43406b830 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa433e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43415a900 con 0x7fa4340cc240 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa432a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa43415e920 0x7fa43406b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51726/0 (socket says 172.21.15.55:51726) 2024-05-10T03:35:10.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.198+0000 7fa432a00640 1 -- 172.21.15.55:0/1904283330 learned_addr learned my addr 172.21.15.55:0/1904283330 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa433400640 1 --2- 172.21.15.55:0/1904283330 >> v2:172.21.15.162:3300/0 conn(0x7fa4340cc240 0x7fa4340ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611040910 0 0) 0x7fa4340b2a30 con 0x7fa43415e920 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa418003520 con 0x7fa43415e920 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209117782 0 0) 0x7fa418003520 con 0x7fa43415e920 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 >> v2:172.21.15.162:3300/0 conn(0x7fa4340cc240 msgr2=0x7fa4340ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 --2- 172.21.15.55:0/1904283330 >> v2:172.21.15.162:3300/0 conn(0x7fa4340cc240 0x7fa4340ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 >> v1:172.21.15.162:6789/0 conn(0x7fa43406b830 legacy=0x7fa4340cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43415b8b0 con 0x7fa43415e920 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa4240019f0 con 0x7fa43415e920 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 >> v1:172.21.15.55:6789/0 conn(0x7fa43415e920 legacy=0x7fa43406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4180039e0 0x7fa418003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa418004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.199+0000 7fa431600640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4180051e0 0x7fa41800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41800c140 con 0x7fa4180047c0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41800c2b0 con 0x7fa4180039e0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41800c470 con 0x7fa4180051e0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa418004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 -- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4180051e0 msgr2=0x7fa41800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432a00640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4180039e0 0x7fa418003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa433400640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4180051e0 0x7fa41800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa4180051e0 0x7fa41800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 -- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4180039e0 msgr2=0x7fa418003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:10.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4180039e0 0x7fa418003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa433400640 1 -- 172.21.15.55:0/1904283330 reap_dead start 2024-05-10T03:35:10.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432000640 1 -- 172.21.15.55:0/1904283330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418003520 con 0x7fa4180047c0 2024-05-10T03:35:10.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.200+0000 7fa432a00640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4180039e0 0x7fa418003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:10.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.202+0000 7fa432000640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa418004bd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa41c001750 tx=0x7fa4180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9649b7e92bcf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.202+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0052d0 con 0x7fa4180047c0 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.202+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c005430 con 0x7fa4180047c0 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.202+0000 7fa431600640 1 -- 172.21.15.55:0/1904283330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c02abc0 con 0x7fa4180047c0 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 -- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 msgr2=0x7fa418004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa418004bd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa41c001750 tx=0x7fa4180057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 -- 172.21.15.55:0/1904283330 shutdown_connections 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 --2- 172.21.15.55:0/1904283330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa418004bd0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 -- 172.21.15.55:0/1904283330 >> 172.21.15.55:0/1904283330 conn(0x7fa43406c750 msgr2=0x7fa4340b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 -- 172.21.15.55:0/1904283330 shutdown_connections 2024-05-10T03:35:10.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.203+0000 7fa433e00640 1 -- 172.21.15.55:0/1904283330 wait complete. 2024-05-10T03:35:10.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.205+0000 7fa433e00640 1 Processor -- start 2024-05-10T03:35:10.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.206+0000 7fa433e00640 1 -- start start 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 0x7fa4340af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa434159cb0 0x7fa4340afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43414e120 con 0x7fa4180047c0 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43413ec40 con 0x7fa4340cae10 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa433e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340b0330 con 0x7fa434159cb0 2024-05-10T03:35:10.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.207+0000 7fa432a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38398/0 (socket says 172.21.15.55:38398) 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 -- 172.21.15.55:0/3444714402 learned_addr learned my addr 172.21.15.55:0/3444714402 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432000640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 0x7fa4340af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa433400640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa434159cb0 0x7fa4340afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 -- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa434159cb0 msgr2=0x7fa4340afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa434159cb0 0x7fa4340afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 -- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 msgr2=0x7fa4340af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 0x7fa4340af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 -- 172.21.15.55:0/3444714402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43413ee90 con 0x7fa4180047c0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432000640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 0x7fa4340af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.208+0000 7fa432a00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa424001fb0 tx=0x7fa42400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.209+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424004b60 con 0x7fa4180047c0 2024-05-10T03:35:10.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.209+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424004cc0 con 0x7fa4180047c0 2024-05-10T03:35:10.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.209+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43413f170 con 0x7fa4180047c0 2024-05-10T03:35:10.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.209+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240072f0 con 0x7fa4180047c0 2024-05-10T03:35:10.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.209+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43413f730 con 0x7fa4180047c0 2024-05-10T03:35:10.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.210+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43404e290 con 0x7fa4180047c0 2024-05-10T03:35:10.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.211+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa424007450 con 0x7fa4180047c0 2024-05-10T03:35:10.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.212+0000 7fa42aa00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 0x7fa41006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.212+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa4240897f0 con 0x7fa4180047c0 2024-05-10T03:35:10.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.213+0000 7fa432000640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 0x7fa41006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.214+0000 7fa432000640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 0x7fa41006d110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa41c032000 tx=0x7fa41c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.217+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa424058340 con 0x7fa4180047c0 2024-05-10T03:35:10.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.413+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4340b2300 con 0x7fa4180047c0 2024-05-10T03:35:10.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.414+0000 7fa42aa00640 1 -- 172.21.15.55:0/3444714402 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa424057ed0 con 0x7fa4180047c0 2024-05-10T03:35:10.419 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 msgr2=0x7fa41006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 0x7fa41006d110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa41c032000 tx=0x7fa41c02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 msgr2=0x7fa4340ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa424001fb0 tx=0x7fa42400b4a0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 shutdown_connections 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa434159cb0 0x7fa4340afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa41006ac00 0x7fa41006d110 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4340cae10 0x7fa4340af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 --2- 172.21.15.55:0/3444714402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4180047c0 0x7fa4340ca8d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 >> 172.21.15.55:0/3444714402 conn(0x7fa43406c750 msgr2=0x7fa434070360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:10.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 shutdown_connections 2024-05-10T03:35:10.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.417+0000 7fa433e00640 1 -- 172.21.15.55:0/3444714402 wait complete. 2024-05-10T03:35:10.449 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:35:10.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.733+0000 7fefeb200640 1 Processor -- start 2024-05-10T03:35:10.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.733+0000 7fefeb200640 1 -- start start 2024-05-10T03:35:10.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.735+0000 7fefeb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b2110 0x7fefec15d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.735+0000 7fefeb200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefec0b1bc0 0x7fefec0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.735+0000 7fefeb200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0ceb00 con 0x7fefec0ce090 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.735+0000 7fefeb200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec15dcd0 con 0x7fefec0b1bc0 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.735+0000 7fefeb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0d2150 con 0x7fefec0b2110 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefec0b1bc0 0x7fefec0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefea800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fefec0ce090 0x7fefec161310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51728/0 (socket says 172.21.15.55:51728) 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefec0b1bc0 0x7fefec0cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38400/0 (socket says 172.21.15.55:38400) 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefea800640 1 -- 172.21.15.55:0/3756415919 learned_addr learned my addr 172.21.15.55:0/3756415919 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9e00640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b2110 0x7fefec15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 -- 172.21.15.55:0/3756415919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b2110 msgr2=0x7fefec15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:10.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b2110 0x7fefec15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 -- 172.21.15.55:0/3756415919 >> v1:172.21.15.55:6789/0 conn(0x7fefec0ce090 legacy=0x7fefec161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe8a00640 1 -- 172.21.15.55:0/3756415919 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669017259 0 0) 0x7fefec0ceb00 con 0x7fefec0ce090 2024-05-10T03:35:10.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.736+0000 7fefe9400640 1 -- 172.21.15.55:0/3756415919 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0b2910 con 0x7fefec0b1bc0 2024-05-10T03:35:10.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.738+0000 7fefe9400640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b1bc0 0x7fefec0cdb50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fefd40097a0 tx=0x7fefd402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39f79b227629c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.738+0000 7fefe8a00640 1 -- 172.21.15.55:0/3756415919 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd402ae50 con 0x7fefec0b1bc0 2024-05-10T03:35:10.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.738+0000 7fefe8a00640 1 -- 172.21.15.55:0/3756415919 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4032440 con 0x7fefec0b1bc0 2024-05-10T03:35:10.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.738+0000 7fefe8a00640 1 -- 172.21.15.55:0/3756415919 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd40327d0 con 0x7fefec0b1bc0 2024-05-10T03:35:10.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 -- 172.21.15.55:0/3756415919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b1bc0 msgr2=0x7fefec0cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b1bc0 0x7fefec0cdb50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fefd40097a0 tx=0x7fefd402a100 comp rx=0 tx=0).stop 2024-05-10T03:35:10.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 -- 172.21.15.55:0/3756415919 shutdown_connections 2024-05-10T03:35:10.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b1bc0 0x7fefec0cdb50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 --2- 172.21.15.55:0/3756415919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b2110 0x7fefec15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 -- 172.21.15.55:0/3756415919 >> 172.21.15.55:0/3756415919 conn(0x7fefec0d30d0 msgr2=0x7fefec0d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:10.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 -- 172.21.15.55:0/3756415919 shutdown_connections 2024-05-10T03:35:10.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.739+0000 7fefeb200640 1 -- 172.21.15.55:0/3756415919 wait complete. 2024-05-10T03:35:10.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.742+0000 7fefeb200640 1 Processor -- start 2024-05-10T03:35:10.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.742+0000 7fefeb200640 1 -- start start 2024-05-10T03:35:10.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b1bc0 0x7fefec0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefec0ce090 0x7fefec0b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec152240 con 0x7fefec0b2110 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0cb6e0 con 0x7fefec0ce090 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefeb200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec15fc20 con 0x7fefec0b1bc0 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefe9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefe9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38406/0 (socket says 172.21.15.55:38406) 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.744+0000 7fefe9400640 1 -- 172.21.15.55:0/3714511615 learned_addr learned my addr 172.21.15.55:0/3714511615 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 -- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b1bc0 msgr2=0x7fefec0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefea800640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefec0ce090 0x7fefec0b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b1bc0 0x7fefec0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 -- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefec0ce090 msgr2=0x7fefec0b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefec0ce090 0x7fefec0b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 -- 172.21.15.55:0/3714511615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009400 con 0x7fefec0b2110 2024-05-10T03:35:10.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe9400640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098c0 tx=0x7fefd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd403d910 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd403da70 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd403de20 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.745+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec1f6fa0 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.746+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec160710 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.747+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec1f6e00 con 0x7fefec0b2110 2024-05-10T03:35:10.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.748+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fefd4039040 con 0x7fefec0b2110 2024-05-10T03:35:10.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.749+0000 7fefe2000640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 0x7fefc806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:10.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.749+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fefd40a9d00 con 0x7fefec0b2110 2024-05-10T03:35:10.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.750+0000 7fefe9e00640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 0x7fefc806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:10.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.751+0000 7fefe9e00640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 0x7fefc806d110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.755+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fefec1f6e00 con 0x7fefec0b2110 2024-05-10T03:35:10.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.958+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefec1f6e00 con 0x7fefec0b2110 2024-05-10T03:35:10.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.958+0000 7fefe2000640 1 -- 172.21.15.55:0/3714511615 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefec1f6e00 con 0x7fefec0b2110 2024-05-10T03:35:10.966 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:35:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.961+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 msgr2=0x7fefc806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.961+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 0x7fefc806d110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fefdc006fd0 tx=0x7fefdc008040 comp rx=0 tx=0).stop 2024-05-10T03:35:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 msgr2=0x7fefec157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098c0 tx=0x7fefd40046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 shutdown_connections 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fefc806ac00 0x7fefc806d110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefec0ce090 0x7fefec0b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefec0b2110 0x7fefec157d70 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 --2- 172.21.15.55:0/3714511615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefec0b1bc0 0x7fefec0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 >> 172.21.15.55:0/3714511615 conn(0x7fefec0d30d0 msgr2=0x7fefec0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 shutdown_connections 2024-05-10T03:35:10.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:10.962+0000 7fefeb200640 1 -- 172.21.15.55:0/3714511615 wait complete. 2024-05-10T03:35:10.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:35:11.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.286+0000 7ff740c00640 1 Processor -- start 2024-05-10T03:35:11.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.286+0000 7ff740c00640 1 -- start start 2024-05-10T03:35:11.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff740c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff73c06b400 0x7ff73c1613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff740c00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff73c162040 0x7ff73c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff740c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c06a680 con 0x7ff73c06ba60 2024-05-10T03:35:11.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff740c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0b5420 con 0x7ff73c06b400 2024-05-10T03:35:11.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff740c00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c06a920 con 0x7ff73c162040 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff739400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff73c06ba60 0x7ff73c161930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51734/0 (socket says 172.21.15.55:51734) 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.289+0000 7ff739400640 1 -- 172.21.15.55:0/955924966 learned_addr learned my addr 172.21.15.55:0/955924966 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff739e00640 1 --2- 172.21.15.55:0/955924966 >> v2:172.21.15.55:3300/0 conn(0x7ff73c06b400 0x7ff73c1613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff73a800640 1 --2- 172.21.15.55:0/955924966 >> v2:172.21.15.162:3300/0 conn(0x7ff73c162040 0x7ff73c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff739e00640 1 -- 172.21.15.55:0/955924966 >> v2:172.21.15.162:3300/0 conn(0x7ff73c162040 msgr2=0x7ff73c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:11.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff739e00640 1 --2- 172.21.15.55:0/955924966 >> v2:172.21.15.162:3300/0 conn(0x7ff73c162040 0x7ff73c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff739e00640 1 -- 172.21.15.55:0/955924966 >> v1:172.21.15.55:6789/0 conn(0x7ff73c06ba60 legacy=0x7ff73c161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.290+0000 7ff739e00640 1 -- 172.21.15.55:0/955924966 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c06ac00 con 0x7ff73c06b400 2024-05-10T03:35:11.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.291+0000 7ff739e00640 1 --2- 172.21.15.55:0/955924966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06b400 0x7ff73c1613f0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cde00 tx=0x7ff72c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ca4fec916377b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.292+0000 7ff738a00640 1 -- 172.21.15.55:0/955924966 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c02ae60 con 0x7ff73c06b400 2024-05-10T03:35:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.292+0000 7ff738a00640 1 -- 172.21.15.55:0/955924966 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c032440 con 0x7ff73c06b400 2024-05-10T03:35:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.292+0000 7ff738a00640 1 -- 172.21.15.55:0/955924966 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0327d0 con 0x7ff73c06b400 2024-05-10T03:35:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.292+0000 7ff740c00640 1 -- 172.21.15.55:0/955924966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06b400 msgr2=0x7ff73c1613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.292+0000 7ff740c00640 1 --2- 172.21.15.55:0/955924966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06b400 0x7ff73c1613f0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cde00 tx=0x7ff72c02a110 comp rx=0 tx=0).stop 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 -- 172.21.15.55:0/955924966 shutdown_connections 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 --2- 172.21.15.55:0/955924966 >> v2:172.21.15.162:3300/0 conn(0x7ff73c162040 0x7ff73c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 --2- 172.21.15.55:0/955924966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06b400 0x7ff73c1613f0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 -- 172.21.15.55:0/955924966 >> 172.21.15.55:0/955924966 conn(0x7ff73c071190 msgr2=0x7ff73c0715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 -- 172.21.15.55:0/955924966 shutdown_connections 2024-05-10T03:35:11.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.293+0000 7ff740c00640 1 -- 172.21.15.55:0/955924966 wait complete. 2024-05-10T03:35:11.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.295+0000 7ff740c00640 1 Processor -- start 2024-05-10T03:35:11.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.295+0000 7ff740c00640 1 -- start start 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff73c162040 0x7ff73c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff73c143d70 0x7ff73c144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c152660 con 0x7ff73c06ba60 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c142700 con 0x7ff73c162040 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff740c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1447a0 con 0x7ff73c143d70 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff739e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff739e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38428/0 (socket says 172.21.15.55:38428) 2024-05-10T03:35:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.297+0000 7ff739e00640 1 -- 172.21.15.55:0/3044048505 learned_addr learned my addr 172.21.15.55:0/3044048505 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739400640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff73c162040 0x7ff73c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff73a800640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff73c143d70 0x7ff73c144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 -- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff73c143d70 msgr2=0x7ff73c144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff73c143d70 0x7ff73c144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 -- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff73c162040 msgr2=0x7ff73c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff73c162040 0x7ff73c158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 -- 172.21.15.55:0/3044048505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c009400 con 0x7ff73c06ba60 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff739e00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009890 tx=0x7ff72c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c030990 con 0x7ff73c06ba60 2024-05-10T03:35:11.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c030b10 con 0x7ff73c06ba60 2024-05-10T03:35:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c03d420 con 0x7ff73c06ba60 2024-05-10T03:35:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.298+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c144cd0 con 0x7ff73c06ba60 2024-05-10T03:35:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.299+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c158640 con 0x7ff73c06ba60 2024-05-10T03:35:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.300+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c067220 con 0x7ff73c06ba60 2024-05-10T03:35:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.301+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff72c039040 con 0x7ff73c06ba60 2024-05-10T03:35:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.302+0000 7ff732000640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 0x7ff71c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.302+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff72c0a9d80 con 0x7ff73c06ba60 2024-05-10T03:35:11.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.303+0000 7ff739400640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 0x7ff71c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.304+0000 7ff739400640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 0x7ff71c06d160 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff728005880 tx=0x7ff728009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.306+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff72c074f20 con 0x7ff73c06ba60 2024-05-10T03:35:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.509+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff73c0afc40 con 0x7ff73c06ba60 2024-05-10T03:35:11.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.510+0000 7ff732000640 1 -- 172.21.15.55:0/3044048505 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff72c078540 con 0x7ff73c06ba60 2024-05-10T03:35:11.514 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 msgr2=0x7ff71c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 0x7ff71c06d160 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff728005880 tx=0x7ff728009040 comp rx=0 tx=0).stop 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 msgr2=0x7ff73c141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009890 tx=0x7ff72c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 shutdown_connections 2024-05-10T03:35:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff73c143d70 0x7ff73c144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff71c06ac50 0x7ff71c06d160 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff73c162040 0x7ff73c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 --2- 172.21.15.55:0/3044048505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c06ba60 0x7ff73c141380 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 >> 172.21.15.55:0/3044048505 conn(0x7ff73c071190 msgr2=0x7ff73c15c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 shutdown_connections 2024-05-10T03:35:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.513+0000 7ff740c00640 1 -- 172.21.15.55:0/3044048505 wait complete. 2024-05-10T03:35:11.546 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:35:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.829+0000 7f4a9a400640 1 Processor -- start 2024-05-10T03:35:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.829+0000 7f4a9a400640 1 -- start start 2024-05-10T03:35:11.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4a940d1ba0 0x7f4a940b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a9415d700 0x7f4a9415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a940cdf70 0x7f4a940ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9415e050 con 0x7f4a940cdf70 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a940b7b50 con 0x7f4a940d1ba0 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a9a400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a940ce8c0 con 0x7f4a9415d700 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a99a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a940cdf70 0x7f4a940ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a99a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a940cdf70 0x7f4a940ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38436/0 (socket says 172.21.15.55:38436) 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.832+0000 7f4a99a00640 1 -- 172.21.15.55:0/1234669814 learned_addr learned my addr 172.21.15.55:0/1234669814 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a93e00640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a9415d700 0x7f4a9415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99000640 1 --2- 172.21.15.55:0/1234669814 >> v2:172.21.15.162:3300/0 conn(0x7f4a940d1ba0 0x7f4a940b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99a00640 1 -- 172.21.15.55:0/1234669814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a9415d700 msgr2=0x7f4a9415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99a00640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a9415d700 0x7f4a9415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99a00640 1 -- 172.21.15.55:0/1234669814 >> v2:172.21.15.162:3300/0 conn(0x7f4a940d1ba0 msgr2=0x7f4a940b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99a00640 1 --2- 172.21.15.55:0/1234669814 >> v2:172.21.15.162:3300/0 conn(0x7f4a940d1ba0 0x7f4a940b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.833+0000 7f4a99a00640 1 -- 172.21.15.55:0/1234669814 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940b6f40 con 0x7f4a940cdf70 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.834+0000 7f4a99a00640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a940ce380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d1470 tx=0x7f4a8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a9a90f948f2b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.834+0000 7f4a93400640 1 -- 172.21.15.55:0/1234669814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c031600 con 0x7f4a940cdf70 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.834+0000 7f4a93400640 1 -- 172.21.15.55:0/1234669814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c02d050 con 0x7f4a940cdf70 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.834+0000 7f4a93400640 1 -- 172.21.15.55:0/1234669814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c031980 con 0x7f4a940cdf70 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.835+0000 7f4a9a400640 1 -- 172.21.15.55:0/1234669814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 msgr2=0x7f4a940ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.835+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a940ce380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d1470 tx=0x7f4a8c02a6a0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 -- 172.21.15.55:0/1234669814 shutdown_connections 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a940ce380 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1234669814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a9415d700 0x7f4a9415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1234669814 >> v2:172.21.15.162:3300/0 conn(0x7f4a940d1ba0 0x7f4a940b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 -- 172.21.15.55:0/1234669814 >> 172.21.15.55:0/1234669814 conn(0x7f4a940d0620 msgr2=0x7f4a940d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 -- 172.21.15.55:0/1234669814 shutdown_connections 2024-05-10T03:35:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.836+0000 7f4a9a400640 1 -- 172.21.15.55:0/1234669814 wait complete. 2024-05-10T03:35:11.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.838+0000 7f4a9a400640 1 Processor -- start 2024-05-10T03:35:11.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.838+0000 7f4a9a400640 1 -- start start 2024-05-10T03:35:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a940d1ba0 0x7f4a940cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 0x7f4a940cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a941f4a50 con 0x7f4a940cdf70 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a940cac80 con 0x7f4a9415d700 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a9a400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a940c8160 con 0x7f4a940d1ba0 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a99000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a99000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38442/0 (socket says 172.21.15.55:38442) 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a99000640 1 -- 172.21.15.55:0/1555580008 learned_addr learned my addr 172.21.15.55:0/1555580008 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99a00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 0x7f4a940cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.840+0000 7f4a93e00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a940d1ba0 0x7f4a940cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 -- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a940d1ba0 msgr2=0x7f4a940cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a940d1ba0 0x7f4a940cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 -- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 msgr2=0x7f4a940cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 0x7f4a940cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 -- 172.21.15.55:0/1555580008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c009b30 con 0x7f4a940cdf70 2024-05-10T03:35:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99a00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 0x7f4a940cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a99000640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800d800 tx=0x7f4a8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880096a0 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88009800 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88009990 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.841+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a940c7570 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.842+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a940c7b10 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.843+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9404e290 con 0x7f4a940cdf70 2024-05-10T03:35:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.844+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a88009af0 con 0x7f4a940cdf70 2024-05-10T03:35:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.845+0000 7f4a90c00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 0x7f4a7406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.845+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4a88089920 con 0x7f4a940cdf70 2024-05-10T03:35:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.846+0000 7f4a93e00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 0x7f4a7406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:11.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.847+0000 7f4a93e00640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 0x7f4a7406d110 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:11.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:11.853+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a880549c0 con 0x7f4a940cdf70 2024-05-10T03:35:12.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.067+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a9415de30 con 0x7f4a940cdf70 2024-05-10T03:35:12.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.068+0000 7f4a90c00640 1 -- 172.21.15.55:0/1555580008 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a88057fe0 con 0x7f4a940cdf70 2024-05-10T03:35:12.072 INFO:teuthology.orchestra.run.smithi055.stderr:listed 0 entries 2024-05-10T03:35:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 msgr2=0x7f4a7406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 0x7f4a7406d110 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80008040 comp rx=0 tx=0).stop 2024-05-10T03:35:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 msgr2=0x7f4a9414e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800d800 tx=0x7f4a8800dcc0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 shutdown_connections 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4a7406ac00 0x7f4a7406d110 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a9415d700 0x7f4a940cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4a940d1ba0 0x7f4a940cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 --2- 172.21.15.55:0/1555580008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a940cdf70 0x7f4a9414e580 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 >> 172.21.15.55:0/1555580008 conn(0x7f4a940d0620 msgr2=0x7f4a9415e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.071+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 shutdown_connections 2024-05-10T03:35:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.072+0000 7f4a9a400640 1 -- 172.21.15.55:0/1555580008 wait complete. 2024-05-10T03:35:12.106 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:35:12.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:12.107 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:12.207 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:35:12.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:12.208 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:35:12.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:35:12.265 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-05-10T03:35:12.398 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.399 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.400 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on skipped message = False 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-05-10T03:35:12.401 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.402 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-05-10T03:35:12.403 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:12.404 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.405 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:12.406 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-05-10T03:35:12.407 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:35:12.408 INFO:teuthology.orchestra.run.smithi162.stdout: mds_sessionmap_keys_per_op = 1 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.417 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:35:12.418 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:35:12.419 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on skipped message = False 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.420 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-10T03:35:12.421 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-10T03:35:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.423 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-10T03:35:12.424 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-10T03:35:12.425 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:12.426 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-10T03:35:12.427 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-10T03:35:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-10T03:35:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:35:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:35:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:35:12.428 INFO:teuthology.orchestra.run.smithi055.stdout: mds_sessionmap_keys_per_op = 1 2024-05-10T03:35:12.457 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:35:12.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:35:12.458 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:35:12.458 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:35:12.458 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:35:12.458 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:35:12.458 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:35:12.459 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:35:12.484 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:35:12.485 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-05-10T03:35:12.487 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:35:12.487 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-05-10T03:35:12.488 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:35:12.488 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-05-10T03:35:12.500 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:35:12.500 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-05-10T03:35:12.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.803+0000 7f03bf000640 1 Processor -- start 2024-05-10T03:35:12.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.804+0000 7fcfe1e00640 1 Processor -- start 2024-05-10T03:35:12.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.804+0000 7f03bf000640 1 -- start start 2024-05-10T03:35:12.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.804+0000 7fcfe1e00640 1 -- start start 2024-05-10T03:35:12.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.804+0000 7f8f41c00640 1 Processor -- start 2024-05-10T03:35:12.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.804+0000 7f8f41c00640 1 -- start start 2024-05-10T03:35:12.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.806+0000 7f8f41c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c1668a0 0x7f8f3c168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.806+0000 7f8f41c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0cee80 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.806+0000 7f8f41c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0ce010 con 0x7f8f3c0d1880 2024-05-10T03:35:12.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.806+0000 7f8f41c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ce2b0 con 0x7f8f3c1668a0 2024-05-10T03:35:12.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f8f3aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8f3c0d12c0 0x7f8f3c0d3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51746/0 (socket says 172.21.15.55:51746) 2024-05-10T03:35:12.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f8f3aa00640 1 -- 172.21.15.55:0/2455685757 learned_addr learned my addr 172.21.15.55:0/2455685757 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:12.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f8f3b400640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c1668a0 0x7f8f3c168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929514045 0 0) 0x7f8f3c0cee80 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f03bf000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03b8071dc0 0x7f03b8141900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f03bf000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f03bf000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b8070bf0 con 0x7f03b8071080 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f03bf000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b8141e40 con 0x7f03b8071dc0 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f03bf000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80cecb0 con 0x7f03b806c100 2024-05-10T03:35:12.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7fcfe1e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcfdc15e820 0x7fcfdc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7fcfe1e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc06ac40 con 0x7fcfdc161dd0 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7fcfe1e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc06a960 con 0x7fcfdc0cd900 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7fcfe1e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc15ec30 con 0x7fcfdc15e820 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.807+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f20003520 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 124590580 0 0) 0x7f8f20003520 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c1668a0 msgr2=0x7f8f3c168cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:12.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c1668a0 0x7f8f3c168cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7fcfdbe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcfdc15e820 0x7fcfdc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 >> v1:172.21.15.162:6789/0 conn(0x7f8f3c0d1880 legacy=0x7f8f3c0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0b36d0 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8f2c001670 con 0x7f8f3c0d12c0 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 >> v1:172.21.15.55:6789/0 conn(0x7f8f3c0d12c0 legacy=0x7f8f3c0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f200039a0 0x7f8f20003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f20004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f20005230 0x7f8f2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ce2b0 con 0x7f8f200047c0 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2000c100 con 0x7f8f200039a0 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2000c2c0 con 0x7f8f20005230 2024-05-10T03:35:12.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f03b7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f03b7400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:32942/0 (socket says 172.21.15.55:32942) 2024-05-10T03:35:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7f03b7400640 1 -- 172.21.15.55:0/2357831659 learned_addr learned my addr 172.21.15.55:0/2357831659 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7fcfdbe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcfdc15e820 0x7fcfdc15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:48552/0 (socket says 172.21.15.55:48552) 2024-05-10T03:35:12.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.808+0000 7fcfdbe00640 1 -- 172.21.15.55:0/4140781583 learned_addr learned my addr 172.21.15.55:0/4140781583 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f03b7e00640 1 --2- 172.21.15.55:0/2357831659 >> v2:172.21.15.162:3300/0 conn(0x7f03b8071dc0 0x7f03b8141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f20004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7fcfdbe00640 1 -- 172.21.15.55:0/4140781583 >> v1:172.21.15.162:6789/0 conn(0x7fcfdc0cd900 legacy=0x7fcfdc0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7fcfdbe00640 1 -- 172.21.15.55:0/4140781583 >> v1:172.21.15.55:6789/0 conn(0x7fcfdc161dd0 legacy=0x7fcfdc06a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7fcfdbe00640 1 -- 172.21.15.55:0/4140781583 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc1612f0 con 0x7fcfdc15e820 2024-05-10T03:35:12.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f8f3b400640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f20005230 0x7f8f2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f03b7400640 1 -- 172.21.15.55:0/2357831659 >> v2:172.21.15.162:3300/0 conn(0x7f03b8071dc0 msgr2=0x7f03b8141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:12.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f03b7400640 1 --2- 172.21.15.55:0/2357831659 >> v2:172.21.15.162:3300/0 conn(0x7f03b8071dc0 0x7f03b8141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f03b7400640 1 -- 172.21.15.55:0/2357831659 >> v1:172.21.15.162:6789/0 conn(0x7f03b8071080 legacy=0x7f03b815f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.809+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f200039a0 0x7f8f20003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f03b7400640 1 -- 172.21.15.55:0/2357831659 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80cc0f0 con 0x7f03b806c100 2024-05-10T03:35:12.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3a000640 1 -- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f20005230 msgr2=0x7f8f2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f20005230 0x7f8f2000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3a000640 1 -- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f200039a0 msgr2=0x7f8f20003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f200039a0 0x7f8f20003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3a000640 1 -- 172.21.15.55:0/2455685757 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20003520 con 0x7f8f200047c0 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.810+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f200039a0 0x7f8f20003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:12.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.811+0000 7fcfdbe00640 1 --2- 172.21.15.55:0/4140781583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc15d670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009870 tx=0x7fcfc802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5eafb9b14bbc04 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.811+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f20004bd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8f28001730 tx=0x7f8f200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0ca97dec683ba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.811+0000 7fcfdb400640 1 -- 172.21.15.55:0/4140781583 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8032440 con 0x7fcfdc15e820 2024-05-10T03:35:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7fcfdb400640 1 -- 172.21.15.55:0/4140781583 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc80325a0 con 0x7fcfdc15e820 2024-05-10T03:35:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7fcfdb400640 1 -- 172.21.15.55:0/4140781583 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8032930 con 0x7fcfdc15e820 2024-05-10T03:35:12.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280052d0 con 0x7f8f200047c0 2024-05-10T03:35:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f03b7400640 1 --2- 172.21.15.55:0/2357831659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009870 tx=0x7f03a402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec190e8811b18fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28005450 con 0x7f8f200047c0 2024-05-10T03:35:12.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f8f39600640 1 -- 172.21.15.55:0/2455685757 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802abc0 con 0x7f8f200047c0 2024-05-10T03:35:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f03b6a00640 1 -- 172.21.15.55:0/2357831659 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4032440 con 0x7f03b806c100 2024-05-10T03:35:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f03b6a00640 1 -- 172.21.15.55:0/2357831659 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40325a0 con 0x7f03b806c100 2024-05-10T03:35:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.812+0000 7f03b6a00640 1 -- 172.21.15.55:0/2357831659 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4032930 con 0x7f03b806c100 2024-05-10T03:35:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 -- 172.21.15.55:0/4140781583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 msgr2=0x7fcfdc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/4140781583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc15d670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009870 tx=0x7fcfc802a270 comp rx=0 tx=0).stop 2024-05-10T03:35:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f8f41c00640 1 -- 172.21.15.55:0/2455685757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 msgr2=0x7f8f20004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2455685757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f20004bd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8f28001730 tx=0x7f8f200057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f8f3b400640 1 -- 172.21.15.55:0/2455685757 reap_dead start 2024-05-10T03:35:12.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 -- 172.21.15.55:0/4140781583 shutdown_connections 2024-05-10T03:35:12.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/4140781583 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc15d670 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 -- 172.21.15.55:0/4140781583 >> 172.21.15.55:0/4140781583 conn(0x7fcfdc0cfd30 msgr2=0x7fcfdc0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:12.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 -- 172.21.15.55:0/4140781583 shutdown_connections 2024-05-10T03:35:12.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f8f41c00640 1 -- 172.21.15.55:0/2455685757 shutdown_connections 2024-05-10T03:35:12.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7fcfe1e00640 1 -- 172.21.15.55:0/4140781583 wait complete. 2024-05-10T03:35:12.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f8f41c00640 1 -- 172.21.15.55:0/2455685757 >> 172.21.15.55:0/2455685757 conn(0x7f8f3c09c5f0 msgr2=0x7f8f3c0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f03bf000640 1 -- 172.21.15.55:0/2357831659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 msgr2=0x7f03b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.813+0000 7f03bf000640 1 --2- 172.21.15.55:0/2357831659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009870 tx=0x7f03a402a1c0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f8f41c00640 1 -- 172.21.15.55:0/2455685757 shutdown_connections 2024-05-10T03:35:12.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f8f41c00640 1 -- 172.21.15.55:0/2455685757 wait complete. 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f03bf000640 1 -- 172.21.15.55:0/2357831659 shutdown_connections 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f03bf000640 1 --2- 172.21.15.55:0/2357831659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b806c100 0x7f03b806c510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f03bf000640 1 --2- 172.21.15.55:0/2357831659 >> v2:172.21.15.162:3300/0 conn(0x7f03b8071dc0 0x7f03b8141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f03bf000640 1 -- 172.21.15.55:0/2357831659 >> 172.21.15.55:0/2357831659 conn(0x7f03b80b0b80 msgr2=0x7f03b80b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.814+0000 7f03bf000640 1 -- 172.21.15.55:0/2357831659 shutdown_connections 2024-05-10T03:35:12.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.815+0000 7f03bf000640 1 -- 172.21.15.55:0/2357831659 wait complete. 2024-05-10T03:35:12.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.817+0000 7fcfe1e00640 1 Processor -- start 2024-05-10T03:35:12.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.817+0000 7fcfe1e00640 1 -- start start 2024-05-10T03:35:12.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.818+0000 7f8f41c00640 1 Processor -- start 2024-05-10T03:35:12.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.818+0000 7f8f41c00640 1 -- start start 2024-05-10T03:35:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.818+0000 7f03bf000640 1 Processor -- start 2024-05-10T03:35:12.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.819+0000 7f03bf000640 1 -- start start 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfdc0cd900 0x7fcfdc159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc14e010 con 0x7fcfdc161dd0 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc14e440 con 0x7fcfdc15e820 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0ae760 con 0x7fcfdc0cd900 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38462/0 (socket says 172.21.15.55:38462) 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfe1400640 1 -- 172.21.15.55:0/1922338796 learned_addr learned my addr 172.21.15.55:0/1922338796 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.820+0000 7fcfdbe00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7fcfe0a00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfdc0cd900 0x7fcfdc159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f3c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 0x7f8f3c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f3c159dc0 0x7f8f3c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1551d0 con 0x7f8f200047c0 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c153430 con 0x7f8f3c159dc0 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f41c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c160d80 con 0x7f8f3c160580 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f8f3a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 0x7f8f3c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.821+0000 7f03bf000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f8f3a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 0x7f8f3c152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:32974/0 (socket says 172.21.15.55:32974) 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f8f3a000640 1 -- 172.21.15.55:0/2778112065 learned_addr learned my addr 172.21.15.55:0/2778112065 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03bf000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b8071080 0x7f03b815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03bf000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03b8071dc0 0x7f03b8143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03bf000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b813abc0 con 0x7f03b806c100 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03bf000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b813a8b0 con 0x7f03b8071dc0 2024-05-10T03:35:12.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03bf000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b815bd80 con 0x7f03b8071080 2024-05-10T03:35:12.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfdbe00640 1 -- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfdc0cd900 msgr2=0x7fcfdc159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfdbe00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfdc0cd900 0x7fcfdc159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfdbe00640 1 -- 172.21.15.55:0/1922338796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 msgr2=0x7fcfdc14cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfdbe00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfdbe00640 1 -- 172.21.15.55:0/1922338796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80094d0 con 0x7fcfdc15e820 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7fcfe1400640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:35:12.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f3c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38494/0 (socket says 172.21.15.55:38494) 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f03b7e00640 1 -- 172.21.15.55:0/2294986089 learned_addr learned my addr 172.21.15.55:0/2294986089 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.822+0000 7f8f3aa00640 1 -- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 msgr2=0x7f8f3c152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 0x7f8f3c152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3b400640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f3c159dc0 0x7f8f3c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3aa00640 1 -- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f3c159dc0 msgr2=0x7f8f3c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f3c159dc0 0x7f8f3c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7400640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b8071080 0x7f03b815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3aa00640 1 -- 172.21.15.55:0/2778112065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c15a710 con 0x7f8f200047c0 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 -- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b8071080 msgr2=0x7f03b815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b8071080 0x7f03b815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 -- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03b8071dc0 msgr2=0x7f03b8143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03b8071dc0 0x7f03b8143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 -- 172.21.15.55:0/2294986089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40094d0 con 0x7f03b806c100 2024-05-10T03:35:12.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f8f3aa00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f3c154c90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d7a0 tx=0x7f8f2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7f03b7e00640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d5f0 tx=0x7f03a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.823+0000 7fcfdbe00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc14dad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0cd8b0 tx=0x7fcfc80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0072d0 con 0x7f8f200047c0 2024-05-10T03:35:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c007430 con 0x7f8f200047c0 2024-05-10T03:35:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0047e0 con 0x7f8f200047c0 2024-05-10T03:35:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80125f0 con 0x7f03b806c100 2024-05-10T03:35:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8012750 con 0x7f03b806c100 2024-05-10T03:35:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8012b00 con 0x7f03b806c100 2024-05-10T03:35:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c158d20 con 0x7f8f200047c0 2024-05-10T03:35:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8030b80 con 0x7fcfdc15e820 2024-05-10T03:35:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8030ce0 con 0x7fcfdc15e820 2024-05-10T03:35:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc803d680 con 0x7fcfdc15e820 2024-05-10T03:35:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b815de50 con 0x7f03b806c100 2024-05-10T03:35:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b815e380 con 0x7f03b806c100 2024-05-10T03:35:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc0ae9b0 con 0x7fcfdc15e820 2024-05-10T03:35:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.824+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1592e0 con 0x7f8f200047c0 2024-05-10T03:35:12.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.825+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc0b2d30 con 0x7fcfdc15e820 2024-05-10T03:35:12.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.825+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c067320 con 0x7f8f200047c0 2024-05-10T03:35:12.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.826+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b8067320 con 0x7f03b806c100 2024-05-10T03:35:12.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.827+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc067240 con 0x7fcfdc15e820 2024-05-10T03:35:12.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.828+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8f2c004940 con 0x7f8f200047c0 2024-05-10T03:35:12.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.828+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f03a8012c60 con 0x7f03b806c100 2024-05-10T03:35:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.829+0000 7f03afe00640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 0x7f039806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.829+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f03a80898c0 con 0x7f03b806c100 2024-05-10T03:35:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.829+0000 7f8f32a00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 0x7f8f1c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.829+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8f2c0890b0 con 0x7f8f200047c0 2024-05-10T03:35:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.830+0000 7f03b7400640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 0x7f039806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.830+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 0x7f8f1c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.830+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcfc8039040 con 0x7fcfdc15e820 2024-05-10T03:35:12.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.831+0000 7fcfd8c00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 0x7fcfbc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.831+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcfc80aa060 con 0x7fcfdc15e820 2024-05-10T03:35:12.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.832+0000 7f8f3a000640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 0x7f8f1c06d110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005040 tx=0x7f8f2802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.832+0000 7f03b7400640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 0x7f039806d110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.832+0000 7fcfe0a00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 0x7fcfbc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.833+0000 7fcfe0a00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 0x7fcfbc06d110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0000f60 tx=0x7fcfd0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.834+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8f2c054130 con 0x7f8f200047c0 2024-05-10T03:35:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.840+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f03a80549f0 con 0x7f03b806c100 2024-05-10T03:35:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.848+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfc8075110 con 0x7fcfdc15e820 2024-05-10T03:35:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.858+0000 7fed9bc00640 1 Processor -- start 2024-05-10T03:35:12.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.858+0000 7fed9bc00640 1 -- start start 2024-05-10T03:35:12.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed9bc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed940712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed9bc00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940cc080 con 0x7fed940cf990 2024-05-10T03:35:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed9bc00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940cfda0 con 0x7fed940ae4f0 2024-05-10T03:35:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed9bc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cc3d0 con 0x7fed94070ec0 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed99200640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fed940cf990 0x7fed9415a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58122/0 (socket says 172.21.15.55:58122) 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.860+0000 7fed99200640 1 -- 172.21.15.55:0/3904074426 learned_addr learned my addr 172.21.15.55:0/3904074426 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93e00640 1 --2- 172.21.15.55:0/3904074426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed940712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93400640 1 -- 172.21.15.55:0/3904074426 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566508934 0 0) 0x7fed940cc080 con 0x7fed940cf990 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93400640 1 -- 172.21.15.55:0/3904074426 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c003520 con 0x7fed940cf990 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93e00640 1 -- 172.21.15.55:0/3904074426 >> v1:172.21.15.162:6789/0 conn(0x7fed940ae4f0 legacy=0x7fed940ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93e00640 1 -- 172.21.15.55:0/3904074426 >> v1:172.21.15.55:6789/0 conn(0x7fed940cf990 legacy=0x7fed9415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.861+0000 7fed93e00640 1 -- 172.21.15.55:0/3904074426 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94071810 con 0x7fed94070ec0 2024-05-10T03:35:12.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.863+0000 7fed93e00640 1 --2- 172.21.15.55:0/3904074426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed940712d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed800097d0 tx=0x7fed8002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678d6cabcd7ede89 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.863+0000 7fed93400640 1 -- 172.21.15.55:0/3904074426 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8002ae40 con 0x7fed94070ec0 2024-05-10T03:35:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.863+0000 7fed93400640 1 -- 172.21.15.55:0/3904074426 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80032440 con 0x7fed94070ec0 2024-05-10T03:35:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.863+0000 7fed93400640 1 -- 172.21.15.55:0/3904074426 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800327b0 con 0x7fed94070ec0 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 -- 172.21.15.55:0/3904074426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 msgr2=0x7fed940712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 --2- 172.21.15.55:0/3904074426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed940712d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed800097d0 tx=0x7fed8002a0f0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 -- 172.21.15.55:0/3904074426 shutdown_connections 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 --2- 172.21.15.55:0/3904074426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed940712d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 -- 172.21.15.55:0/3904074426 >> 172.21.15.55:0/3904074426 conn(0x7fed940b1ad0 msgr2=0x7fed940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.864+0000 7fed9bc00640 1 -- 172.21.15.55:0/3904074426 shutdown_connections 2024-05-10T03:35:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.865+0000 7fed9bc00640 1 -- 172.21.15.55:0/3904074426 wait complete. 2024-05-10T03:35:12.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.867+0000 7fed9bc00640 1 Processor -- start 2024-05-10T03:35:12.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.867+0000 7fed9bc00640 1 -- start start 2024-05-10T03:35:12.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.868+0000 7fed9bc00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed9414f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed9bc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 0x7fed941f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed9bc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed9bc00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cc7d0 con 0x7fed940cf990 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed9bc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940b4160 con 0x7fed940ae4f0 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed9bc00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cde20 con 0x7fed94070ec0 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39754/0 (socket says 172.21.15.55:39754) 2024-05-10T03:35:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 -- 172.21.15.55:0/423139268 learned_addr learned my addr 172.21.15.55:0/423139268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99200640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed9414f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed93e00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 0x7fed941f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 -- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 msgr2=0x7fed9414f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed9414f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 -- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 msgr2=0x7fed941f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 0x7fed941f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed99c00640 1 -- 172.21.15.55:0/423139268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80009400 con 0x7fed940cf990 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.869+0000 7fed93e00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 0x7fed941f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:12.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed99c00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c680 tx=0x7fed8c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c003b70 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c003cd0 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c005310 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed940cdb40 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.870+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940c6440 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.872+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed94067320 con 0x7fed940cf990 2024-05-10T03:35:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.873+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fed8c005470 con 0x7fed940cf990 2024-05-10T03:35:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.874+0000 7fed90c00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 0x7fed7406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.874+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fed8c089360 con 0x7fed940cf990 2024-05-10T03:35:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.874+0000 7fed99200640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 0x7fed7406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:12.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.876+0000 7fed99200640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 0x7fed7406d110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fed880011e0 tx=0x7fed88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:12.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:12.879+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fed8c0542d0 con 0x7fed940cf990 2024-05-10T03:35:13.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.115+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f03b806da40 con 0x7f03b806c100 2024-05-10T03:35:13.120 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:35:13.116+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:35:13.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.134+0000 7f03afe00640 1 -- 172.21.15.55:0/2294986089 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7359 v56) v1 ==== 95+0+0 (secure 0 0 0) 0x7f03a8058010 con 0x7f03b806c100 2024-05-10T03:35:13.138 INFO:teuthology.orchestra.run.smithi055.stderr:failed mds gid 7359 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 msgr2=0x7f039806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 0x7f039806d110 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 msgr2=0x7f03b8147e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f03a800d5f0 tx=0x7f03a800dab0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 shutdown_connections 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f039806ac00 0x7f039806d110 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03b8071dc0 0x7f03b8143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03b8071080 0x7f03b815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 --2- 172.21.15.55:0/2294986089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03b806c100 0x7f03b8147e30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 >> 172.21.15.55:0/2294986089 conn(0x7f03b80b0b80 msgr2=0x7f03b80cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 shutdown_connections 2024-05-10T03:35:13.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.137+0000 7f03bf000640 1 -- 172.21.15.55:0/2294986089 wait complete. 2024-05-10T03:35:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.148+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f8f3c0b2430 con 0x7f8f200047c0 2024-05-10T03:35:13.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.165+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fed940cb8f0 con 0x7fed940cf990 2024-05-10T03:35:13.169 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:35:13.166+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:35:13.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.171+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7fcfdc0d0920 con 0x7fcfdc15e820 2024-05-10T03:35:13.179 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-10T03:35:13.179 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-10T03:35:13.183 INFO:tasks.ceph.mds.c:Started 2024-05-10T03:35:13.183 DEBUG:teuthology.parallel:result is None 2024-05-10T03:35:13.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.183+0000 7f8f32a00640 1 -- 172.21.15.55:0/2778112065 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6139 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7f8f2c019090 con 0x7f8f200047c0 2024-05-10T03:35:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.183+0000 7fed90c00640 1 -- 172.21.15.55:0/423139268 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7356 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7fed8c0578f0 con 0x7fed940cf990 2024-05-10T03:35:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:failed mds gid 7356 2024-05-10T03:35:13.187 INFO:teuthology.orchestra.run.smithi055.stderr:failed mds gid 6139 2024-05-10T03:35:13.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.186+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 msgr2=0x7f8f1c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.186+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 0x7f8f1c06d110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005040 tx=0x7f8f2802e000 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.186+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 msgr2=0x7f8f3c154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.186+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f3c154c90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d7a0 tx=0x7f8f2c00dc60 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 shutdown_connections 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f8f1c06ac00 0x7f8f1c06d110 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f3c159dc0 0x7f8f3c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f3c160580 0x7f8f3c152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 --2- 172.21.15.55:0/2778112065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f200047c0 0x7f8f3c154c90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 >> 172.21.15.55:0/2778112065 conn(0x7f8f3c09c5f0 msgr2=0x7f8f3c161f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 shutdown_connections 2024-05-10T03:35:13.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7f8f41c00640 1 -- 172.21.15.55:0/2778112065 wait complete. 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 msgr2=0x7fed7406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 0x7fed7406d110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fed880011e0 tx=0x7fed88009040 comp rx=0 tx=0).stop 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 msgr2=0x7fed941f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c680 tx=0x7fed8c00cb40 comp rx=0 tx=0).stop 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 shutdown_connections 2024-05-10T03:35:13.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed940cf990 0x7fed941f7c80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fed7406ac00 0x7fed7406d110 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ae4f0 0x7fed941f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 --2- 172.21.15.55:0/423139268 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed94070ec0 0x7fed9414f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 >> 172.21.15.55:0/423139268 conn(0x7fed940b1ad0 msgr2=0x7fed9406e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 shutdown_connections 2024-05-10T03:35:13.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.187+0000 7fed9bc00640 1 -- 172.21.15.55:0/423139268 wait complete. 2024-05-10T03:35:13.196 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:35:13.192+0000 7f777c400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:35:13.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.211+0000 7fcfd8c00640 1 -- 172.21.15.55:0/1922338796 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6142 v60) v1 ==== 95+0+0 (secure 0 0 0) 0x7fcfc8078730 con 0x7fcfdc15e820 2024-05-10T03:35:13.215 INFO:teuthology.orchestra.run.smithi055.stderr:failed mds gid 6142 2024-05-10T03:35:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 msgr2=0x7fcfbc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 0x7fcfbc06d110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0000f60 tx=0x7fcfd0009040 comp rx=0 tx=0).stop 2024-05-10T03:35:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 msgr2=0x7fcfdc14dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc14dad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0cd8b0 tx=0x7fcfc80046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 shutdown_connections 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfdc161dd0 0x7fcfdc14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcfbc06ac00 0x7fcfbc06d110 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfdc15e820 0x7fcfdc14dad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 --2- 172.21.15.55:0/1922338796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfdc0cd900 0x7fcfdc159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 >> 172.21.15.55:0/1922338796 conn(0x7fcfdc0cfd30 msgr2=0x7fcfdc15a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 shutdown_connections 2024-05-10T03:35:13.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.213+0000 7fcfe1e00640 1 -- 172.21.15.55:0/1922338796 wait complete. 2024-05-10T03:35:13.222 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-10T03:35:13.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-10T03:35:13.224 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-10T03:35:13.225 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-10T03:35:13.229 INFO:tasks.ceph.mds.b:Started 2024-05-10T03:35:13.229 DEBUG:teuthology.parallel:result is None 2024-05-10T03:35:13.245 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-10T03:35:13.245 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-10T03:35:13.250 INFO:tasks.ceph.mds.d:Started 2024-05-10T03:35:13.250 DEBUG:teuthology.parallel:result is None 2024-05-10T03:35:13.289 INFO:tasks.ceph.mds.a:Started 2024-05-10T03:35:13.289 DEBUG:teuthology.parallel:result is None 2024-05-10T03:35:13.289 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:13.399 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:35:13.460 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-05-10T03:35:13.471 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-05-10T03:35:13.496 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:35:13.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.527+0000 7fccc3e00640 1 Processor -- start 2024-05-10T03:35:13.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.527+0000 7fccc3e00640 1 -- start start 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fccc40b4670 0x7fccc40d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc40b6080 con 0x7fccc40cff10 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc40d0320 con 0x7fccc40b4670 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40ce740 con 0x7fccc40b86e0 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fccc40cff10 0x7fccc40d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58152/0 (socket says 172.21.15.55:58152) 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fccc40b4670 0x7fccc40d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc3400640 1 -- 172.21.15.55:0/2296503214 learned_addr learned my addr 172.21.15.55:0/2296503214 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.529+0000 7fccc2000640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.530+0000 7fccc2a00640 1 -- 172.21.15.55:0/2296503214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 msgr2=0x7fccc40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.530+0000 7fccc2a00640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.530+0000 7fccc2a00640 1 -- 172.21.15.55:0/2296503214 >> v1:172.21.15.55:6789/0 conn(0x7fccc40cff10 legacy=0x7fccc40d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.530+0000 7fccc2a00640 1 -- 172.21.15.55:0/2296503214 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40cf280 con 0x7fccc40b4670 2024-05-10T03:35:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.530+0000 7fccc1600640 1 -- 172.21.15.55:0/2296503214 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210207840 0 0) 0x7fccc40b6080 con 0x7fccc40cff10 2024-05-10T03:35:13.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.531+0000 7fccc2a00640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40b4670 0x7fccc40d3b90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fccb40097a0 tx=0x7fccb402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace5e98220f5c41e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.531+0000 7fccc1600640 1 -- 172.21.15.55:0/2296503214 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb402ae60 con 0x7fccc40b4670 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.531+0000 7fccc1600640 1 -- 172.21.15.55:0/2296503214 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4032440 con 0x7fccc40b4670 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.531+0000 7fccc1600640 1 -- 172.21.15.55:0/2296503214 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb40327d0 con 0x7fccc40b4670 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 -- 172.21.15.55:0/2296503214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40b4670 msgr2=0x7fccc40d3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40b4670 0x7fccc40d3b90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fccb40097a0 tx=0x7fccb402a110 comp rx=0 tx=0).stop 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 -- 172.21.15.55:0/2296503214 shutdown_connections 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 --2- 172.21.15.55:0/2296503214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40b4670 0x7fccc40d3b90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 -- 172.21.15.55:0/2296503214 >> 172.21.15.55:0/2296503214 conn(0x7fccc40d3070 msgr2=0x7fccc40d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 -- 172.21.15.55:0/2296503214 shutdown_connections 2024-05-10T03:35:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.532+0000 7fccc3e00640 1 -- 172.21.15.55:0/2296503214 wait complete. 2024-05-10T03:35:13.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.534+0000 7fccc3e00640 1 Processor -- start 2024-05-10T03:35:13.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.535+0000 7fccc3e00640 1 -- start start 2024-05-10T03:35:13.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 0x7fccc40b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40cd180 con 0x7fccc40cff10 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40b3110 con 0x7fccc40b4670 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.536+0000 7fccc3e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40cb6d0 con 0x7fccc40b86e0 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39802/0 (socket says 172.21.15.55:39802) 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 -- 172.21.15.55:0/3440954952 learned_addr learned my addr 172.21.15.55:0/3440954952 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc2a00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 0x7fccc40b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc2000640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 -- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 msgr2=0x7fccc40b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 -- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 msgr2=0x7fccc40b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 0x7fccc40b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 -- 172.21.15.55:0/3440954952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb4009400 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc2a00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 0x7fccc40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.537+0000 7fccc3400640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c7b0 tx=0x7fccbc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.538+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00ce60 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.538+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc00a640 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.538+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00a9d0 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.538+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc40b3360 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.538+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc40b3920 con 0x7fccc40cff10 2024-05-10T03:35:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.539+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc40b83d0 con 0x7fccc40cff10 2024-05-10T03:35:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.540+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fccbc00ab30 con 0x7fccc40cff10 2024-05-10T03:35:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.541+0000 7fccbaa00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 0x7fcca406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.541+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fccbc088a50 con 0x7fccc40cff10 2024-05-10T03:35:13.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.541+0000 7fccc2a00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 0x7fcca406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:13.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.542+0000 7fccc2a00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 0x7fcca406d140 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fccb4034000 tx=0x7fccb402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.546+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fccbc0538c0 con 0x7fccc40cff10 2024-05-10T03:35:13.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.779+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccc4067240 con 0x7fccc40cff10 2024-05-10T03:35:13.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.780+0000 7fccbaa00640 1 -- 172.21.15.55:0/3440954952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+1702 (secure 0 0 0) 0x7fccbc018070 con 0x7fccc40cff10 2024-05-10T03:35:13.784 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:13.784 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:13.194050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:13.785 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 60 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 msgr2=0x7fcca406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 0x7fcca406d140 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fccb4034000 tx=0x7fccb402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 msgr2=0x7fccc41589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00c7b0 tx=0x7fccbc00cc70 comp rx=0 tx=0).stop 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 shutdown_connections 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccc40cff10 0x7fccc41589b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fccc40b86e0 0x7fccc40b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fcca406ac30 0x7fcca406d140 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 --2- 172.21.15.55:0/3440954952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccc40b4670 0x7fccc40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:13.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 >> 172.21.15.55:0/3440954952 conn(0x7fccc40d3070 msgr2=0x7fccc406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:13.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.783+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 shutdown_connections 2024-05-10T03:35:13.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:13.784+0000 7fccc3e00640 1 -- 172.21.15.55:0/3440954952 wait complete. 2024-05-10T03:35:14.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:35:13.194050+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:14.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-05-10T03:35:15.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:15.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.502+0000 7f45eb400640 1 Processor -- start 2024-05-10T03:35:15.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.502+0000 7f45eb400640 1 -- start start 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45eb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45e415ef20 0x7f45e40cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45eb400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e40cf560 0x7f45e40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45eb400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40b3f00 con 0x7f45e40cd9d0 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45eb400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e40d0540 con 0x7f45e415ef20 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45eb400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e415f330 con 0x7f45e40cf560 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45e9400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f45e40cd9d0 0x7f45e40cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58160/0 (socket says 172.21.15.55:58160) 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45e8a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45e415ef20 0x7f45e40cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.504+0000 7f45e9400640 1 -- 172.21.15.55:0/2557904551 learned_addr learned my addr 172.21.15.55:0/2557904551 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45dbe00640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e40cf560 0x7f45e40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45e8a00640 1 -- 172.21.15.55:0/2557904551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e40cf560 msgr2=0x7f45e40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45e8a00640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e40cf560 0x7f45e40cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45e8a00640 1 -- 172.21.15.55:0/2557904551 >> v1:172.21.15.55:6789/0 conn(0x7f45e40cd9d0 legacy=0x7f45e40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45e3e00640 1 -- 172.21.15.55:0/2557904551 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550678322 0 0) 0x7f45e40b3f00 con 0x7f45e40cd9d0 2024-05-10T03:35:15.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.505+0000 7f45e8a00640 1 -- 172.21.15.55:0/2557904551 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40ccb00 con 0x7f45e415ef20 2024-05-10T03:35:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.506+0000 7f45e8a00640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e415ef20 0x7f45e40cf020 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f45d40097a0 tx=0x7f45d402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebd4128bb0d3209 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.506+0000 7f45e3e00640 1 -- 172.21.15.55:0/2557904551 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d402ae50 con 0x7f45e415ef20 2024-05-10T03:35:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.506+0000 7f45e3e00640 1 -- 172.21.15.55:0/2557904551 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4032440 con 0x7f45e415ef20 2024-05-10T03:35:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.506+0000 7f45e3e00640 1 -- 172.21.15.55:0/2557904551 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d40327d0 con 0x7f45e415ef20 2024-05-10T03:35:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 -- 172.21.15.55:0/2557904551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e415ef20 msgr2=0x7f45e40cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e415ef20 0x7f45e40cf020 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f45d40097a0 tx=0x7f45d402a100 comp rx=0 tx=0).stop 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 -- 172.21.15.55:0/2557904551 shutdown_connections 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e40cf560 0x7f45e40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 --2- 172.21.15.55:0/2557904551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e415ef20 0x7f45e40cf020 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 -- 172.21.15.55:0/2557904551 >> 172.21.15.55:0/2557904551 conn(0x7f45e40d10b0 msgr2=0x7f45e40d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.507+0000 7f45eb400640 1 -- 172.21.15.55:0/2557904551 shutdown_connections 2024-05-10T03:35:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.508+0000 7f45eb400640 1 -- 172.21.15.55:0/2557904551 wait complete. 2024-05-10T03:35:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.510+0000 7f45eb400640 1 Processor -- start 2024-05-10T03:35:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.510+0000 7f45eb400640 1 -- start start 2024-05-10T03:35:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e40cf560 0x7f45e4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e415ef20 0x7f45e40ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40ca210 con 0x7f45e40cd9d0 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e41f2140 con 0x7f45e40cf560 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45eb400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4157f00 con 0x7f45e415ef20 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45e8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45e8a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39812/0 (socket says 172.21.15.55:39812) 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45e8a00640 1 -- 172.21.15.55:0/1914022450 learned_addr learned my addr 172.21.15.55:0/1914022450 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45e9400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e415ef20 0x7f45e40ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45dbe00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e40cf560 0x7f45e4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.512+0000 7f45e8a00640 1 -- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e415ef20 msgr2=0x7f45e40ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e8a00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e415ef20 0x7f45e40ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e8a00640 1 -- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e40cf560 msgr2=0x7f45e4152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e8a00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e40cf560 0x7f45e4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e8a00640 1 -- 172.21.15.55:0/1914022450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4009400 con 0x7f45e40cd9d0 2024-05-10T03:35:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e8a00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f45d4000c00 tx=0x7f45d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004230 con 0x7f45e40cd9d0 2024-05-10T03:35:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4004390 con 0x7f45e40cd9d0 2024-05-10T03:35:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e4153a10 con 0x7f45e40cd9d0 2024-05-10T03:35:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4003850 con 0x7f45e40cd9d0 2024-05-10T03:35:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.513+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e40c9b50 con 0x7f45e40cd9d0 2024-05-10T03:35:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.515+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e40ca350 con 0x7f45e40cd9d0 2024-05-10T03:35:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.516+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f45d4039050 con 0x7f45e40cd9d0 2024-05-10T03:35:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.517+0000 7f45e1600640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 0x7f45c006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.517+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f45d40a9cc0 con 0x7f45e40cd9d0 2024-05-10T03:35:15.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.518+0000 7f45dbe00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 0x7f45c006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:15.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.519+0000 7f45dbe00640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 0x7f45c006d110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006fd0 tx=0x7f45cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:15.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.521+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f45e40ca350 con 0x7f45e40cd9d0 2024-05-10T03:35:15.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.759+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e40ca350 con 0x7f45e40cd9d0 2024-05-10T03:35:15.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.760+0000 7f45e1600640 1 -- 172.21.15.55:0/1914022450 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4741 (secure 0 0 0) 0x7f45e40ca350 con 0x7f45e40cd9d0 2024-05-10T03:35:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:15.764 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:15.223165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7647},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:15.764 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 63 2024-05-10T03:35:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 msgr2=0x7f45c006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 0x7f45c006d110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006fd0 tx=0x7f45cc008040 comp rx=0 tx=0).stop 2024-05-10T03:35:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 msgr2=0x7f45e41f18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f45d4000c00 tx=0x7f45d40046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 shutdown_connections 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e415ef20 0x7f45e40ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45c006ac00 0x7f45c006d110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e40cf560 0x7f45e4152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 --2- 172.21.15.55:0/1914022450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45e40cd9d0 0x7f45e41f18a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 >> 172.21.15.55:0/1914022450 conn(0x7f45e40d10b0 msgr2=0x7f45e4154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 shutdown_connections 2024-05-10T03:35:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:15.763+0000 7f45eb400640 1 -- 172.21.15.55:0/1914022450 wait complete. 2024-05-10T03:35:16.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:35:15.223165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7647}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7647': {'gid': 7647, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.55:6835/1198619982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 1198619982}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 1198619982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:16.193 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7647:up:reconnect 2024-05-10T03:35:17.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.480+0000 7f7e12800640 1 Processor -- start 2024-05-10T03:35:17.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.480+0000 7f7e12800640 1 -- start start 2024-05-10T03:35:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e12800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7e0c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e12800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7e0c0cad80 0x7f7e0c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e12800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c06eeb0 con 0x7f7e0c071520 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e12800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c06d370 con 0x7f7e0c0cad80 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e12800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c071930 con 0x7f7e0c15dd60 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e0aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7e0c071520 0x7f7e0c0ca670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58164/0 (socket says 172.21.15.55:58164) 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e0aa00640 1 -- 172.21.15.55:0/197204245 learned_addr learned my addr 172.21.15.55:0/197204245 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.483+0000 7f7e0be00640 1 --2- 172.21.15.55:0/197204245 >> v2:172.21.15.55:3300/0 conn(0x7f7e0c0cad80 0x7f7e0c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0b400640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7e0c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568589178 0 0) 0x7f7e0c06eeb0 con 0x7f7e0c071520 2024-05-10T03:35:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df0003520 con 0x7f7e0c071520 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981649504 0 0) 0x7f7df0003520 con 0x7f7e0c071520 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 msgr2=0x7f7e0c070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7e0c070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 >> v2:172.21.15.55:3300/0 conn(0x7f7e0c0cad80 msgr2=0x7f7e0c06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 --2- 172.21.15.55:0/197204245 >> v2:172.21.15.55:3300/0 conn(0x7f7e0c0cad80 0x7f7e0c06dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c06f9a0 con 0x7f7e0c071520 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7df4001670 con 0x7f7e0c071520 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0b400640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7e0c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:17.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 >> v1:172.21.15.55:6789/0 conn(0x7f7e0c071520 legacy=0x7f7e0c0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df0003970 0x7f7df0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7df0004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7df0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c071930 con 0x7f7df00047c0 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df000ba30 con 0x7f7df0003970 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.484+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df000ddd0 con 0x7f7e0c15dd60 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0be00640 1 --2- 172.21.15.55:0/197204245 >> v2:172.21.15.55:3300/0 conn(0x7f7e0c0cad80 0x7f7e0c06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7df0004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0b400640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df0003970 0x7f7df0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 -- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 msgr2=0x7f7df0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c15dd60 0x7f7df0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 -- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df0003970 msgr2=0x7f7df0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df0003970 0x7f7df0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0aa00640 1 -- 172.21.15.55:0/197204245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0003520 con 0x7f7df00047c0 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.485+0000 7f7e0b400640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df0003970 0x7f7df0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:17.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.486+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7df0004bd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0701f0 tx=0x7f7df00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84abdd1c4c0c30e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:17.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4005ad0 con 0x7f7df00047c0 2024-05-10T03:35:17.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df402d070 con 0x7f7df00047c0 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e0a000640 1 -- 172.21.15.55:0/197204245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4005e70 con 0x7f7df00047c0 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e12800640 1 -- 172.21.15.55:0/197204245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 msgr2=0x7f7df0004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e12800640 1 --2- 172.21.15.55:0/197204245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7df0004bd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0701f0 tx=0x7f7df00057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e0be00640 1 -- 172.21.15.55:0/197204245 reap_dead start 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e12800640 1 -- 172.21.15.55:0/197204245 shutdown_connections 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.487+0000 7f7e12800640 1 -- 172.21.15.55:0/197204245 >> 172.21.15.55:0/197204245 conn(0x7f7e0c0b1b10 msgr2=0x7f7e0c0b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.488+0000 7f7e12800640 1 -- 172.21.15.55:0/197204245 shutdown_connections 2024-05-10T03:35:17.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.488+0000 7f7e12800640 1 -- 172.21.15.55:0/197204245 wait complete. 2024-05-10T03:35:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.490+0000 7f7e12800640 1 Processor -- start 2024-05-10T03:35:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.490+0000 7f7e12800640 1 -- start start 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e0c15c160 0x7f7e0c1ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c09f9b0 0x7f7e0c09fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c09f0f0 con 0x7f7df00047c0 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c09f510 con 0x7f7e0c15c160 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e12800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b4e80 con 0x7f7e0c09f9b0 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39844/0 (socket says 172.21.15.55:39844) 2024-05-10T03:35:17.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.492+0000 7f7e0b400640 1 -- 172.21.15.55:0/651603802 learned_addr learned my addr 172.21.15.55:0/651603802 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e0c15c160 0x7f7e0c1ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 -- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c09f9b0 msgr2=0x7f7e0c09fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c09f9b0 0x7f7e0c09fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 -- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e0c15c160 msgr2=0x7f7e0c1ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e0c15c160 0x7f7e0c1ed4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 -- 172.21.15.55:0/651603802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0a0300 con 0x7f7df00047c0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e0b400640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0046e0 tx=0x7f7dfc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc014600 con 0x7f7df00047c0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc004890 con 0x7f7df00047c0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0149c0 con 0x7f7df00047c0 2024-05-10T03:35:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.493+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c09e470 con 0x7f7df00047c0 2024-05-10T03:35:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.494+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c09ea30 con 0x7f7df00047c0 2024-05-10T03:35:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.495+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0cdbd0 con 0x7f7df00047c0 2024-05-10T03:35:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.496+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7dfc014b20 con 0x7f7df00047c0 2024-05-10T03:35:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.497+0000 7f7e03400640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 0x7f7de806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.497+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f7dfc089650 con 0x7f7df00047c0 2024-05-10T03:35:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.498+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 0x7f7de806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:17.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.499+0000 7f7e0aa00640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 0x7f7de806d140 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7df4032000 tx=0x7f7df4009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:17.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.502+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7e0c0cdbd0 con 0x7f7df00047c0 2024-05-10T03:35:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.737+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e0c0cdbd0 con 0x7f7df00047c0 2024-05-10T03:35:17.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.739+0000 7f7e03400640 1 -- 172.21.15.55:0/651603802 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f7e0c0cdbd0 con 0x7f7df00047c0 2024-05-10T03:35:17.742 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:17.743 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:17.243127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7647},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:17.743 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 65 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.741+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 msgr2=0x7f7de806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.741+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 0x7f7de806d140 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7df4032000 tx=0x7f7df4009030 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 msgr2=0x7f7e0c1f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0046e0 tx=0x7f7dfc007700 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 shutdown_connections 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e0c09f9b0 0x7f7e0c09fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7de806ac30 0x7f7de806d140 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e0c15c160 0x7f7e0c1ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 --2- 172.21.15.55:0/651603802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7df00047c0 0x7f7e0c1f3eb0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 >> 172.21.15.55:0/651603802 conn(0x7f7e0c0b1b10 msgr2=0x7f7e0c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 shutdown_connections 2024-05-10T03:35:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:17.742+0000 7f7e12800640 1 -- 172.21.15.55:0/651603802 wait complete. 2024-05-10T03:35:18.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:35:17.243127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7647}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7647': {'gid': 7647, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.55:6835/1198619982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 1198619982}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 1198619982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:18.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T03:35:18.173 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T03:35:18.174 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:35:18.188 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:35:18.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.440+0000 7fde48c00640 1 Processor -- start 2024-05-10T03:35:18.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.440+0000 7fde48c00640 1 -- start start 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde48c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde44070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde48c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde44071ee0 con 0x7fde440706d0 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde48c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde440ce900 con 0x7fde440716d0 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde48c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440ae5c0 con 0x7fde440ce4f0 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde43e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fde440706d0 0x7fde4415bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58168/0 (socket says 172.21.15.55:58168) 2024-05-10T03:35:18.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde43e00640 1 -- 172.21.15.55:0/4148230437 learned_addr learned my addr 172.21.15.55:0/4148230437 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.443+0000 7fde42a00640 1 --2- 172.21.15.55:0/4148230437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde44070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.444+0000 7fde42000640 1 -- 172.21.15.55:0/4148230437 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839659225 0 0) 0x7fde44071ee0 con 0x7fde440706d0 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.444+0000 7fde42000640 1 -- 172.21.15.55:0/4148230437 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde28003520 con 0x7fde440706d0 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.444+0000 7fde42a00640 1 -- 172.21.15.55:0/4148230437 >> v1:172.21.15.162:6789/0 conn(0x7fde440716d0 legacy=0x7fde440adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.444+0000 7fde42a00640 1 -- 172.21.15.55:0/4148230437 >> v1:172.21.15.55:6789/0 conn(0x7fde440706d0 legacy=0x7fde4415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.444+0000 7fde42a00640 1 -- 172.21.15.55:0/4148230437 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44071c00 con 0x7fde440ce4f0 2024-05-10T03:35:18.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.446+0000 7fde42a00640 1 --2- 172.21.15.55:0/4148230437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde44070190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde300097d0 tx=0x7fde3002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e142fc1c647dda6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.446+0000 7fde42000640 1 -- 172.21.15.55:0/4148230437 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30032440 con 0x7fde440ce4f0 2024-05-10T03:35:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.446+0000 7fde42000640 1 -- 172.21.15.55:0/4148230437 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde300325a0 con 0x7fde440ce4f0 2024-05-10T03:35:18.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.446+0000 7fde42000640 1 -- 172.21.15.55:0/4148230437 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30032910 con 0x7fde440ce4f0 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.447+0000 7fde48c00640 1 -- 172.21.15.55:0/4148230437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 msgr2=0x7fde44070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.447+0000 7fde48c00640 1 --2- 172.21.15.55:0/4148230437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde44070190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde300097d0 tx=0x7fde3002a210 comp rx=0 tx=0).stop 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.447+0000 7fde48c00640 1 -- 172.21.15.55:0/4148230437 shutdown_connections 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.447+0000 7fde48c00640 1 --2- 172.21.15.55:0/4148230437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde44070190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.447+0000 7fde48c00640 1 -- 172.21.15.55:0/4148230437 >> 172.21.15.55:0/4148230437 conn(0x7fde440b07b0 msgr2=0x7fde440b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.448+0000 7fde48c00640 1 -- 172.21.15.55:0/4148230437 shutdown_connections 2024-05-10T03:35:18.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.448+0000 7fde48c00640 1 -- 172.21.15.55:0/4148230437 wait complete. 2024-05-10T03:35:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.450+0000 7fde48c00640 1 Processor -- start 2024-05-10T03:35:18.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.450+0000 7fde48c00640 1 -- start start 2024-05-10T03:35:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.451+0000 7fde48c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.451+0000 7fde48c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440716d0 0x7fde44156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.451+0000 7fde48c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde441e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde48c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441f1780 con 0x7fde440706d0 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde48c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441f2730 con 0x7fde440716d0 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde48c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440ac020 con 0x7fde440ce4f0 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39850/0 (socket says 172.21.15.55:39850) 2024-05-10T03:35:18.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 -- 172.21.15.55:0/3338367816 learned_addr learned my addr 172.21.15.55:0/3338367816 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43e00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde441e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 -- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 msgr2=0x7fde441e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde42a00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440716d0 0x7fde44156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde441e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 -- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440716d0 msgr2=0x7fde44156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440716d0 0x7fde44156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43400640 1 -- 172.21.15.55:0/3338367816 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30009400 con 0x7fde440706d0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.452+0000 7fde43e00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde441e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde43400640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c420 tx=0x7fde3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde3b400640 1 -- 172.21.15.55:0/3338367816 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340052d0 con 0x7fde440706d0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde3b400640 1 -- 172.21.15.55:0/3338367816 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34005450 con 0x7fde440706d0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde3b400640 1 -- 172.21.15.55:0/3338367816 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34003830 con 0x7fde440706d0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44156be0 con 0x7fde440706d0 2024-05-10T03:35:18.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.453+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde441f1970 con 0x7fde440706d0 2024-05-10T03:35:18.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.455+0000 7fde3b400640 1 -- 172.21.15.55:0/3338367816 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fde34003990 con 0x7fde440706d0 2024-05-10T03:35:18.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.456+0000 7fde3b400640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 0x7fde2406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.457+0000 7fde3b400640 1 -- 172.21.15.55:0/3338367816 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fde34088bb0 con 0x7fde440706d0 2024-05-10T03:35:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.457+0000 7fde42a00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 0x7fde2406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.458+0000 7fde51194740 1 Processor -- start 2024-05-10T03:35:18.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.458+0000 7fde51194740 1 -- start start 2024-05-10T03:35:18.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.458+0000 7fde42a00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 0x7fde2406d110 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e187cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e0c6a70 con 0x56273dd9d590 2024-05-10T03:35:18.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e0c4200 con 0x56273ddbe270 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56273e0bfaf0 con 0x56273dd953b0 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.459+0000 7fde43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39856/0 (socket says 172.21.15.55:39856) 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 -- 172.21.15.55:0/1957583782 learned_addr learned my addr 172.21.15.55:0/1957583782 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde42a00640 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43400640 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e187cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 -- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 msgr2=0x56273e1a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 -- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 msgr2=0x56273e187cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e187cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 -- 172.21.15.55:0/1957583782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56273e08ab90 con 0x56273dd9d590 2024-05-10T03:35:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.460+0000 7fde43e00640 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fde3c002010 tx=0x7fde3c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06fbf3b6f2c1ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde2f400640 1 -- 172.21.15.55:0/1957583782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00b070 con 0x56273dd9d590 2024-05-10T03:35:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde2f400640 1 -- 172.21.15.55:0/1957583782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c013670 con 0x56273dd9d590 2024-05-10T03:35:18.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde2f400640 1 -- 172.21.15.55:0/1957583782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0139e0 con 0x56273dd9d590 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde51194740 1 -- 172.21.15.55:0/1957583782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 msgr2=0x56273e1a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde51194740 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fde3c002010 tx=0x7fde3c00d520 comp rx=0 tx=0).stop 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde51194740 1 -- 172.21.15.55:0/1957583782 shutdown_connections 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde51194740 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a2f40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.461+0000 7fde51194740 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.462+0000 7fde51194740 1 --2- 172.21.15.55:0/1957583782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e187cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.462+0000 7fde51194740 1 -- 172.21.15.55:0/1957583782 >> 172.21.15.55:0/1957583782 conn(0x56273ddb23c0 msgr2=0x56273e15d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.462+0000 7fde51194740 1 -- 172.21.15.55:0/1957583782 shutdown_connections 2024-05-10T03:35:18.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.462+0000 7fde51194740 1 -- 172.21.15.55:0/1957583782 wait complete. 2024-05-10T03:35:18.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.463+0000 7fde51194740 1 -- start start 2024-05-10T03:35:18.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.463+0000 7fde51194740 1 Processor -- start 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273ddbe270 0x56273e1b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e061220 con 0x56273dd953b0 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e060ec0 con 0x56273dd9d590 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.464+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56273de26f20 con 0x56273ddbe270 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde51194740 10 client.0 _ll_register_callbacks cb 0x56273e0b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:18.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39870/0 (socket says 172.21.15.55:39870) 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 -- 172.21.15.55:0/4268771982 learned_addr learned my addr 172.21.15.55:0/4268771982 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 -- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273ddbe270 msgr2=0x56273e1b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273ddbe270 0x56273e1b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 -- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 msgr2=0x56273e1a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 -- 172.21.15.55:0/4268771982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde28003520 con 0x56273dd953b0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde43400640 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fde34086e00 tx=0x7fde3405b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde3912657127f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.465+0000 7fde23e00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3406f170 con 0x56273dd953b0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fde34055b40 con 0x56273dd953b0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56273e0e9c90 con 0x56273dd953b0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde3405f6d0 con 0x56273dd953b0 2024-05-10T03:35:18.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 10 client.7686 fetch_fsmap learned FSMap version 65 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x56273e0be170 con 0x56273dd953b0 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56273e0e9c90 con 0x56273dd953b0 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fde340534d0 con 0x56273dd953b0 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 10 client.7686 fetch_fsmap finished waiting for FSMap version 65 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 10 client.7686 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 20 client.7686 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273dd7ce40 0x56273e157be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 4 client.7686 mds_command: new command op to 7647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.466+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 --> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56273ddb33c0 con 0x56273dd7ce40 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.467+0000 7fde42a00640 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273dd7ce40 0x56273e157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.467+0000 7fde42a00640 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273dd7ce40 0x56273e157be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f01280bcddc2ce server_cookie=7b662588f0c00a5 in_seq=0 out_seq=0 2024-05-10T03:35:18.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.467+0000 7fde23e00640 10 client.7686 ms_handle_connect on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.468+0000 7fde23e00640 1 -- 172.21.15.55:0/4268771982 <== mds.0 v2:172.21.15.55:6834/1198619982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56273ddb33c0 con 0x56273dd7ce40 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.468+0000 7fde23e00640 10 client.7686 handle_command_reply: tid=0 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.468+0000 7fde51194740 1 client.7686 shutdown 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 msgr2=0x56273e19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fde34086e00 tx=0x7fde3405b5d0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 shutdown_connections 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273dd7ce40 0x56273e157be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273ddbe270 0x56273e1b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e1a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 --2- 172.21.15.55:0/4268771982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd953b0 0x56273e19fac0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde23e00640 0 client.7686 ms_handle_reset on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 >> 172.21.15.55:0/4268771982 conn(0x56273ddb23c0 msgr2=0x56273e15d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 shutdown_connections 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 1 -- 172.21.15.55:0/4268771982 wait complete. 2024-05-10T03:35:18.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.469+0000 7fde51194740 20 client.7686 trim_cache size 0 max 16384 2024-05-10T03:35:18.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.487+0000 7fde51194740 1 Processor -- start 2024-05-10T03:35:18.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.487+0000 7fde51194740 1 -- start start 2024-05-10T03:35:18.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e1a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e0b8a10 con 0x56273dd9d590 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e19e8e0 con 0x56273ddbe270 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.488+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56273e15f6b0 con 0x56273dd953b0 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39874/0 (socket says 172.21.15.55:39874) 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 -- 172.21.15.55:0/1699514271 learned_addr learned my addr 172.21.15.55:0/1699514271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde43400640 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 -- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 msgr2=0x56273e1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde43e00640 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e1a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 -- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 msgr2=0x56273e1a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e1a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 -- 172.21.15.55:0/1699514271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56273e0f3130 con 0x56273dd9d590 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.489+0000 7fde42a00640 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fde300098f0 tx=0x7fde3002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4382ce13736d1ad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde2fe00640 1 -- 172.21.15.55:0/1699514271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30032de0 con 0x56273dd9d590 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde2fe00640 1 -- 172.21.15.55:0/1699514271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde300044e0 con 0x56273dd9d590 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde2fe00640 1 -- 172.21.15.55:0/1699514271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30003850 con 0x56273dd9d590 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 -- 172.21.15.55:0/1699514271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 msgr2=0x56273e1a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fde300098f0 tx=0x7fde3002c000 comp rx=0 tx=0).stop 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 -- 172.21.15.55:0/1699514271 shutdown_connections 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273ddbe270 0x56273e1a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273dd9d590 0x56273e1a1e00 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 --2- 172.21.15.55:0/1699514271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.490+0000 7fde51194740 1 -- 172.21.15.55:0/1699514271 >> 172.21.15.55:0/1699514271 conn(0x56273ddb23c0 msgr2=0x56273e15d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.491+0000 7fde51194740 1 -- 172.21.15.55:0/1699514271 shutdown_connections 2024-05-10T03:35:18.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.491+0000 7fde51194740 1 -- 172.21.15.55:0/1699514271 wait complete. 2024-05-10T03:35:18.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.492+0000 7fde51194740 1 -- start start 2024-05-10T03:35:18.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.492+0000 7fde51194740 1 Processor -- start 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e20cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e20d880 con 0x56273ddbe270 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56273e20cb30 con 0x56273dd9d590 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56273e20cd80 con 0x56273dd953b0 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.493+0000 7fde51194740 10 client.0 _ll_register_callbacks cb 0x7fde340574b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39886/0 (socket says 172.21.15.55:39886) 2024-05-10T03:35:18.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 -- 172.21.15.55:0/1050341394 learned_addr learned my addr 172.21.15.55:0/1050341394 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde42a00640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e20cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43400640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 -- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 msgr2=0x56273e1bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1bb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 -- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 msgr2=0x56273e20cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e20cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.494+0000 7fde43e00640 1 -- 172.21.15.55:0/1050341394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde44071c00 con 0x56273ddbe270 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde43e00640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d560 tx=0x7fde3c0014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f750f2b3e99c13da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde23e00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:18.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c013e60 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7fde3c0072e0 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde3c013670 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde3c0074a0 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde51194740 10 client.7692 fetch_fsmap learned FSMap version 65 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x56273e060430 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.495+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c013670 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fde3c007600 con 0x56273ddbe270 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 10 client.7692 fetch_fsmap finished waiting for FSMap version 65 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 10 client.7692 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 20 client.7692 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273e20f940 0x56273e20fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 4 client.7692 mds_command: new command op to 7647 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:35:18.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 --> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56273ddc46c0 con 0x56273e20f940 2024-05-10T03:35:18.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.496+0000 7fde43400640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273e20f940 0x56273e20fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde43400640 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273e20f940 0x56273e20fd50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ba9d5d3e79c05f server_cookie=87200459f17da62a in_seq=0 out_seq=0 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde23e00640 10 client.7692 ms_handle_connect on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde23e00640 1 -- 172.21.15.55:0/1050341394 <== mds.0 v2:172.21.15.55:6834/1198619982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x56273ddc46c0 con 0x56273e20f940 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde23e00640 10 client.7692 handle_command_reply: tid=0 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 client.7692 shutdown 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 msgr2=0x56273e1bac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d560 tx=0x7fde3c0014d0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 shutdown_connections 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x56273e20f940 0x56273e20fd50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56273ddbe270 0x56273e1bac20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56273dd9d590 0x56273e20cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 --2- 172.21.15.55:0/1050341394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56273dd953b0 0x56273e1bb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 >> 172.21.15.55:0/1050341394 conn(0x56273ddb23c0 msgr2=0x56273e1a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde23e00640 0 client.7692 ms_handle_reset on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 shutdown_connections 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.497+0000 7fde51194740 1 -- 172.21.15.55:0/1050341394 wait complete. 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde51194740 20 client.7692 trim_cache size 0 max 16384 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:18.505 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7647, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 4.2652789159999998, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 65, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 108, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 108, 2024-05-10T03:35:18.506 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 5.0010694469999999 2024-05-10T03:35:18.507 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:18.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 msgr2=0x7fde2406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 0x7fde2406d110 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fde30034000 tx=0x7fde3002e000 comp rx=0 tx=0).stop 2024-05-10T03:35:18.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 msgr2=0x7fde441475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c420 tx=0x7fde3400c8e0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 shutdown_connections 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde440ce4f0 0x7fde441e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fde2406ac00 0x7fde2406d110 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde440716d0 0x7fde44156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 --2- 172.21.15.55:0/3338367816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde440706d0 0x7fde441475e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.498+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 >> 172.21.15.55:0/3338367816 conn(0x7fde440b07b0 msgr2=0x7fde440aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.499+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 shutdown_connections 2024-05-10T03:35:18.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.499+0000 7fde48c00640 1 -- 172.21.15.55:0/3338367816 wait complete. 2024-05-10T03:35:18.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:35:18.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.797+0000 7f45f5400640 1 Processor -- start 2024-05-10T03:35:18.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.798+0000 7f45f5400640 1 -- start start 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45f5400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f0071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45f5400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f45f0070d70 0x7f45f0071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45f5400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f00ccb90 con 0x7f45f006fe90 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45f5400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f00b2e90 con 0x7f45f0070d70 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45f5400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00ccdc0 con 0x7f45f015a6b0 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45e5e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f45f006fe90 0x7f45f015e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58180/0 (socket says 172.21.15.55:58180) 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45e5e00640 1 -- 172.21.15.55:0/2349987067 learned_addr learned my addr 172.21.15.55:0/2349987067 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:18.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.800+0000 7f45ef200640 1 --2- 172.21.15.55:0/2349987067 >> v2:172.21.15.55:3300/0 conn(0x7f45f0070d70 0x7f45f0071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ee800640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f0071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ede00640 1 -- 172.21.15.55:0/2349987067 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707944622 0 0) 0x7f45f00ccb90 con 0x7f45f006fe90 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ede00640 1 -- 172.21.15.55:0/2349987067 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d4005730 con 0x7f45f006fe90 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ef200640 1 -- 172.21.15.55:0/2349987067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 msgr2=0x7f45f0071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ef200640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f0071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ef200640 1 -- 172.21.15.55:0/2349987067 >> v1:172.21.15.55:6789/0 conn(0x7f45f006fe90 legacy=0x7f45f015e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.801+0000 7f45ef200640 1 -- 172.21.15.55:0/2349987067 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00cd0a0 con 0x7f45f0070d70 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.802+0000 7f45ef200640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071d80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009d90 tx=0x7f45e802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7e2c9bb7756c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.802+0000 7f45ede00640 1 -- 172.21.15.55:0/2349987067 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8007740 con 0x7f45f0070d70 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.802+0000 7f45ede00640 1 -- 172.21.15.55:0/2349987067 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e802d050 con 0x7f45f0070d70 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.802+0000 7f45ede00640 1 -- 172.21.15.55:0/2349987067 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8007aa0 con 0x7f45f0070d70 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 -- 172.21.15.55:0/2349987067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 msgr2=0x7f45f0071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071d80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009d90 tx=0x7f45e802a660 comp rx=0 tx=0).stop 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 -- 172.21.15.55:0/2349987067 shutdown_connections 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071d80 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 --2- 172.21.15.55:0/2349987067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f0071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 -- 172.21.15.55:0/2349987067 >> 172.21.15.55:0/2349987067 conn(0x7f45f00b0b80 msgr2=0x7f45f00b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:18.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 -- 172.21.15.55:0/2349987067 shutdown_connections 2024-05-10T03:35:18.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.803+0000 7f45f5400640 1 -- 172.21.15.55:0/2349987067 wait complete. 2024-05-10T03:35:18.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.805+0000 7f45f5400640 1 Processor -- start 2024-05-10T03:35:18.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.806+0000 7f45f5400640 1 -- start start 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45f006fe90 0x7f45f0071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0157750 con 0x7f45f0070d70 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0156040 con 0x7f45f006fe90 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45f5400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0147290 con 0x7f45f015a6b0 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45e5e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39912/0 (socket says 172.21.15.55:39912) 2024-05-10T03:35:18.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.807+0000 7f45e5e00640 1 -- 172.21.15.55:0/1246864026 learned_addr learned my addr 172.21.15.55:0/1246864026 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45ef200640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 -- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 msgr2=0x7f45f015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f015b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 -- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45f006fe90 msgr2=0x7f45f0071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45f006fe90 0x7f45f0071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 -- 172.21.15.55:0/1246864026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8009a70 con 0x7f45f0070d70 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e5e00640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45d8001eb0 tx=0x7f45d800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8005ce0 con 0x7f45f0070d70 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d8005e60 con 0x7f45f0070d70 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80072f0 con 0x7f45f0070d70 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.808+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f0155d90 con 0x7f45f0070d70 2024-05-10T03:35:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.809+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f0156570 con 0x7f45f0070d70 2024-05-10T03:35:18.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.810+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f0067320 con 0x7f45f0070d70 2024-05-10T03:35:18.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.811+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f45d8007450 con 0x7f45f0070d70 2024-05-10T03:35:18.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.812+0000 7f45e7400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 0x7f45cc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:18.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.812+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f45d8088c40 con 0x7f45f0070d70 2024-05-10T03:35:18.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.813+0000 7f45ee800640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 0x7f45cc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:18.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.814+0000 7f45ee800640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 0x7f45cc06d110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45e0006fd0 tx=0x7f45e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:18.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:18.817+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f45d8053b50 con 0x7f45f0070d70 2024-05-10T03:35:19.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.055+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f45f01517e0 con 0x7f45f0070d70 2024-05-10T03:35:19.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.056+0000 7f45e7400640 1 -- 172.21.15.55:0/1246864026 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f45d8017070 con 0x7f45f0070d70 2024-05-10T03:35:19.060 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 msgr2=0x7f45cc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 0x7f45cc06d110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45e0006fd0 tx=0x7f45e0008040 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 msgr2=0x7f45f0071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45d8001eb0 tx=0x7f45d800c820 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 shutdown_connections 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45f015a6b0 0x7f45f015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45f0070d70 0x7f45f0071a50 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f45cc06ac00 0x7f45cc06d110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 --2- 172.21.15.55:0/1246864026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45f006fe90 0x7f45f0071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 >> 172.21.15.55:0/1246864026 conn(0x7f45f00b0b80 msgr2=0x7f45f00cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 shutdown_connections 2024-05-10T03:35:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:19.059+0000 7f45f5400640 1 -- 172.21.15.55:0/1246864026 wait complete. 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:35:19.090 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:19.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.091 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-05-10T03:35:19.145 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:19.146 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42132sec preferred_lft 42132sec 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:35:19.147 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f88b:ecff:fe3a:ea5c/64 scope link 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 86:2b:34:54:9a:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::842b:34ff:fe54:9a75/64 scope link 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 5a:f4:60:71:46:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::58f4:60ff:fe71:4666/64 scope link 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:19.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.148 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:35:19.222 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-05-10T03:35:19.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.226 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-05-10T03:35:19.279 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-05-10T03:35:19.280 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:35:19.280 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-10T03:35:19.280 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-10T03:35:19.280 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:35:19.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.281 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2024-05-10T03:35:19.281 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-05-10T03:35:19.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-10T03:35:19.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-10T03:35:19.282 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-10T03:35:19.282 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-05-10T03:35:19.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.430 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:35:19.481 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:35:19.481 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:35:19.482 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:19.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-05-10T03:35:19.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:19.566 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:19.621 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T03:35:19.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:19.622 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:19.676 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:35:19.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.676 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:19.886 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:19.886 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:19.886 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:19.914 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:19.915 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:19.915 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:19.915 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:19.915 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:35:19.950 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:35:19.952 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-10T03:35:19.952 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:35:19.952 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:35:19.952 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:35:19.952 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:35:19.957 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:35:19.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:19.958 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:20.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:20.038 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:35:20.093 INFO:teuthology.orchestra.run.smithi055.stdout:client5780 2024-05-10T03:35:20.094 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:20.094 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:20.145 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-10T03:35:20.235 INFO:tasks.ceph.mds.a.smithi055.stderr:2024-05-10T03:35:20.230+0000 7fb298600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:35:20.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.432+0000 7f499e000640 1 Processor -- start 2024-05-10T03:35:20.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.432+0000 7f499e000640 1 -- start start 2024-05-10T03:35:20.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.434+0000 7f499e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4998152280 0x7f499815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.434+0000 7f499e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f499815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.434+0000 7f499e000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499815c040 con 0x7f499815ba60 2024-05-10T03:35:20.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.434+0000 7f499e000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499815c320 con 0x7f4998152280 2024-05-10T03:35:20.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.434+0000 7f499e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499815c600 con 0x7f4998152690 2024-05-10T03:35:20.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f499cc00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f499815ba60 0x7f4998071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:58192/0 (socket says 172.21.15.55:58192) 2024-05-10T03:35:20.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f499cc00640 1 -- 172.21.15.55:0/3081097574 learned_addr learned my addr 172.21.15.55:0/3081097574 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f4997e00640 1 --2- 172.21.15.55:0/3081097574 >> v2:172.21.15.162:3300/0 conn(0x7f4998152280 0x7f499815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f499d600640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f499815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665964004 0 0) 0x7f499815c040 con 0x7f499815ba60 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.435+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4980003520 con 0x7f499815ba60 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735098865 0 0) 0x7f4980003520 con 0x7f499815ba60 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 msgr2=0x7f499815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f499815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 >> v2:172.21.15.162:3300/0 conn(0x7f4998152280 msgr2=0x7f499815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 --2- 172.21.15.55:0/3081097574 >> v2:172.21.15.162:3300/0 conn(0x7f4998152280 0x7f499815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499815c8e0 con 0x7f499815ba60 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f498c001670 con 0x7f499815ba60 2024-05-10T03:35:20.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 >> v1:172.21.15.55:6789/0 conn(0x7f499815ba60 legacy=0x7f4998071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f4980003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4980003ef0 0x7f49800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980004d00 0x7f4980005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499815c320 con 0x7f4980003ef0 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499815c600 con 0x7f4980004d00 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498000b9d0 con 0x7f4998152690 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.436+0000 7f4997e00640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4980003ef0 0x7f49800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f499d600640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980004d00 0x7f4980005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f499cc00640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f4980003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f4997e00640 1 -- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 msgr2=0x7f4980003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f4997e00640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998152690 0x7f4980003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f4997e00640 1 -- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980004d00 msgr2=0x7f4980005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f4997e00640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980004d00 0x7f4980005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.437+0000 7f4997e00640 1 -- 172.21.15.55:0/3081097574 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980003520 con 0x7f4980003ef0 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.438+0000 7f4997e00640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4980003ef0 0x7f49800047c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f499806fbb0 tx=0x7f49800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baa46e96182bc7f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.438+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988031400 con 0x7f4980003ef0 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.438+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498802d050 con 0x7f4980003ef0 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.438+0000 7f4997400640 1 -- 172.21.15.55:0/3081097574 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988031780 con 0x7f4980003ef0 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 -- 172.21.15.55:0/3081097574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4980003ef0 msgr2=0x7f49800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 --2- 172.21.15.55:0/3081097574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4980003ef0 0x7f49800047c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f499806fbb0 tx=0x7f49800057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499d600640 1 -- 172.21.15.55:0/3081097574 reap_dead start 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 -- 172.21.15.55:0/3081097574 shutdown_connections 2024-05-10T03:35:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 -- 172.21.15.55:0/3081097574 >> 172.21.15.55:0/3081097574 conn(0x7f49980b07b0 msgr2=0x7f49980b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:20.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 -- 172.21.15.55:0/3081097574 shutdown_connections 2024-05-10T03:35:20.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.439+0000 7f499e000640 1 -- 172.21.15.55:0/3081097574 wait complete. 2024-05-10T03:35:20.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.442+0000 7f499e000640 1 Processor -- start 2024-05-10T03:35:20.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.442+0000 7f499e000640 1 -- start start 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 0x7f49981f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998149620 0x7f4998149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49981f6de0 con 0x7f499815af30 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499814aba0 con 0x7f4980003ef0 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.443+0000 7f499e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499814e2c0 con 0x7f4998149620 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39940/0 (socket says 172.21.15.55:39940) 2024-05-10T03:35:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 -- 172.21.15.55:0/2205294072 learned_addr learned my addr 172.21.15.55:0/2205294072 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f499cc00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 0x7f49981f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f499d600640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998149620 0x7f4998149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 -- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998149620 msgr2=0x7f4998149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998149620 0x7f4998149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 -- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 msgr2=0x7f49981f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 0x7f49981f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f4997e00640 1 -- 172.21.15.55:0/2205294072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499814e5a0 con 0x7f499815af30 2024-05-10T03:35:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.444+0000 7f499cc00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 0x7f49981f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f4997e00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f498802aaa0 tx=0x7f4988004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988003c50 con 0x7f499815af30 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4988003dd0 con 0x7f499815af30 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988044650 con 0x7f499815af30 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49981f6720 con 0x7f499815af30 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.445+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499815b340 con 0x7f499815af30 2024-05-10T03:35:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.446+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4998067240 con 0x7f499815af30 2024-05-10T03:35:20.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.448+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4988038070 con 0x7f499815af30 2024-05-10T03:35:20.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.449+0000 7f4994c00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 0x7f497c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:20.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.449+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f49880a9750 con 0x7f499815af30 2024-05-10T03:35:20.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.449+0000 7f499cc00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 0x7f497c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:20.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.451+0000 7f499cc00640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 0x7f497c06d140 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f498c001820 tx=0x7f498c000fe0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:20.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.453+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4988074710 con 0x7f499815af30 2024-05-10T03:35:20.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.693+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4998151cd0 con 0x7f499815af30 2024-05-10T03:35:20.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.694+0000 7f4994c00640 1 -- 172.21.15.55:0/2205294072 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f498803f070 con 0x7f499815af30 2024-05-10T03:35:20.698 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.696+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 msgr2=0x7f497c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.696+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 0x7f497c06d140 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f498c001820 tx=0x7f498c000fe0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 msgr2=0x7f499814db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f498802aaa0 tx=0x7f4988004730 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 shutdown_connections 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4998149620 0x7f4998149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f499815af30 0x7f499814db90 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f497c06ac30 0x7f497c06d140 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 --2- 172.21.15.55:0/2205294072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4980003ef0 0x7f49981f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 >> 172.21.15.55:0/2205294072 conn(0x7f49980b07b0 msgr2=0x7f499806bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 shutdown_connections 2024-05-10T03:35:20.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:20.697+0000 7f499e000640 1 -- 172.21.15.55:0/2205294072 wait complete. 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-10T03:35:20.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T03:35:20.729 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:20.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:20.729 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42156sec preferred_lft 42156sec 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::142a:3aff:feaa:e80c/64 scope link 2024-05-10T03:35:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 6e:95:e0:62:62:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::6c95:e0ff:fe62:62db/64 scope link 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether c2:0f:29:dd:e1:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c00f:29ff:fedd:e14e/64 scope link 2024-05-10T03:35:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-05-10T03:35:20.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:20.791 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T03:35:20.869 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-05-10T03:35:20.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:20.873 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-05-10T03:35:20.923 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-05-10T03:35:20.923 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T03:35:20.923 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T03:35:20.924 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T03:35:20.924 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T03:35:20.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T03:35:20.924 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-05-10T03:35:21.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:21.072 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:35:21.122 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:35:21.122 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:35:21.123 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:21.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:21.123 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-05-10T03:35:21.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:21.201 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:21.256 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-10T03:35:21.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T03:35:21.257 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:21.305 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-10T03:35:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:21.305 DEBUG:teuthology.orchestra.run.smithi162:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:21.502 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:21.503 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:21.503 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:21.528 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:21.528 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:21.528 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:21.528 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:21.528 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-10T03:35:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-10T03:35:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:invalid new device string format 2024-05-10T03:35:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.162:3300,172.21.15.162:3301" 2024-05-10T03:35:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-10T03:35:21.572 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790" 2024-05-10T03:35:21.572 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.162:6789,172.21.15.162:6790:/ 2024-05-10T03:35:21.572 INFO:teuthology.orchestra.run.smithi162.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=340d5548-dd78-4f40-a3a8-44ee99f8e0d8" will pass to kernel 2024-05-10T03:35:21.574 INFO:teuthology.orchestra.run.smithi162.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-10T03:35:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: You just mounted a file system that supports labels which does not 2024-05-10T03:35:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-10T03:35:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: applications will generate AVC messages and not be allowed access to 2024-05-10T03:35:21.574 INFO:teuthology.orchestra.run.smithi162.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-10T03:35:21.578 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T03:35:21.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:21.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:21.650 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T03:35:21.702 INFO:teuthology.orchestra.run.smithi162.stdout:client7710 2024-05-10T03:35:21.702 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:21.702 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:21.754 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-05-10T03:35:22.074 INFO:teuthology.orchestra.run.smithi055.stdout:[{"id":7710,"entity":{"name":{"type":"client","num":7710},"addr":{"type":"v1","addr":"172.21.15.162:0","nonce":585571976}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":0.49999006699999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99885929142105689,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7710 v1:172.21.15.162:0/585571976","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi162","kernel_version":"6.9.0-rc3-gfb9a6a44107e","root":"/"}},{"id":5780,"entity":{"name":{"type":"client","num":5780},"addr":{"type":"v1","addr":"172.21.15.55:0","nonce":1888309702}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":2.1139580059999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.9951391245980532,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":3,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5780 v1:172.21.15.55:0/1888309702","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi055","kernel_version":"6.9.0-rc3-gfb9a6a44107e","root":"/"}}] 2024-05-10T03:35:22.075 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi162", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.162:0", "nonce": 585571976, "type": "v1" }, "name": { "num": 7710, "type": "client" } }, "id": 7710, "inst": "client.7710 v1:172.21.15.162:0/585571976", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9988592914210569 }, "state": "open", "uptime": 0.499990067 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi055", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.55:0", "nonce": 1888309702, "type": "v1" }, "name": { "num": 5780, "type": "client" } }, "id": 5780, "inst": "client.5780 v1:172.21.15.55:0/1888309702", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 3, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9951391245980532 }, "state": "open", "uptime": 2.113958006 } ] 2024-05-10T03:35:22.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T03:35:22.402 INFO:teuthology.orchestra.run.smithi055.stdout:{"message":"","return_code":0} 2024-05-10T03:35:22.402 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T03:35:22.402 DEBUG:teuthology.orchestra.run.smithi055:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-05-10T03:35:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.578+0000 7fe748a1e280 1 -- start start 2024-05-10T03:35:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.579+0000 7fe748a1e280 1 Processor -- start 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 0x564365698b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 0x564365698580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643655dea00 con 0x564365693180 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643655de780 con 0x564365692880 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.580+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643655de640 con 0x564365692d00 2024-05-10T03:35:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39948/0 (socket says 172.21.15.55:39948) 2024-05-10T03:35:22.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 -- 172.21.15.55:0/2788476247 learned_addr learned my addr 172.21.15.55:0/2788476247 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746000640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 0x564365698b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe745600640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 0x564365698580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 -- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 msgr2=0x564365698580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 0x564365698580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 -- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 msgr2=0x564365698b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 0x564365698b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 -- 172.21.15.55:0/2788476247 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643615029c0 con 0x564365693180 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe745600640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 0x564365698580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746000640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 0x564365698b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.581+0000 7fe746a00640 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x564366346de0 tx=0x5643662ea600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708b5e175970f4fb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe742e00640 1 -- 172.21.15.55:0/2788476247 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564365555600 con 0x564365693180 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe742e00640 1 -- 172.21.15.55:0/2788476247 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56436556c180 con 0x564365693180 2024-05-10T03:35:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe742e00640 1 -- 172.21.15.55:0/2788476247 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643655551e0 con 0x564365693180 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564361502b60 con 0x564365693180 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564361502820 con 0x564365693180 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe748a1e280 4 waiting for MDS map... 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe742e00640 1 -- 172.21.15.55:0/2788476247 <== mon.0 v2:172.21.15.55:3300/0 4 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x564361508380 con 0x564365693180 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe742e00640 1 -- 172.21.15.55:0/2788476247 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x56436555ef00 con 0x564365693180 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.582+0000 7fe748a1e280 4 Got MDS map 65 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.583+0000 7fe748a1e280 10 main: main 2024-05-10T03:35:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.583+0000 7fe748a1e280 4 main: connecting to RADOS... 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.584+0000 7fe748a1e280 1 Processor -- start 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.584+0000 7fe748a1e280 1 -- start start 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365693a80 0x564365699600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564366456480 0x56436569a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x564365557e00 con 0x564366456000 2024-05-10T03:35:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643655de780 con 0x564365693a80 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643655de500 con 0x564366456480 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe746000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe746000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39954/0 (socket says 172.21.15.55:39954) 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.585+0000 7fe746000640 1 -- 172.21.15.55:0/3154991809 learned_addr learned my addr 172.21.15.55:0/3154991809 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 -- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564366456480 msgr2=0x56436569a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564366456480 0x56436569a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe745600640 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365693a80 0x564365699600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 -- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365693a80 msgr2=0x564365699600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365693a80 0x564365699600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 -- 172.21.15.55:0/3154991809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643615024e0 con 0x564366456000 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe746000640 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x5643663462a0 tx=0x5643662ea3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a601c66aead06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe740600640 1 -- 172.21.15.55:0/3154991809 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564365555080 con 0x564366456000 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe740600640 1 -- 172.21.15.55:0/3154991809 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56436556c900 con 0x564366456000 2024-05-10T03:35:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.586+0000 7fe740600640 1 -- 172.21.15.55:0/3154991809 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564365555600 con 0x564366456000 2024-05-10T03:35:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 -- 172.21.15.55:0/3154991809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 msgr2=0x564365699b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x5643663462a0 tx=0x5643662ea3c0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 -- 172.21.15.55:0/3154991809 shutdown_connections 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564366456480 0x56436569a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564366456000 0x564365699b80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 --2- 172.21.15.55:0/3154991809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365693a80 0x564365699600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 -- 172.21.15.55:0/3154991809 >> 172.21.15.55:0/3154991809 conn(0x564365692400 msgr2=0x5643655bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 -- 172.21.15.55:0/3154991809 shutdown_connections 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.587+0000 7fe748a1e280 1 -- 172.21.15.55:0/3154991809 wait complete. 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.588+0000 7fe748a1e280 1 Processor -- start 2024-05-10T03:35:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.588+0000 7fe748a1e280 1 -- start start 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692400 0x56436569b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 0x56436569ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56436630e480 0x56436569a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643655de3c0 con 0x56436630e000 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643655de280 con 0x564365692400 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe748a1e280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643655de140 con 0x56436630e480 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 0x56436569ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 0x56436569ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39966/0 (socket says 172.21.15.55:39966) 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 -- 172.21.15.55:0/879569764 learned_addr learned my addr 172.21.15.55:0/879569764 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 -- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56436630e480 msgr2=0x56436569a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56436630e480 0x56436569a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 -- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692400 msgr2=0x56436569b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692400 0x56436569b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.590+0000 7fe746000640 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564361502d00 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe746000640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 0x56436569ac00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x564366347cb0 tx=0x5643662ea360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe73e800640 1 -- 172.21.15.55:0/879569764 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643655551e0 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe73e800640 1 -- 172.21.15.55:0/879569764 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56436556c480 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe73e800640 1 -- 172.21.15.55:0/879569764 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564365554dc0 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643615024e0 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643615029c0 con 0x56436630e000 2024-05-10T03:35:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.591+0000 7fe748a1e280 4 main: resolving pool 15 2024-05-10T03:35:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.593+0000 7fe73e800640 1 -- 172.21.15.55:0/879569764 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x5643656ba840 con 0x56436630e000 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.594+0000 7fe73e800640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x564366311600 0x56436569b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.594+0000 7fe73e800640 1 -- 172.21.15.55:0/879569764 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x564361508540 con 0x56436630e000 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.594+0000 7fe748a1e280 4 main: creating IoCtx.. 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.594+0000 7fe748a1e280 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x56436630e900 0x56436569a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.594+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v8 -- 0x5643663e5c00 con 0x56436630e900 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x56436630e900 0x56436569a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x564366311600 0x56436569b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x56436630e900 0x56436569a100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 -- 172.21.15.55:0/879569764 <== osd.2 v2:172.21.15.55:6800/35475 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv5 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x5643663d6b40 con 0x56436630e900 2024-05-10T03:35:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v8 -- 0x564365900400 con 0x56436630e900 2024-05-10T03:35:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x564366311600 0x56436569b700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x564366459aa0 tx=0x564366405f20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe745600640 1 -- 172.21.15.55:0/879569764 <== osd.2 v2:172.21.15.55:6800/35475 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=475b] v0'0 uv5 ondisk = 0) v8 ==== 159+0+475 (crc 0 0 0) 0x5643663d6b40 con 0x56436630e900 2024-05-10T03:35:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.596+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- osd_op(unknown.0.0:3 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v8 -- 0x564365900800 con 0x56436630e900 2024-05-10T03:35:22.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe745600640 1 -- 172.21.15.55:0/879569764 <== osd.2 v2:172.21.15.55:6800/35475 3 ==== osd_op_reply(3 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x5643663d6b40 con 0x56436630e900 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "0": { 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "data": { 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions": [ 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7710, 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": { 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 7710 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.601 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.162:0", 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 585571976 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "open", 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "num_leases": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "num_caps": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "request_load_avg": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 0.00099998100000000005, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "requests_in_flight": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "num_completed_requests": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "num_completed_flushes": 0, 2024-05-10T03:35:22.602 INFO:teuthology.orchestra.run.smithi055.stdout: "reconnecting": false, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps": { 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 60 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "release_caps": { 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 60 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps_throttle": { 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 1.5 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps_throttle2o": { 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.603 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 0.5 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "session_cache_liveness": { 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 300 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_acquisition": { 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 30 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "last_trim_completed_requests_tid": 0, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "last_trim_completed_flushes_tid": 0, 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "delegated_inos": [], 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": "client.7710 v1:172.21.15.162:0/585571976", 2024-05-10T03:35:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: "completed_requests": [], 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "prealloc_inos": [], 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "client_metadata": { 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "client_features": { 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "feature_bits": "0x000000000037fbff" 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "metric_spec": { 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "metric_flags": { 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "feature_bits": "0x000000000080ffff" 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.605 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "1", 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi162", 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "kernel_version": "6.9.0-rc3-gfb9a6a44107e", 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 5780, 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": { 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "name": { 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "client", 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "num": 5780 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": { 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-05-10T03:35:22.606 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1888309702 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "open", 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "num_leases": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "num_caps": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "request_load_avg": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 0.00099998100000000005, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "requests_in_flight": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "num_completed_requests": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "num_completed_flushes": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "reconnecting": false, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps": { 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.607 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 60 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "release_caps": { 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 60 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps_throttle": { 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 1.5 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "recall_caps_throttle2o": { 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 0.5 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "session_cache_liveness": { 2024-05-10T03:35:22.608 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 300 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "cap_acquisition": { 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "value": 0, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "halflife": 30 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "last_trim_completed_requests_tid": 0, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "last_trim_completed_flushes_tid": 0, 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "delegated_inos": [], 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "inst": "client.5780 v1:172.21.15.55:0/1888309702", 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "completed_requests": [], 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "prealloc_inos": [], 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "client_metadata": { 2024-05-10T03:35:22.609 INFO:teuthology.orchestra.run.smithi055.stdout: "client_features": { 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "feature_bits": "0x000000000037fbff" 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "metric_spec": { 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "metric_flags": { 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "feature_bits": "0x000000000080ffff" 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "entity_id": "0", 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": "smithi055", 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "kernel_version": "6.9.0-rc3-gfb9a6a44107e", 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: "root": "/" 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-05-10T03:35:22.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:22.611 INFO:teuthology.orchestra.run.smithi055.stdout: "result": 0 2024-05-10T03:35:22.611 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:22.611 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:22.611 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x56436630e900 msgr2=0x56436569a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x56436630e900 0x56436569a100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x564366311600 msgr2=0x56436569b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x564366311600 0x56436569b700 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x564366459aa0 tx=0x564366405f20 comp rx=0 tx=0).stop 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 msgr2=0x56436569ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 --2- 172.21.15.55:0/879569764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56436630e000 0x56436569ac00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x564366347cb0 tx=0x5643662ea360 comp rx=0 tx=0).stop 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe746000640 1 -- 172.21.15.55:0/879569764 reap_dead start 2024-05-10T03:35:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 shutdown_connections 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 >> 172.21.15.55:0/879569764 conn(0x564365693a80 msgr2=0x5643655bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 shutdown_connections 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.597+0000 7fe748a1e280 1 -- 172.21.15.55:0/879569764 wait complete. 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 msgr2=0x564365699080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x564366346de0 tx=0x5643662ea600 comp rx=0 tx=0).stop 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 shutdown_connections 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564365693180 0x564365699080 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564365692d00 0x564365698580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 --2- 172.21.15.55:0/2788476247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564365692880 0x564365698b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 >> 172.21.15.55:0/2788476247 conn(0x564365693600 msgr2=0x5643655bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 shutdown_connections 2024-05-10T03:35:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:22.598+0000 7fe748a1e280 1 -- 172.21.15.55:0/2788476247 wait complete. 2024-05-10T03:35:22.614 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.211835, result: { "0": { "data": { "sessions": [ { "id": 7710, "entity": { "name": { "type": "client", "num": 7710 }, "addr": { "type": "v1", "addr": "172.21.15.162:0", "nonce": 585571976 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099998100000000005, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7710 v1:172.21.15.162:0/585571976", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi162", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } }, { "id": 5780, "entity": { "name": { "type": "client", "num": 5780 }, "addr": { "type": "v1", "addr": "172.21.15.55:0", "nonce": 1888309702 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.00099998100000000005, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5780 v1:172.21.15.55:0/1888309702", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi055", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } } ] }, "result": 0 } } 2024-05-10T03:35:22.615 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [ { "id": 7710, "entity": { "name": { "type": "client", "num": 7710 }, "addr": { "type": "v1", "addr": "172.21.15.162:0", "nonce": 585571976 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999981, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7710 v1:172.21.15.162:0/585571976", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "1", "hostname": "smithi162", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } }, { "id": 5780, "entity": { "name": { "type": "client", "num": 5780 }, "addr": { "type": "v1", "addr": "172.21.15.55:0", "nonce": 1888309702 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.000999981, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.5780 v1:172.21.15.55:0/1888309702", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "entity_id": "0", "hostname": "smithi055", "kernel_version": "6.9.0-rc3-gfb9a6a44107e", "root": "/" } } ] }, "result": 0 } } 2024-05-10T03:35:22.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:22.615 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir bravo) 2024-05-10T03:35:22.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:22.675 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch bravo/file_a) 2024-05-10T03:35:22.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:22.734 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch bravo/file_b) 2024-05-10T03:35:22.812 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-05-10T03:35:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.101+0000 7f0c03200640 1 Processor -- start 2024-05-10T03:35:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.101+0000 7f0c03200640 1 -- start start 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c03200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0bfc0ce7c0 0x7f0bfc154920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c03200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc071620 con 0x7f0bfc06ce40 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c03200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc15ded0 con 0x7f0bfc15e920 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c03200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc15e500 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c01e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0bfc0ce7c0 0x7f0bfc154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c01e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0bfc0ce7c0 0x7f0bfc154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50070/0 (socket says 172.21.15.55:50070) 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c02800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0bfc06ce40 0x7f0bfc06d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52988/0 (socket says 172.21.15.55:52988) 2024-05-10T03:35:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.104+0000 7f0c01e00640 1 -- 172.21.15.55:0/2319871942 learned_addr learned my addr 172.21.15.55:0/2319871942 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.105+0000 7f0c01e00640 1 -- 172.21.15.55:0/2319871942 >> v1:172.21.15.162:6789/0 conn(0x7f0bfc15e920 legacy=0x7f0bfc070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.105+0000 7f0c01e00640 1 -- 172.21.15.55:0/2319871942 >> v1:172.21.15.55:6789/0 conn(0x7f0bfc06ce40 legacy=0x7f0bfc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.105+0000 7f0c01e00640 1 -- 172.21.15.55:0/2319871942 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc071860 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.105+0000 7f0c00a00640 1 -- 172.21.15.55:0/2319871942 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954422207 0 0) 0x7f0bf8001490 con 0x7f0bfc06ce40 2024-05-10T03:35:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.106+0000 7f0c01e00640 1 --2- 172.21.15.55:0/2319871942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc0ce7c0 0x7f0bfc154920 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009870 tx=0x7f0bf002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416d5585dc5f948 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c00a00640 1 -- 172.21.15.55:0/2319871942 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0032440 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c00a00640 1 -- 172.21.15.55:0/2319871942 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf00325a0 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c00a00640 1 -- 172.21.15.55:0/2319871942 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0032930 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c03200640 1 -- 172.21.15.55:0/2319871942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc0ce7c0 msgr2=0x7f0bfc154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c03200640 1 --2- 172.21.15.55:0/2319871942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc0ce7c0 0x7f0bfc154920 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009870 tx=0x7f0bf002a210 comp rx=0 tx=0).stop 2024-05-10T03:35:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c03200640 1 -- 172.21.15.55:0/2319871942 shutdown_connections 2024-05-10T03:35:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c03200640 1 --2- 172.21.15.55:0/2319871942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc0ce7c0 0x7f0bfc154920 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.107+0000 7f0c03200640 1 -- 172.21.15.55:0/2319871942 >> 172.21.15.55:0/2319871942 conn(0x7f0bfc06a930 msgr2=0x7f0bfc0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.108+0000 7f0c03200640 1 -- 172.21.15.55:0/2319871942 shutdown_connections 2024-05-10T03:35:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.108+0000 7f0c03200640 1 -- 172.21.15.55:0/2319871942 wait complete. 2024-05-10T03:35:23.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.110+0000 7f0c03200640 1 Processor -- start 2024-05-10T03:35:23.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.110+0000 7f0c03200640 1 -- start start 2024-05-10T03:35:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bfc06ce40 0x7f0bfc1f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bfc0ce7c0 0x7f0bfc0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc154e60 con 0x7f0bfc15e920 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1f2e80 con 0x7f0bfc06ce40 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.112+0000 7f0c03200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc155260 con 0x7f0bfc0ce7c0 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50072/0 (socket says 172.21.15.55:50072) 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 -- 172.21.15.55:0/2342995619 learned_addr learned my addr 172.21.15.55:0/2342995619 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c01e00640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bfc06ce40 0x7f0bfc1f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c01400640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bfc0ce7c0 0x7f0bfc0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 -- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bfc0ce7c0 msgr2=0x7f0bfc0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bfc0ce7c0 0x7f0bfc0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 -- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bfc06ce40 msgr2=0x7f0bfc1f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bfc06ce40 0x7f0bfc1f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.113+0000 7f0c02800640 1 -- 172.21.15.55:0/2342995619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00094d0 con 0x7f0bfc15e920 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0c02800640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c700 tx=0x7f0bf800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ce10 con 0x7f0bfc15e920 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf800a640 con 0x7f0bfc15e920 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800aa70 con 0x7f0bfc15e920 2024-05-10T03:35:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc1f3090 con 0x7f0bfc15e920 2024-05-10T03:35:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.114+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc1f3370 con 0x7f0bfc15e920 2024-05-10T03:35:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.115+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc0b1180 con 0x7f0bfc15e920 2024-05-10T03:35:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.117+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0bf800abd0 con 0x7f0bfc15e920 2024-05-10T03:35:23.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.117+0000 7f0bf6000640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 0x7f0be006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:23.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.118+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f0bf8088d90 con 0x7f0bfc15e920 2024-05-10T03:35:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.118+0000 7f0c01e00640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 0x7f0be006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:23.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.120+0000 7f0c01e00640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 0x7f0be006d160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0034000 tx=0x7f0bf002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:23.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.123+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0bf8053c80 con 0x7f0bfc15e920 2024-05-10T03:35:23.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:23.364+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f0bfc071620 con 0x7f0bfc15e920 2024-05-10T03:35:24.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.321+0000 7f0bf6000640 1 -- 172.21.15.55:0/2342995619 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v66) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0bfc071620 con 0x7f0bfc15e920 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.324+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 msgr2=0x7f0be006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.324+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 0x7f0be006d160 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0034000 tx=0x7f0bf002e000 comp rx=0 tx=0).stop 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 msgr2=0x7f0bfc0afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c700 tx=0x7f0bf800cbc0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 shutdown_connections 2024-05-10T03:35:24.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bfc15e920 0x7f0bfc0afd40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bfc0ce7c0 0x7f0bfc0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0be006ac50 0x7f0be006d160 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 --2- 172.21.15.55:0/2342995619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bfc06ce40 0x7f0bfc1f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 >> 172.21.15.55:0/2342995619 conn(0x7f0bfc06a930 msgr2=0x7f0bfc0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 shutdown_connections 2024-05-10T03:35:24.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.325+0000 7f0c03200640 1 -- 172.21.15.55:0/2342995619 wait complete. 2024-05-10T03:35:24.369 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.622+0000 7ffb2fe00640 1 Processor -- start 2024-05-10T03:35:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.622+0000 7ffb2fe00640 1 -- start start 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2fe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffb3015ba60 0x7ffb30071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb30152690 0x7ffb300cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3015c040 con 0x7ffb30152280 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2fe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3015c320 con 0x7ffb3015ba60 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2fe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3015c600 con 0x7ffb30152690 2024-05-10T03:35:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffb3015ba60 0x7ffb30071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2ea00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffb3015ba60 0x7ffb30071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50086/0 (socket says 172.21.15.55:50086) 2024-05-10T03:35:24.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2ea00640 1 -- 172.21.15.55:0/20641571 learned_addr learned my addr 172.21.15.55:0/20641571 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:24.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.624+0000 7ffb2e000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ffb30152280 0x7ffb3015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52996/0 (socket says 172.21.15.55:52996) 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2f400640 1 --2- 172.21.15.55:0/20641571 >> v2:172.21.15.162:3300/0 conn(0x7ffb30152690 0x7ffb300cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2ea00640 1 -- 172.21.15.55:0/20641571 >> v2:172.21.15.162:3300/0 conn(0x7ffb30152690 msgr2=0x7ffb300cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2ea00640 1 --2- 172.21.15.55:0/20641571 >> v2:172.21.15.162:3300/0 conn(0x7ffb30152690 0x7ffb300cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2ea00640 1 -- 172.21.15.55:0/20641571 >> v1:172.21.15.55:6789/0 conn(0x7ffb30152280 legacy=0x7ffb3015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2ea00640 1 -- 172.21.15.55:0/20641571 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3015c8e0 con 0x7ffb3015ba60 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.625+0000 7ffb2d600640 1 -- 172.21.15.55:0/20641571 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191242763 0 0) 0x7ffb3015c040 con 0x7ffb30152280 2024-05-10T03:35:24.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.626+0000 7ffb2ea00640 1 --2- 172.21.15.55:0/20641571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb3015ba60 0x7ffb30071920 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ffb300cca40 tx=0x7ffb2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9c5f1bea1e156 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:24.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2d600640 1 -- 172.21.15.55:0/20641571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20032440 con 0x7ffb3015ba60 2024-05-10T03:35:24.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2d600640 1 -- 172.21.15.55:0/20641571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb200325a0 con 0x7ffb3015ba60 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2d600640 1 -- 172.21.15.55:0/20641571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20032930 con 0x7ffb3015ba60 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2fe00640 1 -- 172.21.15.55:0/20641571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb3015ba60 msgr2=0x7ffb30071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/20641571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb3015ba60 0x7ffb30071920 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ffb300cca40 tx=0x7ffb2002a270 comp rx=0 tx=0).stop 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2fe00640 1 -- 172.21.15.55:0/20641571 shutdown_connections 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/20641571 >> v2:172.21.15.162:3300/0 conn(0x7ffb30152690 0x7ffb300cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.627+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/20641571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb3015ba60 0x7ffb30071920 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.628+0000 7ffb2fe00640 1 -- 172.21.15.55:0/20641571 >> 172.21.15.55:0/20641571 conn(0x7ffb300b07b0 msgr2=0x7ffb300b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.628+0000 7ffb2fe00640 1 -- 172.21.15.55:0/20641571 shutdown_connections 2024-05-10T03:35:24.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.628+0000 7ffb2fe00640 1 -- 172.21.15.55:0/20641571 wait complete. 2024-05-10T03:35:24.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.630+0000 7ffb2fe00640 1 Processor -- start 2024-05-10T03:35:24.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.630+0000 7ffb2fe00640 1 -- start start 2024-05-10T03:35:24.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 0x7ffb3013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb3015ba60 0x7ffb300ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300aa9a0 con 0x7ffb30152280 2024-05-10T03:35:24.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300aa370 con 0x7ffb30152690 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300aa5c0 con 0x7ffb3015ba60 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50096/0 (socket says 172.21.15.55:50096) 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.632+0000 7ffb2ea00640 1 -- 172.21.15.55:0/2374298202 learned_addr learned my addr 172.21.15.55:0/2374298202 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2e000640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 0x7ffb3013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 -- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb3015ba60 msgr2=0x7ffb300ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2f400640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb3015ba60 0x7ffb300ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb3015ba60 0x7ffb300ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 -- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 msgr2=0x7ffb3013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 0x7ffb3013ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 -- 172.21.15.55:0/2374298202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200094d0 con 0x7ffb30152280 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2e000640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 0x7ffb3013ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2ea00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20032440 con 0x7ffb30152280 2024-05-10T03:35:24.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20030b80 con 0x7ffb30152280 2024-05-10T03:35:24.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2003d420 con 0x7ffb30152280 2024-05-10T03:35:24.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.633+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb300ad370 con 0x7ffb30152280 2024-05-10T03:35:24.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.634+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3013f930 con 0x7ffb30152280 2024-05-10T03:35:24.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.635+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb30067240 con 0x7ffb30152280 2024-05-10T03:35:24.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.636+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb20039040 con 0x7ffb30152280 2024-05-10T03:35:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.637+0000 7ffb26a00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 0x7ffb1006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.637+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6345+0+0 (secure 0 0 0) 0x7ffb200aa170 con 0x7ffb30152280 2024-05-10T03:35:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.638+0000 7ffb2e000640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 0x7ffb1006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.639+0000 7ffb2e000640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 0x7ffb1006d140 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb301f4be0 tx=0x7ffb1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:24.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.642+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffb200751e0 con 0x7ffb30152280 2024-05-10T03:35:24.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.881+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb30071e10 con 0x7ffb30152280 2024-05-10T03:35:24.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.882+0000 7ffb26a00640 1 -- 172.21.15.55:0/2374298202 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4754 (secure 0 0 0) 0x7ffb20078800 con 0x7ffb30152280 2024-05-10T03:35:24.885 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:24.885 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:24.320978+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:creating","state_seq":1,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:24.885 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 67 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 msgr2=0x7ffb1006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 0x7ffb1006d140 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb301f4be0 tx=0x7ffb1c009040 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 msgr2=0x7ffb300ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb200046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 shutdown_connections 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb3015ba60 0x7ffb300ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ffb1006ac30 0x7ffb1006d140 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb30152690 0x7ffb3013ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 --2- 172.21.15.55:0/2374298202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb30152280 0x7ffb300ace30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 >> 172.21.15.55:0/2374298202 conn(0x7ffb300b07b0 msgr2=0x7ffb30151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 shutdown_connections 2024-05-10T03:35:24.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:24.885+0000 7ffb2fe00640 1 -- 172.21.15.55:0/2374298202 wait complete. 2024-05-10T03:35:25.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:35:24.320978+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7647, 'mds_1': 7623}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7623': {'gid': 7623, 'name': 'c', 'rank': 1, 'incarnation': 67, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.55:6833/609219709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 609219709}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 609219709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7647': {'gid': 7647, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.55:6835/1198619982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 1198619982}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 1198619982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:25.316 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7623:up:creating 2024-05-10T03:35:26.318 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:26.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.612+0000 7f1c3fe00640 1 Processor -- start 2024-05-10T03:35:26.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.612+0000 7f1c3fe00640 1 -- start start 2024-05-10T03:35:26.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.614+0000 7f1c3fe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1c40152280 0x7f1c4015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.614+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c4015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.614+0000 7f1c3fe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4015c040 con 0x7f1c4015ba60 2024-05-10T03:35:26.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.614+0000 7f1c3fe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4015c320 con 0x7f1c40152280 2024-05-10T03:35:26.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015c600 con 0x7f1c40152690 2024-05-10T03:35:26.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3ea00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1c4015ba60 0x7f1c40071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53010/0 (socket says 172.21.15.55:53010) 2024-05-10T03:35:26.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3ea00640 1 -- 172.21.15.55:0/3301834231 learned_addr learned my addr 172.21.15.55:0/3301834231 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3e000640 1 --2- 172.21.15.55:0/3301834231 >> v2:172.21.15.162:3300/0 conn(0x7f1c40152280 0x7f1c4015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237259961 0 0) 0x7f1c4015c040 con 0x7f1c4015ba60 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c24003520 con 0x7f1c4015ba60 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.615+0000 7f1c3f400640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c4015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697349869 0 0) 0x7f1c24003520 con 0x7f1c4015ba60 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 msgr2=0x7f1c4015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c4015fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 >> v2:172.21.15.162:3300/0 conn(0x7f1c40152280 msgr2=0x7f1c4015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:26.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 --2- 172.21.15.55:0/3301834231 >> v2:172.21.15.162:3300/0 conn(0x7f1c40152280 0x7f1c4015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4015c8e0 con 0x7f1c4015ba60 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f1c300019f0 con 0x7f1c4015ba60 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 >> v1:172.21.15.55:6789/0 conn(0x7f1c4015ba60 legacy=0x7f1c40071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c24003ef0 0x7f1c240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c24005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015c320 con 0x7f1c24004d00 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015c600 con 0x7f1c24003ef0 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.616+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2400ba80 con 0x7f1c40152690 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c24005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3e000640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c24003ef0 0x7f1c240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3ea00640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 -- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 msgr2=0x7f1c24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:26.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c40152690 0x7f1c24003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 -- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c24003ef0 msgr2=0x7f1c240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:26.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c24003ef0 0x7f1c240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.617+0000 7f1c3f400640 1 -- 172.21.15.55:0/3301834231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24003520 con 0x7f1c24004d00 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.618+0000 7f1c3f400640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c24005110 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800ba10 tx=0x7f1c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35019eab26e3896 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.618+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3802ade0 con 0x7f1c24004d00 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3802e070 con 0x7f1c24004d00 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3d600640 1 -- 172.21.15.55:0/3301834231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38034620 con 0x7f1c24004d00 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3fe00640 1 -- 172.21.15.55:0/3301834231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 msgr2=0x7f1c24005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/3301834231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c24005110 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800ba10 tx=0x7f1c240057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3f400640 1 -- 172.21.15.55:0/3301834231 reap_dead start 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3fe00640 1 -- 172.21.15.55:0/3301834231 shutdown_connections 2024-05-10T03:35:26.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3fe00640 1 -- 172.21.15.55:0/3301834231 >> 172.21.15.55:0/3301834231 conn(0x7f1c400b07b0 msgr2=0x7f1c400b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:26.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.619+0000 7f1c3fe00640 1 -- 172.21.15.55:0/3301834231 shutdown_connections 2024-05-10T03:35:26.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.620+0000 7f1c3fe00640 1 -- 172.21.15.55:0/3301834231 wait complete. 2024-05-10T03:35:26.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.622+0000 7f1c3fe00640 1 Processor -- start 2024-05-10T03:35:26.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.622+0000 7f1c3fe00640 1 -- start start 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c4006be40 0x7f1c4006c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4014eab0 0x7f1c4014eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401f2300 con 0x7f1c24004d00 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4014f550 con 0x7f1c4014eab0 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3fe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4014a070 con 0x7f1c4006be40 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50118/0 (socket says 172.21.15.55:50118) 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3ea00640 1 -- 172.21.15.55:0/2992070820 learned_addr learned my addr 172.21.15.55:0/2992070820 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3ea00640 1 -- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c4006be40 msgr2=0x7f1c4006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3e000640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c4006be40 0x7f1c4006c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3f400640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4014eab0 0x7f1c4014eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.624+0000 7f1c3ea00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c4006be40 0x7f1c4006c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3ea00640 1 -- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4014eab0 msgr2=0x7f1c4014eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3ea00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4014eab0 0x7f1c4014eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3ea00640 1 -- 172.21.15.55:0/2992070820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4014a330 con 0x7f1c24004d00 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3ea00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1c30002200 tx=0x7f1c3000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30004d60 con 0x7f1c24004d00 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c300070d0 con 0x7f1c24004d00 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4014a610 con 0x7f1c24004d00 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30007440 con 0x7f1c24004d00 2024-05-10T03:35:26.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.625+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c401f2040 con 0x7f1c24004d00 2024-05-10T03:35:26.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.627+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c400b1010 con 0x7f1c24004d00 2024-05-10T03:35:26.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.628+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1c300075a0 con 0x7f1c24004d00 2024-05-10T03:35:26.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.629+0000 7f1c36a00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 0x7f1c1c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:26.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.629+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f1c30089bd0 con 0x7f1c24004d00 2024-05-10T03:35:26.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.629+0000 7f1c3e000640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 0x7f1c1c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:26.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.631+0000 7f1c3e000640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 0x7f1c1c06d110 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ae210 tx=0x7f1c28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:26.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.633+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1c30054a00 con 0x7f1c24004d00 2024-05-10T03:35:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.875+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c40067240 con 0x7f1c24004d00 2024-05-10T03:35:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.876+0000 7f1c36a00640 1 -- 172.21.15.55:0/2992070820 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4752 (secure 0 0 0) 0x7f1c30019070 con 0x7f1c24004d00 2024-05-10T03:35:26.879 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:26.879 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":68,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:25.330023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 68 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 msgr2=0x7f1c1c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 0x7f1c1c06d110 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ae210 tx=0x7f1c28008040 comp rx=0 tx=0).stop 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 msgr2=0x7f1c4014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1c30002200 tx=0x7f1c3000b560 comp rx=0 tx=0).stop 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 shutdown_connections 2024-05-10T03:35:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f1c1c06ac00 0x7f1c1c06d110 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c4014eab0 0x7f1c4014eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c4006be40 0x7f1c4006c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 --2- 172.21.15.55:0/2992070820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c24004d00 0x7f1c4014ae10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 >> 172.21.15.55:0/2992070820 conn(0x7f1c400b07b0 msgr2=0x7f1c40151870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 shutdown_connections 2024-05-10T03:35:26.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:26.879+0000 7f1c3fe00640 1 -- 172.21.15.55:0/2992070820 wait complete. 2024-05-10T03:35:27.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T03:34:55.059838+0000', 'modified': '2024-05-10T03:35:25.330023+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7647, 'mds_1': 7623}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7623': {'gid': 7623, 'name': 'c', 'rank': 1, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.55:6833/609219709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 609219709}, {'type': 'v1', 'addr': '172.21.15.55:6833', 'nonce': 609219709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_7647': {'gid': 7647, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.55:6835/1198619982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 1198619982}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 1198619982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T03:35:27.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-05-10T03:35:27.313 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-10T03:35:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.604+0000 7f30c1200640 1 Processor -- start 2024-05-10T03:35:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.604+0000 7f30c1200640 1 -- start start 2024-05-10T03:35:27.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30c1200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30bc06eb30 0x7f30bc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30c1200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30c1200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30bc06e6a0 con 0x7f30bc071bd0 2024-05-10T03:35:27.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30c1200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc0cfa40 con 0x7f30bc06eb30 2024-05-10T03:35:27.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30c1200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc15bc00 con 0x7f30bc06a820 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30b9e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30bc06eb30 0x7f30bc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30b9400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f30bc071bd0 0x7f30bc06a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53024/0 (socket says 172.21.15.55:53024) 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30b9e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30bc06eb30 0x7f30bc15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50124/0 (socket says 172.21.15.55:50124) 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.607+0000 7f30b9400640 1 -- 172.21.15.55:0/728404741 learned_addr learned my addr 172.21.15.55:0/728404741 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30ba800640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30b9e00640 1 -- 172.21.15.55:0/728404741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 msgr2=0x7f30bc0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30b9e00640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30b9e00640 1 -- 172.21.15.55:0/728404741 >> v1:172.21.15.55:6789/0 conn(0x7f30bc071bd0 legacy=0x7f30bc06a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30b9e00640 1 -- 172.21.15.55:0/728404741 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc160130 con 0x7f30bc06eb30 2024-05-10T03:35:27.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.608+0000 7f30b8a00640 1 -- 172.21.15.55:0/728404741 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203114824 0 0) 0x7f30bc06e6a0 con 0x7f30bc071bd0 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.609+0000 7f30b9e00640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc15b6c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097a0 tx=0x7f30ac02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f44f31052511c97 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.609+0000 7f30b8a00640 1 -- 172.21.15.55:0/728404741 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac02ae50 con 0x7f30bc06eb30 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.609+0000 7f30b8a00640 1 -- 172.21.15.55:0/728404741 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac032440 con 0x7f30bc06eb30 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.609+0000 7f30b8a00640 1 -- 172.21.15.55:0/728404741 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0327d0 con 0x7f30bc06eb30 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 -- 172.21.15.55:0/728404741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 msgr2=0x7f30bc15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc15b6c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097a0 tx=0x7f30ac02a100 comp rx=0 tx=0).stop 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 -- 172.21.15.55:0/728404741 shutdown_connections 2024-05-10T03:35:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 --2- 172.21.15.55:0/728404741 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc15b6c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 -- 172.21.15.55:0/728404741 >> 172.21.15.55:0/728404741 conn(0x7f30bc0b1b40 msgr2=0x7f30bc0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.610+0000 7f30c1200640 1 -- 172.21.15.55:0/728404741 shutdown_connections 2024-05-10T03:35:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.611+0000 7f30c1200640 1 -- 172.21.15.55:0/728404741 wait complete. 2024-05-10T03:35:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.613+0000 7f30c1200640 1 Processor -- start 2024-05-10T03:35:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.613+0000 7f30c1200640 1 -- start start 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.614+0000 7f30c1200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc1545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30c1200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30c1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30bc071bd0 0x7f30bc159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30c1200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc099a70 con 0x7f30bc06eb30 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30c1200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc099ed0 con 0x7f30bc071bd0 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30c1200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc09c1e0 con 0x7f30bc06a820 2024-05-10T03:35:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50128/0 (socket says 172.21.15.55:50128) 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 -- 172.21.15.55:0/1264842252 learned_addr learned my addr 172.21.15.55:0/1264842252 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc1545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 -- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 msgr2=0x7f30bc1545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc1545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 -- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30bc071bd0 msgr2=0x7f30bc159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30bc071bd0 0x7f30bc159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.615+0000 7f30b9400640 1 -- 172.21.15.55:0/1264842252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009400 con 0x7f30bc06eb30 2024-05-10T03:35:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30b9400640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c440 tx=0x7f30a800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30b2000640 1 -- 172.21.15.55:0/1264842252 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80074e0 con 0x7f30bc06eb30 2024-05-10T03:35:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30b2000640 1 -- 172.21.15.55:0/1264842252 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a8007640 con 0x7f30bc06eb30 2024-05-10T03:35:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30b2000640 1 -- 172.21.15.55:0/1264842252 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8004a20 con 0x7f30bc06eb30 2024-05-10T03:35:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc1f7d60 con 0x7f30bc06eb30 2024-05-10T03:35:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.616+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc1f7430 con 0x7f30bc06eb30 2024-05-10T03:35:27.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.619+0000 7f30b2000640 1 -- 172.21.15.55:0/1264842252 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f30a8004b80 con 0x7f30bc06eb30 2024-05-10T03:35:27.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.620+0000 7f30b2000640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 0x7f309c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.621+0000 7f30b2000640 1 -- 172.21.15.55:0/1264842252 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f30a80887b0 con 0x7f30bc06eb30 2024-05-10T03:35:27.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.621+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 0x7f309c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.621+0000 7f30c81c8740 1 Processor -- start 2024-05-10T03:35:27.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.621+0000 7f30c81c8740 1 -- start start 2024-05-10T03:35:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 0x7f309c06d140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f30ac034000 tx=0x7f30ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b31fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b311ea70 con 0x55f3b2df5590 2024-05-10T03:35:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b311c200 con 0x55f3b2ded3b0 2024-05-10T03:35:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3b3117af0 con 0x55f3b2e16270 2024-05-10T03:35:27.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30ba800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30ba800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50140/0 (socket says 172.21.15.55:50140) 2024-05-10T03:35:27.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.623+0000 7f30ba800640 1 -- 172.21.15.55:0/1930157051 learned_addr learned my addr 172.21.15.55:0/1930157051 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30b9400640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b31fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 -- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 msgr2=0x55f3b31dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 -- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 msgr2=0x55f3b31fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b31fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 -- 172.21.15.55:0/1930157051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3b30e2b90 con 0x55f3b2df5590 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30b9400640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b31fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30ba800640 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009eb0 tx=0x7f30b4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562493a99e23ca74 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30a6a00640 1 -- 172.21.15.55:0/1930157051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4012710 con 0x55f3b2df5590 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30a6a00640 1 -- 172.21.15.55:0/1930157051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4012870 con 0x55f3b2df5590 2024-05-10T03:35:27.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.624+0000 7f30a6a00640 1 -- 172.21.15.55:0/1930157051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4012c20 con 0x55f3b2df5590 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 -- 172.21.15.55:0/1930157051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 msgr2=0x55f3b31fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009eb0 tx=0x7f30b4007700 comp rx=0 tx=0).stop 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 -- 172.21.15.55:0/1930157051 shutdown_connections 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31fadf0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b31fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1930157051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.625+0000 7f30c81c8740 1 -- 172.21.15.55:0/1930157051 >> 172.21.15.55:0/1930157051 conn(0x55f3b2e0a3c0 msgr2=0x55f3b31b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.626+0000 7f30c81c8740 1 -- 172.21.15.55:0/1930157051 shutdown_connections 2024-05-10T03:35:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.626+0000 7f30c81c8740 1 -- 172.21.15.55:0/1930157051 wait complete. 2024-05-10T03:35:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.627+0000 7f30c81c8740 1 -- start start 2024-05-10T03:35:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.627+0000 7f30c81c8740 1 Processor -- start 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b30b9220 con 0x55f3b2ded3b0 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b30b8ec0 con 0x55f3b2df5590 2024-05-10T03:35:27.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.628+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3b2e7ef20 con 0x55f3b2e16270 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30c81c8740 10 client.0 _ll_register_callbacks cb 0x55f3b3111160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50142/0 (socket says 172.21.15.55:50142) 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 -- 172.21.15.55:0/1925561693 learned_addr learned my addr 172.21.15.55:0/1925561693 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9400640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 -- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 msgr2=0x55f3b31fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 -- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 msgr2=0x55f3b31e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.629+0000 7f30b9e00640 1 -- 172.21.15.55:0/1925561693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b4009ae0 con 0x55f3b2ded3b0 2024-05-10T03:35:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f30b9e00640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02a6f0 tx=0x7f30ac02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2e1ef3ab2d31a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f309b400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac048670 con 0x55f3b2ded3b0 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f30ac03be20 con 0x55f3b2ded3b0 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3b3141c90 con 0x55f3b2ded3b0 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30ac0487d0 con 0x55f3b2ded3b0 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f30c81c8740 10 client.7764 fetch_fsmap learned FSMap version 68 2024-05-10T03:35:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55f3b3116170 con 0x55f3b2ded3b0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.630+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3b3141c90 con 0x55f3b2ded3b0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 68) v1 ==== 3126+0+0 (secure 0 0 0) 0x7f30ac048930 con 0x55f3b2ded3b0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 10 client.7764 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 10 client.7764 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 20 client.7764 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b2dd4e40 0x55f3b320d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 4 client.7764 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 --> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3b2e0b3c0 con 0x55f3b2dd4e40 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.631+0000 7f30b9400640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b2dd4e40 0x55f3b320d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.632+0000 7f30b9400640 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b2dd4e40 0x55f3b320d4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c46620df086432c server_cookie=445bbef2b53c9650 in_seq=0 out_seq=0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.632+0000 7f309b400640 10 client.7764 ms_handle_connect on v2:172.21.15.55:6832/609219709 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.632+0000 7f309b400640 1 -- 172.21.15.55:0/1925561693 <== mds.1 v2:172.21.15.55:6832/609219709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3b2e0b3c0 con 0x55f3b2dd4e40 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.632+0000 7f309b400640 10 client.7764 handle_command_reply: tid=0 2024-05-10T03:35:27.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.632+0000 7f30c81c8740 1 client.7764 shutdown 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 msgr2=0x55f3b3209030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02a6f0 tx=0x7f30ac02c070 comp rx=0 tx=0).stop 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 shutdown_connections 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b2dd4e40 0x55f3b320d4f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f309b400640 0 client.7764 ms_handle_reset on v2:172.21.15.55:6832/609219709 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 --2- 172.21.15.55:0/1925561693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3209030 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 >> 172.21.15.55:0/1925561693 conn(0x55f3b2e0a3c0 msgr2=0x55f3b31dfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 shutdown_connections 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 1 -- 172.21.15.55:0/1925561693 wait complete. 2024-05-10T03:35:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.633+0000 7f30c81c8740 20 client.7764 trim_cache size 0 max 16384 2024-05-10T03:35:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.651+0000 7f30c81c8740 1 Processor -- start 2024-05-10T03:35:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.651+0000 7f30c81c8740 1 -- start start 2024-05-10T03:35:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.652+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30c81c8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b31f44d0 con 0x55f3b2ded3b0 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b31db500 con 0x55f3b2df5590 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3b320c140 con 0x55f3b2e16270 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50150/0 (socket says 172.21.15.55:50150) 2024-05-10T03:35:27.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 -- 172.21.15.55:0/2006754787 learned_addr learned my addr 172.21.15.55:0/2006754787 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 -- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 msgr2=0x55f3b31dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 -- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 msgr2=0x55f3b31dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.653+0000 7f30b9e00640 1 -- 172.21.15.55:0/2006754787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3b314b130 con 0x55f3b2ded3b0 2024-05-10T03:35:27.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.654+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f30ac03bbb0 tx=0x7f30ac03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1fb86bfc8c4a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.654+0000 7f30a7400640 1 -- 172.21.15.55:0/2006754787 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac048470 con 0x55f3b2ded3b0 2024-05-10T03:35:27.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.654+0000 7f30a7400640 1 -- 172.21.15.55:0/2006754787 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac042050 con 0x55f3b2ded3b0 2024-05-10T03:35:27.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.654+0000 7f30a7400640 1 -- 172.21.15.55:0/2006754787 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac048810 con 0x55f3b2ded3b0 2024-05-10T03:35:27.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 -- 172.21.15.55:0/2006754787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 msgr2=0x55f3b3213760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f30ac03bbb0 tx=0x7f30ac03b000 comp rx=0 tx=0).stop 2024-05-10T03:35:27.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 -- 172.21.15.55:0/2006754787 shutdown_connections 2024-05-10T03:35:27.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31dc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2df5590 0x55f3b31dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2006754787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2ded3b0 0x55f3b3213760 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 -- 172.21.15.55:0/2006754787 >> 172.21.15.55:0/2006754787 conn(0x55f3b2e0a3c0 msgr2=0x55f3b31dfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 -- 172.21.15.55:0/2006754787 shutdown_connections 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.655+0000 7f30c81c8740 1 -- 172.21.15.55:0/2006754787 wait complete. 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.656+0000 7f30c81c8740 1 -- start start 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.656+0000 7f30c81c8740 1 Processor -- start 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b325d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b31d7660 con 0x55f3b2df5590 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3b31d8450 con 0x55f3b2ded3b0 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3b325d010 con 0x55f3b2e16270 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30c81c8740 10 client.0 _ll_register_callbacks cb 0x7f30ac035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:27.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30b9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30b9400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50162/0 (socket says 172.21.15.55:50162) 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.658+0000 7f30b9400640 1 -- 172.21.15.55:0/2331956223 learned_addr learned my addr 172.21.15.55:0/2331956223 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b325d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 -- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 msgr2=0x55f3b31d77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 -- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 msgr2=0x55f3b325d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b325d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 -- 172.21.15.55:0/2331956223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3b3116170 con 0x55f3b2df5590 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30b9400640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f30a80537a0 tx=0x7f30a805b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440a017bc9a7ec96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f309b400640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8086070 con 0x55f3b2df5590 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f30a8086ad0 con 0x55f3b2df5590 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.659+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b4012870 con 0x55f3b2df5590 2024-05-10T03:35:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30a8055640 con 0x55f3b2df5590 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 10 client.7770 fetch_fsmap learned FSMap version 68 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55f3b30b8430 con 0x55f3b2df5590 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b4012870 con 0x55f3b2df5590 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 68) v1 ==== 3126+0+0 (secure 0 0 0) 0x7f30a8059040 con 0x55f3b2df5590 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 10 client.7770 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 10 client.7770 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 20 client.7770 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b3207550 0x55f3b3207960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 4 client.7770 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 --> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f3b2e1c6c0 con 0x55f3b3207550 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.660+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b3207550 0x55f3b3207960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f30b9e00640 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b3207550 0x55f3b3207960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f7e0554f72a684d server_cookie=7a7f89eeac74f1fc in_seq=0 out_seq=0 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f309b400640 10 client.7770 ms_handle_connect on v2:172.21.15.55:6832/609219709 2024-05-10T03:35:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f309b400640 1 -- 172.21.15.55:0/2331956223 <== mds.1 v2:172.21.15.55:6832/609219709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f3b2e1c6c0 con 0x55f3b3207550 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f309b400640 10 client.7770 handle_command_reply: tid=0 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f30c81c8740 1 client.7770 shutdown 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 msgr2=0x55f3b31df400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.661+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f30a80537a0 tx=0x7f30a805b0d0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 shutdown_connections 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:6832/609219709,v1:172.21.15.55:6833/609219709] conn(0x55f3b3207550 0x55f3b3207960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3b2e16270 0x55f3b31d77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f3b2df5590 0x55f3b31df400 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 --2- 172.21.15.55:0/2331956223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3b2ded3b0 0x55f3b325d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 >> 172.21.15.55:0/2331956223 conn(0x55f3b2e0a3c0 msgr2=0x55f3b31daa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f309b400640 0 client.7770 ms_handle_reset on v2:172.21.15.55:6832/609219709 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 shutdown_connections 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 1 -- 172.21.15.55:0/2331956223 wait complete. 2024-05-10T03:35:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.662+0000 7f30c81c8740 20 client.7770 trim_cache size 0 max 16384 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 1, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7623, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 3.3306354169999999, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 68, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 110, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 109, 2024-05-10T03:35:27.675 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 14.261343375999999 2024-05-10T03:35:27.676 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:27.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 msgr2=0x7f309c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 0x7f309c06d140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f30ac034000 tx=0x7f30ac02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:27.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 msgr2=0x7f30bc09b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c440 tx=0x7f30a800c900 comp rx=0 tx=0).stop 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 shutdown_connections 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f309c06ac30 0x7f309c06d140 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30bc071bd0 0x7f30bc159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30bc06eb30 0x7f30bc09b770 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 --2- 172.21.15.55:0/1264842252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f30bc06a820 0x7f30bc1545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 >> 172.21.15.55:0/1264842252 conn(0x7f30bc0b1b40 msgr2=0x7f30bc0ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 shutdown_connections 2024-05-10T03:35:27.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.663+0000 7f30c1200640 1 -- 172.21.15.55:0/1264842252 wait complete. 2024-05-10T03:35:27.694 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-10T03:35:27.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.978+0000 7f0089a00640 1 Processor -- start 2024-05-10T03:35:27.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.978+0000 7f0089a00640 1 -- start start 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.980+0000 7f0089a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f008415af40 0x7f008406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.980+0000 7f0089a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008415df20 con 0x7f008415ab30 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.980+0000 7f0089a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008415a6a0 con 0x7f00840b3200 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.980+0000 7f0089a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008415e720 con 0x7f008415af40 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.980+0000 7f0082000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f008415ab30 0x7f008415d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53028/0 (socket says 172.21.15.55:53028) 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0082000640 1 -- 172.21.15.55:0/637555776 learned_addr learned my addr 172.21.15.55:0/637555776 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:27.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0083400640 1 --2- 172.21.15.55:0/637555776 >> v2:172.21.15.55:3300/0 conn(0x7f008415af40 0x7f008406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0083400640 1 -- 172.21.15.55:0/637555776 >> v1:172.21.15.162:6789/0 conn(0x7f00840b3200 legacy=0x7f00840714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:27.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0083400640 1 -- 172.21.15.55:0/637555776 >> v1:172.21.15.55:6789/0 conn(0x7f008415ab30 legacy=0x7f008415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0083400640 1 -- 172.21.15.55:0/637555776 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008415e160 con 0x7f008415af40 2024-05-10T03:35:27.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.981+0000 7f0081600640 1 -- 172.21.15.55:0/637555776 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936783758 0 0) 0x7f006c001490 con 0x7f008415ab30 2024-05-10T03:35:27.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.982+0000 7f0083400640 1 --2- 172.21.15.55:0/637555776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415af40 0x7f008406c990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097d0 tx=0x7f007c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6707d50fca4c36e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0081600640 1 -- 172.21.15.55:0/637555776 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c032440 con 0x7f008415af40 2024-05-10T03:35:27.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0081600640 1 -- 172.21.15.55:0/637555776 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c0325a0 con 0x7f008415af40 2024-05-10T03:35:27.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0081600640 1 -- 172.21.15.55:0/637555776 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c032930 con 0x7f008415af40 2024-05-10T03:35:27.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0089a00640 1 -- 172.21.15.55:0/637555776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415af40 msgr2=0x7f008406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0089a00640 1 --2- 172.21.15.55:0/637555776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415af40 0x7f008406c990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097d0 tx=0x7f007c02a210 comp rx=0 tx=0).stop 2024-05-10T03:35:27.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0089a00640 1 -- 172.21.15.55:0/637555776 shutdown_connections 2024-05-10T03:35:27.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0089a00640 1 --2- 172.21.15.55:0/637555776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415af40 0x7f008406c990 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.983+0000 7f0089a00640 1 -- 172.21.15.55:0/637555776 >> 172.21.15.55:0/637555776 conn(0x7f00840b0b80 msgr2=0x7f00840b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:27.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.984+0000 7f0089a00640 1 -- 172.21.15.55:0/637555776 shutdown_connections 2024-05-10T03:35:27.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.984+0000 7f0089a00640 1 -- 172.21.15.55:0/637555776 wait complete. 2024-05-10T03:35:27.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.986+0000 7f0089a00640 1 Processor -- start 2024-05-10T03:35:27.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.986+0000 7f0089a00640 1 -- start start 2024-05-10T03:35:27.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00840b3200 0x7f008414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008415af40 0x7f008414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841f2710 con 0x7f008415ab30 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008414d020 con 0x7f00840b3200 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0089a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008414d300 con 0x7f008415af40 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50168/0 (socket says 172.21.15.55:50168) 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082000640 1 -- 172.21.15.55:0/65656712 learned_addr learned my addr 172.21.15.55:0/65656712 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00840b3200 0x7f008414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0083400640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008415af40 0x7f008414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082000640 1 -- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008415af40 msgr2=0x7f008414e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.988+0000 7f0082000640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008415af40 0x7f008414e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0082000640 1 -- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00840b3200 msgr2=0x7f008414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0082000640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00840b3200 0x7f008414be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0082000640 1 -- 172.21.15.55:0/65656712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c009400 con 0x7f008415ab30 2024-05-10T03:35:27.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0082000640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c750 tx=0x7f006c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f007aa00640 1 -- 172.21.15.55:0/65656712 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00ce60 con 0x7f008415ab30 2024-05-10T03:35:27.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f007aa00640 1 -- 172.21.15.55:0/65656712 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c00a640 con 0x7f008415ab30 2024-05-10T03:35:27.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f007aa00640 1 -- 172.21.15.55:0/65656712 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00a9d0 con 0x7f008415ab30 2024-05-10T03:35:27.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008414d5e0 con 0x7f008415ab30 2024-05-10T03:35:27.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.989+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008414dba0 con 0x7f008415ab30 2024-05-10T03:35:27.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.992+0000 7f007aa00640 1 -- 172.21.15.55:0/65656712 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f006c00ab30 con 0x7f008415ab30 2024-05-10T03:35:27.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.993+0000 7f007aa00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 0x7f006006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:27.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.993+0000 7f007aa00640 1 -- 172.21.15.55:0/65656712 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f006c088a70 con 0x7f008415ab30 2024-05-10T03:35:27.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.994+0000 7f0082a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 0x7f006006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:27.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.994+0000 7f0090972740 1 Processor -- start 2024-05-10T03:35:27.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.994+0000 7f0090972740 1 -- start start 2024-05-10T03:35:27.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.995+0000 7f0082a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 0x7f006006d110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0074006fd0 tx=0x7f0074008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.995+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584841350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0090972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58445a590 0x55d584861020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0090972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d584783a70 con 0x55d5844523b0 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d584781200 con 0x55d58447b270 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d58477caf0 con 0x55d58445a590 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50184/0 (socket says 172.21.15.55:50184) 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 -- 172.21.15.55:0/97413314 learned_addr learned my addr 172.21.15.55:0/97413314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082a00640 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584841350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 -- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58445a590 msgr2=0x55d584861020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58445a590 0x55d584861020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 -- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 msgr2=0x55d584841350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584841350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.996+0000 7f0082000640 1 -- 172.21.15.55:0/97413314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d584747b90 con 0x55d5844523b0 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.997+0000 7f0082000640 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f006c05f4b0 tx=0x7f006c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad80ee3f2131da5f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.997+0000 7f0073400640 1 -- 172.21.15.55:0/97413314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c05b160 con 0x55d5844523b0 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.997+0000 7f0073400640 1 -- 172.21.15.55:0/97413314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c05f9e0 con 0x55d5844523b0 2024-05-10T03:35:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.997+0000 7f0073400640 1 -- 172.21.15.55:0/97413314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c055900 con 0x55d5844523b0 2024-05-10T03:35:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 -- 172.21.15.55:0/97413314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 msgr2=0x55d584860ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f006c05f4b0 tx=0x7f006c001250 comp rx=0 tx=0).stop 2024-05-10T03:35:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 -- 172.21.15.55:0/97413314 shutdown_connections 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58445a590 0x55d584861020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d584860ae0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 --2- 172.21.15.55:0/97413314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584841350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 -- 172.21.15.55:0/97413314 >> 172.21.15.55:0/97413314 conn(0x55d58446f3c0 msgr2=0x55d58481a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 -- 172.21.15.55:0/97413314 shutdown_connections 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.998+0000 7f0090972740 1 -- 172.21.15.55:0/97413314 wait complete. 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.999+0000 7f0090972740 1 -- start start 2024-05-10T03:35:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:27.999+0000 7f0090972740 1 Processor -- start 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58445a590 0x55d584843c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 0x55d5848441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58471e220 con 0x55d5844523b0 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58471dec0 con 0x55d58445a590 2024-05-10T03:35:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5844e3f20 con 0x55d58447b270 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0090972740 10 client.0 _ll_register_callbacks cb 0x55d584776160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0082a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0082a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50198/0 (socket says 172.21.15.55:50198) 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.001+0000 7f0082a00640 1 -- 172.21.15.55:0/2099002087 learned_addr learned my addr 172.21.15.55:0/2099002087 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082000640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58445a590 0x55d584843c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0083400640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 0x55d5848441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 -- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 msgr2=0x55d5848441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 0x55d5848441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 -- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58445a590 msgr2=0x55d584843c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58445a590 0x55d584843c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 -- 172.21.15.55:0/2099002087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008414dba0 con 0x55d5844523b0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0083400640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 0x55d5848441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f0082a00640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f007400de30 tx=0x7f0074004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856e471a80457192 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f006be00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.002+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400de60 con 0x55d5844523b0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f00740125f0 con 0x55d5844523b0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5847a6c90 con 0x55d5844523b0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0074005910 con 0x55d5844523b0 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 10 client.7785 fetch_fsmap learned FSMap version 68 2024-05-10T03:35:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d58477b170 con 0x55d5844523b0 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5847a6c90 con 0x55d5844523b0 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 68) v1 ==== 3126+0+0 (secure 0 0 0) 0x7f0074005a70 con 0x55d5844523b0 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 10 client.7785 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 10 client.7785 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 20 client.7785 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d584439e40 0x55d584880300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 4 client.7785 mds_command: new command op to 7647 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.003+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 --> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5844703c0 con 0x55d584439e40 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.004+0000 7f0082000640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d584439e40 0x55d584880300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.004+0000 7f0082000640 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d584439e40 0x55d584880300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6ac654e65e9629 server_cookie=2202eec5d508bbc6 in_seq=0 out_seq=0 2024-05-10T03:35:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.004+0000 7f006be00640 10 client.7785 ms_handle_connect on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f006be00640 1 -- 172.21.15.55:0/2099002087 <== mds.0 v2:172.21.15.55:6834/1198619982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5844703c0 con 0x55d584439e40 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f006be00640 10 client.7785 handle_command_reply: tid=0 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 client.7785 shutdown 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 msgr2=0x55d58483f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f007400de30 tx=0x7f0074004680 comp rx=0 tx=0).stop 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 shutdown_connections 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d584439e40 0x55d584880300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d58447b270 0x55d5848441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58445a590 0x55d584843c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 --2- 172.21.15.55:0/2099002087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d5844523b0 0x55d58483f700 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f006be00640 0 client.7785 ms_handle_reset on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:28.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.005+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 >> 172.21.15.55:0/2099002087 conn(0x55d58446f3c0 msgr2=0x55d584862f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:28.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.006+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 shutdown_connections 2024-05-10T03:35:28.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.006+0000 7f0090972740 1 -- 172.21.15.55:0/2099002087 wait complete. 2024-05-10T03:35:28.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.006+0000 7f0090972740 20 client.7785 trim_cache size 0 max 16384 2024-05-10T03:35:28.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.024+0000 7f0090972740 1 Processor -- start 2024-05-10T03:35:28.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.024+0000 7f0090972740 1 -- start start 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d58483f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584819ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d584843bd0 con 0x55d58445a590 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58485f020 con 0x55d58447b270 2024-05-10T03:35:28.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.025+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d58485f300 con 0x55d5844523b0 2024-05-10T03:35:28.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50214/0 (socket says 172.21.15.55:50214) 2024-05-10T03:35:28.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 -- 172.21.15.55:0/1411342691 learned_addr learned my addr 172.21.15.55:0/1411342691 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:28.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0083400640 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584819ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 -- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 msgr2=0x55d58483f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d58483f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 -- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 msgr2=0x55d584819ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584819ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.026+0000 7f0082000640 1 -- 172.21.15.55:0/1411342691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5847b0130 con 0x55d58445a590 2024-05-10T03:35:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.027+0000 7f0082000640 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f006c002230 tx=0x7f006c057460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c01aee1d4fb87e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:28.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.027+0000 7f0073e00640 1 -- 172.21.15.55:0/1411342691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0760b0 con 0x55d58445a590 2024-05-10T03:35:28.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.027+0000 7f0073e00640 1 -- 172.21.15.55:0/1411342691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c0556a0 con 0x55d58445a590 2024-05-10T03:35:28.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.027+0000 7f0073e00640 1 -- 172.21.15.55:0/1411342691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c053c70 con 0x55d58445a590 2024-05-10T03:35:28.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 -- 172.21.15.55:0/1411342691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 msgr2=0x55d584863040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f006c002230 tx=0x7f006c057460 comp rx=0 tx=0).stop 2024-05-10T03:35:28.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 -- 172.21.15.55:0/1411342691 shutdown_connections 2024-05-10T03:35:28.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d584819ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d584863040 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 --2- 172.21.15.55:0/1411342691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d58483f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 -- 172.21.15.55:0/1411342691 >> 172.21.15.55:0/1411342691 conn(0x55d58446f3c0 msgr2=0x55d584847f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 -- 172.21.15.55:0/1411342691 shutdown_connections 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.028+0000 7f0090972740 1 -- 172.21.15.55:0/1411342691 wait complete. 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.029+0000 7f0090972740 1 -- start start 2024-05-10T03:35:28.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.029+0000 7f0090972740 1 Processor -- start 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d584819ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d58487c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d584861900 con 0x55d58445a590 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d584845970 con 0x55d58447b270 2024-05-10T03:35:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5848cd8e0 con 0x55d5844523b0 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0090972740 10 client.0 _ll_register_callbacks cb 0x7f007400bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0082000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50218/0 (socket says 172.21.15.55:50218) 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.031+0000 7f0082000640 1 -- 172.21.15.55:0/2158338967 learned_addr learned my addr 172.21.15.55:0/2158338967 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 -- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 msgr2=0x55d584819ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d584819ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 -- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 msgr2=0x55d58487c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0083400640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d58487c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d58487c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 -- 172.21.15.55:0/2158338967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d584747b90 con 0x55d58445a590 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0083400640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d58487c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0082000640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f006c0574f0 tx=0x7f006c05a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765eaca7a9b9a23a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f006be00640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c084020 con 0x55d58445a590 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f006c053c70 con 0x55d58445a590 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.032+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f006c05f9e0 con 0x55d58445a590 2024-05-10T03:35:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c053e50 con 0x55d58445a590 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 10 client.7791 fetch_fsmap learned FSMap version 68 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d58471d430 con 0x55d58445a590 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c05f9e0 con 0x55d58445a590 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 68) v1 ==== 3126+0+0 (secure 0 0 0) 0x7f006c05f550 con 0x55d58445a590 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 10 client.7791 fetch_fsmap finished waiting for FSMap version 68 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 10 client.7791 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 20 client.7791 populate_metadata read hostname 'smithi055' 2024-05-10T03:35:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d58487c600 0x55d58487ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 4 client.7791 mds_command: new command op to 7647 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 --> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5844816c0 con 0x55d58487c600 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.033+0000 7f0082a00640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d58487c600 0x55d58487ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f006be00640 10 client.7791 ms_handle_connect on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0082a00640 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d58487c600 0x55d58487ca10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2aaf881a298de64 server_cookie=5e4a6b05d648690d in_seq=0 out_seq=0 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f006be00640 1 -- 172.21.15.55:0/2158338967 <== mds.0 v2:172.21.15.55:6834/1198619982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55d5844816c0 con 0x55d58487c600 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f006be00640 10 client.7791 handle_command_reply: tid=0 2024-05-10T03:35:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0090972740 1 client.7791 shutdown 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 msgr2=0x55d58481a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f006c0574f0 tx=0x7f006c05a070 comp rx=0 tx=0).stop 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 shutdown_connections 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.034+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:6834/1198619982,v1:172.21.15.55:6835/1198619982] conn(0x55d58487c600 0x55d58487ca10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58447b270 0x55d58487c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d58445a590 0x55d58481a410 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 --2- 172.21.15.55:0/2158338967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d5844523b0 0x55d584819ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f006be00640 0 client.7791 ms_handle_reset on v2:172.21.15.55:6834/1198619982 2024-05-10T03:35:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 >> 172.21.15.55:0/2158338967 conn(0x55d58446f3c0 msgr2=0x55d584844030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 shutdown_connections 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 1 -- 172.21.15.55:0/2158338967 wait complete. 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0090972740 20 client.7791 trim_cache size 0 max 16384 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "340d5548-dd78-4f40-a3a8-44ee99f8e0d8", 2024-05-10T03:35:28.045 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7647, 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 13.801613009, 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 68, 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 110, 2024-05-10T03:35:28.046 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 109, 2024-05-10T03:35:28.047 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 14.537404123 2024-05-10T03:35:28.047 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 msgr2=0x7f006006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.035+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 0x7f006006d110 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0074006fd0 tx=0x7f0074008040 comp rx=0 tx=0).stop 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 msgr2=0x7f00841f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c750 tx=0x7f006c00cc10 comp rx=0 tx=0).stop 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 shutdown_connections 2024-05-10T03:35:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f008415af40 0x7f008414e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f008415ab30 0x7f00841f1bb0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f006006ac00 0x7f006006d110 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 --2- 172.21.15.55:0/65656712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00840b3200 0x7f008414be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 >> 172.21.15.55:0/65656712 conn(0x7f00840b0b80 msgr2=0x7f0084070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 shutdown_connections 2024-05-10T03:35:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:28.036+0000 7f0089a00640 1 -- 172.21.15.55:0/65656712 wait complete. 2024-05-10T03:35:28.064 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-10T03:35:28.385 INFO:teuthology.orchestra.run.smithi055.stdout:{"message":"","return_code":0} 2024-05-10T03:35:28.385 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T03:35:28.385 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-05-10T03:35:28.687 INFO:teuthology.orchestra.run.smithi055.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":38,"op_send_bytes":11382,"op_resend":33,"op_reply":38,"op_latency":{"avgcount":38,"sum":0.667986736,"avgtime":0.017578598},"op_inflight":0,"oplen_avg":{"avgcount":38,"sum":60},"op":38,"op_r":0,"op_w":38,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":110,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":14,"omap_rd":0,"omap_del":1}} 2024-05-10T03:35:28.688 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 110, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 14, "op": 38, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 38, "avgtime": 0.017578598, "sum": 0.667986736 }, "op_pg": 0, "op_r": 0, "op_reply": 38, "op_resend": 33, "op_rmw": 0, "op_send": 38, "op_send_bytes": 11382, "op_w": 38, "oplen_avg": { "avgcount": 38, "sum": 60 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-10T03:35:28.688 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json export dir /bravo 1 2024-05-10T03:35:28.996 INFO:teuthology.orchestra.run.smithi055.stdout:{"return_code":0} 2024-05-10T03:35:28.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-10T03:35:28.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-05-10T03:35:29.298 INFO:teuthology.orchestra.run.smithi055.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":38,"op_send_bytes":11382,"op_resend":33,"op_reply":38,"op_latency":{"avgcount":38,"sum":0.667986736,"avgtime":0.017578598},"op_inflight":0,"oplen_avg":{"avgcount":38,"sum":60},"op":38,"op_r":0,"op_w":38,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":110,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":14,"omap_rd":0,"omap_del":1}} 2024-05-10T03:35:29.299 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 110, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 14, "op": 38, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 38, "avgtime": 0.017578598, "sum": 0.667986736 }, "op_pg": 0, "op_r": 0, "op_reply": 38, "op_resend": 33, "op_rmw": 0, "op_send": 38, "op_send_bytes": 11382, "op_w": 38, "oplen_avg": { "avgcount": 38, "sum": 60 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-10T03:35:29.299 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-05-10T03:35:34.300 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-05-10T03:35:34.620 INFO:teuthology.orchestra.run.smithi055.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":14701,"op_resend":33,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.676986552,"avgtime":0.016924663},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":63},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":3,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":110,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":16,"omap_rd":0,"omap_del":1}} 2024-05-10T03:35:34.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 110, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 16, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.016924663, "sum": 0.676986552 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 33, "op_rmw": 0, "op_send": 40, "op_send_bytes": 14701, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 63 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-10T03:35:34.621 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-05-10T03:35:39.622 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-05-10T03:35:39.941 INFO:teuthology.orchestra.run.smithi055.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":42,"op_send_bytes":15035,"op_resend":33,"op_reply":42,"op_latency":{"avgcount":42,"sum":0.684986389,"avgtime":0.016309199},"op_inflight":0,"oplen_avg":{"avgcount":42,"sum":66},"op":42,"op_r":0,"op_w":42,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":4,"osdop_writefull":7,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":110,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":18,"omap_rd":0,"omap_del":1}} 2024-05-10T03:35:39.942 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 110, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 18, "op": 42, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 42, "avgtime": 0.016309199, "sum": 0.684986389 }, "op_pg": 0, "op_r": 0, "op_reply": 42, "op_resend": 33, "op_rmw": 0, "op_send": 42, "op_send_bytes": 15035, "op_w": 42, "oplen_avg": { "avgcount": 42, "sum": 66 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 7, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-05-10T03:35:39.942 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-05-10T03:35:39.942 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:39.942 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:39.996 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:39.996 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:40.055 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.341+0000 7f2d73e00640 1 Processor -- start 2024-05-10T03:35:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.341+0000 7f2d73e00640 1 -- start start 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d6c0cd1b0 0x7f2d6c0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0b1ca0 0x7f2d6c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cf5b0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b20b0 con 0x7f2d6c0cd1b0 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d73e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c06e510 con 0x7f2d6c0b1ca0 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d71e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d71e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55212/0 (socket says 172.21.15.55:55212) 2024-05-10T03:35:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.344+0000 7f2d71e00640 1 -- 172.21.15.55:0/3695525905 learned_addr learned my addr 172.21.15.55:0/3695525905 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71400640 1 --2- 172.21.15.55:0/3695525905 >> v2:172.21.15.162:3300/0 conn(0x7f2d6c0cd1b0 0x7f2d6c0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d70a00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0b1ca0 0x7f2d6c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71e00640 1 -- 172.21.15.55:0/3695525905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0b1ca0 msgr2=0x7f2d6c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71e00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0b1ca0 0x7f2d6c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71e00640 1 -- 172.21.15.55:0/3695525905 >> v2:172.21.15.162:3300/0 conn(0x7f2d6c0cd1b0 msgr2=0x7f2d6c0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71e00640 1 --2- 172.21.15.55:0/3695525905 >> v2:172.21.15.162:3300/0 conn(0x7f2d6c0cd1b0 0x7f2d6c0b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.345+0000 7f2d71e00640 1 -- 172.21.15.55:0/3695525905 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0708c0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.346+0000 7f2d71e00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0b2350 tx=0x7f2d5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadb40221535f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.346+0000 7f2d6be00640 1 -- 172.21.15.55:0/3695525905 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c032440 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.346+0000 7f2d6be00640 1 -- 172.21.15.55:0/3695525905 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c0325a0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.346+0000 7f2d6be00640 1 -- 172.21.15.55:0/3695525905 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c032930 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 -- 172.21.15.55:0/3695525905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 msgr2=0x7f2d6c0ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0b2350 tx=0x7f2d5c02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 -- 172.21.15.55:0/3695525905 shutdown_connections 2024-05-10T03:35:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c0ae530 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 --2- 172.21.15.55:0/3695525905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0b1ca0 0x7f2d6c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 --2- 172.21.15.55:0/3695525905 >> v2:172.21.15.162:3300/0 conn(0x7f2d6c0cd1b0 0x7f2d6c0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 -- 172.21.15.55:0/3695525905 >> 172.21.15.55:0/3695525905 conn(0x7f2d6c06a530 msgr2=0x7f2d6c06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:40.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 -- 172.21.15.55:0/3695525905 shutdown_connections 2024-05-10T03:35:40.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.347+0000 7f2d73e00640 1 -- 172.21.15.55:0/3695525905 wait complete. 2024-05-10T03:35:40.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.350+0000 7f2d73e00640 1 Processor -- start 2024-05-10T03:35:40.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.350+0000 7f2d73e00640 1 -- start start 2024-05-10T03:35:40.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.351+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 0x7f2d6c166e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 0x7f2d6c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d73e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c156ea0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d73e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c14c720 con 0x7f2d6c0b1ca0 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d73e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c14c0b0 con 0x7f2d6c0cd1b0 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55226/0 (socket says 172.21.15.55:55226) 2024-05-10T03:35:40.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 -- 172.21.15.55:0/2277090495 learned_addr learned my addr 172.21.15.55:0/2277090495 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71400640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 0x7f2d6c166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d70a00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 0x7f2d6c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 -- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 msgr2=0x7f2d6c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 0x7f2d6c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 -- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 msgr2=0x7f2d6c166e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.352+0000 7f2d71e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 0x7f2d6c166e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d71e00640 1 -- 172.21.15.55:0/2277090495 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0094d0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d71400640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 0x7f2d6c166e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d70a00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 0x7f2d6c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d71e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0090b0 tx=0x7f2d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c032440 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c032e90 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c030de0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c1573d0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.353+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c14ba60 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.355+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c071a60 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.356+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2d5c039040 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.357+0000 7f2d69600640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 0x7f2d4c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.357+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f2d5c0aa0d0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.357+0000 7f2d71400640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 0x7f2d4c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.359+0000 7f2d71400640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 0x7f2d4c06d110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f50 tx=0x7f2d6400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.362+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2d5c074f90 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.561+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d6c14c220 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.564+0000 7f2d69600640 1 -- 172.21.15.55:0/2277090495 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16733 (secure 0 0 0) 0x7f2d5c0785b0 con 0x7f2d6c0ceea0 2024-05-10T03:35:40.567 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:40.568 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:25.331003+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:40.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 msgr2=0x7f2d4c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 0x7f2d4c06d110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f50 tx=0x7f2d6400b040 comp rx=0 tx=0).stop 2024-05-10T03:35:40.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 msgr2=0x7f2d6c14d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0090b0 tx=0x7f2d5c0046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 shutdown_connections 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ceea0 0x7f2d6c14d460 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d6c0cd1b0 0x7f2d6c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2d4c06ac00 0x7f2d4c06d110 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 --2- 172.21.15.55:0/2277090495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d6c0b1ca0 0x7f2d6c166e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 >> 172.21.15.55:0/2277090495 conn(0x7f2d6c06a530 msgr2=0x7f2d6c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 shutdown_connections 2024-05-10T03:35:40.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.567+0000 7f2d73e00640 1 -- 172.21.15.55:0/2277090495 wait complete. 2024-05-10T03:35:40.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:53.261121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:54.084182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:35:40.596 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:35:40.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.884+0000 7f2e2b200640 1 Processor -- start 2024-05-10T03:35:40.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.885+0000 7f2e2b200640 1 -- start start 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2b200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e240cbfb0 0x7f2e24166120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2b200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e240cd960 con 0x7f2e2415b540 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2b200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e240cdc40 con 0x7f2e240adea0 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2b200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240cd750 con 0x7f2e240cbfb0 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e240cbfb0 0x7f2e24166120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2a800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e240cbfb0 0x7f2e24166120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55234/0 (socket says 172.21.15.55:55234) 2024-05-10T03:35:40.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.887+0000 7f2e2a800640 1 -- 172.21.15.55:0/648905911 learned_addr learned my addr 172.21.15.55:0/648905911 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:40.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.888+0000 7f2e2a800640 1 -- 172.21.15.55:0/648905911 >> v1:172.21.15.162:6789/0 conn(0x7f2e240adea0 legacy=0x7f2e2415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:40.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.888+0000 7f2e2a800640 1 -- 172.21.15.55:0/648905911 >> v1:172.21.15.55:6789/0 conn(0x7f2e2415b540 legacy=0x7f2e240cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.888+0000 7f2e2a800640 1 -- 172.21.15.55:0/648905911 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240cdf20 con 0x7f2e240cbfb0 2024-05-10T03:35:40.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.888+0000 7f2e28a00640 1 -- 172.21.15.55:0/648905911 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673935836 0 0) 0x7f2e240cd960 con 0x7f2e2415b540 2024-05-10T03:35:40.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.889+0000 7f2e2a800640 1 --2- 172.21.15.55:0/648905911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e240cbfb0 0x7f2e24166120 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009e50 tx=0x7f2e2002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb5b22744b507e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e28a00640 1 -- 172.21.15.55:0/648905911 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20031600 con 0x7f2e240cbfb0 2024-05-10T03:35:40.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e28a00640 1 -- 172.21.15.55:0/648905911 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2002d050 con 0x7f2e240cbfb0 2024-05-10T03:35:40.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e28a00640 1 -- 172.21.15.55:0/648905911 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20031980 con 0x7f2e240cbfb0 2024-05-10T03:35:40.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e2b200640 1 -- 172.21.15.55:0/648905911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e240cbfb0 msgr2=0x7f2e24166120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e2b200640 1 --2- 172.21.15.55:0/648905911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e240cbfb0 0x7f2e24166120 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009e50 tx=0x7f2e2002a780 comp rx=0 tx=0).stop 2024-05-10T03:35:40.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e2b200640 1 -- 172.21.15.55:0/648905911 shutdown_connections 2024-05-10T03:35:40.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e2b200640 1 --2- 172.21.15.55:0/648905911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e240cbfb0 0x7f2e24166120 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.890+0000 7f2e2b200640 1 -- 172.21.15.55:0/648905911 >> 172.21.15.55:0/648905911 conn(0x7f2e2406a930 msgr2=0x7f2e240b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:40.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.891+0000 7f2e2b200640 1 -- 172.21.15.55:0/648905911 shutdown_connections 2024-05-10T03:35:40.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.891+0000 7f2e2b200640 1 -- 172.21.15.55:0/648905911 wait complete. 2024-05-10T03:35:40.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.894+0000 7f2e2b200640 1 Processor -- start 2024-05-10T03:35:40.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.894+0000 7f2e2b200640 1 -- start start 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e240adea0 0x7f2e240b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e240b2b20 0x7f2e240b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241562a0 con 0x7f2e2415b540 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24152aa0 con 0x7f2e240adea0 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.895+0000 7f2e2b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240b25f0 con 0x7f2e240b2b20 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55244/0 (socket says 172.21.15.55:55244) 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 -- 172.21.15.55:0/809476888 learned_addr learned my addr 172.21.15.55:0/809476888 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29e00640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e240adea0 0x7f2e240b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e2a800640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e240b2b20 0x7f2e240b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 -- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e240b2b20 msgr2=0x7f2e240b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e240b2b20 0x7f2e240b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 -- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e240adea0 msgr2=0x7f2e240b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:40.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e240adea0 0x7f2e240b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:40.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.896+0000 7f2e29400640 1 -- 172.21.15.55:0/809476888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20009b30 con 0x7f2e2415b540 2024-05-10T03:35:40.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e29400640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400c440 tx=0x7f2e1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14005480 con 0x7f2e2415b540 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e140035f0 con 0x7f2e2415b540 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24152cf0 con 0x7f2e2415b540 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140039a0 con 0x7f2e2415b540 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.897+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e241532b0 con 0x7f2e2415b540 2024-05-10T03:35:40.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.898+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24067240 con 0x7f2e2415b540 2024-05-10T03:35:40.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.900+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2e14003b00 con 0x7f2e2415b540 2024-05-10T03:35:40.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.901+0000 7f2e1e000640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 0x7f2e0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:40.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.901+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f2e14088c30 con 0x7f2e2415b540 2024-05-10T03:35:40.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.902+0000 7f2e29e00640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 0x7f2e0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:40.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.903+0000 7f2e29e00640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 0x7f2e0806d140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:40.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:40.907+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2e14053a60 con 0x7f2e2415b540 2024-05-10T03:35:41.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.104+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e2415adb0 con 0x7f2e2415b540 2024-05-10T03:35:41.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.106+0000 7f2e1e000640 1 -- 172.21.15.55:0/809476888 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e14017030 con 0x7f2e2415b540 2024-05-10T03:35:41.109 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:35:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 msgr2=0x7f2e0806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 0x7f2e0806d140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).stop 2024-05-10T03:35:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 msgr2=0x7f2e24146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400c440 tx=0x7f2e1400c900 comp rx=0 tx=0).stop 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 shutdown_connections 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e240b2b20 0x7f2e240b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e2415b540 0x7f2e24146590 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f2e0806ac30 0x7f2e0806d140 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 --2- 172.21.15.55:0/809476888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e240adea0 0x7f2e240b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.108+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 >> 172.21.15.55:0/809476888 conn(0x7f2e2406a930 msgr2=0x7f2e240b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.109+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 shutdown_connections 2024-05-10T03:35:41.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.109+0000 7f2e2b200640 1 -- 172.21.15.55:0/809476888 wait complete. 2024-05-10T03:35:41.138 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:35:41.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.419+0000 7f3437800640 1 Processor -- start 2024-05-10T03:35:41.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.419+0000 7f3437800640 1 -- start start 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34300cc2c0 0x7f343015c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3430141ae0 0x7f343006c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34300cdd00 0x7f34300ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300cec60 con 0x7f34300cdd00 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343015cbb0 con 0x7f3430141ae0 2024-05-10T03:35:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3437800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34300cbf60 con 0x7f34300cc2c0 2024-05-10T03:35:41.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3435800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34300cdd00 0x7f34300ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.421+0000 7f3435800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34300cdd00 0x7f34300ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55260/0 (socket says 172.21.15.55:55260) 2024-05-10T03:35:41.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 -- 172.21.15.55:0/2343380529 learned_addr learned my addr 172.21.15.55:0/2343380529 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:41.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3434e00640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34300cc2c0 0x7f343015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f342fe00640 1 --2- 172.21.15.55:0/2343380529 >> v2:172.21.15.162:3300/0 conn(0x7f3430141ae0 0x7f343006c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 -- 172.21.15.55:0/2343380529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34300cc2c0 msgr2=0x7f343015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34300cc2c0 0x7f343015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 -- 172.21.15.55:0/2343380529 >> v2:172.21.15.162:3300/0 conn(0x7f3430141ae0 msgr2=0x7f343006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 --2- 172.21.15.55:0/2343380529 >> v2:172.21.15.162:3300/0 conn(0x7f3430141ae0 0x7f343006c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.422+0000 7f3435800640 1 -- 172.21.15.55:0/2343380529 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300cb9e0 con 0x7f34300cdd00 2024-05-10T03:35:41.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.423+0000 7f3435800640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f34300ce110 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3430066fa0 tx=0x7f342002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c9b22c4a7ab2e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f342f400640 1 -- 172.21.15.55:0/2343380529 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342002ae90 con 0x7f34300cdd00 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f342f400640 1 -- 172.21.15.55:0/2343380529 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420032440 con 0x7f34300cdd00 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f342f400640 1 -- 172.21.15.55:0/2343380529 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200327d0 con 0x7f34300cdd00 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 -- 172.21.15.55:0/2343380529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 msgr2=0x7f34300ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f34300ce110 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3430066fa0 tx=0x7f342002a140 comp rx=0 tx=0).stop 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 -- 172.21.15.55:0/2343380529 shutdown_connections 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f34300ce110 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 --2- 172.21.15.55:0/2343380529 >> v2:172.21.15.162:3300/0 conn(0x7f3430141ae0 0x7f343006c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 --2- 172.21.15.55:0/2343380529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34300cc2c0 0x7f343015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.424+0000 7f3437800640 1 -- 172.21.15.55:0/2343380529 >> 172.21.15.55:0/2343380529 conn(0x7f34300ceeb0 msgr2=0x7f34300cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.425+0000 7f3437800640 1 -- 172.21.15.55:0/2343380529 shutdown_connections 2024-05-10T03:35:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.425+0000 7f3437800640 1 -- 172.21.15.55:0/2343380529 wait complete. 2024-05-10T03:35:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.427+0000 7f3437800640 1 Processor -- start 2024-05-10T03:35:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.427+0000 7f3437800640 1 -- start start 2024-05-10T03:35:41.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 0x7f3430137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f343015a5a0 0x7f3430138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3430156480 con 0x7f34300cdd00 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343015d350 con 0x7f3430141ae0 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3437800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343015aac0 con 0x7f343015a5a0 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55272/0 (socket says 172.21.15.55:55272) 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 -- 172.21.15.55:0/133440114 learned_addr learned my addr 172.21.15.55:0/133440114 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f342fe00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 0x7f3430137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 -- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f343015a5a0 msgr2=0x7f3430138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f343015a5a0 0x7f3430138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 -- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 msgr2=0x7f3430137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.429+0000 7f3434e00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 0x7f3430137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f3434e00640 1 -- 172.21.15.55:0/133440114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200094d0 con 0x7f34300cdd00 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f342fe00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 0x7f3430137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:41.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f3434e00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f342800d5f0 tx=0x7f342800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280125f0 con 0x7f34300cdd00 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3428012750 con 0x7f34300cdd00 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428012b00 con 0x7f34300cdd00 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.430+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343015b3f0 con 0x7f34300cdd00 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.431+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34301371c0 con 0x7f34300cdd00 2024-05-10T03:35:41.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.432+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34300cd4b0 con 0x7f34300cdd00 2024-05-10T03:35:41.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.434+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3428012c60 con 0x7f34300cdd00 2024-05-10T03:35:41.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.435+0000 7f342cc00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 0x7f340c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.435+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f3428089990 con 0x7f34300cdd00 2024-05-10T03:35:41.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.436+0000 7f342fe00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 0x7f340c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.437+0000 7f342fe00640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 0x7f340c06d110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f343006ff90 tx=0x7f341800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.440+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3428054860 con 0x7f34300cdd00 2024-05-10T03:35:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.645+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f343015b3f0 con 0x7f34300cdd00 2024-05-10T03:35:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.646+0000 7f342cc00640 1 -- 172.21.15.55:0/133440114 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f343015b3f0 con 0x7f34300cdd00 2024-05-10T03:35:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:35:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 msgr2=0x7f340c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 0x7f340c06d110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f343006ff90 tx=0x7f341800b040 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 msgr2=0x7f3430155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f342800d5f0 tx=0x7f342800dab0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 shutdown_connections 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f343015a5a0 0x7f3430138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f340c06ac00 0x7f340c06d110 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3430141ae0 0x7f3430137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 --2- 172.21.15.55:0/133440114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34300cdd00 0x7f3430155f40 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 >> 172.21.15.55:0/133440114 conn(0x7f34300ceeb0 msgr2=0x7f34300cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.649+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 shutdown_connections 2024-05-10T03:35:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.650+0000 7f3437800640 1 -- 172.21.15.55:0/133440114 wait complete. 2024-05-10T03:35:41.678 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:35:41.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.954+0000 7f6cd7e00640 1 Processor -- start 2024-05-10T03:35:41.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.954+0000 7f6cd7e00640 1 -- start start 2024-05-10T03:35:41.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.956+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd815a540 0x7f6cd80717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cd806d600 0x7f6cd806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd806df50 con 0x7f6cd80b1ab0 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd80cd5a0 con 0x7f6cd806d600 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8070ce0 con 0x7f6cd815a540 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd6000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6cd806d600 0x7f6cd806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6cd80b1ab0 0x7f6cd80b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60512/0 (socket says 172.21.15.55:60512) 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd7400640 1 -- 172.21.15.55:0/2497766945 learned_addr learned my addr 172.21.15.55:0/2497766945 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:41.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.957+0000 7f6cd6a00640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd815a540 0x7f6cd80717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.958+0000 7f6cd6000640 1 -- 172.21.15.55:0/2497766945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd815a540 msgr2=0x7f6cd80717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.958+0000 7f6cd6000640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd815a540 0x7f6cd80717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.958+0000 7f6cd6000640 1 -- 172.21.15.55:0/2497766945 >> v1:172.21.15.55:6789/0 conn(0x7f6cd80b1ab0 legacy=0x7f6cd80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.958+0000 7f6cd6000640 1 -- 172.21.15.55:0/2497766945 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8070fc0 con 0x7f6cd806d600 2024-05-10T03:35:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.959+0000 7f6cd6000640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd806d600 0x7f6cd806da10 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959ecfe2844b63fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.959+0000 7f6cd5600640 1 -- 172.21.15.55:0/2497766945 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032440 con 0x7f6cd806d600 2024-05-10T03:35:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.959+0000 7f6cd5600640 1 -- 172.21.15.55:0/2497766945 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc40325a0 con 0x7f6cd806d600 2024-05-10T03:35:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.959+0000 7f6cd5600640 1 -- 172.21.15.55:0/2497766945 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032930 con 0x7f6cd806d600 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 -- 172.21.15.55:0/2497766945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd806d600 msgr2=0x7f6cd806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd806d600 0x7f6cd806da10 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402a180 comp rx=0 tx=0).stop 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 -- 172.21.15.55:0/2497766945 shutdown_connections 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd806d600 0x7f6cd806da10 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/2497766945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd815a540 0x7f6cd80717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 -- 172.21.15.55:0/2497766945 >> 172.21.15.55:0/2497766945 conn(0x7f6cd80cebc0 msgr2=0x7f6cd80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 -- 172.21.15.55:0/2497766945 shutdown_connections 2024-05-10T03:35:41.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.960+0000 7f6cd7e00640 1 -- 172.21.15.55:0/2497766945 wait complete. 2024-05-10T03:35:41.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.962+0000 7f6cd7e00640 1 Processor -- start 2024-05-10T03:35:41.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.963+0000 7f6cd7e00640 1 -- start start 2024-05-10T03:35:41.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd806d600 0x7f6cd81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd80b1ab0 0x7f6cd81e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80af6d0 con 0x7f6cd815a540 2024-05-10T03:35:41.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81e4080 con 0x7f6cd806d600 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.964+0000 7f6cd7e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81555e0 con 0x7f6cd80b1ab0 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55300/0 (socket says 172.21.15.55:55300) 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 -- 172.21.15.55:0/4247750302 learned_addr learned my addr 172.21.15.55:0/4247750302 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd6000640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd80b1ab0 0x7f6cd81e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd6a00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd806d600 0x7f6cd81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 -- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd80b1ab0 msgr2=0x7f6cd81e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd80b1ab0 0x7f6cd81e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 -- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd806d600 msgr2=0x7f6cd81550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd806d600 0x7f6cd81550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.965+0000 7f6cd7400640 1 -- 172.21.15.55:0/4247750302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009400 con 0x7f6cd815a540 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6cd7400640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000ca20 tx=0x7f6cd000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000abd0 con 0x7f6cd815a540 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd000ad30 con 0x7f6cd815a540 2024-05-10T03:35:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0012600 con 0x7f6cd815a540 2024-05-10T03:35:41.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd81e42d0 con 0x7f6cd815a540 2024-05-10T03:35:41.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.966+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd81f3f80 con 0x7f6cd815a540 2024-05-10T03:35:41.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.967+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8067240 con 0x7f6cd815a540 2024-05-10T03:35:41.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.969+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6cd0007070 con 0x7f6cd815a540 2024-05-10T03:35:41.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.970+0000 7f6ccf400640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:41.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.970+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f6cd00893f0 con 0x7f6cd815a540 2024-05-10T03:35:41.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.970+0000 7f6cd6a00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:41.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.971+0000 7f6cd6a00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006fd0 tx=0x7f6cc8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:41.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:41.974+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6cd0054360 con 0x7f6cd815a540 2024-05-10T03:35:42.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.183+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cd80cde40 con 0x7f6cd815a540 2024-05-10T03:35:42.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.183+0000 7f6ccf400640 1 -- 172.21.15.55:0/4247750302 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6cd0057980 con 0x7f6cd815a540 2024-05-10T03:35:42.186 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:35:42.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 msgr2=0x7f6cb806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006fd0 tx=0x7f6cc8008040 comp rx=0 tx=0).stop 2024-05-10T03:35:42.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 msgr2=0x7f6cd8155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000ca20 tx=0x7f6cd000cee0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 shutdown_connections 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cd815a540 0x7f6cd8155f30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cd80b1ab0 0x7f6cd81e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6cb806ac00 0x7f6cb806d110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 --2- 172.21.15.55:0/4247750302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cd806d600 0x7f6cd81550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.186+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 >> 172.21.15.55:0/4247750302 conn(0x7f6cd80cebc0 msgr2=0x7f6cd81591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.187+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 shutdown_connections 2024-05-10T03:35:42.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.187+0000 7f6cd7e00640 1 -- 172.21.15.55:0/4247750302 wait complete. 2024-05-10T03:35:42.217 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:35:42.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.506+0000 7fafe2200640 1 Processor -- start 2024-05-10T03:35:42.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.506+0000 7fafe2200640 1 -- start start 2024-05-10T03:35:42.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.508+0000 7fafe2200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fafdc06fc00 0x7fafdc159b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.508+0000 7fafe2200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fafdc06cc00 0x7fafdc0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.508+0000 7fafe2200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc070170 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.508+0000 7fafe2200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc070450 con 0x7fafdc06cc00 2024-05-10T03:35:42.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.508+0000 7fafe2200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc070730 con 0x7fafdc06fc00 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fafdc06cc00 0x7fafdc0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fafdc06cc00 0x7fafdc0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55304/0 (socket says 172.21.15.55:55304) 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafd3e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fafdc0b1ab0 0x7fafdc06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:60524/0 (socket says 172.21.15.55:60524) 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 -- 172.21.15.55:0/1436552883 learned_addr learned my addr 172.21.15.55:0/1436552883 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe0e00640 1 --2- 172.21.15.55:0/1436552883 >> v2:172.21.15.162:3300/0 conn(0x7fafdc06fc00 0x7fafdc159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 -- 172.21.15.55:0/1436552883 >> v2:172.21.15.162:3300/0 conn(0x7fafdc06fc00 msgr2=0x7fafdc159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:42.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 --2- 172.21.15.55:0/1436552883 >> v2:172.21.15.162:3300/0 conn(0x7fafdc06fc00 0x7fafdc159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 -- 172.21.15.55:0/1436552883 >> v1:172.21.15.55:6789/0 conn(0x7fafdc0b1ab0 legacy=0x7fafdc06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.509+0000 7fafe1800640 1 -- 172.21.15.55:0/1436552883 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc070a10 con 0x7fafdc06cc00 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.511+0000 7fafe1800640 1 --2- 172.21.15.55:0/1436552883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc06cc00 0x7fafdc0cbe70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fafdc06f0f0 tx=0x7fafd402a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad017be7c73ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.511+0000 7fafdbe00640 1 -- 172.21.15.55:0/1436552883 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4031600 con 0x7fafdc06cc00 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.511+0000 7fafdbe00640 1 -- 172.21.15.55:0/1436552883 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd402d050 con 0x7fafdc06cc00 2024-05-10T03:35:42.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.511+0000 7fafdbe00640 1 -- 172.21.15.55:0/1436552883 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4031980 con 0x7fafdc06cc00 2024-05-10T03:35:42.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 -- 172.21.15.55:0/1436552883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc06cc00 msgr2=0x7fafdc0cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 --2- 172.21.15.55:0/1436552883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc06cc00 0x7fafdc0cbe70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fafdc06f0f0 tx=0x7fafd402a6b0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 -- 172.21.15.55:0/1436552883 shutdown_connections 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 --2- 172.21.15.55:0/1436552883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc06cc00 0x7fafdc0cbe70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 --2- 172.21.15.55:0/1436552883 >> v2:172.21.15.162:3300/0 conn(0x7fafdc06fc00 0x7fafdc159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 -- 172.21.15.55:0/1436552883 >> 172.21.15.55:0/1436552883 conn(0x7fafdc0b07b0 msgr2=0x7fafdc0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 -- 172.21.15.55:0/1436552883 shutdown_connections 2024-05-10T03:35:42.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.512+0000 7fafe2200640 1 -- 172.21.15.55:0/1436552883 wait complete. 2024-05-10T03:35:42.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.515+0000 7fafe2200640 1 Processor -- start 2024-05-10T03:35:42.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.515+0000 7fafe2200640 1 -- start start 2024-05-10T03:35:42.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 0x7fafdc15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafdc0cd460 0x7fafdc148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15b4d0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc148ed0 con 0x7fafdc06fc00 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe2200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc15cb50 con 0x7fafdc0cd460 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafd3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafd3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55310/0 (socket says 172.21.15.55:55310) 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafd3e00640 1 -- 172.21.15.55:0/3801649279 learned_addr learned my addr 172.21.15.55:0/3801649279 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe0e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 0x7fafdc15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.517+0000 7fafe1800640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafdc0cd460 0x7fafdc148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 -- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafdc0cd460 msgr2=0x7fafdc148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafdc0cd460 0x7fafdc148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 -- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 msgr2=0x7fafdc15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 0x7fafdc15af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 -- 172.21.15.55:0/3801649279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009b30 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafe0e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 0x7fafdc15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd3e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c410 tx=0x7fafcc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:42.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.518+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0074e0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.519+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc007640 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.519+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0049e0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.519+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc071390 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.519+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc15a4b0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.520+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc067240 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.522+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fafcc004b40 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.523+0000 7fafd9600640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 0x7fafb806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:42.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.523+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fafcc0890c0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.523+0000 7fafe0e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 0x7fafb806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:42.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.525+0000 7fafe0e00640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 0x7fafb806d110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:42.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.527+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fafcc053ef0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.740+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fafdc06d8c0 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.741+0000 7fafd9600640 1 -- 172.21.15.55:0/3801649279 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v110) v1 ==== 80+0+364 (secure 0 0 0) 0x7fafcc057510 con 0x7fafdc0b1ab0 2024-05-10T03:35:42.744 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.162:6837/476611421 2024-05-11T03:35:13.193337+0000 2024-05-10T03:35:42.744 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.162:6836/476611421 2024-05-11T03:35:13.193337+0000 2024-05-10T03:35:42.744 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6833/2254900234 2024-05-11T03:35:13.167318+0000 2024-05-10T03:35:42.744 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6832/2254900234 2024-05-11T03:35:13.167318+0000 2024-05-10T03:35:42.745 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6835/315649334 2024-05-11T03:35:13.117413+0000 2024-05-10T03:35:42.745 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6834/315649334 2024-05-11T03:35:13.117413+0000 2024-05-10T03:35:42.745 INFO:teuthology.orchestra.run.smithi055.stderr:listed 6 entries 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 msgr2=0x7fafb806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 0x7fafb806d110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4008040 comp rx=0 tx=0).stop 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 msgr2=0x7fafdc0ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c410 tx=0x7fafcc00c8d0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 shutdown_connections 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafdc0cd460 0x7fafdc148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafdc0b1ab0 0x7fafdc0ccf20 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fafb806ac00 0x7fafb806d110 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 --2- 172.21.15.55:0/3801649279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafdc06fc00 0x7fafdc15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:42.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 >> 172.21.15.55:0/3801649279 conn(0x7fafdc0b07b0 msgr2=0x7fafdc15e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:42.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 shutdown_connections 2024-05-10T03:35:42.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:42.744+0000 7fafe2200640 1 -- 172.21.15.55:0/3801649279 wait complete. 2024-05-10T03:35:42.776 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T03:35:42.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:42.776 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T03:35:42.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:35:42.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:42.903 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T03:35:42.968 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:42.968 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:43.009 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:43.010 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:43.097 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:43.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.372+0000 7f03abe00640 1 Processor -- start 2024-05-10T03:35:43.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.372+0000 7f03abe00640 1 -- start start 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.374+0000 7f03abe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03ac071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.374+0000 7f03abe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03ac152280 0x7f03ac15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.374+0000 7f03abe00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac15c040 con 0x7f03ac152690 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.374+0000 7f03abe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac15c320 con 0x7f03ac152280 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.374+0000 7f03abe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac15c600 con 0x7f03ac15ba60 2024-05-10T03:35:43.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03ab400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03ac152690 0x7f03ac0cd7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39284/0 (socket says 172.21.15.55:39284) 2024-05-10T03:35:43.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03ab400640 1 -- 172.21.15.55:0/117744341 learned_addr learned my addr 172.21.15.55:0/117744341 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03aaa00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03ac071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03aa000640 1 --2- 172.21.15.55:0/117744341 >> v2:172.21.15.162:3300/0 conn(0x7f03ac152280 0x7f03ac15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382750712 0 0) 0x7f03ac15c040 con 0x7f03ac152690 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.375+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0390003520 con 0x7f03ac152690 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2841218612 0 0) 0x7f0390003520 con 0x7f03ac152690 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 msgr2=0x7f03ac071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03ac071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 >> v2:172.21.15.162:3300/0 conn(0x7f03ac152280 msgr2=0x7f03ac15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 --2- 172.21.15.55:0/117744341 >> v2:172.21.15.162:3300/0 conn(0x7f03ac152280 0x7f03ac15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac15c8e0 con 0x7f03ac152690 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f03a40019f0 con 0x7f03ac152690 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 >> v1:172.21.15.55:6789/0 conn(0x7f03ac152690 legacy=0x7f03ac0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f0390003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0390004d00 0x7f0390005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac15c320 con 0x7f0390003970 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac15c600 con 0x7f0390004d00 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039000b980 con 0x7f03ac15ba60 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.376+0000 7f03aaa00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f0390003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aa000640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03ab400640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0390004d00 0x7f0390005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aaa00640 1 -- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 msgr2=0x7f03900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aaa00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac15ba60 0x7f03900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aaa00640 1 -- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0390004d00 msgr2=0x7f0390005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aaa00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0390004d00 0x7f0390005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.377+0000 7f03aaa00640 1 -- 172.21.15.55:0/117744341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390003520 con 0x7f0390003970 2024-05-10T03:35:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.378+0000 7f03aaa00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f0390003de0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f039c009a90 tx=0x7f03900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d40a89e6afaed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c031400 con 0x7f0390003970 2024-05-10T03:35:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c02d050 con 0x7f0390003970 2024-05-10T03:35:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03a9600640 1 -- 172.21.15.55:0/117744341 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0317a0 con 0x7f0390003970 2024-05-10T03:35:43.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03abe00640 1 -- 172.21.15.55:0/117744341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 msgr2=0x7f0390003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03abe00640 1 --2- 172.21.15.55:0/117744341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f0390003de0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f039c009a90 tx=0x7f03900057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.379+0000 7f03ab400640 1 -- 172.21.15.55:0/117744341 reap_dead start 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.380+0000 7f03abe00640 1 -- 172.21.15.55:0/117744341 shutdown_connections 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.380+0000 7f03abe00640 1 -- 172.21.15.55:0/117744341 >> 172.21.15.55:0/117744341 conn(0x7f03ac0b07b0 msgr2=0x7f03ac0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.380+0000 7f03abe00640 1 -- 172.21.15.55:0/117744341 shutdown_connections 2024-05-10T03:35:43.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.380+0000 7f03abe00640 1 -- 172.21.15.55:0/117744341 wait complete. 2024-05-10T03:35:43.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.382+0000 7f03abe00640 1 Processor -- start 2024-05-10T03:35:43.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.382+0000 7f03abe00640 1 -- start start 2024-05-10T03:35:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac0ac8e0 0x7f03ac0accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03ac0b2710 0x7f03ac0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac140d60 con 0x7f0390003970 2024-05-10T03:35:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0b3270 con 0x7f03ac0b2710 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03abe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac152e40 con 0x7f03ac0ac8e0 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03aaa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03aaa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34822/0 (socket says 172.21.15.55:34822) 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.384+0000 7f03aaa00640 1 -- 172.21.15.55:0/750572010 learned_addr learned my addr 172.21.15.55:0/750572010 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03ab400640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03ac0b2710 0x7f03ac0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aa000640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac0ac8e0 0x7f03ac0accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 -- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac0ac8e0 msgr2=0x7f03ac0accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac0ac8e0 0x7f03ac0accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 -- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03ac0b2710 msgr2=0x7f03ac0b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03ac0b2710 0x7f03ac0b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 -- 172.21.15.55:0/750572010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0ad230 con 0x7f0390003970 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.385+0000 7f03aaa00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f039c009bb0 tx=0x7f039c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.386+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c03b610 con 0x7f0390003970 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.386+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c03b790 con 0x7f0390003970 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.386+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0b3060 con 0x7f0390003970 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.386+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c03bb40 con 0x7f0390003970 2024-05-10T03:35:43.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.386+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0b34c0 con 0x7f0390003970 2024-05-10T03:35:43.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.387+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac067240 con 0x7f0390003970 2024-05-10T03:35:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.388+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f039c038050 con 0x7f0390003970 2024-05-10T03:35:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.389+0000 7f03a2a00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 0x7f038c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.389+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f039c0a9de0 con 0x7f0390003970 2024-05-10T03:35:43.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.390+0000 7f03aa000640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 0x7f038c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.391+0000 7f03aa000640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 0x7f038c06d110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0b23f0 tx=0x7f0398008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.394+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f039c074e80 con 0x7f0390003970 2024-05-10T03:35:43.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.596+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03ac155db0 con 0x7f0390003970 2024-05-10T03:35:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.598+0000 7f03a2a00640 1 -- 172.21.15.55:0/750572010 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16733 (secure 0 0 0) 0x7f039c036030 con 0x7f0390003970 2024-05-10T03:35:43.602 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:43.602 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:25.331003+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:43.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.601+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 msgr2=0x7f038c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.601+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 0x7f038c06d110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0b23f0 tx=0x7f0398008040 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.601+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 msgr2=0x7f03ac140310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.601+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f039c009bb0 tx=0x7f039c004730 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 shutdown_connections 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f038c06ac00 0x7f038c06d110 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03ac0b2710 0x7f03ac0b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03ac0ac8e0 0x7f03ac0accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 --2- 172.21.15.55:0/750572010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0390003970 0x7f03ac140310 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 >> 172.21.15.55:0/750572010 conn(0x7f03ac0b07b0 msgr2=0x7f03ac06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 shutdown_connections 2024-05-10T03:35:43.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.602+0000 7f03abe00640 1 -- 172.21.15.55:0/750572010 wait complete. 2024-05-10T03:35:43.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:53.261121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:54.084182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:35:43.634 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:35:43.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.918+0000 7fc024600640 1 Processor -- start 2024-05-10T03:35:43.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.918+0000 7fc024600640 1 -- start start 2024-05-10T03:35:43.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc024600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0cf210 0x7fc01c0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc024600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc01c0b6820 0x7fc01c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc024600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c0ced80 con 0x7fc01c0d03f0 2024-05-10T03:35:43.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc024600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c0b4aa0 con 0x7fc01c0b6820 2024-05-10T03:35:43.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc024600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b47c0 con 0x7fc01c0cf210 2024-05-10T03:35:43.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc022600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc01c0b6820 0x7fc01c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc022600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc01c0b6820 0x7fc01c0d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34838/0 (socket says 172.21.15.55:34838) 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.921+0000 7fc022600640 1 -- 172.21.15.55:0/606068255 learned_addr learned my addr 172.21.15.55:0/606068255 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.922+0000 7fc021c00640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0cf210 0x7fc01c0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.922+0000 7fc022600640 1 -- 172.21.15.55:0/606068255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0cf210 msgr2=0x7fc01c0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.922+0000 7fc022600640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0cf210 0x7fc01c0cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.922+0000 7fc022600640 1 -- 172.21.15.55:0/606068255 >> v1:172.21.15.162:6789/0 conn(0x7fc01c0d03f0 legacy=0x7fc01c0b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.922+0000 7fc022600640 1 -- 172.21.15.55:0/606068255 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0cea10 con 0x7fc01c0b6820 2024-05-10T03:35:43.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.923+0000 7fc022600640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c0d1130 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e50 tx=0x7fc01402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530cb4ae0222a015 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.923+0000 7fc01be00640 1 -- 172.21.15.55:0/606068255 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014031600 con 0x7fc01c0b6820 2024-05-10T03:35:43.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.923+0000 7fc01be00640 1 -- 172.21.15.55:0/606068255 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01402d050 con 0x7fc01c0b6820 2024-05-10T03:35:43.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.923+0000 7fc01be00640 1 -- 172.21.15.55:0/606068255 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014031980 con 0x7fc01c0b6820 2024-05-10T03:35:43.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 -- 172.21.15.55:0/606068255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 msgr2=0x7fc01c0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c0d1130 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc014009e50 tx=0x7fc01402a5b0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 -- 172.21.15.55:0/606068255 shutdown_connections 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c0d1130 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 --2- 172.21.15.55:0/606068255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0cf210 0x7fc01c0cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.924+0000 7fc024600640 1 -- 172.21.15.55:0/606068255 >> 172.21.15.55:0/606068255 conn(0x7fc01c09c600 msgr2=0x7fc01c0d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.925+0000 7fc024600640 1 -- 172.21.15.55:0/606068255 shutdown_connections 2024-05-10T03:35:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.925+0000 7fc024600640 1 -- 172.21.15.55:0/606068255 wait complete. 2024-05-10T03:35:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.927+0000 7fc024600640 1 Processor -- start 2024-05-10T03:35:43.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.927+0000 7fc024600640 1 -- start start 2024-05-10T03:35:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc01c0cf210 0x7fc01c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc021c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc021c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34842/0 (socket says 172.21.15.55:34842) 2024-05-10T03:35:43.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0d03f0 0x7fc01c1f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0cb790 con 0x7fc01c0b6820 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c153170 con 0x7fc01c0cf210 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc021c00640 1 -- 172.21.15.55:0/1791065016 learned_addr learned my addr 172.21.15.55:0/1791065016 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c153fd0 con 0x7fc01c0d03f0 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.929+0000 7fc021200640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc01c0cf210 0x7fc01c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 -- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0d03f0 msgr2=0x7fc01c1f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0d03f0 0x7fc01c1f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 -- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc01c0cf210 msgr2=0x7fc01c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc01c0cf210 0x7fc01c0cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014009b30 con 0x7fc01c0b6820 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.930+0000 7fc021c00640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d5d0 tx=0x7fc01000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.931+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010004240 con 0x7fc01c0b6820 2024-05-10T03:35:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.931+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0100043a0 con 0x7fc01c0b6820 2024-05-10T03:35:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.931+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010003830 con 0x7fc01c0b6820 2024-05-10T03:35:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.931+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c152ec0 con 0x7fc01c0b6820 2024-05-10T03:35:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.931+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c1536a0 con 0x7fc01c0b6820 2024-05-10T03:35:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.932+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c067950 con 0x7fc01c0b6820 2024-05-10T03:35:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.933+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc010003990 con 0x7fc01c0b6820 2024-05-10T03:35:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.934+0000 7fc019600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 0x7fbffc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.934+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fc0100897f0 con 0x7fc01c0b6820 2024-05-10T03:35:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.935+0000 7fc021200640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 0x7fbffc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:43.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.936+0000 7fc021200640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 0x7fbffc06d140 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc008006fd0 tx=0x7fc008008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:43.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:43.939+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc0100546b0 con 0x7fc01c0b6820 2024-05-10T03:35:44.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.134+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc01c06c7a0 con 0x7fc01c0b6820 2024-05-10T03:35:44.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.135+0000 7fc019600640 1 -- 172.21.15.55:0/1791065016 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc010057cd0 con 0x7fc01c0b6820 2024-05-10T03:35:44.138 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 msgr2=0x7fbffc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 0x7fbffc06d140 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc008006fd0 tx=0x7fc008008040 comp rx=0 tx=0).stop 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 msgr2=0x7fc01c1f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d5d0 tx=0x7fc01000da90 comp rx=0 tx=0).stop 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 shutdown_connections 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc01c0d03f0 0x7fc01c1f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbffc06ac30 0x7fbffc06d140 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc01c0cf210 0x7fc01c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 --2- 172.21.15.55:0/1791065016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc01c0b6820 0x7fc01c1f34f0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 >> 172.21.15.55:0/1791065016 conn(0x7fc01c09c600 msgr2=0x7fc01c0b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:44.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 shutdown_connections 2024-05-10T03:35:44.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.138+0000 7fc024600640 1 -- 172.21.15.55:0/1791065016 wait complete. 2024-05-10T03:35:44.166 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:35:44.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.441+0000 7fd9f0200640 1 Processor -- start 2024-05-10T03:35:44.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.442+0000 7fd9f0200640 1 -- start start 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e8070d80 0x7fd9e81514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e815b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9e80715b0 0x7fd9e80719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e8151a20 con 0x7fd9e8070d80 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e815ba60 con 0x7fd9e80715b0 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9f0200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8071f00 con 0x7fd9e80ce4b0 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.444+0000 7fd9ed800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e8070d80 0x7fd9e81514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e8070d80 0x7fd9e81514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34854/0 (socket says 172.21.15.55:34854) 2024-05-10T03:35:44.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 -- 172.21.15.55:0/926214258 learned_addr learned my addr 172.21.15.55:0/926214258 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ee200640 1 --2- 172.21.15.55:0/926214258 >> v2:172.21.15.162:3300/0 conn(0x7fd9e80715b0 0x7fd9e80719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ece00640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 -- 172.21.15.55:0/926214258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 msgr2=0x7fd9e815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 -- 172.21.15.55:0/926214258 >> v2:172.21.15.162:3300/0 conn(0x7fd9e80715b0 msgr2=0x7fd9e80719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 --2- 172.21.15.55:0/926214258 >> v2:172.21.15.162:3300/0 conn(0x7fd9e80715b0 0x7fd9e80719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.445+0000 7fd9ed800640 1 -- 172.21.15.55:0/926214258 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e80cec70 con 0x7fd9e8070d80 2024-05-10T03:35:44.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.446+0000 7fd9ed800640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e8070d80 0x7fd9e81514e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd9e806feb0 tx=0x7fd9dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bfff57f52d4121 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:44.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.447+0000 7fd9e7e00640 1 -- 172.21.15.55:0/926214258 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc032440 con 0x7fd9e8070d80 2024-05-10T03:35:44.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.447+0000 7fd9e7e00640 1 -- 172.21.15.55:0/926214258 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0325a0 con 0x7fd9e8070d80 2024-05-10T03:35:44.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.447+0000 7fd9e7e00640 1 -- 172.21.15.55:0/926214258 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc032930 con 0x7fd9e8070d80 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 -- 172.21.15.55:0/926214258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e8070d80 msgr2=0x7fd9e81514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e8070d80 0x7fd9e81514e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd9e806feb0 tx=0x7fd9dc02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 -- 172.21.15.55:0/926214258 shutdown_connections 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 --2- 172.21.15.55:0/926214258 >> v2:172.21.15.162:3300/0 conn(0x7fd9e80715b0 0x7fd9e80719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 --2- 172.21.15.55:0/926214258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e8070d80 0x7fd9e81514e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 -- 172.21.15.55:0/926214258 >> 172.21.15.55:0/926214258 conn(0x7fd9e80ceeb0 msgr2=0x7fd9e80cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 -- 172.21.15.55:0/926214258 shutdown_connections 2024-05-10T03:35:44.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.448+0000 7fd9f0200640 1 -- 172.21.15.55:0/926214258 wait complete. 2024-05-10T03:35:44.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.450+0000 7fd9f0200640 1 Processor -- start 2024-05-10T03:35:44.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.450+0000 7fd9f0200640 1 -- start start 2024-05-10T03:35:44.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e8070d80 0x7fd9e81499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e814a8d0 con 0x7fd9e80715b0 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e814e3b0 con 0x7fd9e8070d80 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9f0200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8149ef0 con 0x7fd9e80ce4b0 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9ece00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9ece00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34868/0 (socket says 172.21.15.55:34868) 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.452+0000 7fd9ece00640 1 -- 172.21.15.55:0/1177865538 learned_addr learned my addr 172.21.15.55:0/1177865538 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ed800640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e8070d80 0x7fd9e81499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ee200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 -- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 msgr2=0x7fd9e814ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e814ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 -- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e8070d80 msgr2=0x7fd9e81499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e8070d80 0x7fd9e81499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 -- 172.21.15.55:0/1177865538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0094d0 con 0x7fd9e80715b0 2024-05-10T03:35:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ee200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e814ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9ece00640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d5f0 tx=0x7fd9d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.453+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40125f0 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.454+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4012750 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.454+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4012b00 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.454+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e814d7a0 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.454+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e814dd60 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.455+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e81525a0 con 0x7fd9e80715b0 2024-05-10T03:35:44.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.456+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd9d4012c60 con 0x7fd9e80715b0 2024-05-10T03:35:44.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.457+0000 7fd9e5600640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 0x7fd9c806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.457+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fd9d4089b10 con 0x7fd9e80715b0 2024-05-10T03:35:44.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.458+0000 7fd9ed800640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 0x7fd9c806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.459+0000 7fd9ed800640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 0x7fd9c806d110 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc034000 tx=0x7fd9dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.462+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd9d40549d0 con 0x7fd9e80715b0 2024-05-10T03:35:44.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.659+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9e8067320 con 0x7fd9e80715b0 2024-05-10T03:35:44.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.660+0000 7fd9e5600640 1 -- 172.21.15.55:0/1177865538 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9d4057ff0 con 0x7fd9e80715b0 2024-05-10T03:35:44.663 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:35:44.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 msgr2=0x7fd9c806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 0x7fd9c806d110 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc034000 tx=0x7fd9dc02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:44.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 msgr2=0x7fd9e814e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d5f0 tx=0x7fd9d400dab0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 shutdown_connections 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e80ce4b0 0x7fd9e814ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e80715b0 0x7fd9e814e840 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd9c806ac00 0x7fd9c806d110 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 --2- 172.21.15.55:0/1177865538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e8070d80 0x7fd9e81499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 >> 172.21.15.55:0/1177865538 conn(0x7fd9e80ceeb0 msgr2=0x7fd9e80aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 shutdown_connections 2024-05-10T03:35:44.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.663+0000 7fd9f0200640 1 -- 172.21.15.55:0/1177865538 wait complete. 2024-05-10T03:35:44.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:35:44.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.978+0000 7fd8c9000640 1 Processor -- start 2024-05-10T03:35:44.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.978+0000 7fd8c9000640 1 -- start start 2024-05-10T03:35:44.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 0x7fd8c4161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c9000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40ccfa0 con 0x7fd8c4070540 2024-05-10T03:35:44.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c9000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40cd280 con 0x7fd8c40cc790 2024-05-10T03:35:44.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cd560 con 0x7fd8c4162a90 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c3e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd8c4070540 0x7fd8c40c9410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39298/0 (socket says 172.21.15.55:39298) 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.981+0000 7fd8c3e00640 1 -- 172.21.15.55:0/3675794286 learned_addr learned my addr 172.21.15.55:0/3675794286 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 0x7fd8c4161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816417895 0 0) 0x7fd8c40ccfa0 con 0x7fd8c4070540 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a8003520 con 0x7fd8c4070540 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051729240 0 0) 0x7fd8a8003520 con 0x7fd8c4070540 2024-05-10T03:35:44.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 msgr2=0x7fd8c4161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 0x7fd8c4161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 >> v1:172.21.15.162:6789/0 conn(0x7fd8c40cc790 legacy=0x7fd8c406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40cd840 con 0x7fd8c4070540 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fd8bc001a10 con 0x7fd8c4070540 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 >> v1:172.21.15.55:6789/0 conn(0x7fd8c4070540 legacy=0x7fd8c40c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a80039a0 0x7fd8a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.982+0000 7fd8c2a00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8a80047c0 0x7fd8a8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c2a00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 0x7fd8a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cd560 con 0x7fd8a80047c0 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a800bab0 con 0x7fd8a80039a0 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a800bc70 con 0x7fd8c4162a90 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8a80047c0 0x7fd8a8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a80039a0 0x7fd8a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 -- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 msgr2=0x7fd8a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c4162a90 0x7fd8a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 -- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a80039a0 msgr2=0x7fd8a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a80039a0 0x7fd8a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8bbe00640 1 -- 172.21.15.55:0/3675794286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8003520 con 0x7fd8a80047c0 2024-05-10T03:35:44.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.983+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8a80039a0 0x7fd8a8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:44.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.984+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8a80047c0 0x7fd8a8004bd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c1dc0 tx=0x7fd8a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee2784ef108edd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:44.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0052d0 con 0x7fd8a80047c0 2024-05-10T03:35:44.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac005430 con 0x7fd8a80047c0 2024-05-10T03:35:44.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c2a00640 1 -- 172.21.15.55:0/3675794286 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac02ab90 con 0x7fd8a80047c0 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c9000640 1 -- 172.21.15.55:0/3675794286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8a80047c0 msgr2=0x7fd8a8004bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3675794286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8a80047c0 0x7fd8a8004bd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c1dc0 tx=0x7fd8a80057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.985+0000 7fd8c3e00640 1 -- 172.21.15.55:0/3675794286 reap_dead start 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.986+0000 7fd8c9000640 1 -- 172.21.15.55:0/3675794286 shutdown_connections 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.986+0000 7fd8c9000640 1 -- 172.21.15.55:0/3675794286 >> 172.21.15.55:0/3675794286 conn(0x7fd8c40cebc0 msgr2=0x7fd8c40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.986+0000 7fd8c9000640 1 -- 172.21.15.55:0/3675794286 shutdown_connections 2024-05-10T03:35:44.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.986+0000 7fd8c9000640 1 -- 172.21.15.55:0/3675794286 wait complete. 2024-05-10T03:35:44.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.988+0000 7fd8c9000640 1 Processor -- start 2024-05-10T03:35:44.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.988+0000 7fd8c9000640 1 -- start start 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c40c14f0 0x7fd8c415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c415c950 0x7fd8c415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4160720 con 0x7fd8c4160bb0 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c415d2a0 con 0x7fd8c40c14f0 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c415fb10 con 0x7fd8c415c950 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34894/0 (socket says 172.21.15.55:34894) 2024-05-10T03:35:44.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c3400640 1 -- 172.21.15.55:0/3975107402 learned_addr learned my addr 172.21.15.55:0/3975107402 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:44.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c3400640 1 -- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c415c950 msgr2=0x7fd8c415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.990+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c415c950 0x7fd8c415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c3400640 1 -- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c40c14f0 msgr2=0x7fd8c415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:44.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c40c14f0 0x7fd8c415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c3400640 1 -- 172.21.15.55:0/3975107402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c415fdf0 con 0x7fd8c4160bb0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c3400640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40108e0 tx=0x7fd8b4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400ec20 con 0x7fd8c4160bb0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400ed80 con 0x7fd8c4160bb0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b401c600 con 0x7fd8c4160bb0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c41600d0 con 0x7fd8c4160bb0 2024-05-10T03:35:44.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.991+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c40bf580 con 0x7fd8c4160bb0 2024-05-10T03:35:44.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.992+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c4067240 con 0x7fd8c4160bb0 2024-05-10T03:35:44.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.994+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd8b401c760 con 0x7fd8c4160bb0 2024-05-10T03:35:44.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.995+0000 7fd8bb400640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 0x7fd8a006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:44.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.995+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fd8b4089c30 con 0x7fd8c4160bb0 2024-05-10T03:35:44.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.996+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 0x7fd8a006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:44.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:44.997+0000 7fd8bbe00640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 0x7fd8a006d110 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac032000 tx=0x7fd8ac02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:45.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.000+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd8b4054cd0 con 0x7fd8c4160bb0 2024-05-10T03:35:45.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.206+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8c4161c60 con 0x7fd8c4160bb0 2024-05-10T03:35:45.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.207+0000 7fd8bb400640 1 -- 172.21.15.55:0/3975107402 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8b40582f0 con 0x7fd8c4160bb0 2024-05-10T03:35:45.210 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.210+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 msgr2=0x7fd8a006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.210+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 0x7fd8a006d110 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac032000 tx=0x7fd8ac02c000 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 msgr2=0x7fd8c40bf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40108e0 tx=0x7fd8b4010da0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 shutdown_connections 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8c415c950 0x7fd8c415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd8a006ac00 0x7fd8a006d110 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8c40c14f0 0x7fd8c415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 --2- 172.21.15.55:0/3975107402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8c4160bb0 0x7fd8c40bf040 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 >> 172.21.15.55:0/3975107402 conn(0x7fd8c40cebc0 msgr2=0x7fd8c406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 shutdown_connections 2024-05-10T03:35:45.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.211+0000 7fd8c9000640 1 -- 172.21.15.55:0/3975107402 wait complete. 2024-05-10T03:35:45.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T03:35:45.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.519+0000 7fba80000640 1 Processor -- start 2024-05-10T03:35:45.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.519+0000 7fba80000640 1 -- start start 2024-05-10T03:35:45.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba80000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba780b3200 0x7fba780714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:45.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba80000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7815df20 con 0x7fba7815ab30 2024-05-10T03:35:45.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba80000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7815a6a0 con 0x7fba7815af40 2024-05-10T03:35:45.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba80000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7815e720 con 0x7fba780b3200 2024-05-10T03:35:45.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba7d600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba780b3200 0x7fba780714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:45.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba7d600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba780b3200 0x7fba780714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34898/0 (socket says 172.21.15.55:34898) 2024-05-10T03:35:45.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.522+0000 7fba7d600640 1 -- 172.21.15.55:0/3233729454 learned_addr learned my addr 172.21.15.55:0/3233729454 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:45.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.523+0000 7fba7d600640 1 -- 172.21.15.55:0/3233729454 >> v1:172.21.15.162:6789/0 conn(0x7fba7815af40 legacy=0x7fba7806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:45.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.523+0000 7fba7d600640 1 -- 172.21.15.55:0/3233729454 >> v1:172.21.15.55:6789/0 conn(0x7fba7815ab30 legacy=0x7fba7815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.523+0000 7fba7d600640 1 -- 172.21.15.55:0/3233729454 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7815e160 con 0x7fba780b3200 2024-05-10T03:35:45.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.524+0000 7fba7d600640 1 --2- 172.21.15.55:0/3233729454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba780b3200 0x7fba780714c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00af90 tx=0x7fba6c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e502c1449e9834 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:45.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.524+0000 7fba77e00640 1 -- 172.21.15.55:0/3233729454 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c008500 con 0x7fba780b3200 2024-05-10T03:35:45.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.524+0000 7fba77e00640 1 -- 172.21.15.55:0/3233729454 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c008660 con 0x7fba780b3200 2024-05-10T03:35:45.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.524+0000 7fba77e00640 1 -- 172.21.15.55:0/3233729454 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0089f0 con 0x7fba780b3200 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 -- 172.21.15.55:0/3233729454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba780b3200 msgr2=0x7fba780714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 --2- 172.21.15.55:0/3233729454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba780b3200 0x7fba780714c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00af90 tx=0x7fba6c02b8a0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 -- 172.21.15.55:0/3233729454 shutdown_connections 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 --2- 172.21.15.55:0/3233729454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba780b3200 0x7fba780714c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 -- 172.21.15.55:0/3233729454 >> 172.21.15.55:0/3233729454 conn(0x7fba780b0b80 msgr2=0x7fba780b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 -- 172.21.15.55:0/3233729454 shutdown_connections 2024-05-10T03:35:45.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.525+0000 7fba80000640 1 -- 172.21.15.55:0/3233729454 wait complete. 2024-05-10T03:35:45.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.528+0000 7fba80000640 1 Processor -- start 2024-05-10T03:35:45.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.528+0000 7fba80000640 1 -- start start 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba780b3200 0x7fba781fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba7815af40 0x7fba7815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7814af90 con 0x7fba7815ab30 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba781fbd70 con 0x7fba780b3200 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.529+0000 7fba80000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba781fc580 con 0x7fba7815af40 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34910/0 (socket says 172.21.15.55:34910) 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 -- 172.21.15.55:0/3585396470 learned_addr learned my addr 172.21.15.55:0/3585396470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7d600640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba780b3200 0x7fba781fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 -- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba7815af40 msgr2=0x7fba7815c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba7815af40 0x7fba7815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 -- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba780b3200 msgr2=0x7fba781fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba780b3200 0x7fba781fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 -- 172.21.15.55:0/3585396470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c00ac40 con 0x7fba7815ab30 2024-05-10T03:35:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.530+0000 7fba7cc00640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c3f0 tx=0x7fba6400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.531+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640052d0 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.531+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64005430 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.531+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64003830 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.531+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba781fbb40 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.531+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba781fc2a0 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.532+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba78067320 con 0x7fba7815ab30 2024-05-10T03:35:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.533+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fba64003990 con 0x7fba7815ab30 2024-05-10T03:35:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.534+0000 7fba75600640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 0x7fba5806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.534+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fba64088910 con 0x7fba7815ab30 2024-05-10T03:35:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.535+0000 7fba7d600640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 0x7fba5806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.536+0000 7fba7d600640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 0x7fba5806d110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fba6c035000 tx=0x7fba6c00a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:45.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.539+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fba64053860 con 0x7fba7815ab30 2024-05-10T03:35:45.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.751+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fba7815df20 con 0x7fba7815ab30 2024-05-10T03:35:45.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.752+0000 7fba75600640 1 -- 172.21.15.55:0/3585396470 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v110) v1 ==== 80+0+364 (secure 0 0 0) 0x7fba64017180 con 0x7fba7815ab30 2024-05-10T03:35:45.755 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.162:6837/476611421 2024-05-11T03:35:13.193337+0000 2024-05-10T03:35:45.755 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.162:6836/476611421 2024-05-11T03:35:13.193337+0000 2024-05-10T03:35:45.755 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6833/2254900234 2024-05-11T03:35:13.167318+0000 2024-05-10T03:35:45.755 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6832/2254900234 2024-05-11T03:35:13.167318+0000 2024-05-10T03:35:45.756 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6835/315649334 2024-05-11T03:35:13.117413+0000 2024-05-10T03:35:45.756 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6834/315649334 2024-05-11T03:35:13.117413+0000 2024-05-10T03:35:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:listed 6 entries 2024-05-10T03:35:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.754+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 msgr2=0x7fba5806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.754+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 0x7fba5806d110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fba6c035000 tx=0x7fba6c00a000 comp rx=0 tx=0).stop 2024-05-10T03:35:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.754+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 msgr2=0x7fba78148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.754+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c3f0 tx=0x7fba6400c8b0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 shutdown_connections 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba7815af40 0x7fba7815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba7815ab30 0x7fba78148480 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fba5806ac00 0x7fba5806d110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 --2- 172.21.15.55:0/3585396470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba780b3200 0x7fba781fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 >> 172.21.15.55:0/3585396470 conn(0x7fba780b0b80 msgr2=0x7fba780707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 shutdown_connections 2024-05-10T03:35:45.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:45.755+0000 7fba80000640 1 -- 172.21.15.55:0/3585396470 wait complete. 2024-05-10T03:35:45.784 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-10T03:35:45.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:45.785 DEBUG:teuthology.orchestra.run.smithi162:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-10T03:35:45.890 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:35:45.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:45.890 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-10T03:35:45.941 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-05-10T03:35:46.256 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-05-10T03:35:46.257 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-05-10T03:35:46.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-10T03:35:46.582 INFO:teuthology.orchestra.run.smithi055.stdout:{"message":"","return_code":0} 2024-05-10T03:35:46.582 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-10T03:35:46.582 DEBUG:teuthology.orchestra.run.smithi055:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-05-10T03:35:46.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.759+0000 7f76a031c280 1 -- start start 2024-05-10T03:35:46.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.759+0000 7f76a031c280 1 Processor -- start 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc6880 0x5567c0dccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c0dc7180 0x5567c0dcd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12a00 con 0x5567c0dc6d00 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12780 con 0x5567c0dc7180 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12640 con 0x5567c0dc6880 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f769d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f769d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34934/0 (socket says 172.21.15.55:34934) 2024-05-10T03:35:46.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.761+0000 7f769d000640 1 -- 172.21.15.55:0/1018508038 learned_addr learned my addr 172.21.15.55:0/1018508038 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769da00640 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc6880 0x5567c0dccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 -- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc6880 msgr2=0x5567c0dccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769e400640 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c0dc7180 0x5567c0dcd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc6880 0x5567c0dccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 -- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c0dc7180 msgr2=0x5567c0dcd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:46.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c0dc7180 0x5567c0dcd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 -- 172.21.15.55:0/1018508038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567bcc369c0 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769d000640 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x5567c1a6c360 tx=0x5567c0cec2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5659ad6059a5de91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769a800640 1 -- 172.21.15.55:0/1018508038 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c89080 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769a800640 1 -- 172.21.15.55:0/1018508038 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567c1a22300 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f769a800640 1 -- 172.21.15.55:0/1018508038 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c89600 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567bcc36d00 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5567bcc369c0 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.762+0000 7f76a031c280 4 waiting for MDS map... 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.763+0000 7f769a800640 1 -- 172.21.15.55:0/1018508038 <== mon.0 v2:172.21.15.55:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x5567bcc3c380 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.763+0000 7f769a800640 1 -- 172.21.15.55:0/1018508038 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 69) v1 ==== 3130+0+0 (secure 0 0 0) 0x5567c0c92f00 con 0x5567c0dc6d00 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.763+0000 7f76a031c280 4 Got MDS map 69 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.763+0000 7f76a031c280 10 main: main 2024-05-10T03:35:46.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.763+0000 7f76a031c280 4 main: connecting to RADOS... 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.764+0000 7f76a031c280 1 Processor -- start 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.764+0000 7f76a031c280 1 -- start start 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc7a80 0x5567c0dcd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 0x5567c0dcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0c8be00 con 0x5567c1b76480 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12780 con 0x5567c1b76000 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12a00 con 0x5567c0dc7a80 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34944/0 (socket says 172.21.15.55:34944) 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 -- 172.21.15.55:0/1335335817 learned_addr learned my addr 172.21.15.55:0/1335335817 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769e400640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc7a80 0x5567c0dcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 -- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc7a80 msgr2=0x5567c0dcd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc7a80 0x5567c0dcd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.766+0000 7f769d000640 1 -- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 msgr2=0x5567c0dcdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f769da00640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 0x5567c0dcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f769d000640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 0x5567c0dcdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f769d000640 1 -- 172.21.15.55:0/1335335817 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567bcc36b60 con 0x5567c1b76480 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f769da00640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 0x5567c0dcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f769d000640 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x5567c1a6d0e0 tx=0x5567c0cec360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecdc1e3f1b38238 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f7698000640 1 -- 172.21.15.55:0/1335335817 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c891e0 con 0x5567c1b76480 2024-05-10T03:35:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f7698000640 1 -- 172.21.15.55:0/1335335817 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567c1a22d80 con 0x5567c1b76480 2024-05-10T03:35:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.767+0000 7f7698000640 1 -- 172.21.15.55:0/1335335817 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c88f20 con 0x5567c1b76480 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 -- 172.21.15.55:0/1335335817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 msgr2=0x5567c0dce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x5567c1a6d0e0 tx=0x5567c0cec360 comp rx=0 tx=0).stop 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 -- 172.21.15.55:0/1335335817 shutdown_connections 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c1b76480 0x5567c0dce100 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1b76000 0x5567c0dcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 --2- 172.21.15.55:0/1335335817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc7a80 0x5567c0dcd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 -- 172.21.15.55:0/1335335817 >> 172.21.15.55:0/1335335817 conn(0x5567c0dc6400 msgr2=0x5567c0d7f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 -- 172.21.15.55:0/1335335817 shutdown_connections 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.768+0000 7f76a031c280 1 -- 172.21.15.55:0/1335335817 wait complete. 2024-05-10T03:35:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.769+0000 7f76a031c280 1 Processor -- start 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.769+0000 7f76a031c280 1 -- start start 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 0x5567c0dcf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1a38000 0x5567c0dcec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c1a38480 0x5567c0dce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12500 con 0x5567c0dc6400 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d123c0 con 0x5567c1a38000 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f76a031c280 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c0d12280 con 0x5567c1a38480 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 0x5567c0dcf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 0x5567c0dcf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34950/0 (socket says 172.21.15.55:34950) 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 -- 172.21.15.55:0/1046356275 learned_addr learned my addr 172.21.15.55:0/1046356275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 -- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c1a38480 msgr2=0x5567c0dce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c1a38480 0x5567c0dce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 -- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1a38000 msgr2=0x5567c0dcec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c1a38000 0x5567c0dcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.771+0000 7f769e400640 1 -- 172.21.15.55:0/1046356275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567bcc36d00 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f769e400640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 0x5567c0dcf180 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x5567c1b792c0 tx=0x5567c1b28cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f7696200640 1 -- 172.21.15.55:0/1046356275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c891e0 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f7696200640 1 -- 172.21.15.55:0/1046356275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567c1a22600 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5567bcc369c0 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f7696200640 1 -- 172.21.15.55:0/1046356275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567c0c88f20 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567bcc36b60 con 0x5567c0dc6400 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.772+0000 7f76a031c280 4 main: resolving pool 15 2024-05-10T03:35:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.774+0000 7f7696200640 1 -- 172.21.15.55:0/1046356275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x5567c0def080 con 0x5567c0dc6400 2024-05-10T03:35:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.775+0000 7f7696200640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x5567c1a3b600 0x5567c0dcf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.775+0000 7f7696200640 1 -- 172.21.15.55:0/1046356275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x5567bcc3c540 con 0x5567c0dc6400 2024-05-10T03:35:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.775+0000 7f76a031c280 4 main: creating IoCtx.. 2024-05-10T03:35:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.775+0000 7f76a031c280 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x5567c1a38900 0x5567c0dce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.775+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- osd_op(unknown.0.0:1 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5567c1b0bc00 con 0x5567c1a38900 2024-05-10T03:35:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.776+0000 7f769d000640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x5567c1a38900 0x5567c0dce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.776+0000 7f769da00640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x5567c1a3b600 0x5567c0dcf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.776+0000 7f769d000640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x5567c1a38900 0x5567c0dce100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f769d000640 1 -- 172.21.15.55:0/1046356275 <== osd.2 v2:172.21.15.55:6800/35475 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv10 ondisk = 0) v8 ==== 159+0+14 (crc 0 0 0) 0x5567c1b04b40 con 0x5567c1a38900 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- osd_op(unknown.0.0:2 15.b 15:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5567c101ac00 con 0x5567c1a38900 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f769da00640 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x5567c1a3b600 0x5567c0dcf700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x5567c0d5c030 tx=0x5567c0fa0060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f769d000640 1 -- 172.21.15.55:0/1046356275 <== osd.2 v2:172.21.15.55:6800/35475 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv10 ondisk = 0) v8 ==== 159+0+5 (crc 0 0 0) 0x5567c1b04b40 con 0x5567c1a38900 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stdout: "0": { 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stdout: "data": { 2024-05-10T03:35:46.780 INFO:teuthology.orchestra.run.smithi055.stdout: "sessions": [] 2024-05-10T03:35:46.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-05-10T03:35:46.781 INFO:teuthology.orchestra.run.smithi055.stdout: "result": 0 2024-05-10T03:35:46.781 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-05-10T03:35:46.781 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:35:46.781 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:46.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x5567c1a38900 msgr2=0x5567c0dce100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x5567c1a38900 0x5567c0dce100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x5567c1a3b600 msgr2=0x5567c0dcf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x5567c1a3b600 0x5567c0dcf700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x5567c0d5c030 tx=0x5567c0fa0060 comp rx=0 tx=0).stop 2024-05-10T03:35:46.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 msgr2=0x5567c0dcf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 --2- 172.21.15.55:0/1046356275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6400 0x5567c0dcf180 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x5567c1b792c0 tx=0x5567c1b28cc0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f769da00640 1 -- 172.21.15.55:0/1046356275 reap_dead start 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 shutdown_connections 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.777+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 >> 172.21.15.55:0/1046356275 conn(0x5567c0dc7a80 msgr2=0x5567c0d7f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 shutdown_connections 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1046356275 wait complete. 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 msgr2=0x5567c0dcc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x5567c1a6c360 tx=0x5567c0cec2a0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 shutdown_connections 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5567c0dc7180 0x5567c0dcd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5567c0dc6d00 0x5567c0dcc580 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 --2- 172.21.15.55:0/1018508038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5567c0dc6880 0x5567c0dccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:46.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 >> 172.21.15.55:0/1018508038 conn(0x5567c0dc7600 msgr2=0x5567c0d7d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:46.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 shutdown_connections 2024-05-10T03:35:46.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:46.778+0000 7f76a031c280 1 -- 172.21.15.55:0/1018508038 wait complete. 2024-05-10T03:35:46.793 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.210630, result: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-05-10T03:35:46.793 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-05-10T03:35:46.793 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-10T03:35:46.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:35:46.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-10T03:35:47.024 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-10T03:35:47.025 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:47.025 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:47.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:35:47.085 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:47.085 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:47.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:35:47.143 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:47.143 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:47.206 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:35:47.206 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:47.206 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:47.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:35:47.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:47.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.550+0000 7fef27e00640 1 Processor -- start 2024-05-10T03:35:47.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.550+0000 7fef27e00640 1 -- start start 2024-05-10T03:35:47.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.552+0000 7fef27e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef280ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.552+0000 7fef27e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280ceda0 con 0x7fef280d01b0 2024-05-10T03:35:47.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.552+0000 7fef27e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280b65d0 con 0x7fef280b8630 2024-05-10T03:35:47.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.552+0000 7fef27e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280cefd0 con 0x7fef280b44f0 2024-05-10T03:35:47.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.553+0000 7fef26000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fef280d01b0 0x7fef280b6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39334/0 (socket says 172.21.15.55:39334) 2024-05-10T03:35:47.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.553+0000 7fef26000640 1 -- 172.21.15.55:0/3484966357 learned_addr learned my addr 172.21.15.55:0/3484966357 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:47.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.553+0000 7fef27400640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef280ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.553+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838380313 0 0) 0x7fef280ceda0 con 0x7fef280d01b0 2024-05-10T03:35:47.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef0c003520 con 0x7fef280d01b0 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454529599 0 0) 0x7fef0c003520 con 0x7fef280d01b0 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 msgr2=0x7fef280ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef280ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 >> v1:172.21.15.162:6789/0 conn(0x7fef280b8630 legacy=0x7fef280cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280b5f80 con 0x7fef280d01b0 2024-05-10T03:35:47.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fef14001670 con 0x7fef280d01b0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 >> v1:172.21.15.55:6789/0 conn(0x7fef280d01b0 legacy=0x7fef280b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef0c0047c0 0x7fef0c004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef0c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280cefd0 con 0x7fef0c0039a0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c00ba30 con 0x7fef0c0047c0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c00bbf0 con 0x7fef280b44f0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.554+0000 7fef26a00640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26000640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef0c0047c0 0x7fef0c004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef27400640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef0c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26a00640 1 -- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 msgr2=0x7fef0c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26a00640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef0c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26a00640 1 -- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef0c0047c0 msgr2=0x7fef0c004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:47.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26a00640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef0c0047c0 0x7fef0c004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26a00640 1 -- 172.21.15.55:0/3484966357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c003520 con 0x7fef0c0039a0 2024-05-10T03:35:47.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef27400640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef280b44f0 0x7fef0c005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:47.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.555+0000 7fef26000640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef0c0047c0 0x7fef0c004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.556+0000 7fef26a00640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef0c003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fef280a0fa0 tx=0x7fef0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25e8ec95269db6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18031600 con 0x7fef0c0039a0 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1802d050 con 0x7fef0c0039a0 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef25600640 1 -- 172.21.15.55:0/3484966357 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180319c0 con 0x7fef0c0039a0 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef27e00640 1 -- 172.21.15.55:0/3484966357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 msgr2=0x7fef0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef27e00640 1 --2- 172.21.15.55:0/3484966357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef0c003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fef280a0fa0 tx=0x7fef0c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.557+0000 7fef27400640 1 -- 172.21.15.55:0/3484966357 reap_dead start 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.558+0000 7fef27e00640 1 -- 172.21.15.55:0/3484966357 shutdown_connections 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.558+0000 7fef27e00640 1 -- 172.21.15.55:0/3484966357 >> 172.21.15.55:0/3484966357 conn(0x7fef280d3090 msgr2=0x7fef280d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.558+0000 7fef27e00640 1 -- 172.21.15.55:0/3484966357 shutdown_connections 2024-05-10T03:35:47.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.558+0000 7fef27e00640 1 -- 172.21.15.55:0/3484966357 wait complete. 2024-05-10T03:35:47.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.560+0000 7fef27e00640 1 Processor -- start 2024-05-10T03:35:47.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.560+0000 7fef27e00640 1 -- start start 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef28151480 0x7fef28154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef28151890 0x7fef28155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28157030 con 0x7fef0c0039a0 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef281f4b30 con 0x7fef28151890 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef27e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef28153e50 con 0x7fef28151480 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef26a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef26a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34966/0 (socket says 172.21.15.55:34966) 2024-05-10T03:35:47.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.562+0000 7fef26a00640 1 -- 172.21.15.55:0/1568949273 learned_addr learned my addr 172.21.15.55:0/1568949273 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26000640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef28151480 0x7fef28154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 -- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef28151480 msgr2=0x7fef28154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef27400640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef28151890 0x7fef28155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef28151480 0x7fef28154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 -- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef28151890 msgr2=0x7fef28155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef28151890 0x7fef28155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 -- 172.21.15.55:0/1568949273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef281540a0 con 0x7fef0c0039a0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef26a00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fef1802a9c0 tx=0x7fef180015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180054a0 con 0x7fef0c0039a0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18005600 con 0x7fef0c0039a0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.563+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18007050 con 0x7fef0c0039a0 2024-05-10T03:35:47.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.564+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef28154380 con 0x7fef0c0039a0 2024-05-10T03:35:47.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.564+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef28154940 con 0x7fef0c0039a0 2024-05-10T03:35:47.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.565+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef281f34a0 con 0x7fef0c0039a0 2024-05-10T03:35:47.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.566+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fef180071b0 con 0x7fef0c0039a0 2024-05-10T03:35:47.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.567+0000 7fef1ea00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 0x7fef0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:47.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.567+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fef180aa850 con 0x7fef0c0039a0 2024-05-10T03:35:47.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.568+0000 7fef26000640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 0x7fef0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:47.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.569+0000 7fef26000640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 0x7fef0806d140 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef140017a0 tx=0x7fef14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:47.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.572+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fef18079110 con 0x7fef0c0039a0 2024-05-10T03:35:47.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.813+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef28157030 con 0x7fef0c0039a0 2024-05-10T03:35:47.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.814+0000 7fef1ea00640 1 -- 172.21.15.55:0/1568949273 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fef18001b30 con 0x7fef0c0039a0 2024-05-10T03:35:47.817 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:47.817 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:34.375414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:47.817 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 69 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 msgr2=0x7fef0806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 0x7fef0806d140 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef140017a0 tx=0x7fef14009040 comp rx=0 tx=0).stop 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 msgr2=0x7fef2815f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fef1802a9c0 tx=0x7fef180015b0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 shutdown_connections 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fef0806ac30 0x7fef0806d140 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef28151890 0x7fef28155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef28151480 0x7fef28154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 --2- 172.21.15.55:0/1568949273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef0c0039a0 0x7fef2815f740 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:47.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 >> 172.21.15.55:0/1568949273 conn(0x7fef280d3090 msgr2=0x7fef2806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:47.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 shutdown_connections 2024-05-10T03:35:47.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:47.817+0000 7fef27e00640 1 -- 172.21.15.55:0/1568949273 wait complete. 2024-05-10T03:35:47.849 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:48.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.130+0000 7f3d63e00640 1 Processor -- start 2024-05-10T03:35:48.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.130+0000 7f3d63e00640 1 -- start start 2024-05-10T03:35:48.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.132+0000 7f3d63e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 0x7f3d640b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d63e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d63e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6415b2c0 con 0x7f3d6406d600 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d63e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640b39f0 con 0x7f3d64154b10 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d63e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415b550 con 0x7f3d6415a970 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d5a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.133+0000 7f3d62a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 0x7f3d640b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47908/0 (socket says 172.21.15.55:47908) 2024-05-10T03:35:48.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 -- 172.21.15.55:0/193374328 learned_addr learned my addr 172.21.15.55:0/193374328 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 -- 172.21.15.55:0/193374328 >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 msgr2=0x7f3d640b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 --2- 172.21.15.55:0/193374328 >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 0x7f3d640b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 -- 172.21.15.55:0/193374328 >> v1:172.21.15.162:6789/0 conn(0x7f3d6406d600 legacy=0x7f3d6406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d5a000640 1 -- 172.21.15.55:0/193374328 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6415b830 con 0x7f3d6415a970 2024-05-10T03:35:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.134+0000 7f3d62a00640 1 --2- 172.21.15.55:0/193374328 >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 0x7f3d640b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:35:48.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.136+0000 7f3d5a000640 1 --2- 172.21.15.55:0/193374328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009870 tx=0x7f3d4c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a8b5604595328c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.137+0000 7f3d62000640 1 -- 172.21.15.55:0/193374328 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c032440 con 0x7f3d6415a970 2024-05-10T03:35:48.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.137+0000 7f3d62000640 1 -- 172.21.15.55:0/193374328 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0325a0 con 0x7f3d6415a970 2024-05-10T03:35:48.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.137+0000 7f3d62000640 1 -- 172.21.15.55:0/193374328 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c032930 con 0x7f3d6415a970 2024-05-10T03:35:48.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.137+0000 7f3d63e00640 1 -- 172.21.15.55:0/193374328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 msgr2=0x7f3d6415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.137+0000 7f3d63e00640 1 --2- 172.21.15.55:0/193374328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009870 tx=0x7f3d4c02a250 comp rx=0 tx=0).stop 2024-05-10T03:35:48.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 -- 172.21.15.55:0/193374328 shutdown_connections 2024-05-10T03:35:48.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 --2- 172.21.15.55:0/193374328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d6415a970 0x7f3d6415ad80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 --2- 172.21.15.55:0/193374328 >> v2:172.21.15.162:3300/0 conn(0x7f3d64154b10 0x7f3d640b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 -- 172.21.15.55:0/193374328 >> 172.21.15.55:0/193374328 conn(0x7f3d640b07b0 msgr2=0x7f3d640b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 -- 172.21.15.55:0/193374328 shutdown_connections 2024-05-10T03:35:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.138+0000 7f3d63e00640 1 -- 172.21.15.55:0/193374328 wait complete. 2024-05-10T03:35:48.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.141+0000 7f3d63e00640 1 Processor -- start 2024-05-10T03:35:48.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.141+0000 7f3d63e00640 1 -- start start 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d64154b10 0x7f3d64149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d6415a970 0x7f3d6415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641f3470 con 0x7f3d6406d600 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641f0540 con 0x7f3d6415a970 2024-05-10T03:35:48.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.143+0000 7f3d63e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6414ee20 con 0x7f3d64154b10 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d5a000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d64154b10 0x7f3d64149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34970/0 (socket says 172.21.15.55:34970) 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d63400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d6415a970 0x7f3d6415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 -- 172.21.15.55:0/1314775693 learned_addr learned my addr 172.21.15.55:0/1314775693 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 -- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d64154b10 msgr2=0x7f3d64149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d64154b10 0x7f3d64149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 -- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d6415a970 msgr2=0x7f3d6415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d6415a970 0x7f3d6415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.144+0000 7f3d62a00640 1 -- 172.21.15.55:0/1314775693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54009400 con 0x7f3d6406d600 2024-05-10T03:35:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d62a00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e910 tx=0x7f3d5400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400ca00 con 0x7f3d6406d600 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5400cb60 con 0x7f3d6406d600 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c0094d0 con 0x7f3d6406d600 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400ced0 con 0x7f3d6406d600 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.145+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6414f630 con 0x7f3d6406d600 2024-05-10T03:35:48.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.147+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640cf9f0 con 0x7f3d6406d600 2024-05-10T03:35:48.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.148+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3d54013470 con 0x7f3d6406d600 2024-05-10T03:35:48.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.149+0000 7f3d5be00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 0x7f3d4006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.149+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f3d54089b20 con 0x7f3d6406d600 2024-05-10T03:35:48.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.150+0000 7f3d5a000640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 0x7f3d4006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.151+0000 7f3d5a000640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 0x7f3d4006d140 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c034000 tx=0x7f3d4c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.154+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3d54054aa0 con 0x7f3d6406d600 2024-05-10T03:35:48.313 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:35:48.321 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:35:48.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.359+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d64067240 con 0x7f3d6406d600 2024-05-10T03:35:48.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.361+0000 7f3d5be00640 1 -- 172.21.15.55:0/1314775693 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16733 (secure 0 0 0) 0x7f3d54021090 con 0x7f3d6406d600 2024-05-10T03:35:48.363 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:48.364 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:25.331003+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 msgr2=0x7f3d4006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 0x7f3d4006d140 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c034000 tx=0x7f3d4c02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 msgr2=0x7f3d641f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e910 tx=0x7f3d5400edd0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 shutdown_connections 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3d4006ac30 0x7f3d4006d140 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d6415a970 0x7f3d6415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3d64154b10 0x7f3d64149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 --2- 172.21.15.55:0/1314775693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6406d600 0x7f3d641f3890 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.363+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 >> 172.21.15.55:0/1314775693 conn(0x7f3d640b07b0 msgr2=0x7f3d640b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:48.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.364+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 shutdown_connections 2024-05-10T03:35:48.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.364+0000 7f3d63e00640 1 -- 172.21.15.55:0/1314775693 wait complete. 2024-05-10T03:35:48.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T03:29:33.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T03:34:53.261121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T03:34:54.084182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T03:35:48.391 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T03:35:48.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.744+0000 7f7687e00640 1 Processor -- start 2024-05-10T03:35:48.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.744+0000 7f7687e00640 1 -- start start 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.746+0000 7f7687e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76881480e0 0x7f768806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7687e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768806d190 0x7f7688070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7687e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76880b3200 con 0x7f768814ac20 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7687e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76880b34e0 con 0x7f76881480e0 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7687e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76880b37c0 con 0x7f768806d190 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76881480e0 0x7f768806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76881480e0 0x7f768806fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34982/0 (socket says 172.21.15.55:34982) 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 -- 172.21.15.55:0/3684860565 learned_addr learned my addr 172.21.15.55:0/3684860565 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 -- 172.21.15.55:0/3684860565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768806d190 msgr2=0x7f7688070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 --2- 172.21.15.55:0/3684860565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768806d190 0x7f7688070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 -- 172.21.15.55:0/3684860565 >> v1:172.21.15.162:6789/0 conn(0x7f768814ac20 legacy=0x7f7688165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.747+0000 7f7686a00640 1 -- 172.21.15.55:0/3684860565 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880b3aa0 con 0x7f76881480e0 2024-05-10T03:35:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.748+0000 7f7686a00640 1 --2- 172.21.15.55:0/3684860565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f768806fcf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17097c630364f6e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.749+0000 7f7686000640 1 -- 172.21.15.55:0/3684860565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767802ae90 con 0x7f76881480e0 2024-05-10T03:35:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.749+0000 7f7686000640 1 -- 172.21.15.55:0/3684860565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678032440 con 0x7f76881480e0 2024-05-10T03:35:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.749+0000 7f7686000640 1 -- 172.21.15.55:0/3684860565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780327d0 con 0x7f76881480e0 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 -- 172.21.15.55:0/3684860565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 msgr2=0x7f768806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 --2- 172.21.15.55:0/3684860565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f768806fcf0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7678009870 tx=0x7f767802a180 comp rx=0 tx=0).stop 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 -- 172.21.15.55:0/3684860565 shutdown_connections 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 --2- 172.21.15.55:0/3684860565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768806d190 0x7f7688070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 --2- 172.21.15.55:0/3684860565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f768806fcf0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 -- 172.21.15.55:0/3684860565 >> 172.21.15.55:0/3684860565 conn(0x7f768806a930 msgr2=0x7f76880b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 -- 172.21.15.55:0/3684860565 shutdown_connections 2024-05-10T03:35:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.750+0000 7f7687e00640 1 -- 172.21.15.55:0/3684860565 wait complete. 2024-05-10T03:35:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.752+0000 7f7687e00640 1 Processor -- start 2024-05-10T03:35:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.753+0000 7f7687e00640 1 -- start start 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f768806d190 0x7f7688145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768814ac20 0x7f76881fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688163cc0 con 0x7f76881480e0 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76880cdaa0 con 0x7f768806d190 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.754+0000 7f7687e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7688145580 con 0x7f768814ac20 2024-05-10T03:35:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34984/0 (socket says 172.21.15.55:34984) 2024-05-10T03:35:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 -- 172.21.15.55:0/271693935 learned_addr learned my addr 172.21.15.55:0/271693935 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f7687400640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768814ac20 0x7f76881fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f7686a00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f768806d190 0x7f7688145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 -- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768814ac20 msgr2=0x7f76881fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768814ac20 0x7f76881fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 -- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f768806d190 msgr2=0x7f7688145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f768806d190 0x7f7688145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 -- 172.21.15.55:0/271693935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780094d0 con 0x7f76881480e0 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.755+0000 7f767e000640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f767000b3e0 tx=0x7f767000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.756+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670011830 con 0x7f76881480e0 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.756+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76700119b0 con 0x7f76881480e0 2024-05-10T03:35:48.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.756+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670011d60 con 0x7f76881480e0 2024-05-10T03:35:48.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.756+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76880cdcf0 con 0x7f76881480e0 2024-05-10T03:35:48.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.756+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688161ea0 con 0x7f76881480e0 2024-05-10T03:35:48.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.757+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7688071460 con 0x7f76881480e0 2024-05-10T03:35:48.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.758+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f767001a400 con 0x7f76881480e0 2024-05-10T03:35:48.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.759+0000 7f767f400640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 0x7f766406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:48.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.759+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f7670089700 con 0x7f76881480e0 2024-05-10T03:35:48.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.760+0000 7f7686a00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 0x7f766406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:48.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.761+0000 7f7686a00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 0x7f766406d160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7678034000 tx=0x7f767802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:48.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.764+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7670054640 con 0x7f76881480e0 2024-05-10T03:35:48.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.962+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7688067240 con 0x7f76881480e0 2024-05-10T03:35:48.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.963+0000 7f767f400640 1 -- 172.21.15.55:0/271693935 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7670057c60 con 0x7f76881480e0 2024-05-10T03:35:48.966 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 msgr2=0x7f766406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 0x7f766406d160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7678034000 tx=0x7f767802e000 comp rx=0 tx=0).stop 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 msgr2=0x7f76881fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f767000b3e0 tx=0x7f767000b8a0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 shutdown_connections 2024-05-10T03:35:48.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f768814ac20 0x7f76881fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76881480e0 0x7f76881fb840 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f766406ac50 0x7f766406d160 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 --2- 172.21.15.55:0/271693935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f768806d190 0x7f7688145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 >> 172.21.15.55:0/271693935 conn(0x7f768806a930 msgr2=0x7f76880afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 shutdown_connections 2024-05-10T03:35:48.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:48.966+0000 7f7687e00640 1 -- 172.21.15.55:0/271693935 wait complete. 2024-05-10T03:35:48.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T03:35:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.281+0000 7ff351600640 1 Processor -- start 2024-05-10T03:35:49.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.282+0000 7ff351600640 1 -- start start 2024-05-10T03:35:49.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff351600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff34c15b540 0x7ff34c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff351600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c0cbde0 0x7ff34c070530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff351600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34c0cc1f0 con 0x7ff34c0adea0 2024-05-10T03:35:49.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff351600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34c0cd750 con 0x7ff34c15b540 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff351600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0cd960 con 0x7ff34c0cbde0 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff34be00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff34c0adea0 0x7ff34c15c430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39342/0 (socket says 172.21.15.55:39342) 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff34be00640 1 -- 172.21.15.55:0/3374556944 learned_addr learned my addr 172.21.15.55:0/3374556944 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.284+0000 7ff343e00640 1 --2- 172.21.15.55:0/3374556944 >> v2:172.21.15.55:3300/0 conn(0x7ff34c15b540 0x7ff34c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff350c00640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c0cbde0 0x7ff34c070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff34b400640 1 -- 172.21.15.55:0/3374556944 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792461875 0 0) 0x7ff34c0cc1f0 con 0x7ff34c0adea0 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff34b400640 1 -- 172.21.15.55:0/3374556944 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff330003520 con 0x7ff34c0adea0 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff343e00640 1 -- 172.21.15.55:0/3374556944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c0cbde0 msgr2=0x7ff34c070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:49.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff343e00640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c0cbde0 0x7ff34c070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff343e00640 1 -- 172.21.15.55:0/3374556944 >> v1:172.21.15.55:6789/0 conn(0x7ff34c0adea0 legacy=0x7ff34c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.285+0000 7ff343e00640 1 -- 172.21.15.55:0/3374556944 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c0cdc40 con 0x7ff34c15b540 2024-05-10T03:35:49.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.286+0000 7ff343e00640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c15b540 0x7ff34c0cb8a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70974ad5f708df98 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.287+0000 7ff34b400640 1 -- 172.21.15.55:0/3374556944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334032440 con 0x7ff34c15b540 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.287+0000 7ff34b400640 1 -- 172.21.15.55:0/3374556944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3340325a0 con 0x7ff34c15b540 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.287+0000 7ff34b400640 1 -- 172.21.15.55:0/3374556944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334032930 con 0x7ff34c15b540 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 -- 172.21.15.55:0/3374556944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c15b540 msgr2=0x7ff34c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c15b540 0x7ff34c0cb8a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a180 comp rx=0 tx=0).stop 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 -- 172.21.15.55:0/3374556944 shutdown_connections 2024-05-10T03:35:49.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c0cbde0 0x7ff34c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 --2- 172.21.15.55:0/3374556944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c15b540 0x7ff34c0cb8a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 -- 172.21.15.55:0/3374556944 >> 172.21.15.55:0/3374556944 conn(0x7ff34c06a930 msgr2=0x7ff34c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:49.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 -- 172.21.15.55:0/3374556944 shutdown_connections 2024-05-10T03:35:49.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.288+0000 7ff351600640 1 -- 172.21.15.55:0/3374556944 wait complete. 2024-05-10T03:35:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.291+0000 7ff351600640 1 Processor -- start 2024-05-10T03:35:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.291+0000 7ff351600640 1 -- start start 2024-05-10T03:35:49.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34c0cbde0 0x7ff34c1f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c15b540 0x7ff34c1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c156350 con 0x7ff34c0adea0 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c145890 con 0x7ff34c0cbde0 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.293+0000 7ff351600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0acdd0 con 0x7ff34c15b540 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34992/0 (socket says 172.21.15.55:34992) 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 -- 172.21.15.55:0/2473311444 learned_addr learned my addr 172.21.15.55:0/2473311444 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff350c00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c15b540 0x7ff34c1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff343e00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34c0cbde0 0x7ff34c1f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 -- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c15b540 msgr2=0x7ff34c1f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c15b540 0x7ff34c1f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 -- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34c0cbde0 msgr2=0x7ff34c1f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34c0cbde0 0x7ff34c1f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.294+0000 7ff34be00640 1 -- 172.21.15.55:0/2473311444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334009400 con 0x7ff34c0adea0 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff34be00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c3e0 tx=0x7ff33c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c005ce0 con 0x7ff34c0adea0 2024-05-10T03:35:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c005e40 con 0x7ff34c0adea0 2024-05-10T03:35:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0072d0 con 0x7ff34c0adea0 2024-05-10T03:35:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c0a8a10 con 0x7ff34c0adea0 2024-05-10T03:35:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.295+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c0ad300 con 0x7ff34c0adea0 2024-05-10T03:35:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.297+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34c0cc8b0 con 0x7ff34c0adea0 2024-05-10T03:35:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.298+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff33c0074f0 con 0x7ff34c0adea0 2024-05-10T03:35:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.298+0000 7ff348c00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 0x7ff32806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.299+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7ff33c088a60 con 0x7ff34c0adea0 2024-05-10T03:35:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.299+0000 7ff343e00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 0x7ff32806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.301+0000 7ff343e00640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 0x7ff32806d110 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.303+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff33c053940 con 0x7ff34c0adea0 2024-05-10T03:35:49.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.514+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff34c0a8a10 con 0x7ff34c0adea0 2024-05-10T03:35:49.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.517+0000 7ff348c00640 1 -- 172.21.15.55:0/2473311444 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff34c0a8a10 con 0x7ff34c0adea0 2024-05-10T03:35:49.520 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 msgr2=0x7ff32806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 0x7ff32806d110 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 msgr2=0x7ff34c0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c3e0 tx=0x7ff33c00c8a0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 shutdown_connections 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff34c15b540 0x7ff34c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7ff32806ac00 0x7ff32806d110 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34c0cbde0 0x7ff34c1f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 --2- 172.21.15.55:0/2473311444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff34c0adea0 0x7ff34c0a84d0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.520+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 >> 172.21.15.55:0/2473311444 conn(0x7ff34c06a930 msgr2=0x7ff34c0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:49.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.521+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 shutdown_connections 2024-05-10T03:35:49.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.521+0000 7ff351600640 1 -- 172.21.15.55:0/2473311444 wait complete. 2024-05-10T03:35:49.553 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T03:35:49.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.829+0000 7f318e600640 1 Processor -- start 2024-05-10T03:35:49.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.830+0000 7f318e600640 1 -- start start 2024-05-10T03:35:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.832+0000 7f318e600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31880b45f0 0x7f31880ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.832+0000 7f318e600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31880ceda0 0x7f31880cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.832+0000 7f318e600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880d03d0 con 0x7f31880cffc0 2024-05-10T03:35:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.832+0000 7f318e600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880b4a00 con 0x7f31880ceda0 2024-05-10T03:35:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.832+0000 7f318e600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880b5ee0 con 0x7f31880b45f0 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f31880ceda0 0x7f31880cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f318dc00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f31880cffc0 0x7f31880d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39354/0 (socket says 172.21.15.55:39354) 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f318dc00640 1 -- 172.21.15.55:0/1201423969 learned_addr learned my addr 172.21.15.55:0/1201423969 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f318d200640 1 --2- 172.21.15.55:0/1201423969 >> v2:172.21.15.162:3300/0 conn(0x7f31880b45f0 0x7f31880ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187e00640 1 -- 172.21.15.55:0/1201423969 >> v2:172.21.15.162:3300/0 conn(0x7f31880b45f0 msgr2=0x7f31880ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187e00640 1 --2- 172.21.15.55:0/1201423969 >> v2:172.21.15.162:3300/0 conn(0x7f31880b45f0 0x7f31880ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187e00640 1 -- 172.21.15.55:0/1201423969 >> v1:172.21.15.55:6789/0 conn(0x7f31880cffc0 legacy=0x7f31880d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187e00640 1 -- 172.21.15.55:0/1201423969 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880cf2e0 con 0x7f31880ceda0 2024-05-10T03:35:49.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.833+0000 7f3187400640 1 -- 172.21.15.55:0/1201423969 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124605648 0 0) 0x7f31880d03d0 con 0x7f31880cffc0 2024-05-10T03:35:49.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.834+0000 7f3187e00640 1 --2- 172.21.15.55:0/1201423969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880ceda0 0x7f31880cfa80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f318806a970 tx=0x7f317402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8186c835699c49a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.835+0000 7f3187400640 1 -- 172.21.15.55:0/1201423969 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317402ae60 con 0x7f31880ceda0 2024-05-10T03:35:49.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.835+0000 7f3187400640 1 -- 172.21.15.55:0/1201423969 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3174032440 con 0x7f31880ceda0 2024-05-10T03:35:49.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.835+0000 7f3187400640 1 -- 172.21.15.55:0/1201423969 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740327d0 con 0x7f31880ceda0 2024-05-10T03:35:49.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.835+0000 7f318e600640 1 -- 172.21.15.55:0/1201423969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880ceda0 msgr2=0x7f31880cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.835+0000 7f318e600640 1 --2- 172.21.15.55:0/1201423969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880ceda0 0x7f31880cfa80 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f318806a970 tx=0x7f317402a110 comp rx=0 tx=0).stop 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 -- 172.21.15.55:0/1201423969 shutdown_connections 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 --2- 172.21.15.55:0/1201423969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880ceda0 0x7f31880cfa80 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 --2- 172.21.15.55:0/1201423969 >> v2:172.21.15.162:3300/0 conn(0x7f31880b45f0 0x7f31880ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 -- 172.21.15.55:0/1201423969 >> 172.21.15.55:0/1201423969 conn(0x7f31880d3090 msgr2=0x7f31880d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 -- 172.21.15.55:0/1201423969 shutdown_connections 2024-05-10T03:35:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.836+0000 7f318e600640 1 -- 172.21.15.55:0/1201423969 wait complete. 2024-05-10T03:35:49.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.838+0000 7f318e600640 1 Processor -- start 2024-05-10T03:35:49.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.838+0000 7f318e600640 1 -- start start 2024-05-10T03:35:49.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31880ceda0 0x7f318809ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31880cffc0 0x7f318809dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318814c770 con 0x7f31880b45f0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318809ccd0 con 0x7f31880cffc0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318e600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318809cf20 con 0x7f31880ceda0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318d200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35012/0 (socket says 172.21.15.55:35012) 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f318d200640 1 -- 172.21.15.55:0/3030215399 learned_addr learned my addr 172.21.15.55:0/3030215399 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.840+0000 7f3187e00640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31880ceda0 0x7f318809ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 -- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31880ceda0 msgr2=0x7f318809ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31880ceda0 0x7f318809ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 -- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31880cffc0 msgr2=0x7f318809dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318dc00640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31880cffc0 0x7f318809dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31880cffc0 0x7f318809dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:49.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 -- 172.21.15.55:0/3030215399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174009400 con 0x7f31880b45f0 2024-05-10T03:35:49.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318d200640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f317c00e870 tx=0x7f317c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0094f0 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c00c940 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318809d200 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.841+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00ccf0 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.842+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318809d7c0 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.843+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31880d2d50 con 0x7f31880b45f0 2024-05-10T03:35:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.844+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f317c013470 con 0x7f31880b45f0 2024-05-10T03:35:49.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.845+0000 7f3184c00640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 0x7f316806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:49.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.845+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f317c0897b0 con 0x7f31880b45f0 2024-05-10T03:35:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.845+0000 7f3187e00640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 0x7f316806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:49.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.847+0000 7f3187e00640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 0x7f316806d110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3174034000 tx=0x7f317402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:49.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:49.850+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f317c054670 con 0x7f31880b45f0 2024-05-10T03:35:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.051+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3188067240 con 0x7f31880b45f0 2024-05-10T03:35:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.052+0000 7f3184c00640 1 -- 172.21.15.55:0/3030215399 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f317c057c90 con 0x7f31880b45f0 2024-05-10T03:35:50.056 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-05-10T03:35:50.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 msgr2=0x7f316806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 0x7f316806d110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3174034000 tx=0x7f317402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 msgr2=0x7f318814b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f317c00e870 tx=0x7f317c00ed30 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 shutdown_connections 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f316806ac00 0x7f316806d110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31880cffc0 0x7f318809dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31880ceda0 0x7f318809ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 --2- 172.21.15.55:0/3030215399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31880b45f0 0x7f318814b3b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.055+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 >> 172.21.15.55:0/3030215399 conn(0x7f31880d3090 msgr2=0x7f318806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.056+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 shutdown_connections 2024-05-10T03:35:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.056+0000 7f318e600640 1 -- 172.21.15.55:0/3030215399 wait complete. 2024-05-10T03:35:50.087 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:50.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.358+0000 7f3a37200640 1 Processor -- start 2024-05-10T03:35:50.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.359+0000 7f3a37200640 1 -- start start 2024-05-10T03:35:50.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.361+0000 7f3a37200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a30161400 0x7f3a3015d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.361+0000 7f3a37200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a300b2e30 0x7f3a300b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.361+0000 7f3a37200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a300b3780 con 0x7f3a3006c9d0 2024-05-10T03:35:50.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.361+0000 7f3a37200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a300ac8e0 con 0x7f3a30161400 2024-05-10T03:35:50.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.361+0000 7f3a37200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300acbc0 con 0x7f3a300b2e30 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a30161400 0x7f3a3015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a30161400 0x7f3a3015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35024/0 (socket says 172.21.15.55:35024) 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 -- 172.21.15.55:0/4008572931 learned_addr learned my addr 172.21.15.55:0/4008572931 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a35200640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a300b2e30 0x7f3a300b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 -- 172.21.15.55:0/4008572931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a300b2e30 msgr2=0x7f3a300b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a300b2e30 0x7f3a300b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 -- 172.21.15.55:0/4008572931 >> v1:172.21.15.162:6789/0 conn(0x7f3a3006c9d0 legacy=0x7f3a300adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.362+0000 7f3a27400640 1 -- 172.21.15.55:0/4008572931 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300acea0 con 0x7f3a30161400 2024-05-10T03:35:50.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.364+0000 7f3a27400640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a3015d810 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009870 tx=0x7f3a1802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba719c62080827f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.364+0000 7f3a2f400640 1 -- 172.21.15.55:0/4008572931 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18032440 con 0x7f3a30161400 2024-05-10T03:35:50.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.364+0000 7f3a2f400640 1 -- 172.21.15.55:0/4008572931 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a180325a0 con 0x7f3a30161400 2024-05-10T03:35:50.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.364+0000 7f3a2f400640 1 -- 172.21.15.55:0/4008572931 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18032930 con 0x7f3a30161400 2024-05-10T03:35:50.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 -- 172.21.15.55:0/4008572931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 msgr2=0x7f3a3015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a3015d810 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009870 tx=0x7f3a1802a250 comp rx=0 tx=0).stop 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 -- 172.21.15.55:0/4008572931 shutdown_connections 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a300b2e30 0x7f3a300b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 --2- 172.21.15.55:0/4008572931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a3015d810 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 -- 172.21.15.55:0/4008572931 >> 172.21.15.55:0/4008572931 conn(0x7f3a300b0b80 msgr2=0x7f3a300b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 -- 172.21.15.55:0/4008572931 shutdown_connections 2024-05-10T03:35:50.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.365+0000 7f3a37200640 1 -- 172.21.15.55:0/4008572931 wait complete. 2024-05-10T03:35:50.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.367+0000 7f3a37200640 1 Processor -- start 2024-05-10T03:35:50.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.367+0000 7f3a37200640 1 -- start start 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 0x7f3a30149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a300b2e30 0x7f3a30157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3015e000 con 0x7f3a30161400 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301580e0 con 0x7f3a300b2e30 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a37200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301f4340 con 0x7f3a3006c9d0 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a35200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a35200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35038/0 (socket says 172.21.15.55:35038) 2024-05-10T03:35:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.369+0000 7f3a35200640 1 -- 172.21.15.55:0/3873475809 learned_addr learned my addr 172.21.15.55:0/3873475809 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a2fe00640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 0x7f3a30149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 -- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 msgr2=0x7f3a30149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 0x7f3a30149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 -- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a300b2e30 msgr2=0x7f3a30157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a300b2e30 0x7f3a30157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 -- 172.21.15.55:0/3873475809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180094d0 con 0x7f3a30161400 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a2fe00640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 0x7f3a30149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a35200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3a28007e60 tx=0x7f3a2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28014420 con 0x7f3a30161400 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a280145a0 con 0x7f3a30161400 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a301f4870 con 0x7f3a30161400 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.370+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28014950 con 0x7f3a30161400 2024-05-10T03:35:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.371+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3015d940 con 0x7f3a30161400 2024-05-10T03:35:50.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.372+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3015e140 con 0x7f3a30161400 2024-05-10T03:35:50.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.373+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a28014ab0 con 0x7f3a30161400 2024-05-10T03:35:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.374+0000 7f3a2cc00640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 0x7f3a0c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.374+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f3a28089f00 con 0x7f3a30161400 2024-05-10T03:35:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.375+0000 7f3a2fe00640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 0x7f3a0c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.376+0000 7f3a2fe00640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 0x7f3a0c06d110 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000f60 tx=0x7f3a20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.378+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a3015e140 con 0x7f3a30161400 2024-05-10T03:35:50.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.619+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a3015e140 con 0x7f3a30161400 2024-05-10T03:35:50.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.621+0000 7f3a2cc00640 1 -- 172.21.15.55:0/3873475809 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f3a3015e140 con 0x7f3a30161400 2024-05-10T03:35:50.625 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:50.625 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:34.375414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:50.626 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 69 2024-05-10T03:35:50.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.623+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 msgr2=0x7f3a0c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.623+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 0x7f3a0c06d110 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000f60 tx=0x7f3a20009040 comp rx=0 tx=0).stop 2024-05-10T03:35:50.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.623+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 msgr2=0x7f3a301f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3a28007e60 tx=0x7f3a2800e4b0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 shutdown_connections 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a30161400 0x7f3a301f3880 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3a0c06ac00 0x7f3a0c06d110 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a300b2e30 0x7f3a30157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 --2- 172.21.15.55:0/3873475809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a3006c9d0 0x7f3a30149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 >> 172.21.15.55:0/3873475809 conn(0x7f3a300b0b80 msgr2=0x7f3a30071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:50.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 shutdown_connections 2024-05-10T03:35:50.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.624+0000 7f3a37200640 1 -- 172.21.15.55:0/3873475809 wait complete. 2024-05-10T03:35:50.653 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:50.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.929+0000 7f76c2600640 1 Processor -- start 2024-05-10T03:35:50.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.929+0000 7f76c2600640 1 -- start start 2024-05-10T03:35:50.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76c2600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76bc0cf730 0x7f76bc0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76c2600640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc0b3f00 con 0x7f76bc0cfb40 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76c2600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc0ccb00 con 0x7f76bc15ef20 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76c2600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76bc0d0540 con 0x7f76bc0cf730 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76baa00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76bc0cf730 0x7f76bc0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76bbe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f76bc0cfb40 0x7f76bc0b4bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39368/0 (socket says 172.21.15.55:39368) 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76bbe00640 1 -- 172.21.15.55:0/3805155390 learned_addr learned my addr 172.21.15.55:0/3805155390 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76baa00640 1 -- 172.21.15.55:0/3805155390 >> v1:172.21.15.162:6789/0 conn(0x7f76bc15ef20 legacy=0x7f76bc0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:50.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.932+0000 7f76baa00640 1 -- 172.21.15.55:0/3805155390 >> v1:172.21.15.55:6789/0 conn(0x7f76bc0cfb40 legacy=0x7f76bc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.933+0000 7f76baa00640 1 -- 172.21.15.55:0/3805155390 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0ccde0 con 0x7f76bc0cf730 2024-05-10T03:35:50.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.933+0000 7f76ba000640 1 -- 172.21.15.55:0/3805155390 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447441118 0 0) 0x7f76bc0b3f00 con 0x7f76bc0cfb40 2024-05-10T03:35:50.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.934+0000 7f76baa00640 1 --2- 172.21.15.55:0/3805155390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc0b4670 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097a0 tx=0x7f76a802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b03f73a16aa2cd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.934+0000 7f76ba000640 1 -- 172.21.15.55:0/3805155390 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a802ae50 con 0x7f76bc0cf730 2024-05-10T03:35:50.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.934+0000 7f76ba000640 1 -- 172.21.15.55:0/3805155390 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a8032440 con 0x7f76bc0cf730 2024-05-10T03:35:50.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.934+0000 7f76ba000640 1 -- 172.21.15.55:0/3805155390 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80327d0 con 0x7f76bc0cf730 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 -- 172.21.15.55:0/3805155390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 msgr2=0x7f76bc0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 --2- 172.21.15.55:0/3805155390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc0b4670 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097a0 tx=0x7f76a802a140 comp rx=0 tx=0).stop 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 -- 172.21.15.55:0/3805155390 shutdown_connections 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 --2- 172.21.15.55:0/3805155390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc0b4670 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 -- 172.21.15.55:0/3805155390 >> 172.21.15.55:0/3805155390 conn(0x7f76bc0d10b0 msgr2=0x7f76bc0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 -- 172.21.15.55:0/3805155390 shutdown_connections 2024-05-10T03:35:50.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.935+0000 7f76c2600640 1 -- 172.21.15.55:0/3805155390 wait complete. 2024-05-10T03:35:50.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.937+0000 7f76c2600640 1 Processor -- start 2024-05-10T03:35:50.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.937+0000 7f76c2600640 1 -- start start 2024-05-10T03:35:50.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76bc0cfb40 0x7f76bc1f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76bc15ef20 0x7f76bc0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc1f3ec0 con 0x7f76bc0cf730 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc0b1b60 con 0x7f76bc0cfb40 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76c2600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc15aac0 con 0x7f76bc15ef20 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76bb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76bb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35066/0 (socket says 172.21.15.55:35066) 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.939+0000 7f76bb400640 1 -- 172.21.15.55:0/3285756430 learned_addr learned my addr 172.21.15.55:0/3285756430 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:50.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 -- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76bc15ef20 msgr2=0x7f76bc0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76bc15ef20 0x7f76bc0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 -- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76bc0cfb40 msgr2=0x7f76bc1f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76baa00640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76bc0cfb40 0x7f76bc1f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76bc0cfb40 0x7f76bc1f3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 -- 172.21.15.55:0/3285756430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009400 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76bb400640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b3f0 tx=0x7f76ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac011830 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac011990 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.940+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac011d40 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.941+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc0c7e40 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.941+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc0c8400 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.942+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc067950 con 0x7f76bc0cf730 2024-05-10T03:35:50.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.943+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f76ac01a400 con 0x7f76bc0cf730 2024-05-10T03:35:50.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.944+0000 7f76b3400640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 0x7f769c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:50.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.944+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f76ac089500 con 0x7f76bc0cf730 2024-05-10T03:35:50.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.945+0000 7f76baa00640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 0x7f769c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:50.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.946+0000 7f76baa00640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 0x7f769c06d140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034000 tx=0x7f76a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:50.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:50.951+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f76ac0543c0 con 0x7f76bc0cf730 2024-05-10T03:35:51.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.149+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76bc15e870 con 0x7f76bc0cf730 2024-05-10T03:35:51.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.150+0000 7f76b3400640 1 -- 172.21.15.55:0/3285756430 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16733 (secure 0 0 0) 0x7f76ac0579e0 con 0x7f76bc0cf730 2024-05-10T03:35:51.155 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:51.155 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:25.331003+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:51.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 msgr2=0x7f769c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 0x7f769c06d140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034000 tx=0x7f76a802e000 comp rx=0 tx=0).stop 2024-05-10T03:35:51.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 msgr2=0x7f76bc153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b3f0 tx=0x7f76ac00b8b0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 shutdown_connections 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76bc15ef20 0x7f76bc0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f769c06ac30 0x7f769c06d140 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76bc0cfb40 0x7f76bc1f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 --2- 172.21.15.55:0/3285756430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76bc0cf730 0x7f76bc153770 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.153+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 >> 172.21.15.55:0/3285756430 conn(0x7f76bc0d10b0 msgr2=0x7f76bc0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.154+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 shutdown_connections 2024-05-10T03:35:51.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.154+0000 7f76c2600640 1 -- 172.21.15.55:0/3285756430 wait complete. 2024-05-10T03:35:51.186 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:35:51.186 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.454+0000 7fe0cfe00640 1 Processor -- start 2024-05-10T03:35:51.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.454+0000 7fe0cfe00640 1 -- start start 2024-05-10T03:35:51.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00b8630 0x7fe0d00b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0d00ce860 0x7fe0d00cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe0d00cffe0 0x7fe0d00d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00b4a30 con 0x7fe0d00cffe0 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00b65d0 con 0x7fe0d00ce860 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00d03f0 con 0x7fe0d00b8630 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe0d00cffe0 0x7fe0d00d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cf400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe0d00cffe0 0x7fe0d00d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35082/0 (socket says 172.21.15.55:35082) 2024-05-10T03:35:51.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cf400640 1 -- 172.21.15.55:0/1379297943 learned_addr learned my addr 172.21.15.55:0/1379297943 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0cea00640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00b8630 0x7fe0d00b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.457+0000 7fe0ce000640 1 --2- 172.21.15.55:0/1379297943 >> v2:172.21.15.162:3300/0 conn(0x7fe0d00ce860 0x7fe0d00cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.458+0000 7fe0cf400640 1 -- 172.21.15.55:0/1379297943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00b8630 msgr2=0x7fe0d00b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.458+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00b8630 0x7fe0d00b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.458+0000 7fe0cf400640 1 -- 172.21.15.55:0/1379297943 >> v2:172.21.15.162:3300/0 conn(0x7fe0d00ce860 msgr2=0x7fe0d00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:51.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.458+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1379297943 >> v2:172.21.15.162:3300/0 conn(0x7fe0d00ce860 0x7fe0d00cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.458+0000 7fe0cf400640 1 -- 172.21.15.55:0/1379297943 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00cefd0 con 0x7fe0d00cffe0 2024-05-10T03:35:51.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.459+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d00d1090 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fe0d006a880 tx=0x7fe0c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977f109b503cc664 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:51.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.459+0000 7fe0cd600640 1 -- 172.21.15.55:0/1379297943 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8032440 con 0x7fe0d00cffe0 2024-05-10T03:35:51.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cd600640 1 -- 172.21.15.55:0/1379297943 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c80325a0 con 0x7fe0d00cffe0 2024-05-10T03:35:51.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cd600640 1 -- 172.21.15.55:0/1379297943 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8032930 con 0x7fe0d00cffe0 2024-05-10T03:35:51.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1379297943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 msgr2=0x7fe0d00d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d00d1090 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fe0d006a880 tx=0x7fe0c802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1379297943 shutdown_connections 2024-05-10T03:35:51.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d00d1090 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1379297943 >> v2:172.21.15.162:3300/0 conn(0x7fe0d00ce860 0x7fe0d00cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1379297943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00b8630 0x7fe0d00b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.460+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1379297943 >> 172.21.15.55:0/1379297943 conn(0x7fe0d00d3090 msgr2=0x7fe0d00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:51.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.461+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1379297943 shutdown_connections 2024-05-10T03:35:51.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.461+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1379297943 wait complete. 2024-05-10T03:35:51.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.463+0000 7fe0cfe00640 1 Processor -- start 2024-05-10T03:35:51.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.463+0000 7fe0cfe00640 1 -- start start 2024-05-10T03:35:51.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0d00b8630 0x7fe0d00cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00ce860 0x7fe0d01f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0158110 con 0x7fe0d00cffe0 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00cc230 con 0x7fe0d00b8630 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cfe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00cd090 con 0x7fe0d00ce860 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cf400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35090/0 (socket says 172.21.15.55:35090) 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cf400640 1 -- 172.21.15.55:0/1863707122 learned_addr learned my addr 172.21.15.55:0/1863707122 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:51.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cea00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0d00b8630 0x7fe0d00cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0ce000640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00ce860 0x7fe0d01f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cf400640 1 -- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00ce860 msgr2=0x7fe0d01f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.465+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00ce860 0x7fe0d01f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cf400640 1 -- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0d00b8630 msgr2=0x7fe0d00cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0d00b8630 0x7fe0d00cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cf400640 1 -- 172.21.15.55:0/1863707122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80094d0 con 0x7fe0d00cffe0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cf400640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe0c802a220 tx=0x7fe0c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8032440 con 0x7fe0d00cffe0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c8030b80 con 0x7fe0d00cffe0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c803d440 con 0x7fe0d00cffe0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d00cc480 con 0x7fe0d00cffe0 2024-05-10T03:35:51.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.466+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d00cca40 con 0x7fe0d00cffe0 2024-05-10T03:35:51.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.468+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d00cd9e0 con 0x7fe0d00cffe0 2024-05-10T03:35:51.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.469+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe0c8039040 con 0x7fe0d00cffe0 2024-05-10T03:35:51.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.470+0000 7fe0c6a00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 0x7fe0ac06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:51.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.470+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fe0c80aa210 con 0x7fe0d00cffe0 2024-05-10T03:35:51.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.470+0000 7fe0cea00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 0x7fe0ac06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:51.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.472+0000 7fe0cea00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 0x7fe0ac06d110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0006fd0 tx=0x7fe0c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.476+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe0c8075180 con 0x7fe0d00cffe0 2024-05-10T03:35:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.721+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0d00d1e90 con 0x7fe0d00cffe0 2024-05-10T03:35:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.722+0000 7fe0c6a00640 1 -- 172.21.15.55:0/1863707122 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fe0c80787a0 con 0x7fe0d00cffe0 2024-05-10T03:35:51.726 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:51.726 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:34.375414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:51.727 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 69 2024-05-10T03:35:51.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.725+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 msgr2=0x7fe0ac06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.725+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 0x7fe0ac06d110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0006fd0 tx=0x7fe0c0008040 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.725+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 msgr2=0x7fe0d01f7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.725+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe0c802a220 tx=0x7fe0c80046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 shutdown_connections 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d00cffe0 0x7fe0d01f7bf0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0d00ce860 0x7fe0d01f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe0ac06ac00 0x7fe0ac06d110 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 --2- 172.21.15.55:0/1863707122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0d00b8630 0x7fe0d00cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 >> 172.21.15.55:0/1863707122 conn(0x7fe0d00d3090 msgr2=0x7fe0d0073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 shutdown_connections 2024-05-10T03:35:51.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:51.726+0000 7fe0cfe00640 1 -- 172.21.15.55:0/1863707122 wait complete. 2024-05-10T03:35:51.757 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:35:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.042+0000 7fdcc9000640 1 Processor -- start 2024-05-10T03:35:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.043+0000 7fdcc9000640 1 -- start start 2024-05-10T03:35:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.045+0000 7fdcc9000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdcc4071dc0 0x7fdcc4141900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.045+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.045+0000 7fdcc9000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc4070bf0 con 0x7fdcc406c100 2024-05-10T03:35:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.045+0000 7fdcc9000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc406c920 con 0x7fdcc4071dc0 2024-05-10T03:35:52.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.045+0000 7fdcc9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4141e40 con 0x7fdcc406c510 2024-05-10T03:35:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.046+0000 7fdcc2800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.046+0000 7fdcc1e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdcc4071dc0 0x7fdcc4141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.046+0000 7fdcc2800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:47988/0 (socket says 172.21.15.55:47988) 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.046+0000 7fdcc2800640 1 -- 172.21.15.55:0/268197786 learned_addr learned my addr 172.21.15.55:0/268197786 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.047+0000 7fdcc1e00640 1 -- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 msgr2=0x7fdcc40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.047+0000 7fdcc1e00640 1 --2- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.047+0000 7fdcc1e00640 1 -- 172.21.15.55:0/268197786 >> v1:172.21.15.162:6789/0 conn(0x7fdcc406c100 legacy=0x7fdcc4071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.047+0000 7fdcc1e00640 1 -- 172.21.15.55:0/268197786 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40cc0f0 con 0x7fdcc4071dc0 2024-05-10T03:35:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.048+0000 7fdcc1e00640 1 --2- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc4071dc0 0x7fdcc4141900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40097a0 tx=0x7fdcb402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c679772f8726180 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.049+0000 7fdcc1400640 1 -- 172.21.15.55:0/268197786 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb402ae50 con 0x7fdcc4071dc0 2024-05-10T03:35:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.049+0000 7fdcc1400640 1 -- 172.21.15.55:0/268197786 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4032440 con 0x7fdcc4071dc0 2024-05-10T03:35:52.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.049+0000 7fdcc1400640 1 -- 172.21.15.55:0/268197786 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40327d0 con 0x7fdcc4071dc0 2024-05-10T03:35:52.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.049+0000 7fdcc9000640 1 -- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc4071dc0 msgr2=0x7fdcc4141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.049+0000 7fdcc9000640 1 --2- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc4071dc0 0x7fdcc4141900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40097a0 tx=0x7fdcb402a100 comp rx=0 tx=0).stop 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 -- 172.21.15.55:0/268197786 shutdown_connections 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 --2- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 --2- 172.21.15.55:0/268197786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc4071dc0 0x7fdcc4141900 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 -- 172.21.15.55:0/268197786 >> 172.21.15.55:0/268197786 conn(0x7fdcc40b0b80 msgr2=0x7fdcc40b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 -- 172.21.15.55:0/268197786 shutdown_connections 2024-05-10T03:35:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.050+0000 7fdcc9000640 1 -- 172.21.15.55:0/268197786 wait complete. 2024-05-10T03:35:52.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.053+0000 7fdcc9000640 1 Processor -- start 2024-05-10T03:35:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.053+0000 7fdcc9000640 1 -- start start 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc406c100 0x7fdcc41f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc41f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41f4bb0 con 0x7fdcc4071dc0 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40b2430 con 0x7fdcc406c100 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.055+0000 7fdcc9000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40b2680 con 0x7fdcc406c510 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35094/0 (socket says 172.21.15.55:35094) 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 -- 172.21.15.55:0/2691484760 learned_addr learned my addr 172.21.15.55:0/2691484760 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcb9400640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc41f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc1e00640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc406c100 0x7fdcc41f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 -- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 msgr2=0x7fdcc41f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc41f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 -- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc406c100 msgr2=0x7fdcc41f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc406c100 0x7fdcc41f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.056+0000 7fdcc2800640 1 -- 172.21.15.55:0/2691484760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009400 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcc2800640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00ec10 tx=0x7fdcbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00cb20 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc00f040 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003990 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc40b2960 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.057+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc41f79d0 con 0x7fdcc4071dc0 2024-05-10T03:35:52.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.058+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc415ed60 con 0x7fdcc4071dc0 2024-05-10T03:35:52.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.059+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdcbc003d00 con 0x7fdcc4071dc0 2024-05-10T03:35:52.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.060+0000 7fdcbaa00640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 0x7fdca006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.061+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fdcbc089e40 con 0x7fdcc4071dc0 2024-05-10T03:35:52.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.061+0000 7fdcc1e00640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 0x7fdca006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.063+0000 7fdcc1e00640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 0x7fdca006d110 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4034000 tx=0x7fdcb402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.065+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdcbc054c70 con 0x7fdcc4071dc0 2024-05-10T03:35:52.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.302+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcc40b2e80 con 0x7fdcc4071dc0 2024-05-10T03:35:52.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.303+0000 7fdcbaa00640 1 -- 172.21.15.55:0/2691484760 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fdcc40b2e80 con 0x7fdcc4071dc0 2024-05-10T03:35:52.307 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:52.307 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":69,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T03:34:55.059838+0000","modified":"2024-05-10T03:35:34.375414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7647,"mds_1":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":1,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6833/609219709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":609219709},{"type":"v1","addr":"172.21.15.55:6833","nonce":609219709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7647":{"gid":7647,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.55:6835/1198619982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1198619982},{"type":"v1","addr":"172.21.15.55:6835","nonce":1198619982}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-05-10T03:35:52.307 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 69 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 msgr2=0x7fdca006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 0x7fdca006d110 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4034000 tx=0x7fdcb402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 msgr2=0x7fdcc415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00ec10 tx=0x7fdcbc00c490 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 shutdown_connections 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdcc4071dc0 0x7fdcc415ce70 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcc406c510 0x7fdcc41f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.306+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdca006ac00 0x7fdca006d110 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.307+0000 7fdcc9000640 1 --2- 172.21.15.55:0/2691484760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcc406c100 0x7fdcc41f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.307+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 >> 172.21.15.55:0/2691484760 conn(0x7fdcc40b0b80 msgr2=0x7fdcc4144b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.307+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 shutdown_connections 2024-05-10T03:35:52.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.307+0000 7fdcc9000640 1 -- 172.21.15.55:0/2691484760 wait complete. 2024-05-10T03:35:52.337 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:35:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.616+0000 7fdff9600640 1 Processor -- start 2024-05-10T03:35:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.616+0000 7fdff9600640 1 -- start start 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdff9600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdff4152280 0x7fdff415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdff9600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdff9600640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff415c040 con 0x7fdff415ba60 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdff9600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff415c320 con 0x7fdff4152280 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdff9600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff415c600 con 0x7fdff4152690 2024-05-10T03:35:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdfeb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdff4152280 0x7fdff415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdfeb400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdff4152280 0x7fdff415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35110/0 (socket says 172.21.15.55:35110) 2024-05-10T03:35:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.618+0000 7fdfeb400640 1 -- 172.21.15.55:0/2153248709 learned_addr learned my addr 172.21.15.55:0/2153248709 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.619+0000 7fdff8c00640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.619+0000 7fdfeb400640 1 -- 172.21.15.55:0/2153248709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 msgr2=0x7fdff415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.619+0000 7fdfeb400640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff415fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.619+0000 7fdfeb400640 1 -- 172.21.15.55:0/2153248709 >> v1:172.21.15.162:6789/0 conn(0x7fdff415ba60 legacy=0x7fdff4071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.619+0000 7fdfeb400640 1 -- 172.21.15.55:0/2153248709 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff415c8e0 con 0x7fdff4152280 2024-05-10T03:35:52.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.620+0000 7fdfeb400640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff415f5a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f42f83c4e9c06b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.620+0000 7fdff3400640 1 -- 172.21.15.55:0/2153248709 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc032440 con 0x7fdff4152280 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.620+0000 7fdff3400640 1 -- 172.21.15.55:0/2153248709 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc0325a0 con 0x7fdff4152280 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.620+0000 7fdff3400640 1 -- 172.21.15.55:0/2153248709 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc032930 con 0x7fdff4152280 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 -- 172.21.15.55:0/2153248709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 msgr2=0x7fdff415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff415f5a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009870 tx=0x7fdfdc02a210 comp rx=0 tx=0).stop 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 -- 172.21.15.55:0/2153248709 shutdown_connections 2024-05-10T03:35:52.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff415fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 --2- 172.21.15.55:0/2153248709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff415f5a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 -- 172.21.15.55:0/2153248709 >> 172.21.15.55:0/2153248709 conn(0x7fdff40b07b0 msgr2=0x7fdff40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 -- 172.21.15.55:0/2153248709 shutdown_connections 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.621+0000 7fdff9600640 1 -- 172.21.15.55:0/2153248709 wait complete. 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.624+0000 7fdff9600640 1 Processor -- start 2024-05-10T03:35:52.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.624+0000 7fdff9600640 1 -- start start 2024-05-10T03:35:52.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff41e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 0x7fdff40b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41e8f70 con 0x7fdff4152280 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41e6390 con 0x7fdff415ba60 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.625+0000 7fdff9600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41e9190 con 0x7fdff4152690 2024-05-10T03:35:52.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdfeb400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff41e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35126/0 (socket says 172.21.15.55:35126) 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 -- 172.21.15.55:0/4190835467 learned_addr learned my addr 172.21.15.55:0/4190835467 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff8c00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 0x7fdff40b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 -- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 msgr2=0x7fdff41e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff41e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 -- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 msgr2=0x7fdff40b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 0x7fdff40b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.626+0000 7fdff3e00640 1 -- 172.21.15.55:0/4190835467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc0094d0 con 0x7fdff4152280 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff8c00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 0x7fdff40b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff3e00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b9f0 tx=0x7fdfe400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400cbd0 con 0x7fdff4152280 2024-05-10T03:35:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400cd30 con 0x7fdff4152280 2024-05-10T03:35:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4003bb0 con 0x7fdff4152280 2024-05-10T03:35:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff413e5b0 con 0x7fdff4152280 2024-05-10T03:35:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.627+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff413eb70 con 0x7fdff4152280 2024-05-10T03:35:52.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.628+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff4067240 con 0x7fdff4152280 2024-05-10T03:35:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.630+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdfe4014440 con 0x7fdff4152280 2024-05-10T03:35:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.631+0000 7fdff0c00640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 0x7fdfd006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.631+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fdfe40893a0 con 0x7fdff4152280 2024-05-10T03:35:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.631+0000 7fdfeb400640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 0x7fdfd006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.633+0000 7fdfeb400640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 0x7fdfd006d140 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc034000 tx=0x7fdfdc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:52.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.636+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdfe4054320 con 0x7fdff4152280 2024-05-10T03:35:52.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.837+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdff40cbb00 con 0x7fdff4152280 2024-05-10T03:35:52.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.840+0000 7fdff0c00640 1 -- 172.21.15.55:0/4190835467 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16733 (secure 0 0 0) 0x7fdfe4057940 con 0x7fdff4152280 2024-05-10T03:35:52.844 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:52.844 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:25.331003+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-10T03:34:53.261121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-10T03:34:54.084182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:35:52.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 msgr2=0x7fdfd006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 0x7fdfd006d140 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc034000 tx=0x7fdfdc02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:52.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 msgr2=0x7fdff41e59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:52.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400b9f0 tx=0x7fdfe400beb0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 shutdown_connections 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fdfd006ac30 0x7fdfd006d140 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff415ba60 0x7fdff40b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff4152690 0x7fdff41e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 --2- 172.21.15.55:0/4190835467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff4152280 0x7fdff41e59c0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.843+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 >> 172.21.15.55:0/4190835467 conn(0x7fdff40b07b0 msgr2=0x7fdff4073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.844+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 shutdown_connections 2024-05-10T03:35:52.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:52.844+0000 7fdff9600640 1 -- 172.21.15.55:0/4190835467 wait complete. 2024-05-10T03:35:52.875 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T03:35:53.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.148+0000 7fd380400640 1 Processor -- start 2024-05-10T03:35:53.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.148+0000 7fd380400640 1 -- start start 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd380400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd37806c100 0x7fd37806c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd380400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378070bf0 con 0x7fd378071080 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd380400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3780cc0f0 con 0x7fd378071dc0 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd380400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3780cc3b0 con 0x7fd37806c100 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd37d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd37806c100 0x7fd37806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd37e400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd378071080 0x7fd3780cdca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51962/0 (socket says 172.21.15.55:51962) 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd37d000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd37806c100 0x7fd37806c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40660/0 (socket says 172.21.15.55:40660) 2024-05-10T03:35:53.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.151+0000 7fd37e400640 1 -- 172.21.15.55:0/514225981 learned_addr learned my addr 172.21.15.55:0/514225981 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.152+0000 7fd37d000640 1 -- 172.21.15.55:0/514225981 >> v1:172.21.15.162:6789/0 conn(0x7fd378071dc0 legacy=0x7fd378141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.152+0000 7fd37d000640 1 -- 172.21.15.55:0/514225981 >> v1:172.21.15.55:6789/0 conn(0x7fd378071080 legacy=0x7fd3780cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.152+0000 7fd377e00640 1 -- 172.21.15.55:0/514225981 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469728118 0 0) 0x7fd378070bf0 con 0x7fd378071080 2024-05-10T03:35:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.152+0000 7fd37d000640 1 -- 172.21.15.55:0/514225981 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780cc690 con 0x7fd37806c100 2024-05-10T03:35:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.153+0000 7fd37d000640 1 --2- 172.21.15.55:0/514225981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd37806c100 0x7fd37806c510 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097a0 tx=0x7fd36402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624a8d4c0fdc6909 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.153+0000 7fd377e00640 1 -- 172.21.15.55:0/514225981 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36402ae50 con 0x7fd37806c100 2024-05-10T03:35:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.153+0000 7fd377e00640 1 -- 172.21.15.55:0/514225981 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd364032440 con 0x7fd37806c100 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 -- 172.21.15.55:0/514225981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd37806c100 msgr2=0x7fd37806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 --2- 172.21.15.55:0/514225981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd37806c100 0x7fd37806c510 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097a0 tx=0x7fd36402a140 comp rx=0 tx=0).stop 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 -- 172.21.15.55:0/514225981 shutdown_connections 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 --2- 172.21.15.55:0/514225981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd37806c100 0x7fd37806c510 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 -- 172.21.15.55:0/514225981 >> 172.21.15.55:0/514225981 conn(0x7fd3780b0b80 msgr2=0x7fd3780b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 -- 172.21.15.55:0/514225981 shutdown_connections 2024-05-10T03:35:53.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.154+0000 7fd380400640 1 -- 172.21.15.55:0/514225981 wait complete. 2024-05-10T03:35:53.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.157+0000 7fd380400640 1 Processor -- start 2024-05-10T03:35:53.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.157+0000 7fd380400640 1 -- start start 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd37806c100 0x7fd37815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 0x7fd3781f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781f1790 con 0x7fd378071dc0 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781378a0 con 0x7fd378071080 2024-05-10T03:35:53.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.158+0000 7fd380400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815d9b0 con 0x7fd37806c100 2024-05-10T03:35:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40664/0 (socket says 172.21.15.55:40664) 2024-05-10T03:35:53.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 -- 172.21.15.55:0/2716648450 learned_addr learned my addr 172.21.15.55:0/2716648450 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37d000640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 0x7fd3781f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37da00640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd37806c100 0x7fd37815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 -- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd37806c100 msgr2=0x7fd37815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd37806c100 0x7fd37815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 -- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 msgr2=0x7fd3781f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 0x7fd3781f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37e400640 1 -- 172.21.15.55:0/2716648450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364009400 con 0x7fd378071dc0 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.159+0000 7fd37d000640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 0x7fd3781f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:53.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd37e400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c410 tx=0x7fd37000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700074e0 con 0x7fd378071dc0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd370007640 con 0x7fd378071dc0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37815dc90 con 0x7fd378071dc0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370004a20 con 0x7fd378071dc0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.160+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378137af0 con 0x7fd378071dc0 2024-05-10T03:35:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.162+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378067320 con 0x7fd378071dc0 2024-05-10T03:35:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.163+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd370004b80 con 0x7fd378071dc0 2024-05-10T03:35:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.164+0000 7fd375600640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 0x7fd35806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.164+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fd370089430 con 0x7fd378071dc0 2024-05-10T03:35:53.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.165+0000 7fd37da00640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 0x7fd35806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.166+0000 7fd37da00640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 0x7fd35806d110 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009f20 tx=0x7fd36c0094d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.169+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd370054320 con 0x7fd378071dc0 2024-05-10T03:35:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.407+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd378070430 con 0x7fd378071dc0 2024-05-10T03:35:53.412 INFO:tasks.ceph.mon.a.smithi055.stderr:2024-05-10T03:35:53.409+0000 7f777f600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T03:35:53.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.430+0000 7fd375600640 1 -- 172.21.15.55:0/2716648450 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd370057940 con 0x7fd378071dc0 2024-05-10T03:35:53.434 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T03:35:53.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.433+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 msgr2=0x7fd35806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.433+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 0x7fd35806d110 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009f20 tx=0x7fd36c0094d0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.433+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 msgr2=0x7fd3781f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c410 tx=0x7fd37000c8d0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 shutdown_connections 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd378071dc0 0x7fd3781f52b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd35806ac00 0x7fd35806d110 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd378071080 0x7fd3781f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 --2- 172.21.15.55:0/2716648450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd37806c100 0x7fd37815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 >> 172.21.15.55:0/2716648450 conn(0x7fd3780b0b80 msgr2=0x7fd378143290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 shutdown_connections 2024-05-10T03:35:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.434+0000 7fd380400640 1 -- 172.21.15.55:0/2716648450 wait complete. 2024-05-10T03:35:53.445 INFO:tasks.ceph.mds.a.smithi055.stderr: -5038> 2024-05-10T03:35:20.230+0000 7fb298600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T03:35:53.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T03:35:53.582 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2024-05-10T03:35:53.586 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2024-05-10T03:35:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.730+0000 7f486e000640 1 Processor -- start 2024-05-10T03:35:53.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.730+0000 7f486e000640 1 -- start start 2024-05-10T03:35:53.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.732+0000 7f486e000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4868071cf0 0x7f48680c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.732+0000 7f486e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48680ac8f0 0x7f48680c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.732+0000 7f486e000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680cd300 con 0x7f486815d840 2024-05-10T03:35:53.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f486e000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48680c9770 con 0x7f4868071cf0 2024-05-10T03:35:53.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f486e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868070d90 con 0x7f48680ac8f0 2024-05-10T03:35:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f486cc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4868071cf0 0x7f48680c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f486cc00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4868071cf0 0x7f48680c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40712/0 (socket says 172.21.15.55:40712) 2024-05-10T03:35:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f486cc00640 1 -- 172.21.15.55:0/3188680213 learned_addr learned my addr 172.21.15.55:0/3188680213 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.733+0000 7f4867e00640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48680ac8f0 0x7f48680c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.734+0000 7f486cc00640 1 -- 172.21.15.55:0/3188680213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48680ac8f0 msgr2=0x7f48680c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:53.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.734+0000 7f486cc00640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48680ac8f0 0x7f48680c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.734+0000 7f486cc00640 1 -- 172.21.15.55:0/3188680213 >> v1:172.21.15.162:6789/0 conn(0x7f486815d840 legacy=0x7f4868161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:53.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.734+0000 7f486cc00640 1 -- 172.21.15.55:0/3188680213 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868071070 con 0x7f4868071cf0 2024-05-10T03:35:53.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.736+0000 7f486cc00640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4868071cf0 0x7f48680c9230 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f485c009870 tx=0x7f485c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f931c3a080142f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.737+0000 7f4867400640 1 -- 172.21.15.55:0/3188680213 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c032440 con 0x7f4868071cf0 2024-05-10T03:35:53.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.737+0000 7f4867400640 1 -- 172.21.15.55:0/3188680213 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c0325a0 con 0x7f4868071cf0 2024-05-10T03:35:53.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.737+0000 7f4867400640 1 -- 172.21.15.55:0/3188680213 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c032930 con 0x7f4868071cf0 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 -- 172.21.15.55:0/3188680213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4868071cf0 msgr2=0x7f48680c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4868071cf0 0x7f48680c9230 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f485c009870 tx=0x7f485c02a1e0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 -- 172.21.15.55:0/3188680213 shutdown_connections 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48680ac8f0 0x7f48680c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 --2- 172.21.15.55:0/3188680213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4868071cf0 0x7f48680c9230 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 -- 172.21.15.55:0/3188680213 >> 172.21.15.55:0/3188680213 conn(0x7f48680cebe0 msgr2=0x7f48680ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 -- 172.21.15.55:0/3188680213 shutdown_connections 2024-05-10T03:35:53.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.738+0000 7f486e000640 1 -- 172.21.15.55:0/3188680213 wait complete. 2024-05-10T03:35:53.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.741+0000 7f486e000640 1 Processor -- start 2024-05-10T03:35:53.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.741+0000 7f486e000640 1 -- start start 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 0x7f4868157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 0x7f48680b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868159180 con 0x7f48680ac8f0 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868160bd0 con 0x7f4868071cf0 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.742+0000 7f486e000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680b29f0 con 0x7f486815d840 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40726/0 (socket says 172.21.15.55:40726) 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 -- 172.21.15.55:0/733337026 learned_addr learned my addr 172.21.15.55:0/733337026 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f486cc00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 0x7f4868157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f486d600640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 0x7f48680b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 -- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 msgr2=0x7f48680b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 0x7f48680b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 -- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 msgr2=0x7f4868157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 0x7f4868157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f4867e00640 1 -- 172.21.15.55:0/733337026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c0094d0 con 0x7f48680ac8f0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f486cc00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 0x7f4868157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.743+0000 7f486d600640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 0x7f48680b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f4867e00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f485800d620 tx=0x7f485800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800b620 con 0x7f48680ac8f0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485800b780 con 0x7f48680ac8f0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800bb30 con 0x7f48680ac8f0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48681582b0 con 0x7f48680ac8f0 2024-05-10T03:35:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.744+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48681587e0 con 0x7f48680ac8f0 2024-05-10T03:35:53.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.745+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4868067240 con 0x7f48680ac8f0 2024-05-10T03:35:53.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.747+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f485800bc90 con 0x7f48680ac8f0 2024-05-10T03:35:53.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.748+0000 7f4864c00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 0x7f484c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:53.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.748+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f4858089c10 con 0x7f48680ac8f0 2024-05-10T03:35:53.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.748+0000 7f486cc00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 0x7f484c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:53.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.750+0000 7f486cc00640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 0x7f484c06d110 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f485c034000 tx=0x7f485c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:53.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.753+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4858054a20 con 0x7f48680ac8f0 2024-05-10T03:35:53.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:53.993+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f48680cceb0 con 0x7f48680ac8f0 2024-05-10T03:35:54.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.553+0000 7f4864c00640 1 -- 172.21.15.55:0/733337026 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4858058040 con 0x7f48680ac8f0 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 msgr2=0x7f484c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 0x7f484c06d110 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f485c034000 tx=0x7f485c02e000 comp rx=0 tx=0).stop 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 msgr2=0x7f486815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f485800d620 tx=0x7f485800dae0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 shutdown_connections 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f486815d840 0x7f48680b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48680ac8f0 0x7f486815e6b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f484c06ac00 0x7f484c06d110 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 --2- 172.21.15.55:0/733337026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4868071cf0 0x7f4868157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.557+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 >> 172.21.15.55:0/733337026 conn(0x7f48680cebe0 msgr2=0x7f486806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:54.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.558+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 shutdown_connections 2024-05-10T03:35:54.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.558+0000 7f486e000640 1 -- 172.21.15.55:0/733337026 wait complete. 2024-05-10T03:35:54.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T03:35:54.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.878+0000 7fa58c800640 1 Processor -- start 2024-05-10T03:35:54.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.878+0000 7fa58c800640 1 -- start start 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa58c800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa5840b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa58c800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5840cdd50 0x7fa5840ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa58c800640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5840b4480 con 0x7fa58406d770 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa58c800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5840b2c90 con 0x7fa5840cdd50 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa58c800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b2ea0 con 0x7fa5840cc5f0 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa589e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa58406d770 0x7fa58415c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51972/0 (socket says 172.21.15.55:51972) 2024-05-10T03:35:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.880+0000 7fa589e00640 1 -- 172.21.15.55:0/1595924708 learned_addr learned my addr 172.21.15.55:0/1595924708 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:54.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa58a800640 1 --2- 172.21.15.55:0/1595924708 >> v2:172.21.15.162:3300/0 conn(0x7fa5840cdd50 0x7fa5840ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa5840b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494007450 0 0) 0x7fa5840b4480 con 0x7fa58406d770 2024-05-10T03:35:54.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003520 con 0x7fa58406d770 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958588675 0 0) 0x7fa56c003520 con 0x7fa58406d770 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 msgr2=0x7fa5840b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa5840b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 >> v2:172.21.15.162:3300/0 conn(0x7fa5840cdd50 msgr2=0x7fa5840ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 --2- 172.21.15.55:0/1595924708 >> v2:172.21.15.162:3300/0 conn(0x7fa5840cdd50 0x7fa5840ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840b3180 con 0x7fa58406d770 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fa578001670 con 0x7fa58406d770 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa5840b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:54.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.881+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 >> v1:172.21.15.55:6789/0 conn(0x7fa58406d770 legacy=0x7fa58415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa56c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa56c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c004d00 0x7fa56c005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b2c90 con 0x7fa56c003ed0 2024-05-10T03:35:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840b2ea0 con 0x7fa56c004d00 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00ba80 con 0x7fa5840cc5f0 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa56c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589e00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa56c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa58a800640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c004d00 0x7fa56c005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 -- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 msgr2=0x7fa56c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa56c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 -- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c004d00 msgr2=0x7fa56c005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c004d00 0x7fa56c005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589400640 1 -- 172.21.15.55:0/1595924708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c003520 con 0x7fa56c003ed0 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa589e00640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cc5f0 0x7fa56c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.882+0000 7fa58a800640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa56c004d00 0x7fa56c005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.883+0000 7fa589400640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa56c0047c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa5740098d0 tx=0x7fa56c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd088123fec3ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57402ad00 con 0x7fa56c003ed0 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57402ae60 con 0x7fa56c003ed0 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa588a00640 1 -- 172.21.15.55:0/1595924708 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574004250 con 0x7fa56c003ed0 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa58c800640 1 -- 172.21.15.55:0/1595924708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 msgr2=0x7fa56c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa58c800640 1 --2- 172.21.15.55:0/1595924708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa56c0047c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa5740098d0 tx=0x7fa56c0057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.884+0000 7fa58a800640 1 -- 172.21.15.55:0/1595924708 reap_dead start 2024-05-10T03:35:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.885+0000 7fa58c800640 1 -- 172.21.15.55:0/1595924708 shutdown_connections 2024-05-10T03:35:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.885+0000 7fa58c800640 1 -- 172.21.15.55:0/1595924708 >> 172.21.15.55:0/1595924708 conn(0x7fa5840b0c60 msgr2=0x7fa5840b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.885+0000 7fa58c800640 1 -- 172.21.15.55:0/1595924708 shutdown_connections 2024-05-10T03:35:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.885+0000 7fa58c800640 1 -- 172.21.15.55:0/1595924708 wait complete. 2024-05-10T03:35:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.887+0000 7fa58c800640 1 Processor -- start 2024-05-10T03:35:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.887+0000 7fa58c800640 1 -- start start 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 0x7fa584157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cb7c0 0x7fa584157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840c4780 con 0x7fa56c003ed0 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840cbd80 con 0x7fa5840c4080 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa58c800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58413fac0 con 0x7fa5840cb7c0 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa589e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa589e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40754/0 (socket says 172.21.15.55:40754) 2024-05-10T03:35:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.889+0000 7fa589e00640 1 -- 172.21.15.55:0/1880542366 learned_addr learned my addr 172.21.15.55:0/1880542366 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:54.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 -- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cb7c0 msgr2=0x7fa584157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589400640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 0x7fa584157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa58a800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cb7c0 0x7fa584157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cb7c0 0x7fa584157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 -- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 msgr2=0x7fa584157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 0x7fa584157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 -- 172.21.15.55:0/1880542366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5841537d0 con 0x7fa56c003ed0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589400640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 0x7fa584157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa589e00640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa5780018e0 tx=0x7fa57800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780074e0 con 0x7fa56c003ed0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578007640 con 0x7fa56c003ed0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780049e0 con 0x7fa56c003ed0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.890+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa584152ea0 con 0x7fa56c003ed0 2024-05-10T03:35:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.891+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584153460 con 0x7fa56c003ed0 2024-05-10T03:35:54.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.892+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5840cca00 con 0x7fa56c003ed0 2024-05-10T03:35:54.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.893+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa578004b40 con 0x7fa56c003ed0 2024-05-10T03:35:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.894+0000 7fa57e000640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 0x7fa56806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.894+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fa5780890a0 con 0x7fa56c003ed0 2024-05-10T03:35:54.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.895+0000 7fa589400640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 0x7fa56806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:54.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.896+0000 7fa589400640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 0x7fa56806d110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa574033000 tx=0x7fa57402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:54.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:54.898+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa578053e30 con 0x7fa56c003ed0 2024-05-10T03:35:55.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.102+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa584067460 con 0x7fa56c003ed0 2024-05-10T03:35:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.590+0000 7fa57e000640 1 -- 172.21.15.55:0/1880542366 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa5780190d0 con 0x7fa56c003ed0 2024-05-10T03:35:55.594 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-05-10T03:35:55.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 msgr2=0x7fa56806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:55.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 0x7fa56806d110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa574033000 tx=0x7fa57402e000 comp rx=0 tx=0).stop 2024-05-10T03:35:55.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 msgr2=0x7fa58413f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:55.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa5780018e0 tx=0x7fa57800dc70 comp rx=0 tx=0).stop 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 shutdown_connections 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5840cb7c0 0x7fa584157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fa56806ac00 0x7fa56806d110 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5840c4080 0x7fa584157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 --2- 172.21.15.55:0/1880542366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa56c003ed0 0x7fa58413f580 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 >> 172.21.15.55:0/1880542366 conn(0x7fa5840b0c60 msgr2=0x7fa584144910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:55.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.595+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 shutdown_connections 2024-05-10T03:35:55.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.596+0000 7fa58c800640 1 -- 172.21.15.55:0/1880542366 wait complete. 2024-05-10T03:35:55.631 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T03:35:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.911+0000 7f787c600640 1 Processor -- start 2024-05-10T03:35:55.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.911+0000 7f787c600640 1 -- start start 2024-05-10T03:35:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78740b4810 0x7f78740cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78740d1300 0x7f78740d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f78740cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740b8d50 con 0x7f78740b4810 2024-05-10T03:35:55.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740b4c20 con 0x7f78740d1300 2024-05-10T03:35:55.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f787c600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b6100 con 0x7f78740d1c50 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f7879c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78740b4810 0x7f78740cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f7879c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78740b4810 0x7f78740cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40760/0 (socket says 172.21.15.55:40760) 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.914+0000 7f7879c00640 1 -- 172.21.15.55:0/3504174792 learned_addr learned my addr 172.21.15.55:0/3504174792 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f787a600640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f78740cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879200640 1 --2- 172.21.15.55:0/3504174792 >> v2:172.21.15.162:3300/0 conn(0x7f78740d1300 0x7f78740d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879c00640 1 -- 172.21.15.55:0/3504174792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 msgr2=0x7f78740cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879c00640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f78740cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879c00640 1 -- 172.21.15.55:0/3504174792 >> v2:172.21.15.162:3300/0 conn(0x7f78740d1300 msgr2=0x7f78740d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879c00640 1 --2- 172.21.15.55:0/3504174792 >> v2:172.21.15.162:3300/0 conn(0x7f78740d1300 0x7f78740d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.915+0000 7f7879c00640 1 -- 172.21.15.55:0/3504174792 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740b66c0 con 0x7f78740b4810 2024-05-10T03:35:55.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.917+0000 7f7879c00640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f78740cea80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f787406d350 tx=0x7f786802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f483f55a34b6089c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:55.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.917+0000 7f7873e00640 1 -- 172.21.15.55:0/3504174792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786802ae90 con 0x7f78740b4810 2024-05-10T03:35:55.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.917+0000 7f7873e00640 1 -- 172.21.15.55:0/3504174792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868032440 con 0x7f78740b4810 2024-05-10T03:35:55.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.917+0000 7f7873e00640 1 -- 172.21.15.55:0/3504174792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680327d0 con 0x7f78740b4810 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 -- 172.21.15.55:0/3504174792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 msgr2=0x7f78740cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f78740cea80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f787406d350 tx=0x7f786802a180 comp rx=0 tx=0).stop 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 -- 172.21.15.55:0/3504174792 shutdown_connections 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f78740cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 --2- 172.21.15.55:0/3504174792 >> v2:172.21.15.162:3300/0 conn(0x7f78740d1300 0x7f78740d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 --2- 172.21.15.55:0/3504174792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f78740cea80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 -- 172.21.15.55:0/3504174792 >> 172.21.15.55:0/3504174792 conn(0x7f78740d32b0 msgr2=0x7f78740d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 -- 172.21.15.55:0/3504174792 shutdown_connections 2024-05-10T03:35:55.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.918+0000 7f787c600640 1 -- 172.21.15.55:0/3504174792 wait complete. 2024-05-10T03:35:55.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.921+0000 7f787c600640 1 Processor -- start 2024-05-10T03:35:55.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.922+0000 7f787c600640 1 -- start start 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78740d1300 0x7f787415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f7874160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7874160f10 con 0x7f78740b4810 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b1fd0 con 0x7f78740d1300 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f787c600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787409dc50 con 0x7f78740d1c50 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f7879c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f7879c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40766/0 (socket says 172.21.15.55:40766) 2024-05-10T03:35:55.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.924+0000 7f7879c00640 1 -- 172.21.15.55:0/3537310141 learned_addr learned my addr 172.21.15.55:0/3537310141 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:55.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879200640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78740d1300 0x7f787415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f787a600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f7874160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 -- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 msgr2=0x7f7874160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f7874160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 -- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78740d1300 msgr2=0x7f787415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78740d1300 0x7f787415fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 -- 172.21.15.55:0/3537310141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680094d0 con 0x7f78740b4810 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f787a600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f7874160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7879c00640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f786802a1c0 tx=0x7f78680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868030850 con 0x7f78740b4810 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78680309b0 con 0x7f78740b4810 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868030d60 con 0x7f78740b4810 2024-05-10T03:35:55.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.925+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787409dea0 con 0x7f78740b4810 2024-05-10T03:35:55.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.926+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78740a02b0 con 0x7f78740b4810 2024-05-10T03:35:55.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.927+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7874067460 con 0x7f78740b4810 2024-05-10T03:35:55.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.928+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7868039040 con 0x7f78740b4810 2024-05-10T03:35:55.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.929+0000 7f7871600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 0x7f785406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:55.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.929+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6079+0+0 (secure 0 0 0) 0x7f78680aa060 con 0x7f78740b4810 2024-05-10T03:35:55.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.930+0000 7f7879200640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 0x7f785406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:55.934 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.931+0000 7f7879200640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 0x7f785406d140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7860006fd0 tx=0x7f7860008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:55.934+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7868075040 con 0x7f78740b4810 2024-05-10T03:35:56.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.146+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f78740ce860 con 0x7f78740b4810 2024-05-10T03:35:56.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.586+0000 7f7871600640 1 -- 172.21.15.55:0/3537310141 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7868078660 con 0x7f78740b4810 2024-05-10T03:35:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-05-10T03:35:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.592+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 msgr2=0x7f785406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.592+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 0x7f785406d140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7860006fd0 tx=0x7f7860008040 comp rx=0 tx=0).stop 2024-05-10T03:35:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.592+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 msgr2=0x7f7874151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.592+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f786802a1c0 tx=0x7f78680046c0 comp rx=0 tx=0).stop 2024-05-10T03:35:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 shutdown_connections 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78740d1c50 0x7f7874160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f785406ac30 0x7f785406d140 secure :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7860006fd0 tx=0x7f7860008040 comp rx=0 tx=0).stop 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78740d1300 0x7f787415fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 --2- 172.21.15.55:0/3537310141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78740b4810 0x7f7874151830 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 >> 172.21.15.55:0/3537310141 conn(0x7f78740d32b0 msgr2=0x7f787406ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.593+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 shutdown_connections 2024-05-10T03:35:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:56.594+0000 7f787c600640 1 -- 172.21.15.55:0/3537310141 wait complete. 2024-05-10T03:35:56.638 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:35:56.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:35:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on skipped message = False 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-05-10T03:35:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-05-10T03:35:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-05-10T03:35:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-05-10T03:35:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-05-10T03:35:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-05-10T03:35:56.771 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-05-10T03:35:56.772 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-05-10T03:35:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2024-05-10T03:35:56.774 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.775 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.776 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on skipped message = False 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = 340d5548-dd78-4f40-a3a8-44ee99f8e0d8 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2024-05-10T03:35:56.777 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2024-05-10T03:35:56.778 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2024-05-10T03:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 20 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 20 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 10 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:56.780 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2024-05-10T03:35:56.781 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2024-05-10T03:35:56.782 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:35:56.783 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds balancer = 20 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2024-05-10T03:35:56.784 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2024-05-10T03:35:56.785 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2024-05-10T03:35:56.785 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2024-05-10T03:35:56.785 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2024-05-10T03:35:56.802 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-10T03:35:57.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.096+0000 7f5d83200640 1 Processor -- start 2024-05-10T03:35:57.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.097+0000 7f5d83200640 1 -- start start 2024-05-10T03:35:57.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.098+0000 7f5d83200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d7c0cc2d0 0x7f5d7c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.098+0000 7f5d83200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d7c0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.098+0000 7f5d83200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c06c120 con 0x7f5d7c141900 2024-05-10T03:35:57.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d83200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ce860 con 0x7f5d7c0cc2d0 2024-05-10T03:35:57.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d83200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ceb40 con 0x7f5d7c141d10 2024-05-10T03:35:57.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d81e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5d7c141900 0x7f5d7c0cbbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51988/0 (socket says 172.21.15.55:51988) 2024-05-10T03:35:57.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d81e00640 1 -- 172.21.15.55:0/85622699 learned_addr learned my addr 172.21.15.55:0/85622699 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d81400640 1 --2- 172.21.15.55:0/85622699 >> v2:172.21.15.162:3300/0 conn(0x7f5d7c0cc2d0 0x7f5d7c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.099+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785287300 0 0) 0x7f5d7c06c120 con 0x7f5d7c141900 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d64003520 con 0x7f5d7c141900 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d82800640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d7c0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499412455 0 0) 0x7f5d64003520 con 0x7f5d7c141900 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 msgr2=0x7f5d7c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d7c0cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 >> v2:172.21.15.162:3300/0 conn(0x7f5d7c0cc2d0 msgr2=0x7f5d7c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 --2- 172.21.15.55:0/85622699 >> v2:172.21.15.162:3300/0 conn(0x7f5d7c0cc2d0 0x7f5d7c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c06c370 con 0x7f5d7c141900 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f5d70001670 con 0x7f5d7c141900 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 >> v1:172.21.15.55:6789/0 conn(0x7f5d7c141900 legacy=0x7f5d7c0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d640039e0 0x7f5d64003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d64004d00 0x7f5d64005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ce860 con 0x7f5d64004d00 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ceb40 con 0x7f5d640039e0 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.100+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6400bb30 con 0x7f5d7c141d10 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d64004d00 0x7f5d64005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d81400640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d81e00640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d640039e0 0x7f5d64003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 -- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 msgr2=0x7f5d640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c141d10 0x7f5d640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 -- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d640039e0 msgr2=0x7f5d64003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d640039e0 0x7f5d64003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.101+0000 7f5d82800640 1 -- 172.21.15.55:0/85622699 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64003520 con 0x7f5d64004d00 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.102+0000 7f5d82800640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d64004d00 0x7f5d64005110 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06edf0 tx=0x7f5d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9084316eeb8476de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7802f630 con 0x7f5d64004d00 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7802f790 con 0x7f5d64004d00 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d80a00640 1 -- 172.21.15.55:0/85622699 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7802fb40 con 0x7f5d64004d00 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d83200640 1 -- 172.21.15.55:0/85622699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d64004d00 msgr2=0x7f5d64005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d83200640 1 --2- 172.21.15.55:0/85622699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d64004d00 0x7f5d64005110 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06edf0 tx=0x7f5d640057f0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d82800640 1 -- 172.21.15.55:0/85622699 reap_dead start 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d83200640 1 -- 172.21.15.55:0/85622699 shutdown_connections 2024-05-10T03:35:57.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d83200640 1 -- 172.21.15.55:0/85622699 >> 172.21.15.55:0/85622699 conn(0x7f5d7c0b1290 msgr2=0x7f5d7c06a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:57.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.103+0000 7f5d83200640 1 -- 172.21.15.55:0/85622699 shutdown_connections 2024-05-10T03:35:57.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.104+0000 7f5d83200640 1 -- 172.21.15.55:0/85622699 wait complete. 2024-05-10T03:35:57.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.106+0000 7f5d83200640 1 Processor -- start 2024-05-10T03:35:57.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.106+0000 7f5d83200640 1 -- start start 2024-05-10T03:35:57.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d64004d00 0x7f5d7c13c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 0x7f5d7c1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c155fa0 con 0x7f5d7c13cf30 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c13cd50 con 0x7f5d64004d00 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d83200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0af570 con 0x7f5d7c155830 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d82800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d82800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40774/0 (socket says 172.21.15.55:40774) 2024-05-10T03:35:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d82800640 1 -- 172.21.15.55:0/3336959176 learned_addr learned my addr 172.21.15.55:0/3336959176 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d81e00640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d64004d00 0x7f5d7c13c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.108+0000 7f5d81400640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 0x7f5d7c1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 -- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 msgr2=0x7f5d7c1534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 0x7f5d7c1534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 -- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d64004d00 msgr2=0x7f5d7c13c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d64004d00 0x7f5d7c13c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 -- 172.21.15.55:0/3336959176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0af850 con 0x7f5d7c13cf30 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d81400640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 0x7f5d7c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d82800640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007420 tx=0x7f5d78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7803e070 con 0x7f5d7c13cf30 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d78038660 con 0x7f5d7c13cf30 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780389d0 con 0x7f5d7c13cf30 2024-05-10T03:35:57.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.109+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c0afb30 con 0x7f5d7c13cf30 2024-05-10T03:35:57.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.110+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c15a7b0 con 0x7f5d7c13cf30 2024-05-10T03:35:57.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.111+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c15a970 con 0x7f5d7c13cf30 2024-05-10T03:35:57.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.112+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5d78038b30 con 0x7f5d7c13cf30 2024-05-10T03:35:57.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.113+0000 7f5d76000640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 0x7f5d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.113+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f5d780aa510 con 0x7f5d7c13cf30 2024-05-10T03:35:57.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.114+0000 7f5d81e00640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 0x7f5d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.115+0000 7f5d81e00640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 0x7f5d6006d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d70001eb0 tx=0x7f5d70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.120+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5d78075610 con 0x7f5d7c13cf30 2024-05-10T03:35:57.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.321+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5d7c04e290 con 0x7f5d7c13cf30 2024-05-10T03:35:57.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.331+0000 7f5d76000640 1 -- 172.21.15.55:0/3336959176 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5d78009bc0 con 0x7f5d7c13cf30 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 msgr2=0x7f5d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 0x7f5d6006d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d70001eb0 tx=0x7f5d70009040 comp rx=0 tx=0).stop 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 msgr2=0x7f5d7c153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007420 tx=0x7f5d78004ab0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 shutdown_connections 2024-05-10T03:35:57.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d7c13cf30 0x7f5d7c153a10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d7c155830 0x7f5d7c1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f5d6006ac80 0x7f5d6006d190 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 --2- 172.21.15.55:0/3336959176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d64004d00 0x7f5d7c13c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.334+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 >> 172.21.15.55:0/3336959176 conn(0x7f5d7c0b1290 msgr2=0x7f5d7c13f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.335+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 shutdown_connections 2024-05-10T03:35:57.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.335+0000 7f5d83200640 1 -- 172.21.15.55:0/3336959176 wait complete. 2024-05-10T03:35:57.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-05-10T03:35:57.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.644+0000 7fbd1b600640 1 Processor -- start 2024-05-10T03:35:57.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.644+0000 7fbd1b600640 1 -- start start 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd14161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 0x7fbd140719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd140adbc0 0x7fbd14071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd141623a0 con 0x7fbd140adbc0 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140cdd60 con 0x7fbd14161c70 2024-05-10T03:35:57.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.646+0000 7fbd1b600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140ce040 con 0x7fbd140ad7b0 2024-05-10T03:35:57.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.647+0000 7fbd19600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd140adbc0 0x7fbd14071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.647+0000 7fbd19600640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd140adbc0 0x7fbd14071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40790/0 (socket says 172.21.15.55:40790) 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.647+0000 7fbd19600640 1 -- 172.21.15.55:0/664416716 learned_addr learned my addr 172.21.15.55:0/664416716 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.647+0000 7fbd18c00640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd14161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.647+0000 7fbd13e00640 1 --2- 172.21.15.55:0/664416716 >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 0x7fbd140719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd19600640 1 -- 172.21.15.55:0/664416716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 msgr2=0x7fbd14161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd19600640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd14161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd19600640 1 -- 172.21.15.55:0/664416716 >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 msgr2=0x7fbd140719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd19600640 1 --2- 172.21.15.55:0/664416716 >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 0x7fbd140719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd19600640 1 -- 172.21.15.55:0/664416716 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140ce320 con 0x7fbd140adbc0 2024-05-10T03:35:57.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.648+0000 7fbd13e00640 1 --2- 172.21.15.55:0/664416716 >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 0x7fbd140719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:35:57.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.650+0000 7fbd19600640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd14071f20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406d200 tx=0x7fbd0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f1655614a3aa96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.650+0000 7fbd13400640 1 -- 172.21.15.55:0/664416716 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c02ae90 con 0x7fbd140adbc0 2024-05-10T03:35:57.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.650+0000 7fbd13400640 1 -- 172.21.15.55:0/664416716 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c032440 con 0x7fbd140adbc0 2024-05-10T03:35:57.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.650+0000 7fbd13400640 1 -- 172.21.15.55:0/664416716 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0327d0 con 0x7fbd140adbc0 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 -- 172.21.15.55:0/664416716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 msgr2=0x7fbd14071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd14071f20 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fbd1406d200 tx=0x7fbd0c02a140 comp rx=0 tx=0).stop 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 -- 172.21.15.55:0/664416716 shutdown_connections 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd14071f20 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 --2- 172.21.15.55:0/664416716 >> v2:172.21.15.162:3300/0 conn(0x7fbd14161c70 0x7fbd140719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 --2- 172.21.15.55:0/664416716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd14161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 -- 172.21.15.55:0/664416716 >> 172.21.15.55:0/664416716 conn(0x7fbd140cbb90 msgr2=0x7fbd140cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:57.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 -- 172.21.15.55:0/664416716 shutdown_connections 2024-05-10T03:35:57.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.651+0000 7fbd1b600640 1 -- 172.21.15.55:0/664416716 wait complete. 2024-05-10T03:35:57.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.654+0000 7fbd1b600640 1 Processor -- start 2024-05-10T03:35:57.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.654+0000 7fbd1b600640 1 -- start start 2024-05-10T03:35:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd141f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 0x7fbd141f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14142ee0 con 0x7fbd140adbc0 2024-05-10T03:35:57.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14144b20 con 0x7fbd14161c70 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd1b600640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd141450b0 con 0x7fbd140ad7b0 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd13e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd13e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40798/0 (socket says 172.21.15.55:40798) 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd13e00640 1 -- 172.21.15.55:0/3169060167 learned_addr learned my addr 172.21.15.55:0/3169060167 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.656+0000 7fbd18c00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd141f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd19600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 0x7fbd141f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 -- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 msgr2=0x7fbd141f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd141f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 -- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 msgr2=0x7fbd141f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 0x7fbd141f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 -- 172.21.15.55:0/3169060167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0094d0 con 0x7fbd140adbc0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd19600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 0x7fbd141f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd13e00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d7d0 tx=0x7fbd0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00004030 con 0x7fbd140adbc0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00004190 con 0x7fbd140adbc0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.657+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd141f7810 con 0x7fbd140adbc0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.658+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00004520 con 0x7fbd140adbc0 2024-05-10T03:35:57.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.658+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd141f7ab0 con 0x7fbd140adbc0 2024-05-10T03:35:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.659+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd14066f60 con 0x7fbd140adbc0 2024-05-10T03:35:57.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.660+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd000096a0 con 0x7fbd140adbc0 2024-05-10T03:35:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.661+0000 7fbd10c00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 0x7fbcf406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:35:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.662+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fbd00089470 con 0x7fbd140adbc0 2024-05-10T03:35:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.662+0000 7fbd18c00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 0x7fbcf406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:35:57.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.663+0000 7fbd18c00640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 0x7fbcf406d140 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbd04004450 tx=0x7fbd04003a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:35:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.667+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbd00054660 con 0x7fbd140adbc0 2024-05-10T03:35:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:57.941+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) v1 -- 0x7fbd140cb350 con 0x7fbd140adbc0 2024-05-10T03:35:58.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.619+0000 7fbd10c00640 1 -- 172.21.15.55:0/3169060167 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v380) v1 ==== 145+0+0 (secure 0 0 0) 0x7fbd00057c80 con 0x7fbd140adbc0 2024-05-10T03:35:58.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 msgr2=0x7fbcf406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:58.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 0x7fbcf406d140 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbd04004450 tx=0x7fbd04003a30 comp rx=0 tx=0).stop 2024-05-10T03:35:58.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 msgr2=0x7fbd141585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:35:58.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d7d0 tx=0x7fbd0000dc90 comp rx=0 tx=0).stop 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 shutdown_connections 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fbcf406ac30 0x7fbcf406d140 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd14161c70 0x7fbd141f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd140adbc0 0x7fbd141585e0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 --2- 172.21.15.55:0/3169060167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd140ad7b0 0x7fbd141f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:35:58.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 >> 172.21.15.55:0/3169060167 conn(0x7fbd140cbb90 msgr2=0x7fbd140af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:35:58.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 shutdown_connections 2024-05-10T03:35:58.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:35:58.623+0000 7fbd1b600640 1 -- 172.21.15.55:0/3169060167 wait complete. 2024-05-10T03:35:58.662 INFO:tasks.cephfs_test_runner:That when many sessions are updated, they are correctly ... ok 2024-05-10T03:35:58.662 INFO:tasks.cephfs_test_runner: 2024-05-10T03:35:58.662 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-10T03:35:58.663 INFO:tasks.cephfs_test_runner:Ran 6 tests in 310.553s 2024-05-10T03:35:58.663 INFO:tasks.cephfs_test_runner: 2024-05-10T03:35:58.663 INFO:tasks.cephfs_test_runner:OK (skipped=2) 2024-05-10T03:35:58.663 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-10T03:35:58.675 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T03:35:58.684 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T03:35:58.684 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:58.685 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:58.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:35:58.743 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-05-10T03:35:58.743 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:58.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-10T03:35:58.800 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:58.800 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:58.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:35:58.859 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-05-10T03:35:58.859 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T03:35:58.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-10T03:35:58.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:58.915 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:35:58.976 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T03:35:58.976 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-10T03:35:58.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:58.978 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T03:35:59.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.072 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-10T03:35:59.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.148 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-10T03:35:59.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.272 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-05-10T03:35:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-05-10T03:35:59.438 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2024-05-10T03:35:59.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:35:59.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.442 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:35:59.499 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-10T03:35:59.499 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-10T03:35:59.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-10T03:35:59.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-10T03:35:59.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.664 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-05-10T03:35:59.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.775 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-05-10T03:35:59.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T03:35:59.817 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-05-10T03:35:59.930 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2024-05-10T03:35:59.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:35:59.933 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-10T03:35:59.943 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-10T03:35:59.943 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:36:00.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.237+0000 7f80a5000640 1 Processor -- start 2024-05-10T03:36:00.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.238+0000 7f80a5000640 1 -- start start 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f80a5000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f80a006d7e0 0x7f80a00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f80a5000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a006d350 con 0x7f80a0144710 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f80a5000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a00b36d0 con 0x7f80a0144120 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f80a5000640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a0144530 con 0x7f80a006d7e0 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f809f400640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f80a0144710 0x7f80a00b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52000/0 (socket says 172.21.15.55:52000) 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f809f400640 1 -- 172.21.15.55:0/3044683116 learned_addr learned my addr 172.21.15.55:0/3044683116 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.240+0000 7f809fe00640 1 --2- 172.21.15.55:0/3044683116 >> v2:172.21.15.162:3300/0 conn(0x7f80a006d7e0 0x7f80a00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530942298 0 0) 0x7f80a006d350 con 0x7f80a0144710 2024-05-10T03:36:00.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8084003520 con 0x7f80a0144710 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749059532 0 0) 0x7f8084003520 con 0x7f80a0144710 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 >> v2:172.21.15.162:3300/0 conn(0x7f80a006d7e0 msgr2=0x7f80a00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 --2- 172.21.15.55:0/3044683116 >> v2:172.21.15.162:3300/0 conn(0x7f80a006d7e0 0x7f80a00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 >> v1:172.21.15.162:6789/0 conn(0x7f80a0144120 legacy=0x7f80a00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a006c740 con 0x7f80a0144710 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f8090001670 con 0x7f80a0144710 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 >> v1:172.21.15.55:6789/0 conn(0x7f80a0144710 legacy=0x7f80a00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 0x7f8084003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80840047c0 0x7f8084004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8084005360 0x7f808400b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0144530 con 0x7f80840039e0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808400c2c0 con 0x7f8084005360 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.241+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808400c480 con 0x7f80840047c0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 0x7f8084003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809fe00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8084005360 0x7f808400b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f8096a00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80840047c0 0x7f8084004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 -- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80840047c0 msgr2=0x7f8084004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80840047c0 0x7f8084004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 -- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8084005360 msgr2=0x7f808400b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8084005360 0x7f808400b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809f400640 1 -- 172.21.15.55:0/3044683116 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084003520 con 0x7f80840039e0 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809fe00640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8084005360 0x7f808400b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:00.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.242+0000 7f809fe00640 1 -- 172.21.15.55:0/3044683116 reap_dead start 2024-05-10T03:36:00.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.243+0000 7f809f400640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 0x7f8084003e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f809000c210 tx=0x7f80840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f626de670ddb8d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.243+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090005ad0 con 0x7f80840039e0 2024-05-10T03:36:00.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.243+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809002d070 con 0x7f80840039e0 2024-05-10T03:36:00.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.243+0000 7f809ea00640 1 -- 172.21.15.55:0/3044683116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090007070 con 0x7f80840039e0 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 -- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 msgr2=0x7f8084003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 0x7f8084003e50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f809000c210 tx=0x7f80840057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 -- 172.21.15.55:0/3044683116 shutdown_connections 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 --2- 172.21.15.55:0/3044683116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80840039e0 0x7f8084003e50 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 -- 172.21.15.55:0/3044683116 >> 172.21.15.55:0/3044683116 conn(0x7f80a006a110 msgr2=0x7f80a006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 -- 172.21.15.55:0/3044683116 shutdown_connections 2024-05-10T03:36:00.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.244+0000 7f80a5000640 1 -- 172.21.15.55:0/3044683116 wait complete. 2024-05-10T03:36:00.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.247+0000 7f80a5000640 1 Processor -- start 2024-05-10T03:36:00.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.247+0000 7f80a5000640 1 -- start start 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80840039e0 0x7f80a0141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80a01419a0 0x7f80a0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a01f3f60 con 0x7f80a015ac20 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0140110 con 0x7f80840039e0 2024-05-10T03:36:00.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f80a5000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a01403d0 con 0x7f80a01419a0 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f8096a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f8096a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40812/0 (socket says 172.21.15.55:40812) 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f8096a00640 1 -- 172.21.15.55:0/1149546248 learned_addr learned my addr 172.21.15.55:0/1149546248 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f809f400640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80840039e0 0x7f80a0141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f809fe00640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80a01419a0 0x7f80a0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f8096a00640 1 -- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80a01419a0 msgr2=0x7f80a0141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.249+0000 7f8096a00640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80a01419a0 0x7f80a0141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8096a00640 1 -- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80840039e0 msgr2=0x7f80a0141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8096a00640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80840039e0 0x7f80a0141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8096a00640 1 -- 172.21.15.55:0/1149546248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a01406b0 con 0x7f80a015ac20 2024-05-10T03:36:00.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8096a00640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f808800d480 tx=0x7f808800d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088005480 con 0x7f80a015ac20 2024-05-10T03:36:00.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088003610 con 0x7f80a015ac20 2024-05-10T03:36:00.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a0140990 con 0x7f80a015ac20 2024-05-10T03:36:00.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880039a0 con 0x7f80a015ac20 2024-05-10T03:36:00.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.250+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a01416d0 con 0x7f80a015ac20 2024-05-10T03:36:00.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.252+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a0066ce0 con 0x7f80a015ac20 2024-05-10T03:36:00.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.253+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8088003b00 con 0x7f80a015ac20 2024-05-10T03:36:00.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.253+0000 7f8097e00640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 0x7f807c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.254+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f8088089270 con 0x7f80a015ac20 2024-05-10T03:36:00.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.254+0000 7f809f400640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 0x7f807c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.256+0000 7f809f400640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 0x7f807c06d110 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8090032000 tx=0x7f809002d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.259+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f80880544d0 con 0x7f80a015ac20 2024-05-10T03:36:00.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.456+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80a006cd00 con 0x7f807c06ac00 2024-05-10T03:36:00.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.463+0000 7f8097e00640 1 -- 172.21.15.55:0/1149546248 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48696 (secure 0 0 0) 0x7f80900016d0 con 0x7f807c06ac00 2024-05-10T03:36:00.467 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:00.467 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 msgr2=0x7f807c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 0x7f807c06d110 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8090032000 tx=0x7f809002d000 comp rx=0 tx=0).stop 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 msgr2=0x7f80a01f3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f808800d480 tx=0x7f808800d940 comp rx=0 tx=0).stop 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 shutdown_connections 2024-05-10T03:36:00.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80a01419a0 0x7f80a0141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80a015ac20 0x7f80a01f3350 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f807c06ac00 0x7f807c06d110 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 --2- 172.21.15.55:0/1149546248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80840039e0 0x7f80a0141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.465+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 >> 172.21.15.55:0/1149546248 conn(0x7f80a006a110 msgr2=0x7f80a006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.466+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 shutdown_connections 2024-05-10T03:36:00.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.466+0000 7f80a5000640 1 -- 172.21.15.55:0/1149546248 wait complete. 2024-05-10T03:36:00.495 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-05-10T03:35:59.099517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261028,"kb_used_data":12088,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713692,"statfs":{"total":773094113280,"available":772826820608,"internally_reserved":0,"allocated":12378112,"data_stored":6314589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26094,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-170,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002532"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-10T03:35:55.618556+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:35:55.618556+0000","last_peered":"2024-05-10T03:35:55.618556+0000","last_clean":"2024-05-10T03:35:55.618556+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:35:55.618556+0000","last_undegraded":"2024-05-10T03:35:55.618556+0000","last_fullsized":"2024-05-10T03:35:55.618556+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1986560,"data_stored":1229751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.327,"15min":1.351},"min":{"1min":0.78100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8140000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.1830000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1459999999999999,"15min":1.143},"min":{"1min":0.78400000000000003,"5min":0.65400000000000003,"15min":0.59299999999999997},"max":{"1min":1.4850000000000001,"5min":1.853,"15min":1.853},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2649999999999999},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.679,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.095}]},{"osd":2,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2869999999999999,"15min":1.357},"min":{"1min":0.90000000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.9079999999999999,"5min":2.1349999999999998,"15min":3.0289999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2929999999999999,"15min":1.3999999999999999},"min":{"1min":0.80800000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9930000000000001,"5min":2.4649999999999999,"15min":2.6000000000000001},"last":1.6699999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.222,"15min":2.222},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.355,"15min":1.335},"min":{"1min":0.94099999999999995,"5min":0.71899999999999997,"15min":0.70799999999999996},"max":{"1min":1.675,"5min":2.367,"15min":2.367},"last":1.899}]},{"osd":4,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3740000000000001,"15min":1.4450000000000001},"min":{"1min":0.61099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.7110000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.498},{"interface":"front","average":{"1min":1.429,"5min":1.49,"15min":1.4970000000000001},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.0680000000000001,"5min":2.323,"15min":2.323},"last":1.7410000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.5169999999999999},"min":{"1min":0.70199999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4660000000000002,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.788},{"interface":"front","average":{"1min":1.625,"5min":1.6619999999999999,"15min":1.6870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.601,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.649}]},{"osd":6,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.641,"15min":1.7010000000000001},"min":{"1min":0.73899999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.639,"15min":1.6359999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.2250000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.8380000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.1839999999999999,"15min":1.1080000000000001},"min":{"1min":0.80200000000000005,"5min":0.57799999999999996,"15min":0.48699999999999999},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2110000000000001,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.7170000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.4399999999999999},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.60399999999999998},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":4.1429999999999998},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3919999999999999,"15min":1.2809999999999999},"min":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.53900000000000003},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.083},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3400000000000001,"15min":1.474},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.569},"last":1.659}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4119999999999999,"15min":1.5680000000000001},"min":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.488,"5min":3.488,"15min":4.0869999999999997},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3759999999999999,"15min":1.327},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.0209999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.468,"15min":1.4399999999999999},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5009999999999999,"15min":1.5640000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.966,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.7729999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.488,"15min":1.4199999999999999},"min":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.345},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.635,"15min":1.627},"min":{"1min":0.70499999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.415}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.655,"15min":1.6679999999999999},"min":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1200000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.6659999999999999,"15min":1.6599999999999999},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.64000000000000001},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1619999999999999,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7410000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.448}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.153,"15min":1.143},"min":{"1min":0.89400000000000002,"5min":0.66300000000000003,"15min":0.57699999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.206,"15min":1.22},"min":{"1min":0.86499999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.954},"last":1.081}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.216,"15min":1.2170000000000001},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.7490000000000001,"15min":2.359},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.304,"15min":1.3300000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7909999999999999,"5min":1.913,"15min":1.913},"last":1.3740000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.347,"15min":1.3520000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4139999999999999,"15min":1.3240000000000001},"min":{"1min":1.1850000000000001,"5min":0.76800000000000002,"15min":0.68200000000000005},"max":{"1min":1.9390000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.321}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.91300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5429999999999999,"15min":1.587},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":3.7759999999999998},"last":1.266}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5329999999999999,"15min":1.518},"min":{"1min":0.875,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9670000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.595,"5min":1.556,"15min":1.554},"min":{"1min":0.86299999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.508}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.732,"15min":1.79},"min":{"1min":1.167,"5min":0.877,"15min":0.753},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":4.0899999999999999},"last":1.609},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.653,"15min":1.6240000000000001},"min":{"1min":1.4299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.036,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.129}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.153},{"interface":"front","average":{"1min":1.238,"5min":1.151,"15min":1.1439999999999999},"min":{"1min":0.72299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.681,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.3500000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2130000000000001,"15min":1.1719999999999999},"min":{"1min":0.80600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.679,"5min":1.734,"15min":2.3839999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.2490000000000001,"15min":1.2729999999999999},"min":{"1min":0.997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1619999999999999},"last":1.3220000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.242,"15min":1.2709999999999999},"min":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.702,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.3740000000000001,"15min":1.399},"min":{"1min":0.94099999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.29},"last":1.415}]},{"osd":3,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.341,"15min":1.4370000000000001},"min":{"1min":0.753,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8859999999999999,"5min":2.379,"15min":2.6779999999999999},"last":0.876},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5269999999999999,"15min":1.5680000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.7709999999999999},"last":1.252}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4810000000000001,"15min":1.4990000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.512},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4119999999999999,"15min":1.5209999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0579999999999998,"5min":2.5510000000000002,"15min":2.8109999999999999},"last":1.113}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.5249999999999999,"15min":1.5720000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.141,"5min":2.4769999999999999,"15min":2.8530000000000002},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.619,"15min":1.742},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.597,"5min":2.597,"15min":5.3529999999999998},"last":1.631}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5880000000000001,"15min":1.6679999999999999},"min":{"1min":0.63600000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.177,"5min":2.6160000000000001,"15min":3.3479999999999999},"last":1.488},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.694,"15min":1.7290000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.556,"5min":2.6349999999999998,"15min":2.8700000000000001},"last":1.9159999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.95999999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2130000000000001,"5min":1.6200000000000001,"15min":1.708},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.603},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.625,"5min":0.377,"15min":0.377},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.3140000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1559999999999999,"15min":1.177},"min":{"1min":0.63100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8360000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.27},{"interface":"front","average":{"1min":1.349,"5min":1.2390000000000001,"15min":1.335},"min":{"1min":0.72299999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.762,"5min":1.919,"15min":2.2599999999999998},"last":1.534}]},{"osd":4,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3540000000000001,"15min":1.371},"min":{"1min":0.93799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.881,"5min":2.0910000000000002,"15min":2.3610000000000002},"last":1.206},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4299999999999999,"15min":1.3919999999999999},"min":{"1min":0.95499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0790000000000002,"5min":2.1349999999999998,"15min":2.464},"last":1.1120000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4810000000000001,"15min":1.6299999999999999},"min":{"1min":0.97799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.839,"5min":2.4399999999999999,"15min":3.0470000000000002},"last":1.655},{"interface":"front","average":{"1min":1.456,"5min":1.446,"15min":1.478},"min":{"1min":0.998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9399999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.544,"15min":1.569},"min":{"1min":0.88200000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.7569999999999999},"min":{"1min":0.92900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9319999999999999,"5min":2.4729999999999999,"15min":3.2280000000000002},"last":1.8009999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.51,"15min":1.5720000000000001},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.0670000000000002,"5min":2.2229999999999999,"15min":2.4399999999999999},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5780000000000001,"15min":1.6379999999999999},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.1429999999999998,"5min":2.29,"15min":2.6190000000000002},"last":1.6140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.88,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.434,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.643,"5min":1.746,"15min":1.746},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1699999999999999},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.1030000000000002},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.052,"15min":1.1060000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":2.169},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1930000000000001,"15min":1.244},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8380000000000001,"5min":1.9910000000000001,"15min":2.2229999999999999},"last":1.45}]},{"osd":4,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.506},"min":{"1min":0.71299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.8050000000000002},"last":1.847},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3580000000000001,"15min":1.504},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.6150000000000002},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9950000000000001,"5min":2.173,"15min":2.508},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5589999999999999,"15min":1.607},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.379,"5min":2.379,"15min":2.6669999999999998},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.504,"15min":1.542},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.7280000000000002},"last":1.619},{"interface":"front","average":{"1min":1.52,"5min":1.4950000000000001,"15min":1.569},"min":{"1min":0.87,"5min":0.81899999999999995,"15min":0.78200000000000003},"max":{"1min":2.1720000000000002,"5min":2.3799999999999999,"15min":2.8860000000000001},"last":1.353}]},{"osd":7,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.579,"15min":1.639},"min":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2759999999999998,"5min":2.3149999999999999,"15min":2.8410000000000002},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.5700000000000001,"15min":1.649},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.58999999999999997},"max":{"1min":2.0870000000000002,"5min":2.468,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.89600000000000002,"15min":0.91700000000000004},"min":{"1min":0.42699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.607,"15min":1.607},"last":3.1970000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.45800000000000002},"max":{"1min":1.4770000000000001,"5min":1.996,"15min":2.2949999999999999},"last":1.4590000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96299999999999997,"15min":0.998},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":1.6910000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.171},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.627,"5min":2.0720000000000001,"15min":2.444},"last":4.7439999999999998}]},{"osd":3,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2250000000000001,"15min":1.3029999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.4129999999999998},"last":3.8090000000000002},{"interface":"front","average":{"1min":1.272,"5min":1.196,"15min":1.2749999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.7799999999999998},"last":1.3799999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.508},"min":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0230000000000001,"5min":2.3159999999999998,"15min":2.5299999999999998},"last":3.9780000000000002},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4390000000000001,"15min":1.554},"min":{"1min":0.93899999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.9610000000000001,"5min":2.1139999999999999,"15min":4.2949999999999999},"last":4.0880000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.544},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9350000000000001,"5min":2.2450000000000001,"15min":3.8029999999999999},"last":3.6299999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.603},"min":{"1min":0.90600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.1200000000000001,"5min":2.3290000000000002,"15min":2.6190000000000002},"last":5.54}]},{"osd":6,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.5469999999999999,"15min":1.6859999999999999},"min":{"1min":1.111,"5min":0.94999999999999996,"15min":0.91100000000000003},"max":{"1min":2.1549999999999998,"5min":2.4470000000000001,"15min":3.48},"last":3.1040000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5329999999999999,"15min":1.6870000000000001},"min":{"1min":1.028,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0720000000000001,"5min":2.1230000000000002,"15min":3.6880000000000002},"last":3.4900000000000002}]},{"osd":7,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6100000000000001,"15min":1.708},"min":{"1min":0.87,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.0409999999999999,"5min":2.2709999999999999,"15min":3.2869999999999999},"last":9.7850000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.5920000000000001,"15min":1.74},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.27,"5min":2.5449999999999999,"15min":4.0140000000000002},"last":1.8340000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.875,"15min":0.995},"min":{"1min":0.56899999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":2.1309999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.048,"5min":0.93899999999999995,"15min":1.137},"min":{"1min":0.77400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.532,"5min":1.6120000000000001,"15min":2.9390000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0609999999999999,"15min":1.1910000000000001},"min":{"1min":0.72799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.72,"5min":1.72,"15min":2.4390000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.0620000000000001,"15min":1.1879999999999999},"min":{"1min":0.755,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7190000000000001,"5min":1.8400000000000001,"15min":3.0009999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.014,"15min":1.089},"min":{"1min":0.66700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.9850000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2270000000000001,"15min":1.3069999999999999},"min":{"1min":0.68899999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.097,"15min":2.4750000000000001},"last":1.202}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.335,"15min":1.5649999999999999},"min":{"1min":1.071,"5min":0.745,"15min":0.745},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":3.1040000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3460000000000001,"15min":1.4179999999999999},"min":{"1min":0.872,"5min":0.84099999999999997,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":3.0790000000000002},"last":1.601}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.3759999999999999,"15min":1.4570000000000001},"min":{"1min":0.95199999999999996,"5min":0.80700000000000005,"15min":0.57899999999999996},"max":{"1min":1.841,"5min":1.9710000000000001,"15min":2.4889999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.444,"15min":1.5920000000000001},"min":{"1min":1.028,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0299999999999998,"5min":2.2120000000000002,"15min":2.9279999999999999},"last":1.7909999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.552,"15min":1.6279999999999999},"min":{"1min":1.0600000000000001,"5min":0.98099999999999998,"15min":0.90900000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.7189999999999999},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.5029999999999999,"15min":1.6919999999999999},"min":{"1min":1.167,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":3.0539999999999998},"last":1.6259999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5209999999999999,"15min":1.7370000000000001},"min":{"1min":1.3029999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.117,"5min":2.117,"15min":3.149},"last":1.738},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5249999999999999,"15min":1.569},"min":{"1min":1.2709999999999999,"5min":0.86899999999999999,"15min":0.63700000000000001},"max":{"1min":2.0409999999999999,"5min":2.1509999999999998,"15min":2.3860000000000001},"last":2.0209999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:36:00.497 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:36:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.773+0000 7f0ac8c00640 1 Processor -- start 2024-05-10T03:36:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.774+0000 7f0ac8c00640 1 -- start start 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.776+0000 7f0ac8c00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ac40b2550 0x7f0ac406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.776+0000 7f0ac8c00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac406ad20 con 0x7f0ac415fe50 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.776+0000 7f0ac8c00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac406a110 con 0x7f0ac40b2140 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.776+0000 7f0ac8c00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac406a3f0 con 0x7f0ac40b2550 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.777+0000 7f0ac1e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0ac415fe50 0x7f0ac4141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52008/0 (socket says 172.21.15.55:52008) 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.777+0000 7f0ac1e00640 1 -- 172.21.15.55:0/2969596140 learned_addr learned my addr 172.21.15.55:0/2969596140 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.777+0000 7f0ac2800640 1 --2- 172.21.15.55:0/2969596140 >> v2:172.21.15.55:3300/0 conn(0x7f0ac40b2550 0x7f0ac406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.777+0000 7f0ac2800640 1 -- 172.21.15.55:0/2969596140 >> v1:172.21.15.162:6789/0 conn(0x7f0ac40b2140 legacy=0x7f0ac4142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.777+0000 7f0ac2800640 1 -- 172.21.15.55:0/2969596140 >> v1:172.21.15.55:6789/0 conn(0x7f0ac415fe50 legacy=0x7f0ac4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.778+0000 7f0ac2800640 1 -- 172.21.15.55:0/2969596140 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac406a6d0 con 0x7f0ac40b2550 2024-05-10T03:36:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.778+0000 7f0ac1400640 1 -- 172.21.15.55:0/2969596140 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590288537 0 0) 0x7f0ac406ad20 con 0x7f0ac415fe50 2024-05-10T03:36:00.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.779+0000 7f0ac2800640 1 --2- 172.21.15.55:0/2969596140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2550 0x7f0ac406b1b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009d90 tx=0x7f0abc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7d898043a7e724 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.779+0000 7f0ac1400640 1 -- 172.21.15.55:0/2969596140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc007740 con 0x7f0ac40b2550 2024-05-10T03:36:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.779+0000 7f0ac1400640 1 -- 172.21.15.55:0/2969596140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc0078a0 con 0x7f0ac40b2550 2024-05-10T03:36:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.779+0000 7f0ac1400640 1 -- 172.21.15.55:0/2969596140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc007c30 con 0x7f0ac40b2550 2024-05-10T03:36:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 -- 172.21.15.55:0/2969596140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2550 msgr2=0x7f0ac406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/2969596140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2550 0x7f0ac406b1b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009d90 tx=0x7f0abc02a780 comp rx=0 tx=0).stop 2024-05-10T03:36:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 -- 172.21.15.55:0/2969596140 shutdown_connections 2024-05-10T03:36:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/2969596140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2550 0x7f0ac406b1b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 -- 172.21.15.55:0/2969596140 >> 172.21.15.55:0/2969596140 conn(0x7f0ac40cbb90 msgr2=0x7f0ac40cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 -- 172.21.15.55:0/2969596140 shutdown_connections 2024-05-10T03:36:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.780+0000 7f0ac8c00640 1 -- 172.21.15.55:0/2969596140 wait complete. 2024-05-10T03:36:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.782+0000 7f0ac8c00640 1 Processor -- start 2024-05-10T03:36:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.782+0000 7f0ac8c00640 1 -- start start 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 0x7f0ac40ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac415fe50 0x7f0ac40ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40a9310 con 0x7f0ac40b2140 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40acc00 con 0x7f0ac40b2550 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac8c00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac415eb90 con 0x7f0ac415fe50 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac1e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40840/0 (socket says 172.21.15.55:40840) 2024-05-10T03:36:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.784+0000 7f0ac1e00640 1 -- 172.21.15.55:0/1331858775 learned_addr learned my addr 172.21.15.55:0/1331858775 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:00.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac2800640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac415fe50 0x7f0ac40ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 -- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac415fe50 msgr2=0x7f0ac40ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0abbe00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 0x7f0ac40ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac415fe50 0x7f0ac40ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 -- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 msgr2=0x7f0ac40ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 0x7f0ac40ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 -- 172.21.15.55:0/1331858775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc009a70 con 0x7f0ac40b2140 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0abbe00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 0x7f0ac40ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac1e00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c420 tx=0x7f0ab400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40074e0 con 0x7f0ac40b2140 2024-05-10T03:36:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4007640 con 0x7f0ac40b2140 2024-05-10T03:36:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40a9c70 con 0x7f0ac40b2140 2024-05-10T03:36:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.785+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004a00 con 0x7f0ac40b2140 2024-05-10T03:36:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.786+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac40a8970 con 0x7f0ac40b2140 2024-05-10T03:36:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.787+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4066f60 con 0x7f0ac40b2140 2024-05-10T03:36:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.788+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ab4004b60 con 0x7f0ac40b2140 2024-05-10T03:36:00.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.789+0000 7f0aba000640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 0x7f0aa006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:00.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.789+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f0ab40891a0 con 0x7f0ac40b2140 2024-05-10T03:36:00.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.789+0000 7f0abbe00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 0x7f0aa006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.791+0000 7f0abbe00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 0x7f0aa006d140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0ac41333f0 tx=0x7f0aac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.793+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ab4054350 con 0x7f0ac40b2140 2024-05-10T03:36:00.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.986+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ac40668e0 con 0x7f0aa006ac30 2024-05-10T03:36:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.993+0000 7f0aba000640 1 -- 172.21.15.55:0/1331858775 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48696 (secure 0 0 0) 0x7f0ac40668e0 con 0x7f0aa006ac30 2024-05-10T03:36:00.997 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.995+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 msgr2=0x7f0aa006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 0x7f0aa006d140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0ac41333f0 tx=0x7f0aac009040 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 msgr2=0x7f0ac40a9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400c420 tx=0x7f0ab400c8e0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 shutdown_connections 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ac415fe50 0x7f0ac40ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0aa006ac30 0x7f0aa006d140 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ac40b2550 0x7f0ac40ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 --2- 172.21.15.55:0/1331858775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac40b2140 0x7f0ac40a9730 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 >> 172.21.15.55:0/1331858775 conn(0x7f0ac40cbb90 msgr2=0x7f0ac40ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 shutdown_connections 2024-05-10T03:36:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:00.996+0000 7f0ac8c00640 1 -- 172.21.15.55:0/1331858775 wait complete. 2024-05-10T03:36:01.026 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-05-10T03:35:59.099517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261028,"kb_used_data":12088,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713692,"statfs":{"total":773094113280,"available":772826820608,"internally_reserved":0,"allocated":12378112,"data_stored":6314589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26094,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-170,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002532"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-10T03:35:55.618556+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:35:55.618556+0000","last_peered":"2024-05-10T03:35:55.618556+0000","last_clean":"2024-05-10T03:35:55.618556+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:35:55.618556+0000","last_undegraded":"2024-05-10T03:35:55.618556+0000","last_fullsized":"2024-05-10T03:35:55.618556+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1986560,"data_stored":1229751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.327,"15min":1.351},"min":{"1min":0.78100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8140000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.1830000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1459999999999999,"15min":1.143},"min":{"1min":0.78400000000000003,"5min":0.65400000000000003,"15min":0.59299999999999997},"max":{"1min":1.4850000000000001,"5min":1.853,"15min":1.853},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2649999999999999},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.679,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.095}]},{"osd":2,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2869999999999999,"15min":1.357},"min":{"1min":0.90000000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.9079999999999999,"5min":2.1349999999999998,"15min":3.0289999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2929999999999999,"15min":1.3999999999999999},"min":{"1min":0.80800000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9930000000000001,"5min":2.4649999999999999,"15min":2.6000000000000001},"last":1.6699999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.222,"15min":2.222},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.355,"15min":1.335},"min":{"1min":0.94099999999999995,"5min":0.71899999999999997,"15min":0.70799999999999996},"max":{"1min":1.675,"5min":2.367,"15min":2.367},"last":1.899}]},{"osd":4,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3740000000000001,"15min":1.4450000000000001},"min":{"1min":0.61099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.7110000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.498},{"interface":"front","average":{"1min":1.429,"5min":1.49,"15min":1.4970000000000001},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.0680000000000001,"5min":2.323,"15min":2.323},"last":1.7410000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.5169999999999999},"min":{"1min":0.70199999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4660000000000002,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.788},{"interface":"front","average":{"1min":1.625,"5min":1.6619999999999999,"15min":1.6870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.601,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.649}]},{"osd":6,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.641,"15min":1.7010000000000001},"min":{"1min":0.73899999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.639,"15min":1.6359999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.2250000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.8380000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.1839999999999999,"15min":1.1080000000000001},"min":{"1min":0.80200000000000005,"5min":0.57799999999999996,"15min":0.48699999999999999},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2110000000000001,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.7170000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.4399999999999999},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.60399999999999998},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":4.1429999999999998},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3919999999999999,"15min":1.2809999999999999},"min":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.53900000000000003},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.083},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3400000000000001,"15min":1.474},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.569},"last":1.659}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4119999999999999,"15min":1.5680000000000001},"min":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.488,"5min":3.488,"15min":4.0869999999999997},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3759999999999999,"15min":1.327},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.0209999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.468,"15min":1.4399999999999999},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5009999999999999,"15min":1.5640000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.966,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.7729999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.488,"15min":1.4199999999999999},"min":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.345},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.635,"15min":1.627},"min":{"1min":0.70499999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.415}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.655,"15min":1.6679999999999999},"min":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1200000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.6659999999999999,"15min":1.6599999999999999},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.64000000000000001},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1619999999999999,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7410000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.448}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.153,"15min":1.143},"min":{"1min":0.89400000000000002,"5min":0.66300000000000003,"15min":0.57699999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.206,"15min":1.22},"min":{"1min":0.86499999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.954},"last":1.081}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.216,"15min":1.2170000000000001},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.7490000000000001,"15min":2.359},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.304,"15min":1.3300000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7909999999999999,"5min":1.913,"15min":1.913},"last":1.3740000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.347,"15min":1.3520000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4139999999999999,"15min":1.3240000000000001},"min":{"1min":1.1850000000000001,"5min":0.76800000000000002,"15min":0.68200000000000005},"max":{"1min":1.9390000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.321}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.91300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5429999999999999,"15min":1.587},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":3.7759999999999998},"last":1.266}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5329999999999999,"15min":1.518},"min":{"1min":0.875,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9670000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.595,"5min":1.556,"15min":1.554},"min":{"1min":0.86299999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.508}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.732,"15min":1.79},"min":{"1min":1.167,"5min":0.877,"15min":0.753},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":4.0899999999999999},"last":1.609},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.653,"15min":1.6240000000000001},"min":{"1min":1.4299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.036,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.129}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.153},{"interface":"front","average":{"1min":1.238,"5min":1.151,"15min":1.1439999999999999},"min":{"1min":0.72299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.681,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.3500000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2130000000000001,"15min":1.1719999999999999},"min":{"1min":0.80600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.679,"5min":1.734,"15min":2.3839999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.2490000000000001,"15min":1.2729999999999999},"min":{"1min":0.997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1619999999999999},"last":1.3220000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.242,"15min":1.2709999999999999},"min":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.702,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.3740000000000001,"15min":1.399},"min":{"1min":0.94099999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.29},"last":1.415}]},{"osd":3,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.341,"15min":1.4370000000000001},"min":{"1min":0.753,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8859999999999999,"5min":2.379,"15min":2.6779999999999999},"last":0.876},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5269999999999999,"15min":1.5680000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.7709999999999999},"last":1.252}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4810000000000001,"15min":1.4990000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.512},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4119999999999999,"15min":1.5209999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0579999999999998,"5min":2.5510000000000002,"15min":2.8109999999999999},"last":1.113}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.5249999999999999,"15min":1.5720000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.141,"5min":2.4769999999999999,"15min":2.8530000000000002},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.619,"15min":1.742},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.597,"5min":2.597,"15min":5.3529999999999998},"last":1.631}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5880000000000001,"15min":1.6679999999999999},"min":{"1min":0.63600000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.177,"5min":2.6160000000000001,"15min":3.3479999999999999},"last":1.488},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.694,"15min":1.7290000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.556,"5min":2.6349999999999998,"15min":2.8700000000000001},"last":1.9159999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.95999999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2130000000000001,"5min":1.6200000000000001,"15min":1.708},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.603},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.625,"5min":0.377,"15min":0.377},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.3140000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1559999999999999,"15min":1.177},"min":{"1min":0.63100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8360000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.27},{"interface":"front","average":{"1min":1.349,"5min":1.2390000000000001,"15min":1.335},"min":{"1min":0.72299999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.762,"5min":1.919,"15min":2.2599999999999998},"last":1.534}]},{"osd":4,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3540000000000001,"15min":1.371},"min":{"1min":0.93799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.881,"5min":2.0910000000000002,"15min":2.3610000000000002},"last":1.206},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4299999999999999,"15min":1.3919999999999999},"min":{"1min":0.95499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0790000000000002,"5min":2.1349999999999998,"15min":2.464},"last":1.1120000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4810000000000001,"15min":1.6299999999999999},"min":{"1min":0.97799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.839,"5min":2.4399999999999999,"15min":3.0470000000000002},"last":1.655},{"interface":"front","average":{"1min":1.456,"5min":1.446,"15min":1.478},"min":{"1min":0.998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9399999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.544,"15min":1.569},"min":{"1min":0.88200000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.7569999999999999},"min":{"1min":0.92900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9319999999999999,"5min":2.4729999999999999,"15min":3.2280000000000002},"last":1.8009999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.51,"15min":1.5720000000000001},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.0670000000000002,"5min":2.2229999999999999,"15min":2.4399999999999999},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5780000000000001,"15min":1.6379999999999999},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.1429999999999998,"5min":2.29,"15min":2.6190000000000002},"last":1.6140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.88,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.434,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.643,"5min":1.746,"15min":1.746},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1699999999999999},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.1030000000000002},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.052,"15min":1.1060000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":2.169},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1930000000000001,"15min":1.244},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8380000000000001,"5min":1.9910000000000001,"15min":2.2229999999999999},"last":1.45}]},{"osd":4,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.506},"min":{"1min":0.71299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.8050000000000002},"last":1.847},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3580000000000001,"15min":1.504},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.6150000000000002},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9950000000000001,"5min":2.173,"15min":2.508},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5589999999999999,"15min":1.607},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.379,"5min":2.379,"15min":2.6669999999999998},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.504,"15min":1.542},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.7280000000000002},"last":1.619},{"interface":"front","average":{"1min":1.52,"5min":1.4950000000000001,"15min":1.569},"min":{"1min":0.87,"5min":0.81899999999999995,"15min":0.78200000000000003},"max":{"1min":2.1720000000000002,"5min":2.3799999999999999,"15min":2.8860000000000001},"last":1.353}]},{"osd":7,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.579,"15min":1.639},"min":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2759999999999998,"5min":2.3149999999999999,"15min":2.8410000000000002},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.5700000000000001,"15min":1.649},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.58999999999999997},"max":{"1min":2.0870000000000002,"5min":2.468,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.89600000000000002,"15min":0.91700000000000004},"min":{"1min":0.42699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.607,"15min":1.607},"last":3.1970000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.45800000000000002},"max":{"1min":1.4770000000000001,"5min":1.996,"15min":2.2949999999999999},"last":1.4590000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96299999999999997,"15min":0.998},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":1.6910000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.171},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.627,"5min":2.0720000000000001,"15min":2.444},"last":4.7439999999999998}]},{"osd":3,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2250000000000001,"15min":1.3029999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.4129999999999998},"last":3.8090000000000002},{"interface":"front","average":{"1min":1.272,"5min":1.196,"15min":1.2749999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.7799999999999998},"last":1.3799999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.508},"min":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0230000000000001,"5min":2.3159999999999998,"15min":2.5299999999999998},"last":3.9780000000000002},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4390000000000001,"15min":1.554},"min":{"1min":0.93899999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.9610000000000001,"5min":2.1139999999999999,"15min":4.2949999999999999},"last":4.0880000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.544},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9350000000000001,"5min":2.2450000000000001,"15min":3.8029999999999999},"last":3.6299999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.603},"min":{"1min":0.90600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.1200000000000001,"5min":2.3290000000000002,"15min":2.6190000000000002},"last":5.54}]},{"osd":6,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.5469999999999999,"15min":1.6859999999999999},"min":{"1min":1.111,"5min":0.94999999999999996,"15min":0.91100000000000003},"max":{"1min":2.1549999999999998,"5min":2.4470000000000001,"15min":3.48},"last":3.1040000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5329999999999999,"15min":1.6870000000000001},"min":{"1min":1.028,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0720000000000001,"5min":2.1230000000000002,"15min":3.6880000000000002},"last":3.4900000000000002}]},{"osd":7,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6100000000000001,"15min":1.708},"min":{"1min":0.87,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.0409999999999999,"5min":2.2709999999999999,"15min":3.2869999999999999},"last":9.7850000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.5920000000000001,"15min":1.74},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.27,"5min":2.5449999999999999,"15min":4.0140000000000002},"last":1.8340000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.875,"15min":0.995},"min":{"1min":0.56899999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":2.1309999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.048,"5min":0.93899999999999995,"15min":1.137},"min":{"1min":0.77400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.532,"5min":1.6120000000000001,"15min":2.9390000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0609999999999999,"15min":1.1910000000000001},"min":{"1min":0.72799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.72,"5min":1.72,"15min":2.4390000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.0620000000000001,"15min":1.1879999999999999},"min":{"1min":0.755,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7190000000000001,"5min":1.8400000000000001,"15min":3.0009999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.014,"15min":1.089},"min":{"1min":0.66700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.9850000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2270000000000001,"15min":1.3069999999999999},"min":{"1min":0.68899999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.097,"15min":2.4750000000000001},"last":1.202}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.335,"15min":1.5649999999999999},"min":{"1min":1.071,"5min":0.745,"15min":0.745},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":3.1040000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3460000000000001,"15min":1.4179999999999999},"min":{"1min":0.872,"5min":0.84099999999999997,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":3.0790000000000002},"last":1.601}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.3759999999999999,"15min":1.4570000000000001},"min":{"1min":0.95199999999999996,"5min":0.80700000000000005,"15min":0.57899999999999996},"max":{"1min":1.841,"5min":1.9710000000000001,"15min":2.4889999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.444,"15min":1.5920000000000001},"min":{"1min":1.028,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0299999999999998,"5min":2.2120000000000002,"15min":2.9279999999999999},"last":1.7909999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.552,"15min":1.6279999999999999},"min":{"1min":1.0600000000000001,"5min":0.98099999999999998,"15min":0.90900000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.7189999999999999},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.5029999999999999,"15min":1.6919999999999999},"min":{"1min":1.167,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":3.0539999999999998},"last":1.6259999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5209999999999999,"15min":1.7370000000000001},"min":{"1min":1.3029999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.117,"5min":2.117,"15min":3.149},"last":1.738},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5249999999999999,"15min":1.569},"min":{"1min":1.2709999999999999,"5min":0.86899999999999999,"15min":0.63700000000000001},"max":{"1min":2.0409999999999999,"5min":2.1509999999999998,"15min":2.3860000000000001},"last":2.0209999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:36:01.028 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-10T03:36:01.028 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:36:01.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.309+0000 7f0d17e00640 1 Processor -- start 2024-05-10T03:36:01.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.309+0000 7f0d17e00640 1 -- start start 2024-05-10T03:36:01.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0d18071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d17e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180b39f0 con 0x7f0d181612f0 2024-05-10T03:36:01.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d17e00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180b3cd0 con 0x7f0d181549e0 2024-05-10T03:36:01.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d17e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180b3fb0 con 0x7f0d1806d930 2024-05-10T03:36:01.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d0e000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0d181612f0 0x7f0d18161700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52022/0 (socket says 172.21.15.55:52022) 2024-05-10T03:36:01.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d0e000640 1 -- 172.21.15.55:0/429707640 learned_addr learned my addr 172.21.15.55:0/429707640 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.312+0000 7f0d17400640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0d18071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326298074 0 0) 0x7f0d180b39f0 con 0x7f0d181612f0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003520 con 0x7f0d181612f0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408064554 0 0) 0x7f0cfc003520 con 0x7f0d181612f0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 msgr2=0x7f0d18071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0d18071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 >> v1:172.21.15.162:6789/0 conn(0x7f0d181549e0 legacy=0x7f0d180b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180b4290 con 0x7f0d181612f0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f0d00001670 con 0x7f0d181612f0 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 >> v1:172.21.15.55:6789/0 conn(0x7f0d181612f0 legacy=0x7f0d18161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0cfc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc003f90 0x7f0cfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc004d00 0x7f0cfc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180b3fb0 con 0x7f0cfc004d00 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bc30 con 0x7f0cfc003f90 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.313+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bdf0 con 0x7f0d1806d930 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.314+0000 7f0d17400640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc004d00 0x7f0cfc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.314+0000 7f0d0e000640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc003f90 0x7f0cfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.314+0000 7f0d16a00640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0cfc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d17400640 1 -- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 msgr2=0x7f0cfc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d17400640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0cfc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d17400640 1 -- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc003f90 msgr2=0x7f0cfc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d17400640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cfc003f90 0x7f0cfc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d17400640 1 -- 172.21.15.55:0/429707640 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc003520 con 0x7f0cfc004d00 2024-05-10T03:36:01.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.315+0000 7f0d16a00640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d1806d930 0x7f0cfc003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T03:36:01.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.316+0000 7f0d17400640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc004d00 0x7f0cfc005190 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009a90 tx=0x7f0cfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b288c16996aae59 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.316+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10031400 con 0x7f0cfc004d00 2024-05-10T03:36:01.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.316+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1002d050 con 0x7f0cfc004d00 2024-05-10T03:36:01.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.316+0000 7f0d16000640 1 -- 172.21.15.55:0/429707640 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10031780 con 0x7f0cfc004d00 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.317+0000 7f0d17e00640 1 -- 172.21.15.55:0/429707640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc004d00 msgr2=0x7f0cfc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.317+0000 7f0d17e00640 1 --2- 172.21.15.55:0/429707640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc004d00 0x7f0cfc005190 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009a90 tx=0x7f0cfc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.317+0000 7f0d17400640 1 -- 172.21.15.55:0/429707640 reap_dead start 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.317+0000 7f0d17e00640 1 -- 172.21.15.55:0/429707640 shutdown_connections 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.317+0000 7f0d17e00640 1 -- 172.21.15.55:0/429707640 >> 172.21.15.55:0/429707640 conn(0x7f0d1806a110 msgr2=0x7f0d1806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.318+0000 7f0d17e00640 1 -- 172.21.15.55:0/429707640 shutdown_connections 2024-05-10T03:36:01.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.318+0000 7f0d17e00640 1 -- 172.21.15.55:0/429707640 wait complete. 2024-05-10T03:36:01.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.320+0000 7f0d17e00640 1 Processor -- start 2024-05-10T03:36:01.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.320+0000 7f0d17e00640 1 -- start start 2024-05-10T03:36:01.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cfc004d00 0x7f0d180ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d18152830 0x7f0d181436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180c6820 con 0x7f0d180cc350 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18145290 con 0x7f0d18152830 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d17e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180cd500 con 0x7f0cfc004d00 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d0e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d0e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40852/0 (socket says 172.21.15.55:40852) 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.322+0000 7f0d0e000640 1 -- 172.21.15.55:0/2610940960 learned_addr learned my addr 172.21.15.55:0/2610940960 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d16a00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cfc004d00 0x7f0d180ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d17400640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d18152830 0x7f0d181436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 -- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cfc004d00 msgr2=0x7f0d180ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cfc004d00 0x7f0d180ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 -- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d18152830 msgr2=0x7f0d181436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d18152830 0x7f0d181436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 -- 172.21.15.55:0/2610940960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180cd750 con 0x7f0d180cc350 2024-05-10T03:36:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.323+0000 7f0d0e000640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0d000072c0 tx=0x7f0d00007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.324+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000f8e0 con 0x7f0d180cc350 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.324+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0000fa40 con 0x7f0d180cc350 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.324+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000fdf0 con 0x7f0d180cc350 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.324+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d180cda30 con 0x7f0d180cc350 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.324+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180cdff0 con 0x7f0d180cc350 2024-05-10T03:36:01.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.325+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1804e290 con 0x7f0d180cc350 2024-05-10T03:36:01.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.326+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d0001d460 con 0x7f0d180cc350 2024-05-10T03:36:01.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.327+0000 7f0d0fe00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 0x7f0cf406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.327+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f0d00089470 con 0x7f0d180cc350 2024-05-10T03:36:01.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.328+0000 7f0d16a00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 0x7f0cf406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.329+0000 7f0d16a00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 0x7f0cf406d110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0d08007fd0 tx=0x7f0d08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.332+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0d000546a0 con 0x7f0d180cc350 2024-05-10T03:36:01.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.532+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d1815f540 con 0x7f0cf406ac00 2024-05-10T03:36:01.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.538+0000 7f0d0fe00640 1 -- 172.21.15.55:0/2610940960 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48696 (secure 0 0 0) 0x7f0d1815f540 con 0x7f0cf406ac00 2024-05-10T03:36:01.542 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:36:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.540+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 msgr2=0x7f0cf406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 0x7f0cf406d110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0d08007fd0 tx=0x7f0d08009040 comp rx=0 tx=0).stop 2024-05-10T03:36:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 msgr2=0x7f0d181522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0d000072c0 tx=0x7f0d00007780 comp rx=0 tx=0).stop 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 shutdown_connections 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0cf406ac00 0x7f0cf406d110 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d18152830 0x7f0d181436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d180cc350 0x7f0d181522f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 --2- 172.21.15.55:0/2610940960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cfc004d00 0x7f0d180ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 >> 172.21.15.55:0/2610940960 conn(0x7f0d1806a110 msgr2=0x7f0d181606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 shutdown_connections 2024-05-10T03:36:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.541+0000 7f0d17e00640 1 -- 172.21.15.55:0/2610940960 wait complete. 2024-05-10T03:36:01.574 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-05-10T03:36:01.100154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261028,"kb_used_data":12088,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713692,"statfs":{"total":773094113280,"available":772826820608,"internally_reserved":0,"allocated":12378112,"data_stored":6314589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26094,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-170,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.681718"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-10T03:35:55.618556+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:35:55.618556+0000","last_peered":"2024-05-10T03:35:55.618556+0000","last_clean":"2024-05-10T03:35:55.618556+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:35:55.618556+0000","last_undegraded":"2024-05-10T03:35:55.618556+0000","last_fullsized":"2024-05-10T03:35:55.618556+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1986560,"data_stored":1229751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.327,"15min":1.351},"min":{"1min":0.78100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8140000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.1830000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1459999999999999,"15min":1.143},"min":{"1min":0.78400000000000003,"5min":0.65400000000000003,"15min":0.59299999999999997},"max":{"1min":1.4850000000000001,"5min":1.853,"15min":1.853},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2649999999999999},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.679,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.095}]},{"osd":2,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2869999999999999,"15min":1.357},"min":{"1min":0.90000000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.9079999999999999,"5min":2.1349999999999998,"15min":3.0289999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2929999999999999,"15min":1.3999999999999999},"min":{"1min":0.80800000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9930000000000001,"5min":2.4649999999999999,"15min":2.6000000000000001},"last":1.6699999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.222,"15min":2.222},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.355,"15min":1.335},"min":{"1min":0.94099999999999995,"5min":0.71899999999999997,"15min":0.70799999999999996},"max":{"1min":1.675,"5min":2.367,"15min":2.367},"last":1.899}]},{"osd":4,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3740000000000001,"15min":1.4450000000000001},"min":{"1min":0.61099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.7110000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.498},{"interface":"front","average":{"1min":1.429,"5min":1.49,"15min":1.4970000000000001},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.0680000000000001,"5min":2.323,"15min":2.323},"last":1.7410000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.5169999999999999},"min":{"1min":0.70199999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4660000000000002,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.788},{"interface":"front","average":{"1min":1.625,"5min":1.6619999999999999,"15min":1.6870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.601,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.649}]},{"osd":6,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.641,"15min":1.7010000000000001},"min":{"1min":0.73899999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.639,"15min":1.6359999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.2250000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.8380000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.1839999999999999,"15min":1.1080000000000001},"min":{"1min":0.80200000000000005,"5min":0.57799999999999996,"15min":0.48699999999999999},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2110000000000001,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.7170000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.4399999999999999},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.60399999999999998},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":4.1429999999999998},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3919999999999999,"15min":1.2809999999999999},"min":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.53900000000000003},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.083},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3400000000000001,"15min":1.474},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.569},"last":1.659}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4119999999999999,"15min":1.5680000000000001},"min":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.488,"5min":3.488,"15min":4.0869999999999997},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3759999999999999,"15min":1.327},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.0209999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.468,"15min":1.4399999999999999},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5009999999999999,"15min":1.5640000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.966,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.7729999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.488,"15min":1.4199999999999999},"min":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.345},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.635,"15min":1.627},"min":{"1min":0.70499999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.415}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.655,"15min":1.6679999999999999},"min":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1200000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.6659999999999999,"15min":1.6599999999999999},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.64000000000000001},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1619999999999999,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7410000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.448}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.153,"15min":1.143},"min":{"1min":0.89400000000000002,"5min":0.66300000000000003,"15min":0.57699999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.206,"15min":1.22},"min":{"1min":0.86499999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.954},"last":1.081}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.216,"15min":1.2170000000000001},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.7490000000000001,"15min":2.359},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.304,"15min":1.3300000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7909999999999999,"5min":1.913,"15min":1.913},"last":1.3740000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.347,"15min":1.3520000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4139999999999999,"15min":1.3240000000000001},"min":{"1min":1.1850000000000001,"5min":0.76800000000000002,"15min":0.68200000000000005},"max":{"1min":1.9390000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.321}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.91300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5429999999999999,"15min":1.587},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":3.7759999999999998},"last":1.266}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5329999999999999,"15min":1.518},"min":{"1min":0.875,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9670000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.595,"5min":1.556,"15min":1.554},"min":{"1min":0.86299999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.508}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.732,"15min":1.79},"min":{"1min":1.167,"5min":0.877,"15min":0.753},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":4.0899999999999999},"last":1.609},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.653,"15min":1.6240000000000001},"min":{"1min":1.4299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.036,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.129}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.153},{"interface":"front","average":{"1min":1.238,"5min":1.151,"15min":1.1439999999999999},"min":{"1min":0.72299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.681,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.3500000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2130000000000001,"15min":1.1719999999999999},"min":{"1min":0.80600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.679,"5min":1.734,"15min":2.3839999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.2490000000000001,"15min":1.2729999999999999},"min":{"1min":0.997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1619999999999999},"last":1.3220000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.242,"15min":1.2709999999999999},"min":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.702,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.472,"5min":1.3740000000000001,"15min":1.399},"min":{"1min":0.94099999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.29},"last":1.415}]},{"osd":3,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.341,"15min":1.4370000000000001},"min":{"1min":0.753,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8859999999999999,"5min":2.379,"15min":2.6779999999999999},"last":0.876},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5269999999999999,"15min":1.5680000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.7709999999999999},"last":1.252}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4810000000000001,"15min":1.4990000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.512},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4119999999999999,"15min":1.5209999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0579999999999998,"5min":2.5510000000000002,"15min":2.8109999999999999},"last":1.113}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.5249999999999999,"15min":1.5720000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.141,"5min":2.4769999999999999,"15min":2.8530000000000002},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.619,"15min":1.742},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.597,"5min":2.597,"15min":5.3529999999999998},"last":1.631}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5880000000000001,"15min":1.6679999999999999},"min":{"1min":0.63600000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.177,"5min":2.6160000000000001,"15min":3.3479999999999999},"last":1.488},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.694,"15min":1.7290000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.556,"5min":2.6349999999999998,"15min":2.8700000000000001},"last":1.9159999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.95999999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2130000000000001,"5min":1.6200000000000001,"15min":1.708},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.603},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.625,"5min":0.377,"15min":0.377},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.3140000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1559999999999999,"15min":1.177},"min":{"1min":0.63100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8360000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.27},{"interface":"front","average":{"1min":1.349,"5min":1.2390000000000001,"15min":1.335},"min":{"1min":0.72299999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.762,"5min":1.919,"15min":2.2599999999999998},"last":1.534}]},{"osd":4,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3540000000000001,"15min":1.371},"min":{"1min":0.93799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.881,"5min":2.0910000000000002,"15min":2.3610000000000002},"last":1.206},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4299999999999999,"15min":1.3919999999999999},"min":{"1min":0.95499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0790000000000002,"5min":2.1349999999999998,"15min":2.464},"last":1.1120000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4810000000000001,"15min":1.6299999999999999},"min":{"1min":0.97799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.839,"5min":2.4399999999999999,"15min":3.0470000000000002},"last":1.655},{"interface":"front","average":{"1min":1.456,"5min":1.446,"15min":1.478},"min":{"1min":0.998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9399999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.3660000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.544,"15min":1.569},"min":{"1min":0.88200000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.7569999999999999},"min":{"1min":0.92900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9319999999999999,"5min":2.4729999999999999,"15min":3.2280000000000002},"last":1.8009999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.51,"15min":1.5720000000000001},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.0670000000000002,"5min":2.2229999999999999,"15min":2.4399999999999999},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5780000000000001,"15min":1.6379999999999999},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.1429999999999998,"5min":2.29,"15min":2.6190000000000002},"last":1.6140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1392640,"data_stored":639383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.88,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.434,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.643,"5min":1.746,"15min":1.746},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1699999999999999},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.1030000000000002},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.052,"15min":1.1060000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":2.169},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1930000000000001,"15min":1.244},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8380000000000001,"5min":1.9910000000000001,"15min":2.2229999999999999},"last":1.45}]},{"osd":4,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.506},"min":{"1min":0.71299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.8050000000000002},"last":1.847},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3580000000000001,"15min":1.504},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.6150000000000002},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9950000000000001,"5min":2.173,"15min":2.508},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5589999999999999,"15min":1.607},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.379,"5min":2.379,"15min":2.6669999999999998},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.504,"15min":1.542},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.7280000000000002},"last":1.619},{"interface":"front","average":{"1min":1.52,"5min":1.4950000000000001,"15min":1.569},"min":{"1min":0.87,"5min":0.81899999999999995,"15min":0.78200000000000003},"max":{"1min":2.1720000000000002,"5min":2.3799999999999999,"15min":2.8860000000000001},"last":1.353}]},{"osd":7,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.579,"15min":1.639},"min":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2759999999999998,"5min":2.3149999999999999,"15min":2.8410000000000002},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.5700000000000001,"15min":1.649},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.58999999999999997},"max":{"1min":2.0870000000000002,"5min":2.468,"15min":2.7690000000000001},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.89600000000000002,"15min":0.91700000000000004},"min":{"1min":0.42699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.607,"15min":1.607},"last":3.1970000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.45800000000000002},"max":{"1min":1.4770000000000001,"5min":1.996,"15min":2.2949999999999999},"last":1.4590000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96299999999999997,"15min":0.998},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":1.6910000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.171},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.627,"5min":2.0720000000000001,"15min":2.444},"last":4.7439999999999998}]},{"osd":3,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2250000000000001,"15min":1.3029999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.4129999999999998},"last":3.8090000000000002},{"interface":"front","average":{"1min":1.272,"5min":1.196,"15min":1.2749999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.7799999999999998},"last":1.3799999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.508},"min":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0230000000000001,"5min":2.3159999999999998,"15min":2.5299999999999998},"last":3.9780000000000002},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4390000000000001,"15min":1.554},"min":{"1min":0.93899999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.9610000000000001,"5min":2.1139999999999999,"15min":4.2949999999999999},"last":4.0880000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.544},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9350000000000001,"5min":2.2450000000000001,"15min":3.8029999999999999},"last":3.6299999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.603},"min":{"1min":0.90600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.1200000000000001,"5min":2.3290000000000002,"15min":2.6190000000000002},"last":5.54}]},{"osd":6,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.5469999999999999,"15min":1.6859999999999999},"min":{"1min":1.111,"5min":0.94999999999999996,"15min":0.91100000000000003},"max":{"1min":2.1549999999999998,"5min":2.4470000000000001,"15min":3.48},"last":3.1040000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5329999999999999,"15min":1.6870000000000001},"min":{"1min":1.028,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0720000000000001,"5min":2.1230000000000002,"15min":3.6880000000000002},"last":3.4900000000000002}]},{"osd":7,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6100000000000001,"15min":1.708},"min":{"1min":0.87,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.0409999999999999,"5min":2.2709999999999999,"15min":3.2869999999999999},"last":9.7850000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.5920000000000001,"15min":1.74},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.27,"5min":2.5449999999999999,"15min":4.0140000000000002},"last":1.8340000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.875,"15min":0.995},"min":{"1min":0.56899999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":2.1309999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.048,"5min":0.93899999999999995,"15min":1.137},"min":{"1min":0.77400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.532,"5min":1.6120000000000001,"15min":2.9390000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0609999999999999,"15min":1.1910000000000001},"min":{"1min":0.72799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.72,"5min":1.72,"15min":2.4390000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.0620000000000001,"15min":1.1879999999999999},"min":{"1min":0.755,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7190000000000001,"5min":1.8400000000000001,"15min":3.0009999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.014,"15min":1.089},"min":{"1min":0.66700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.9850000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2270000000000001,"15min":1.3069999999999999},"min":{"1min":0.68899999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.097,"15min":2.4750000000000001},"last":1.202}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.335,"15min":1.5649999999999999},"min":{"1min":1.071,"5min":0.745,"15min":0.745},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":3.1040000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3460000000000001,"15min":1.4179999999999999},"min":{"1min":0.872,"5min":0.84099999999999997,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":3.0790000000000002},"last":1.601}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.3759999999999999,"15min":1.4570000000000001},"min":{"1min":0.95199999999999996,"5min":0.80700000000000005,"15min":0.57899999999999996},"max":{"1min":1.841,"5min":1.9710000000000001,"15min":2.4889999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.444,"15min":1.5920000000000001},"min":{"1min":1.028,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0299999999999998,"5min":2.2120000000000002,"15min":2.9279999999999999},"last":1.7909999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.552,"15min":1.6279999999999999},"min":{"1min":1.0600000000000001,"5min":0.98099999999999998,"15min":0.90900000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.7189999999999999},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.5029999999999999,"15min":1.6919999999999999},"min":{"1min":1.167,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":3.0539999999999998},"last":1.6259999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5209999999999999,"15min":1.7370000000000001},"min":{"1min":1.3029999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.117,"5min":2.117,"15min":3.149},"last":1.738},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5249999999999999,"15min":1.569},"min":{"1min":1.2709999999999999,"5min":0.86899999999999999,"15min":0.63700000000000001},"max":{"1min":2.0409999999999999,"5min":2.1509999999999998,"15min":2.3860000000000001},"last":2.0209999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:36:01.576 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T03:36:01.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.859+0000 7f3a1f800640 1 Processor -- start 2024-05-10T03:36:01.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.859+0000 7f3a1f800640 1 -- start start 2024-05-10T03:36:01.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.861+0000 7f3a1f800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a1806b6e0 0x7f3a180ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.861+0000 7f3a1f800640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a181659c0 0x7f3a18165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.861+0000 7f3a1f800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180715e0 con 0x7f3a18162380 2024-05-10T03:36:01.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.861+0000 7f3a1f800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a180711c0 con 0x7f3a1806b6e0 2024-05-10T03:36:01.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.861+0000 7f3a1f800640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1806baf0 con 0x7f3a181659c0 2024-05-10T03:36:01.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a1806b6e0 0x7f3a180ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a1806b6e0 0x7f3a180ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40860/0 (socket says 172.21.15.55:40860) 2024-05-10T03:36:01.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 -- 172.21.15.55:0/2820874278 learned_addr learned my addr 172.21.15.55:0/2820874278 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a0fe00640 1 --2- 172.21.15.55:0/2820874278 >> v2:172.21.15.162:3300/0 conn(0x7f3a181659c0 0x7f3a18165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 -- 172.21.15.55:0/2820874278 >> v2:172.21.15.162:3300/0 conn(0x7f3a181659c0 msgr2=0x7f3a18165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 --2- 172.21.15.55:0/2820874278 >> v2:172.21.15.162:3300/0 conn(0x7f3a181659c0 0x7f3a18165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 -- 172.21.15.55:0/2820874278 >> v1:172.21.15.162:6789/0 conn(0x7f3a18162380 legacy=0x7f3a18162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.862+0000 7f3a1ce00640 1 -- 172.21.15.55:0/2820874278 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a18166310 con 0x7f3a1806b6e0 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.863+0000 7f3a1ce00640 1 --2- 172.21.15.55:0/2820874278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1806b6e0 0x7f3a180ce3b0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3a1806d1d0 tx=0x7f3a0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82656ce7980e42b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.864+0000 7f3a17e00640 1 -- 172.21.15.55:0/2820874278 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08032440 con 0x7f3a1806b6e0 2024-05-10T03:36:01.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.864+0000 7f3a17e00640 1 -- 172.21.15.55:0/2820874278 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a080325a0 con 0x7f3a1806b6e0 2024-05-10T03:36:01.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.864+0000 7f3a17e00640 1 -- 172.21.15.55:0/2820874278 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08032930 con 0x7f3a1806b6e0 2024-05-10T03:36:01.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 -- 172.21.15.55:0/2820874278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1806b6e0 msgr2=0x7f3a180ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 --2- 172.21.15.55:0/2820874278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1806b6e0 0x7f3a180ce3b0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3a1806d1d0 tx=0x7f3a0802a1a0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 -- 172.21.15.55:0/2820874278 shutdown_connections 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 --2- 172.21.15.55:0/2820874278 >> v2:172.21.15.162:3300/0 conn(0x7f3a181659c0 0x7f3a18165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 --2- 172.21.15.55:0/2820874278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1806b6e0 0x7f3a180ce3b0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 -- 172.21.15.55:0/2820874278 >> 172.21.15.55:0/2820874278 conn(0x7f3a180cbb90 msgr2=0x7f3a180cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 -- 172.21.15.55:0/2820874278 shutdown_connections 2024-05-10T03:36:01.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.865+0000 7f3a1f800640 1 -- 172.21.15.55:0/2820874278 wait complete. 2024-05-10T03:36:01.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.867+0000 7f3a1f800640 1 Processor -- start 2024-05-10T03:36:01.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.868+0000 7f3a1f800640 1 -- start start 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1806b6e0 0x7f3a1815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a18162380 0x7f3a1815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1814c610 con 0x7f3a181659c0 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815e7f0 con 0x7f3a18162380 2024-05-10T03:36:01.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1f800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1815f650 con 0x7f3a1806b6e0 2024-05-10T03:36:01.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.869+0000 7f3a1d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40866/0 (socket says 172.21.15.55:40866) 2024-05-10T03:36:01.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 -- 172.21.15.55:0/3077463833 learned_addr learned my addr 172.21.15.55:0/3077463833 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:01.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1ce00640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1806b6e0 0x7f3a1815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a0fe00640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a18162380 0x7f3a1815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 -- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1806b6e0 msgr2=0x7f3a1815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1806b6e0 0x7f3a1815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 -- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a18162380 msgr2=0x7f3a1815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a18162380 0x7f3a1815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 -- 172.21.15.55:0/3077463833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080094d0 con 0x7f3a181659c0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.870+0000 7f3a1d800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b970 tx=0x7f3a1000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.871+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000cb40 con 0x7f3a181659c0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.871+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000cca0 con 0x7f3a181659c0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.871+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10014680 con 0x7f3a181659c0 2024-05-10T03:36:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.871+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1815ea40 con 0x7f3a181659c0 2024-05-10T03:36:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.871+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1815f000 con 0x7f3a181659c0 2024-05-10T03:36:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.872+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1815f800 con 0x7f3a181659c0 2024-05-10T03:36:01.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.873+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3a100147e0 con 0x7f3a181659c0 2024-05-10T03:36:01.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.874+0000 7f3a15600640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 0x7f39f406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:01.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.875+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f3a100891a0 con 0x7f3a181659c0 2024-05-10T03:36:01.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.875+0000 7f3a1ce00640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 0x7f39f406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:01.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.877+0000 7f3a1ce00640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 0x7f39f406d110 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3a08034000 tx=0x7f3a0802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:01.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:01.879+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3a1815f800 con 0x7f3a181659c0 2024-05-10T03:36:02.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.079+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a18066f60 con 0x7f3a181659c0 2024-05-10T03:36:02.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.081+0000 7f3a15600640 1 -- 172.21.15.55:0/3077463833 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13249 (secure 0 0 0) 0x7f3a10054350 con 0x7f3a181659c0 2024-05-10T03:36:02.085 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:02.086 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":113,"fsid":"340d5548-dd78-4f40-a3a8-44ee99f8e0d8","created":"2024-05-10T03:29:17.736882+0000","modified":"2024-05-10T03:35:56.574899+0000","last_up_change":"2024-05-10T03:29:31.198368+0000","last_in_change":"2024-05-10T03:29:21.862076+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T03:29:33.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"097e253b-6207-4031-9378-098510bfc2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":35473},{"type":"v1","addr":"172.21.15.55:6817","nonce":35473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":35473},{"type":"v1","addr":"172.21.15.55:6819","nonce":35473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6823","nonce":35473},{"type":"v1","addr":"172.21.15.55:6824","nonce":35473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":35473},{"type":"v1","addr":"172.21.15.55:6821","nonce":35473}]},"public_addr":"172.21.15.55:6817/35473","cluster_addr":"172.21.15.55:6819/35473","heartbeat_back_addr":"172.21.15.55:6824/35473","heartbeat_front_addr":"172.21.15.55:6821/35473","state":["exists","up"]},{"osd":1,"uuid":"f73e2adc-a6b9-45d7-abc3-cb47949005ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":35474},{"type":"v1","addr":"172.21.15.55:6825","nonce":35474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":35474},{"type":"v1","addr":"172.21.15.55:6827","nonce":35474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":35474},{"type":"v1","addr":"172.21.15.55:6831","nonce":35474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":35474},{"type":"v1","addr":"172.21.15.55:6829","nonce":35474}]},"public_addr":"172.21.15.55:6825/35474","cluster_addr":"172.21.15.55:6827/35474","heartbeat_back_addr":"172.21.15.55:6831/35474","heartbeat_front_addr":"172.21.15.55:6829/35474","state":["exists","up"]},{"osd":2,"uuid":"de765a8c-93e6-4aba-a97e-06bfb1c026d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":35475},{"type":"v1","addr":"172.21.15.55:6801","nonce":35475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":35475},{"type":"v1","addr":"172.21.15.55:6803","nonce":35475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":35475},{"type":"v1","addr":"172.21.15.55:6807","nonce":35475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":35475},{"type":"v1","addr":"172.21.15.55:6805","nonce":35475}]},"public_addr":"172.21.15.55:6801/35475","cluster_addr":"172.21.15.55:6803/35475","heartbeat_back_addr":"172.21.15.55:6807/35475","heartbeat_front_addr":"172.21.15.55:6805/35475","state":["exists","up"]},{"osd":3,"uuid":"60999af4-c269-4ee9-8303-8441b4aed779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":35476},{"type":"v1","addr":"172.21.15.55:6809","nonce":35476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":35476},{"type":"v1","addr":"172.21.15.55:6811","nonce":35476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":35476},{"type":"v1","addr":"172.21.15.55:6815","nonce":35476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":35476},{"type":"v1","addr":"172.21.15.55:6813","nonce":35476}]},"public_addr":"172.21.15.55:6809/35476","cluster_addr":"172.21.15.55:6811/35476","heartbeat_back_addr":"172.21.15.55:6815/35476","heartbeat_front_addr":"172.21.15.55:6813/35476","state":["exists","up"]},{"osd":4,"uuid":"af007dd1-0644-42f1-9149-ae514bb1d110","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":35542},{"type":"v1","addr":"172.21.15.162:6801","nonce":35542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":35542},{"type":"v1","addr":"172.21.15.162:6803","nonce":35542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":35542},{"type":"v1","addr":"172.21.15.162:6807","nonce":35542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":35542},{"type":"v1","addr":"172.21.15.162:6805","nonce":35542}]},"public_addr":"172.21.15.162:6801/35542","cluster_addr":"172.21.15.162:6803/35542","heartbeat_back_addr":"172.21.15.162:6807/35542","heartbeat_front_addr":"172.21.15.162:6805/35542","state":["exists","up"]},{"osd":5,"uuid":"9bcde535-4e7a-4394-8fd3-bf35c240183e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":35543},{"type":"v1","addr":"172.21.15.162:6809","nonce":35543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":35543},{"type":"v1","addr":"172.21.15.162:6811","nonce":35543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":35543},{"type":"v1","addr":"172.21.15.162:6815","nonce":35543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":35543},{"type":"v1","addr":"172.21.15.162:6813","nonce":35543}]},"public_addr":"172.21.15.162:6809/35543","cluster_addr":"172.21.15.162:6811/35543","heartbeat_back_addr":"172.21.15.162:6815/35543","heartbeat_front_addr":"172.21.15.162:6813/35543","state":["exists","up"]},{"osd":6,"uuid":"877cf826-8802-4a9f-815c-d81c9aafff27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":35544},{"type":"v1","addr":"172.21.15.162:6817","nonce":35544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":35544},{"type":"v1","addr":"172.21.15.162:6819","nonce":35544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":35544},{"type":"v1","addr":"172.21.15.162:6823","nonce":35544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":35544},{"type":"v1","addr":"172.21.15.162:6821","nonce":35544}]},"public_addr":"172.21.15.162:6817/35544","cluster_addr":"172.21.15.162:6819/35544","heartbeat_back_addr":"172.21.15.162:6823/35544","heartbeat_front_addr":"172.21.15.162:6821/35544","state":["exists","up"]},{"osd":7,"uuid":"0051db81-3299-4de5-a2af-a56a1e4c9ba7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":35545},{"type":"v1","addr":"172.21.15.162:6825","nonce":35545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":35545},{"type":"v1","addr":"172.21.15.162:6827","nonce":35545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":35545},{"type":"v1","addr":"172.21.15.162:6831","nonce":35545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":35545},{"type":"v1","addr":"172.21.15.162:6829","nonce":35545}]},"public_addr":"172.21.15.162:6825/35545","cluster_addr":"172.21.15.162:6827/35545","heartbeat_back_addr":"172.21.15.162:6831/35545","heartbeat_front_addr":"172.21.15.162:6829/35545","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.566340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.647294+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.601459+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.557913+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.298612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.319008+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.315980+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T03:29:27.366701+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:6835/1198619982":"2024-05-11T03:35:53.410043+0000","172.21.15.162:6837/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6834/1198619982":"2024-05-11T03:35:53.410043+0000","172.21.15.162:6836/476611421":"2024-05-11T03:35:13.193337+0000","172.21.15.55:6832/609219709":"2024-05-11T03:35:53.410015+0000","172.21.15.55:6833/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6832/2254900234":"2024-05-11T03:35:13.167318+0000","172.21.15.55:6835/315649334":"2024-05-11T03:35:13.117413+0000","172.21.15.55:6833/609219709":"2024-05-11T03:35:53.410015+0000","172.21.15.55:6834/315649334":"2024-05-11T03:35:13.117413+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T03:36:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 msgr2=0x7f39f406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 0x7f39f406d110 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3a08034000 tx=0x7f3a0802e000 comp rx=0 tx=0).stop 2024-05-10T03:36:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 msgr2=0x7f3a18160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b970 tx=0x7f3a1000be30 comp rx=0 tx=0).stop 2024-05-10T03:36:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 shutdown_connections 2024-05-10T03:36:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a181659c0 0x7f3a18160020 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f39f406ac00 0x7f39f406d110 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a18162380 0x7f3a1815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 --2- 172.21.15.55:0/3077463833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1806b6e0 0x7f3a1815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.084+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 >> 172.21.15.55:0/3077463833 conn(0x7f3a180cbb90 msgr2=0x7f3a180b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.085+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 shutdown_connections 2024-05-10T03:36:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:02.085+0000 7f3a1f800640 1 -- 172.21.15.55:0/3077463833 wait complete. 2024-05-10T03:36:03.117 INFO:tasks.ceph:Scrubbing osd.0 2024-05-10T03:36:03.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.416+0000 7f0913200640 1 Processor -- start 2024-05-10T03:36:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.416+0000 7f0913200640 1 -- start start 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.418+0000 7f0913200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09140ce520 0x7f091415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.418+0000 7f0913200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f09140d1330 0x7f091406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.418+0000 7f0913200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140b2240 con 0x7f09140b1cf0 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f0913200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140b2490 con 0x7f09140ce520 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f0913200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140d2480 con 0x7f09140d1330 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09140ce520 0x7f091415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09140ce520 0x7f091415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54648/0 (socket says 172.21.15.55:54648) 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 -- 172.21.15.55:0/3215225165 learned_addr learned my addr 172.21.15.55:0/3215225165 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f0912800640 1 --2- 172.21.15.55:0/3215225165 >> v2:172.21.15.162:3300/0 conn(0x7f09140d1330 0x7f091406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 -- 172.21.15.55:0/3215225165 >> v2:172.21.15.162:3300/0 conn(0x7f09140d1330 msgr2=0x7f091406a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 --2- 172.21.15.55:0/3215225165 >> v2:172.21.15.162:3300/0 conn(0x7f09140d1330 0x7f091406a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 -- 172.21.15.55:0/3215225165 >> v1:172.21.15.162:6789/0 conn(0x7f09140b1cf0 legacy=0x7f09140cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:03.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.419+0000 7f090be00640 1 -- 172.21.15.55:0/3215225165 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140d2760 con 0x7f09140ce520 2024-05-10T03:36:03.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.421+0000 7f090be00640 1 --2- 172.21.15.55:0/3215225165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140ce520 0x7f091415d7d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f09140ca1c0 tx=0x7f08fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d89f6d1e32f9120 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.421+0000 7f0911400640 1 -- 172.21.15.55:0/3215225165 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc032440 con 0x7f09140ce520 2024-05-10T03:36:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.421+0000 7f0911400640 1 -- 172.21.15.55:0/3215225165 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc0325a0 con 0x7f09140ce520 2024-05-10T03:36:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.421+0000 7f0911400640 1 -- 172.21.15.55:0/3215225165 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc032930 con 0x7f09140ce520 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 -- 172.21.15.55:0/3215225165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140ce520 msgr2=0x7f091415d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 --2- 172.21.15.55:0/3215225165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140ce520 0x7f091415d7d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f09140ca1c0 tx=0x7f08fc02a230 comp rx=0 tx=0).stop 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 -- 172.21.15.55:0/3215225165 shutdown_connections 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 --2- 172.21.15.55:0/3215225165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140ce520 0x7f091415d7d0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 --2- 172.21.15.55:0/3215225165 >> v2:172.21.15.162:3300/0 conn(0x7f09140d1330 0x7f091406a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 -- 172.21.15.55:0/3215225165 >> 172.21.15.55:0/3215225165 conn(0x7f09140d0850 msgr2=0x7f09140d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 -- 172.21.15.55:0/3215225165 shutdown_connections 2024-05-10T03:36:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.422+0000 7f0913200640 1 -- 172.21.15.55:0/3215225165 wait complete. 2024-05-10T03:36:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.425+0000 7f0913200640 1 Processor -- start 2024-05-10T03:36:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.425+0000 7f0913200640 1 -- start start 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09140b1cf0 0x7f0914154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 0x7f0914150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0914150780 0x7f091415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914155e40 con 0x7f09140d1330 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914150ff0 con 0x7f09140b1cf0 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0913200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09141505a0 con 0x7f0914150780 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 0x7f0914150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 0x7f0914150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54656/0 (socket says 172.21.15.55:54656) 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 -- 172.21.15.55:0/897101263 learned_addr learned my addr 172.21.15.55:0/897101263 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0911e00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09140b1cf0 0x7f0914154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f0912800640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0914150780 0x7f091415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 -- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0914150780 msgr2=0x7f091415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0914150780 0x7f091415c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 -- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09140b1cf0 msgr2=0x7f0914154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.427+0000 7f090be00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09140b1cf0 0x7f0914154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f090be00640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0094d0 con 0x7f09140d1330 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f0911e00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09140b1cf0 0x7f0914154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f090be00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 0x7f0914150060 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f08fc000c00 tx=0x7f08fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc032440 con 0x7f09140d1330 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc030b80 con 0x7f09140d1330 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc03b400 con 0x7f09140d1330 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091415f9d0 con 0x7f09140d1330 2024-05-10T03:36:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.428+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091415ff70 con 0x7f09140d1330 2024-05-10T03:36:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.429+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09140d1e30 con 0x7f09140d1330 2024-05-10T03:36:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.431+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08fc039040 con 0x7f09140d1330 2024-05-10T03:36:03.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f090a000640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08f006ac00 0x7f08f006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f08fc0a9f00 con 0x7f09140d1330 2024-05-10T03:36:03.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f090a000640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7f08f00711d0 0x7f08f0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08f0073d30 con 0x7f08f00711d0 2024-05-10T03:36:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08fc0ac480 con 0x7f09140d1330 2024-05-10T03:36:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f0912800640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7f08f00711d0 0x7f08f0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.432+0000 7f0911e00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08f006ac00 0x7f08f006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.433+0000 7f0912800640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7f08f00711d0 0x7f08f0073660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.434+0000 7f0911e00640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08f006ac00 0x7f08f006d110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f091406c200 tx=0x7f0904008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.434+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== osd.0 v2:172.21.15.55:6816/35473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f08f0073d30 con 0x7f08f00711d0 2024-05-10T03:36:03.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.448+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 --> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f091406a1b0 con 0x7f08f00711d0 2024-05-10T03:36:03.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f090a000640 1 -- 172.21.15.55:0/897101263 <== osd.0 v2:172.21.15.55:6816/35473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f090c0089c0 con 0x7f08f00711d0 2024-05-10T03:36:03.452 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:03.453 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:03.453 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7f08f00711d0 msgr2=0x7f08f0073660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:6816/35473,v1:172.21.15.55:6817/35473] conn(0x7f08f00711d0 0x7f08f0073660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08f006ac00 msgr2=0x7f08f006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f08f006ac00 0x7f08f006d110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f091406c200 tx=0x7f0904008040 comp rx=0 tx=0).stop 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 msgr2=0x7f0914150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 --2- 172.21.15.55:0/897101263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09140d1330 0x7f0914150060 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f08fc000c00 tx=0x7f08fc0046c0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0912800640 1 -- 172.21.15.55:0/897101263 reap_dead start 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 shutdown_connections 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.449+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 >> 172.21.15.55:0/897101263 conn(0x7f09140d0850 msgr2=0x7f09140b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.450+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 shutdown_connections 2024-05-10T03:36:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.450+0000 7f0913200640 1 -- 172.21.15.55:0/897101263 wait complete. 2024-05-10T03:36:03.479 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-10T03:36:03.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.753+0000 7fc81de00640 1 Processor -- start 2024-05-10T03:36:03.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.753+0000 7fc81de00640 1 -- start start 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc81de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 0x7fc8180ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc81de00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180715e0 con 0x7fc8181659c0 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc81de00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8181667f0 con 0x7fc8180b2700 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc81de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180711c0 con 0x7fc81806b6e0 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc80e000640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc8181659c0 0x7fc818165dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42730/0 (socket says 172.21.15.55:42730) 2024-05-10T03:36:03.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.755+0000 7fc80e000640 1 -- 172.21.15.55:0/2909571610 learned_addr learned my addr 172.21.15.55:0/2909571610 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816a00640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 0x7fc8180ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823468390 0 0) 0x7fc800001490 con 0x7fc8181659c0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7fc003520 con 0x7fc8181659c0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892899203 0 0) 0x7fc7fc003520 con 0x7fc8181659c0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 msgr2=0x7fc8180ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 0x7fc8180ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 >> v1:172.21.15.162:6789/0 conn(0x7fc8180b2700 legacy=0x7fc8180b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818070260 con 0x7fc8181659c0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.756+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fc800001b80 con 0x7fc8181659c0 2024-05-10T03:36:03.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 >> v1:172.21.15.55:6789/0 conn(0x7fc8181659c0 legacy=0x7fc818165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 0x7fc7fc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7fc003f20 0x7fc7fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180711c0 con 0x7fc7fc004d00 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc00bbc0 con 0x7fc7fc003f20 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc00bd80 con 0x7fc81806b6e0 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc817400640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc80e000640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7fc003f20 0x7fc7fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc817400640 1 -- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 msgr2=0x7fc7fc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc817400640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806b6e0 0x7fc7fc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc817400640 1 -- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7fc003f20 msgr2=0x7fc7fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.757+0000 7fc817400640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7fc003f20 0x7fc7fc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.758+0000 7fc817400640 1 -- 172.21.15.55:0/2909571610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc003520 con 0x7fc7fc004d00 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc817400640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005110 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc810001720 tx=0x7fc7fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f274113f7ae91c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8100052d0 con 0x7fc7fc004d00 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc810005430 con 0x7fc7fc004d00 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc816000640 1 -- 172.21.15.55:0/2909571610 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81002abc0 con 0x7fc7fc004d00 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc81de00640 1 -- 172.21.15.55:0/2909571610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7fc004d00 msgr2=0x7fc7fc005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.759+0000 7fc81de00640 1 --2- 172.21.15.55:0/2909571610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7fc004d00 0x7fc7fc005110 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc810001720 tx=0x7fc7fc0057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.760+0000 7fc817400640 1 -- 172.21.15.55:0/2909571610 reap_dead start 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.760+0000 7fc81de00640 1 -- 172.21.15.55:0/2909571610 shutdown_connections 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.760+0000 7fc81de00640 1 -- 172.21.15.55:0/2909571610 >> 172.21.15.55:0/2909571610 conn(0x7fc8180cbb90 msgr2=0x7fc8180cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.760+0000 7fc81de00640 1 -- 172.21.15.55:0/2909571610 shutdown_connections 2024-05-10T03:36:03.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.760+0000 7fc81de00640 1 -- 172.21.15.55:0/2909571610 wait complete. 2024-05-10T03:36:03.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.762+0000 7fc81de00640 1 Processor -- start 2024-05-10T03:36:03.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.763+0000 7fc81de00640 1 -- start start 2024-05-10T03:36:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181512c0 0x7fc818155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 0x7fc818157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818158600 con 0x7fc818157bc0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181516d0 con 0x7fc8181441c0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc81de00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815a1f0 con 0x7fc8181512c0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.764+0000 7fc817400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54672/0 (socket says 172.21.15.55:54672) 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 -- 172.21.15.55:0/1720457121 learned_addr learned my addr 172.21.15.55:0/1720457121 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc816a00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181512c0 0x7fc818155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc80e000640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 0x7fc818157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 -- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181512c0 msgr2=0x7fc818155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181512c0 0x7fc818155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 -- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 msgr2=0x7fc818157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 0x7fc818157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 -- 172.21.15.55:0/1720457121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818159850 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc80e000640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 0x7fc818157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.765+0000 7fc817400640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc810001840 tx=0x7fc810001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.766+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810007240 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.766+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8100073a0 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.766+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818159b30 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.766+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810007a30 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.766+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8181561b0 con 0x7fc818157bc0 2024-05-10T03:36:03.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.767+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81815a330 con 0x7fc818157bc0 2024-05-10T03:36:03.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.768+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc8100067d0 con 0x7fc818157bc0 2024-05-10T03:36:03.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.770+0000 7fc80fe00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 0x7fc7f406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:03.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.770+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fc8100a8260 con 0x7fc818157bc0 2024-05-10T03:36:03.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.770+0000 7fc816a00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 0x7fc7f406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:03.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.772+0000 7fc816a00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 0x7fc7f406d110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8181ec5b0 tx=0x7fc808008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.775+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc81815a330 con 0x7fc818157bc0 2024-05-10T03:36:03.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.973+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc818070b00 con 0x7fc7f406ac00 2024-05-10T03:36:03.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.975+0000 7fc80fe00640 1 -- 172.21.15.55:0/1720457121 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc808004020 con 0x7fc7f406ac00 2024-05-10T03:36:03.979 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 0 to deep-scrub 2024-05-10T03:36:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.977+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 msgr2=0x7fc7f406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.977+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 0x7fc7f406d110 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8181ec5b0 tx=0x7fc808008040 comp rx=0 tx=0).stop 2024-05-10T03:36:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.977+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 msgr2=0x7fc8181f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.977+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc810001840 tx=0x7fc810001120 comp rx=0 tx=0).stop 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 shutdown_connections 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc818157bc0 0x7fc8181f8030 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fc7f406ac00 0x7fc7f406d110 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8181441c0 0x7fc818157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 --2- 172.21.15.55:0/1720457121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181512c0 0x7fc818155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 >> 172.21.15.55:0/1720457121 conn(0x7fc8180cbb90 msgr2=0x7fc8180b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 shutdown_connections 2024-05-10T03:36:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:03.978+0000 7fc81de00640 1 -- 172.21.15.55:0/1720457121 wait complete. 2024-05-10T03:36:04.011 INFO:tasks.ceph:Scrubbing osd.1 2024-05-10T03:36:04.011 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.296+0000 7f9416a00640 1 Processor -- start 2024-05-10T03:36:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.296+0000 7f9416a00640 1 -- start start 2024-05-10T03:36:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9416a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94100cdc70 0x7f94100cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9416a00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100b4330 con 0x7f941006b930 2024-05-10T03:36:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9416a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941015b0e0 con 0x7f94100ce600 2024-05-10T03:36:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9416a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941015b3c0 con 0x7f94100cdc70 2024-05-10T03:36:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9414a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f941006b930 0x7f941015e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42740/0 (socket says 172.21.15.55:42740) 2024-05-10T03:36:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f9414a00640 1 -- 172.21.15.55:0/3479500541 learned_addr learned my addr 172.21.15.55:0/3479500541 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.299+0000 7f940fe00640 1 --2- 172.21.15.55:0/3479500541 >> v2:172.21.15.162:3300/0 conn(0x7f94100cdc70 0x7f94100cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521997918 0 0) 0x7f94100b4330 con 0x7f941006b930 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f4003520 con 0x7f941006b930 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940fe00640 1 -- 172.21.15.55:0/3479500541 >> v1:172.21.15.162:6789/0 conn(0x7f94100ce600 legacy=0x7f941006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940fe00640 1 -- 172.21.15.55:0/3479500541 >> v1:172.21.15.55:6789/0 conn(0x7f941006b930 legacy=0x7f941015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278194756 0 0) 0x7f93f4003520 con 0x7f941006b930 2024-05-10T03:36:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.300+0000 7f940fe00640 1 -- 172.21.15.55:0/3479500541 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941015b6a0 con 0x7f94100cdc70 2024-05-10T03:36:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.302+0000 7f940fe00640 1 --2- 172.21.15.55:0/3479500541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100cdc70 0x7f94100cce40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f94000097a0 tx=0x7f940002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00b7313bf36067f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.302+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080019f0 con 0x7f94100cdc70 2024-05-10T03:36:04.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.302+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940002abc0 con 0x7f94100cdc70 2024-05-10T03:36:04.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.302+0000 7f940ea00640 1 -- 172.21.15.55:0/3479500541 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400032440 con 0x7f94100cdc70 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 -- 172.21.15.55:0/3479500541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100cdc70 msgr2=0x7f94100cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 --2- 172.21.15.55:0/3479500541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100cdc70 0x7f94100cce40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f94000097a0 tx=0x7f940002a130 comp rx=0 tx=0).stop 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 -- 172.21.15.55:0/3479500541 shutdown_connections 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 --2- 172.21.15.55:0/3479500541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100cdc70 0x7f94100cce40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 -- 172.21.15.55:0/3479500541 >> 172.21.15.55:0/3479500541 conn(0x7f94100b0cf0 msgr2=0x7f94100b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 -- 172.21.15.55:0/3479500541 shutdown_connections 2024-05-10T03:36:04.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.303+0000 7f9416a00640 1 -- 172.21.15.55:0/3479500541 wait complete. 2024-05-10T03:36:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.306+0000 7f9416a00640 1 Processor -- start 2024-05-10T03:36:04.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.306+0000 7f9416a00640 1 -- start start 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 0x7f941009cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100ce600 0x7f9410070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9410070fd0 0x7f94101f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100c9ba0 con 0x7f94100cdc70 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941009d130 con 0x7f94100ce600 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.307+0000 7f9416a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941009adf0 con 0x7f9410070fd0 2024-05-10T03:36:04.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 0x7f941009cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 0x7f941009cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54682/0 (socket says 172.21.15.55:54682) 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 -- 172.21.15.55:0/677004633 learned_addr learned my addr 172.21.15.55:0/677004633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940f400640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100ce600 0x7f9410070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f9414a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9410070fd0 0x7f94101f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 -- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9410070fd0 msgr2=0x7f94101f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9410070fd0 0x7f94101f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 -- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100ce600 msgr2=0x7f9410070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94100ce600 0x7f9410070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f940fe00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400009400 con 0x7f94100cdc70 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.308+0000 7f9414a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9410070fd0 0x7f94101f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:04.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f940fe00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 0x7f941009cbf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94000098c0 tx=0x7f94000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400006e10 con 0x7f94100cdc70 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940003d630 con 0x7f94100cdc70 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940003d9a0 con 0x7f94100cdc70 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94101f34c0 con 0x7f94100cdc70 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.309+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94101f3710 con 0x7f94100cdc70 2024-05-10T03:36:04.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.310+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94100cd380 con 0x7f94100cdc70 2024-05-10T03:36:04.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.311+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f940003db00 con 0x7f94100cdc70 2024-05-10T03:36:04.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.312+0000 7f9407e00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93f006ac00 0x7f93f006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f94000a9cc0 con 0x7f94100cdc70 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f9407e00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7f93f00711d0 0x7f93f0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93f0073d30 con 0x7f93f00711d0 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94000ac2a0 con 0x7f94100cdc70 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f940f400640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93f006ac00 0x7f93f006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.313+0000 7f9414a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7f93f00711d0 0x7f93f0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.314+0000 7f9414a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7f93f00711d0 0x7f93f0073660 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.314+0000 7f940f400640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93f006ac00 0x7f93f006d110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f94100cceb0 tx=0x7f93fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.315+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== osd.1 v2:172.21.15.55:6822/35474 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f93f0073d30 con 0x7f93f00711d0 2024-05-10T03:36:04.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.331+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 --> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94100c9ce0 con 0x7f93f00711d0 2024-05-10T03:36:04.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.332+0000 7f9407e00640 1 -- 172.21.15.55:0/677004633 <== osd.1 v2:172.21.15.55:6822/35474 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f94100c9ce0 con 0x7f93f00711d0 2024-05-10T03:36:04.336 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:04.336 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:04.336 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.332+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7f93f00711d0 msgr2=0x7f93f0073660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.332+0000 7f9416a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:6822/35474,v1:172.21.15.55:6825/35474] conn(0x7f93f00711d0 0x7f93f0073660 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93f006ac00 msgr2=0x7f93f006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f93f006ac00 0x7f93f006d110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f94100cceb0 tx=0x7f93fc009040 comp rx=0 tx=0).stop 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 msgr2=0x7f941009cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 --2- 172.21.15.55:0/677004633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94100cdc70 0x7f941009cbf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f94000098c0 tx=0x7f94000046c0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9414a00640 1 -- 172.21.15.55:0/677004633 reap_dead start 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 shutdown_connections 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 >> 172.21.15.55:0/677004633 conn(0x7f94100b0cf0 msgr2=0x7f9410071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 shutdown_connections 2024-05-10T03:36:04.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.333+0000 7f9416a00640 1 -- 172.21.15.55:0/677004633 wait complete. 2024-05-10T03:36:04.366 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-10T03:36:04.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.647+0000 7f4eae800640 1 Processor -- start 2024-05-10T03:36:04.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.647+0000 7f4eae800640 1 -- start start 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4eae800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 0x7f4ea8141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4eae800640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ea80b2140 0x7f4ea8142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4eae800640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea806ad20 con 0x7f4ea80b2550 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4eae800640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea8141f50 con 0x7f4ea80b2140 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4eae800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea806a110 con 0x7f4ea815fe50 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4ea6a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ea80b2140 0x7f4ea8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4ea6a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ea80b2140 0x7f4ea8142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54696/0 (socket says 172.21.15.55:54696) 2024-05-10T03:36:04.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.649+0000 7f4ea6a00640 1 -- 172.21.15.55:0/3124991221 learned_addr learned my addr 172.21.15.55:0/3124991221 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea7400640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 0x7f4ea8141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea6a00640 1 -- 172.21.15.55:0/3124991221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 msgr2=0x7f4ea8141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea6a00640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 0x7f4ea8141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea6a00640 1 -- 172.21.15.55:0/3124991221 >> v1:172.21.15.162:6789/0 conn(0x7f4ea80b2550 legacy=0x7f4ea806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea6a00640 1 -- 172.21.15.55:0/3124991221 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea806a3f0 con 0x7f4ea80b2140 2024-05-10T03:36:04.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.650+0000 7f4ea7400640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 0x7f4ea8141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:04.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.651+0000 7f4ea6a00640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2140 0x7f4ea8142440 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009870 tx=0x7f4e9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb350742e035bde server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.658+0000 7f4ea6000640 1 -- 172.21.15.55:0/3124991221 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90032440 con 0x7f4ea80b2140 2024-05-10T03:36:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.658+0000 7f4ea6000640 1 -- 172.21.15.55:0/3124991221 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e900325a0 con 0x7f4ea80b2140 2024-05-10T03:36:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.658+0000 7f4ea6000640 1 -- 172.21.15.55:0/3124991221 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90032930 con 0x7f4ea80b2140 2024-05-10T03:36:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 -- 172.21.15.55:0/3124991221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2140 msgr2=0x7f4ea8142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2140 0x7f4ea8142440 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009870 tx=0x7f4e9002a2a0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 -- 172.21.15.55:0/3124991221 shutdown_connections 2024-05-10T03:36:04.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2140 0x7f4ea8142440 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 --2- 172.21.15.55:0/3124991221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea815fe50 0x7f4ea8141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.659+0000 7f4eae800640 1 -- 172.21.15.55:0/3124991221 >> 172.21.15.55:0/3124991221 conn(0x7f4ea80cbb90 msgr2=0x7f4ea80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:04.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.660+0000 7f4eae800640 1 -- 172.21.15.55:0/3124991221 shutdown_connections 2024-05-10T03:36:04.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.660+0000 7f4eae800640 1 -- 172.21.15.55:0/3124991221 wait complete. 2024-05-10T03:36:04.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.662+0000 7f4eae800640 1 Processor -- start 2024-05-10T03:36:04.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.662+0000 7f4eae800640 1 -- start start 2024-05-10T03:36:04.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea80b2140 0x7f4ea812d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 0x7f4ea815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea812db90 con 0x7f4ea80b2550 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea815dfa0 con 0x7f4ea815fe50 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4eae800640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea812d6c0 con 0x7f4ea80b2140 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54710/0 (socket says 172.21.15.55:54710) 2024-05-10T03:36:04.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4ea6a00640 1 -- 172.21.15.55:0/1861274227 learned_addr learned my addr 172.21.15.55:0/1861274227 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:04.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.664+0000 7f4ea7400640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea80b2140 0x7f4ea812d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 -- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea80b2140 msgr2=0x7f4ea812d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea80b2140 0x7f4ea812d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea7e00640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 0x7f4ea815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 -- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 msgr2=0x7f4ea815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 0x7f4ea815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 -- 172.21.15.55:0/1861274227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900094d0 con 0x7f4ea80b2550 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea7e00640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 0x7f4ea815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4ea6a00640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009870 tx=0x7f4e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90030b00 con 0x7f4ea80b2550 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.665+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e90030c60 con 0x7f4ea80b2550 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.666+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9003d420 con 0x7f4ea80b2550 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.666+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea815d3b0 con 0x7f4ea80b2550 2024-05-10T03:36:04.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.666+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea815d950 con 0x7f4ea80b2550 2024-05-10T03:36:04.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.667+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea8066f60 con 0x7f4ea80b2550 2024-05-10T03:36:04.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.668+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4e90039040 con 0x7f4ea80b2550 2024-05-10T03:36:04.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.669+0000 7f4e9f400640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 0x7f4e8406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:04.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.669+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f4e900a9e60 con 0x7f4ea80b2550 2024-05-10T03:36:04.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.670+0000 7f4ea7400640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 0x7f4e8406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:04.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.671+0000 7f4ea7400640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 0x7f4e8406d110 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b840 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:04.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.674+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4e900750a0 con 0x7f4ea80b2550 2024-05-10T03:36:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.878+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea8067220 con 0x7f4e8406ac00 2024-05-10T03:36:04.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.879+0000 7f4e9f400640 1 -- 172.21.15.55:0/1861274227 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ea8067220 con 0x7f4e8406ac00 2024-05-10T03:36:04.883 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 1 to deep-scrub 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 msgr2=0x7f4e8406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 0x7f4e8406d110 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b840 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 msgr2=0x7f4ea815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009870 tx=0x7f4e900046c0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 shutdown_connections 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f4e8406ac00 0x7f4e8406d110 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ea815fe50 0x7f4ea815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea80b2550 0x7f4ea815ae60 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 --2- 172.21.15.55:0/1861274227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ea80b2140 0x7f4ea812d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 >> 172.21.15.55:0/1861274227 conn(0x7f4ea80cbb90 msgr2=0x7f4ea80b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 shutdown_connections 2024-05-10T03:36:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:04.882+0000 7f4eae800640 1 -- 172.21.15.55:0/1861274227 wait complete. 2024-05-10T03:36:04.914 INFO:tasks.ceph:Scrubbing osd.2 2024-05-10T03:36:04.914 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.203+0000 7fd584a00640 1 Processor -- start 2024-05-10T03:36:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.203+0000 7fd584a00640 1 -- start start 2024-05-10T03:36:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.205+0000 7fd584a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 0x7fd58006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.205+0000 7fd584a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd58006b840 0x7fd5800cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.205+0000 7fd584a00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd580071b80 con 0x7fd580071e70 2024-05-10T03:36:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.205+0000 7fd584a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd580072280 con 0x7fd58006b840 2024-05-10T03:36:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.205+0000 7fd584a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58015bc20 con 0x7fd58015f210 2024-05-10T03:36:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.206+0000 7fd57de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 0x7fd58006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.206+0000 7fd57f200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd58006b840 0x7fd5800cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.206+0000 7fd57de00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 0x7fd58006b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.55:55914/0 (socket says 172.21.15.55:55914) 2024-05-10T03:36:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.206+0000 7fd57de00640 1 -- 172.21.15.55:0/2028718047 learned_addr learned my addr 172.21.15.55:0/2028718047 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.207+0000 7fd57f200640 1 -- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 msgr2=0x7fd58006b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.207+0000 7fd57f200640 1 --2- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 0x7fd58006b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.207+0000 7fd57f200640 1 -- 172.21.15.55:0/2028718047 >> v1:172.21.15.162:6789/0 conn(0x7fd580071e70 legacy=0x7fd58015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.207+0000 7fd57f200640 1 -- 172.21.15.55:0/2028718047 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58015ca60 con 0x7fd58006b840 2024-05-10T03:36:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.209+0000 7fd57f200640 1 --2- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58006b840 0x7fd5800cb900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5780097a0 tx=0x7fd57802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e87ac69407233f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.209+0000 7fd57d400640 1 -- 172.21.15.55:0/2028718047 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57802ae50 con 0x7fd58006b840 2024-05-10T03:36:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.209+0000 7fd57d400640 1 -- 172.21.15.55:0/2028718047 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578032440 con 0x7fd58006b840 2024-05-10T03:36:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.209+0000 7fd57d400640 1 -- 172.21.15.55:0/2028718047 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780327d0 con 0x7fd58006b840 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 -- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58006b840 msgr2=0x7fd5800cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 --2- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58006b840 0x7fd5800cb900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5780097a0 tx=0x7fd57802a100 comp rx=0 tx=0).stop 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 -- 172.21.15.55:0/2028718047 shutdown_connections 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 --2- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd58015f210 0x7fd58006b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 --2- 172.21.15.55:0/2028718047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58006b840 0x7fd5800cb900 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 -- 172.21.15.55:0/2028718047 >> 172.21.15.55:0/2028718047 conn(0x7fd58006c070 msgr2=0x7fd58006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 -- 172.21.15.55:0/2028718047 shutdown_connections 2024-05-10T03:36:05.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.210+0000 7fd584a00640 1 -- 172.21.15.55:0/2028718047 wait complete. 2024-05-10T03:36:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.214+0000 7fd584a00640 1 Processor -- start 2024-05-10T03:36:05.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.214+0000 7fd584a00640 1 -- start start 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 0x7fd5800acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd580071e70 0x7fd5801f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58015f210 0x7fd580137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800b0000 con 0x7fd58006b840 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800b2f20 con 0x7fd58015f210 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd584a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800b3a10 con 0x7fd580071e70 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd57e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 0x7fd5800acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd57e800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 0x7fd5800acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54722/0 (socket says 172.21.15.55:54722) 2024-05-10T03:36:05.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.216+0000 7fd57e800640 1 -- 172.21.15.55:0/4225902190 learned_addr learned my addr 172.21.15.55:0/4225902190 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57f200640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58015f210 0x7fd580137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57de00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd580071e70 0x7fd5801f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 -- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd580071e70 msgr2=0x7fd5801f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd580071e70 0x7fd5801f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 -- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58015f210 msgr2=0x7fd580137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58015f210 0x7fd580137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009400 con 0x7fd58006b840 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57de00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd580071e70 0x7fd5801f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.217+0000 7fd57e800640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 0x7fd5800acb80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c400 tx=0x7fd57000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.218+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5700052d0 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.218+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5801f3b80 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.218+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570005430 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.218+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570003830 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.218+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5800b3450 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.219+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd580145120 con 0x7fd58006b840 2024-05-10T03:36:05.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.220+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd570003990 con 0x7fd58006b840 2024-05-10T03:36:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.221+0000 7fd576a00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd55c06ac00 0x7fd55c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.221+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fd570088a60 con 0x7fd58006b840 2024-05-10T03:36:05.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.221+0000 7fd576a00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fd55c0711d0 0x7fd55c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.221+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd55c073d30 con 0x7fd55c0711d0 2024-05-10T03:36:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.221+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57008d2a0 con 0x7fd58006b840 2024-05-10T03:36:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.222+0000 7fd57de00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd55c06ac00 0x7fd55c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.222+0000 7fd57f200640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fd55c0711d0 0x7fd55c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.223+0000 7fd57f200640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fd55c0711d0 0x7fd55c073660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.223+0000 7fd57de00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd55c06ac00 0x7fd55c06d110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd568009440 tx=0x7fd56800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.223+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== osd.2 v2:172.21.15.55:6800/35475 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd55c073d30 con 0x7fd55c0711d0 2024-05-10T03:36:05.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.238+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 --> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd58015fed0 con 0x7fd55c0711d0 2024-05-10T03:36:05.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd576a00640 1 -- 172.21.15.55:0/4225902190 <== osd.2 v2:172.21.15.55:6800/35475 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd578006e10 con 0x7fd55c0711d0 2024-05-10T03:36:05.243 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:05.243 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:05.243 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fd55c0711d0 msgr2=0x7fd55c073660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:6800/35475,v1:172.21.15.55:6801/35475] conn(0x7fd55c0711d0 0x7fd55c073660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd55c06ac00 msgr2=0x7fd55c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fd55c06ac00 0x7fd55c06d110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd568009440 tx=0x7fd56800b040 comp rx=0 tx=0).stop 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 msgr2=0x7fd5800acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd584a00640 1 --2- 172.21.15.55:0/4225902190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd58006b840 0x7fd5800acb80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c400 tx=0x7fd57000c8c0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.239+0000 7fd57f200640 1 -- 172.21.15.55:0/4225902190 reap_dead start 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.240+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 shutdown_connections 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.240+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 >> 172.21.15.55:0/4225902190 conn(0x7fd58006c070 msgr2=0x7fd580141720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.240+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 shutdown_connections 2024-05-10T03:36:05.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.240+0000 7fd584a00640 1 -- 172.21.15.55:0/4225902190 wait complete. 2024-05-10T03:36:05.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-10T03:36:05.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.559+0000 7f9bb3200640 1 Processor -- start 2024-05-10T03:36:05.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.560+0000 7f9bb3200640 1 -- start start 2024-05-10T03:36:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.561+0000 7f9bb3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 0x7f9bb4144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bb406d7e0 0x7f9bb40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb3200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9bb40b2fc0 0x7f9bb415eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb406d350 con 0x7f9bb406d7e0 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb3200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb406c740 con 0x7f9bb40b2fc0 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb406ca20 con 0x7f9bb4144710 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bb406d7e0 0x7f9bb40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bb406d7e0 0x7f9bb40b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54732/0 (socket says 172.21.15.55:54732) 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 -- 172.21.15.55:0/4219170750 learned_addr learned my addr 172.21.15.55:0/4219170750 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:05.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb1e00640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 0x7f9bb4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9bb2800640 1 --2- 172.21.15.55:0/4219170750 >> v2:172.21.15.162:3300/0 conn(0x7f9bb40b2fc0 0x7f9bb415eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 -- 172.21.15.55:0/4219170750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 msgr2=0x7f9bb4144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 0x7f9bb4144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 -- 172.21.15.55:0/4219170750 >> v2:172.21.15.162:3300/0 conn(0x7f9bb40b2fc0 msgr2=0x7f9bb415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 --2- 172.21.15.55:0/4219170750 >> v2:172.21.15.162:3300/0 conn(0x7f9bb40b2fc0 0x7f9bb415eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.562+0000 7f9babe00640 1 -- 172.21.15.55:0/4219170750 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb406cd00 con 0x7f9bb406d7e0 2024-05-10T03:36:05.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.563+0000 7f9bb1e00640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 0x7f9bb4144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:05.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.564+0000 7f9babe00640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb406d7e0 0x7f9bb40b2a80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b5200 tx=0x7f9b9c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8c8c925690effe server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.564+0000 7f9bb1400640 1 -- 172.21.15.55:0/4219170750 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c02ae90 con 0x7f9bb406d7e0 2024-05-10T03:36:05.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.564+0000 7f9bb1400640 1 -- 172.21.15.55:0/4219170750 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c032440 con 0x7f9bb406d7e0 2024-05-10T03:36:05.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.564+0000 7f9bb1400640 1 -- 172.21.15.55:0/4219170750 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0327d0 con 0x7f9bb406d7e0 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 -- 172.21.15.55:0/4219170750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb406d7e0 msgr2=0x7f9bb40b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb406d7e0 0x7f9bb40b2a80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b5200 tx=0x7f9b9c02a140 comp rx=0 tx=0).stop 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 -- 172.21.15.55:0/4219170750 shutdown_connections 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4219170750 >> v2:172.21.15.162:3300/0 conn(0x7f9bb40b2fc0 0x7f9bb415eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb406d7e0 0x7f9bb40b2a80 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4219170750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb4144710 0x7f9bb4144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 -- 172.21.15.55:0/4219170750 >> 172.21.15.55:0/4219170750 conn(0x7f9bb406a110 msgr2=0x7f9bb406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 -- 172.21.15.55:0/4219170750 shutdown_connections 2024-05-10T03:36:05.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.565+0000 7f9bb3200640 1 -- 172.21.15.55:0/4219170750 wait complete. 2024-05-10T03:36:05.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.567+0000 7f9bb3200640 1 Processor -- start 2024-05-10T03:36:05.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.567+0000 7f9bb3200640 1 -- start start 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb406d7e0 0x7f9bb4136460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb40b2fc0 0x7f9bb415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb41369a0 con 0x7f9bb4144710 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb415dbc0 con 0x7f9bb40b2fc0 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4138a80 con 0x7f9bb406d7e0 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54748/0 (socket says 172.21.15.55:54748) 2024-05-10T03:36:05.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.569+0000 7f9bb2800640 1 -- 172.21.15.55:0/4042307517 learned_addr learned my addr 172.21.15.55:0/4042307517 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9babe00640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb40b2fc0 0x7f9bb415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb1e00640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb406d7e0 0x7f9bb4136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 -- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb406d7e0 msgr2=0x7f9bb4136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb406d7e0 0x7f9bb4136460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 -- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb40b2fc0 msgr2=0x7f9bb415d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb40b2fc0 0x7f9bb415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 -- 172.21.15.55:0/4042307517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0094d0 con 0x7f9bb4144710 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9bb2800640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00cc60 tx=0x7f9bac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.570+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00ce70 con 0x7f9bb4144710 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.571+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac003990 con 0x7f9bb4144710 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.571+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003d20 con 0x7f9bb4144710 2024-05-10T03:36:05.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.571+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb4136dd0 con 0x7f9bb4144710 2024-05-10T03:36:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.571+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4070290 con 0x7f9bb4144710 2024-05-10T03:36:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.572+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb4066ce0 con 0x7f9bb4144710 2024-05-10T03:36:05.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.573+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9bac003e80 con 0x7f9bb4144710 2024-05-10T03:36:05.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.574+0000 7f9baa000640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 0x7f9b9006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:05.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.574+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f9bac089750 con 0x7f9bb4144710 2024-05-10T03:36:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.574+0000 7f9bb1e00640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 0x7f9b9006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:05.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.576+0000 7f9bb1e00640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 0x7f9b9006d140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b4ad0 tx=0x7f9ba4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:05.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.579+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9bac054900 con 0x7f9bb4144710 2024-05-10T03:36:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.780+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb40cec70 con 0x7f9b9006ac30 2024-05-10T03:36:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.781+0000 7f9baa000640 1 -- 172.21.15.55:0/4042307517 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9bb40cec70 con 0x7f9b9006ac30 2024-05-10T03:36:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 2 to deep-scrub 2024-05-10T03:36:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.783+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 msgr2=0x7f9b9006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.783+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 0x7f9b9006d140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40b4ad0 tx=0x7f9ba4008040 comp rx=0 tx=0).stop 2024-05-10T03:36:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.783+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 msgr2=0x7f9bb4071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:05.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.783+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00cc60 tx=0x7f9bac00a5c0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 shutdown_connections 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bb4144710 0x7f9bb4071050 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f9b9006ac30 0x7f9b9006d140 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9bb40b2fc0 0x7f9bb415d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 --2- 172.21.15.55:0/4042307517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9bb406d7e0 0x7f9bb4136460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 >> 172.21.15.55:0/4042307517 conn(0x7f9bb406a110 msgr2=0x7f9bb40cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 shutdown_connections 2024-05-10T03:36:05.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:05.784+0000 7f9bb3200640 1 -- 172.21.15.55:0/4042307517 wait complete. 2024-05-10T03:36:05.813 INFO:tasks.ceph:Scrubbing osd.3 2024-05-10T03:36:05.814 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:06.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.094+0000 7f633b200640 1 Processor -- start 2024-05-10T03:36:06.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.094+0000 7f633b200640 1 -- start start 2024-05-10T03:36:06.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f633c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633b200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0b44a0 con 0x7f633c0ce380 2024-05-10T03:36:06.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633b200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0b3a00 con 0x7f633c0b35f0 2024-05-10T03:36:06.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0b3ee0 con 0x7f633c15e0e0 2024-05-10T03:36:06.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633a800640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f633c0ce380 0x7f633c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42750/0 (socket says 172.21.15.55:42750) 2024-05-10T03:36:06.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f633a800640 1 -- 172.21.15.55:0/572981249 learned_addr learned my addr 172.21.15.55:0/572981249 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:06.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.097+0000 7f6339400640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f633c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915071692 0 0) 0x7f633c0b44a0 con 0x7f633c0ce380 2024-05-10T03:36:06.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6320003520 con 0x7f633c0ce380 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2780302599 0 0) 0x7f6320003520 con 0x7f633c0ce380 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 msgr2=0x7f633c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f633c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 >> v1:172.21.15.162:6789/0 conn(0x7f633c0b35f0 legacy=0x7f633c1407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0b41c0 con 0x7f633c0ce380 2024-05-10T03:36:06.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f6334001a10 con 0x7f633c0ce380 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 >> v1:172.21.15.55:6789/0 conn(0x7f633c0ce380 legacy=0x7f633c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200039a0 0x7f6320003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f63200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f6320005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0b3ee0 con 0x7f6320004d00 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632000bb40 con 0x7f63200039a0 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.098+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632000df10 con 0x7f633c15e0e0 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f6320005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f6339e00640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200039a0 0x7f6320003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f6339400640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f63200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 -- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 msgr2=0x7f63200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c15e0e0 0x7f63200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 -- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200039a0 msgr2=0x7f6320003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200039a0 0x7f6320003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.099+0000 7f633a800640 1 -- 172.21.15.55:0/572981249 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6320003520 con 0x7f6320004d00 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.100+0000 7f633a800640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f6320005110 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f633c15ea50 tx=0x7f63200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7750514416cf0c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633402b9d0 con 0x7f6320004d00 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633402bb30 con 0x7f6320004d00 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f6338a00640 1 -- 172.21.15.55:0/572981249 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334034470 con 0x7f6320004d00 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f633b200640 1 -- 172.21.15.55:0/572981249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 msgr2=0x7f6320005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f633b200640 1 --2- 172.21.15.55:0/572981249 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f6320005110 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f633c15ea50 tx=0x7f63200057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.101+0000 7f633a800640 1 -- 172.21.15.55:0/572981249 reap_dead start 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.102+0000 7f633b200640 1 -- 172.21.15.55:0/572981249 shutdown_connections 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.102+0000 7f633b200640 1 -- 172.21.15.55:0/572981249 >> 172.21.15.55:0/572981249 conn(0x7f633c06c070 msgr2=0x7f633c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.102+0000 7f633b200640 1 -- 172.21.15.55:0/572981249 shutdown_connections 2024-05-10T03:36:06.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.102+0000 7f633b200640 1 -- 172.21.15.55:0/572981249 wait complete. 2024-05-10T03:36:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.104+0000 7f633b200640 1 Processor -- start 2024-05-10T03:36:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.104+0000 7f633b200640 1 -- start start 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f633c13dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c13f100 0x7f633c1f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c141860 0x7f633c141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c13d020 con 0x7f6320004d00 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c13e7a0 con 0x7f633c13f100 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f633b200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1f3ef0 con 0x7f633c141860 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f6339e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f633c13dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f6339e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f633c13dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54766/0 (socket says 172.21.15.55:54766) 2024-05-10T03:36:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.106+0000 7f6339e00640 1 -- 172.21.15.55:0/1468627672 learned_addr learned my addr 172.21.15.55:0/1468627672 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 -- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c141860 msgr2=0x7f633c141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f633c141860 0x7f633c141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 -- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c13f100 msgr2=0x7f633c1f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c13f100 0x7f633c1f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c140e10 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6339e00640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f633c13dd90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e550 tx=0x7f632c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0052e0 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c005440 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c13e2d0 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c003810 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.107+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1f4420 con 0x7f6320004d00 2024-05-10T03:36:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.108+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f633c143f80 con 0x7f6320004d00 2024-05-10T03:36:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.110+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f632c003970 con 0x7f6320004d00 2024-05-10T03:36:06.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6332000640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631c06ac00 0x7f631c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f632c089ac0 con 0x7f6320004d00 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6332000640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f631c071220 0x7f631c0736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f631c073d80 con 0x7f631c071220 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f632c089e30 con 0x7f6320004d00 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f6339400640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631c06ac00 0x7f631c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.111+0000 7f633a800640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f631c071220 0x7f631c0736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.112+0000 7f633a800640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f631c071220 0x7f631c0736b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.113+0000 7f6339400640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631c06ac00 0x7f631c06d110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f633c13dce0 tx=0x7f6328008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.113+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== osd.3 v2:172.21.15.55:6808/35476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f631c073d80 con 0x7f631c071220 2024-05-10T03:36:06.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.128+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 --> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f633c0cd5c0 con 0x7f631c071220 2024-05-10T03:36:06.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.128+0000 7f6332000640 1 -- 172.21.15.55:0/1468627672 <== osd.3 v2:172.21.15.55:6808/35476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f633c0cd5c0 con 0x7f631c071220 2024-05-10T03:36:06.132 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:06.132 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:06.132 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f631c071220 msgr2=0x7f631c0736b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:6808/35476,v1:172.21.15.55:6809/35476] conn(0x7f631c071220 0x7f631c0736b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631c06ac00 msgr2=0x7f631c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f631c06ac00 0x7f631c06d110 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f633c13dce0 tx=0x7f6328008040 comp rx=0 tx=0).stop 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 msgr2=0x7f633c13dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 --2- 172.21.15.55:0/1468627672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6320004d00 0x7f633c13dd90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e550 tx=0x7f632c00ea10 comp rx=0 tx=0).stop 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633a800640 1 -- 172.21.15.55:0/1468627672 reap_dead start 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 shutdown_connections 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 >> 172.21.15.55:0/1468627672 conn(0x7f633c06c070 msgr2=0x7f633c06afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 shutdown_connections 2024-05-10T03:36:06.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.129+0000 7f633b200640 1 -- 172.21.15.55:0/1468627672 wait complete. 2024-05-10T03:36:06.156 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-10T03:36:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.437+0000 7fe096000640 1 Processor -- start 2024-05-10T03:36:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.437+0000 7fe096000640 1 -- start start 2024-05-10T03:36:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.439+0000 7fe096000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 0x7fe09015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.439+0000 7fe096000640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900d25a0 con 0x7fe0900b2170 2024-05-10T03:36:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.439+0000 7fe096000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900b28d0 con 0x7fe0901618b0 2024-05-10T03:36:06.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.439+0000 7fe096000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900d2b20 con 0x7fe0900b6680 2024-05-10T03:36:06.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.440+0000 7fe08fe00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe0900b2170 0x7fe0901611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42764/0 (socket says 172.21.15.55:42764) 2024-05-10T03:36:06.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.440+0000 7fe08fe00640 1 -- 172.21.15.55:0/276838667 learned_addr learned my addr 172.21.15.55:0/276838667 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:06.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.440+0000 7fe094c00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 0x7fe09015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.440+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236008982 0 0) 0x7fe0900d25a0 con 0x7fe0900b2170 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.440+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe078003520 con 0x7fe0900b2170 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2445563529 0 0) 0x7fe078003520 con 0x7fe0900b2170 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 msgr2=0x7fe09015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 0x7fe09015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 >> v1:172.21.15.162:6789/0 conn(0x7fe0901618b0 legacy=0x7fe09006aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0900d2840 con 0x7fe0900b2170 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7fe07c001670 con 0x7fe0900b2170 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 >> v1:172.21.15.55:6789/0 conn(0x7fe0900b2170 legacy=0x7fe0901611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe078003a10 0x7fe078003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0780047c0 0x7fe078004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 0x7fe078005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900d2b20 con 0x7fe078003a10 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07800bb10 con 0x7fe0780047c0 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07800bcd0 con 0x7fe0900b6680 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.441+0000 7fe094c00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe078003a10 0x7fe078003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe094c00640 1 -- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 msgr2=0x7fe078005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe094c00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0900b6680 0x7fe078005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe08fe00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0780047c0 0x7fe078004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe094c00640 1 -- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0780047c0 msgr2=0x7fe078004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe094c00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0780047c0 0x7fe078004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.442+0000 7fe094c00640 1 -- 172.21.15.55:0/276838667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078003520 con 0x7fe078003a10 2024-05-10T03:36:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.443+0000 7fe094c00640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe078003a10 0x7fe078003e80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe084009ac0 tx=0x7fe0780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451bdfdd3d9e1a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.443+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084031400 con 0x7fe078003a10 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.443+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08402d050 con 0x7fe078003a10 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.443+0000 7fe08f400640 1 -- 172.21.15.55:0/276838667 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084031780 con 0x7fe078003a10 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 -- 172.21.15.55:0/276838667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe078003a10 msgr2=0x7fe078003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 --2- 172.21.15.55:0/276838667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe078003a10 0x7fe078003e80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe084009ac0 tx=0x7fe0780057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe095600640 1 -- 172.21.15.55:0/276838667 reap_dead start 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 -- 172.21.15.55:0/276838667 shutdown_connections 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 -- 172.21.15.55:0/276838667 >> 172.21.15.55:0/276838667 conn(0x7fe0900b5730 msgr2=0x7fe0900d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 -- 172.21.15.55:0/276838667 shutdown_connections 2024-05-10T03:36:06.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.444+0000 7fe096000640 1 -- 172.21.15.55:0/276838667 wait complete. 2024-05-10T03:36:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.446+0000 7fe096000640 1 Processor -- start 2024-05-10T03:36:06.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.447+0000 7fe096000640 1 -- start start 2024-05-10T03:36:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe078003a10 0x7fe0901e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09014b3c0 0x7fe0901e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09015db20 con 0x7fe09014b7d0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901f4530 con 0x7fe078003a10 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe096000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09015ef20 con 0x7fe09014b3c0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe095600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe095600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54778/0 (socket says 172.21.15.55:54778) 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.448+0000 7fe095600640 1 -- 172.21.15.55:0/2637444383 learned_addr learned my addr 172.21.15.55:0/2637444383 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe094c00640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe078003a10 0x7fe0901e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe08fe00640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09014b3c0 0x7fe0901e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 -- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09014b3c0 msgr2=0x7fe0901e9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09014b3c0 0x7fe0901e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 -- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe078003a10 msgr2=0x7fe0901e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe078003a10 0x7fe0901e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 -- 172.21.15.55:0/2637444383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0901e7f00 con 0x7fe09014b7d0 2024-05-10T03:36:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe095600640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe09006c220 tx=0x7fe08800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088005d40 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08800ced0 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800a850 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.449+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09015f730 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.450+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09015f450 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.451+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe090067740 con 0x7fe09014b7d0 2024-05-10T03:36:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.452+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe08800a9b0 con 0x7fe09014b7d0 2024-05-10T03:36:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.453+0000 7fe08cc00640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 0x7fe07006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.453+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fe088088550 con 0x7fe09014b7d0 2024-05-10T03:36:06.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.453+0000 7fe094c00640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 0x7fe07006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.455+0000 7fe094c00640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 0x7fe07006d140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe084033000 tx=0x7fe084008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.457+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe088053660 con 0x7fe09014b7d0 2024-05-10T03:36:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.657+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe09006a590 con 0x7fe07006ac30 2024-05-10T03:36:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.658+0000 7fe08cc00640 1 -- 172.21.15.55:0/2637444383 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe084004210 con 0x7fe07006ac30 2024-05-10T03:36:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 3 to deep-scrub 2024-05-10T03:36:06.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.660+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 msgr2=0x7fe07006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.660+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 0x7fe07006d140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe084033000 tx=0x7fe084008030 comp rx=0 tx=0).stop 2024-05-10T03:36:06.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.660+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 msgr2=0x7fe0901e6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.660+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe09006c220 tx=0x7fe08800cb90 comp rx=0 tx=0).stop 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 shutdown_connections 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe09014b7d0 0x7fe0901e6fc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe09014b3c0 0x7fe0901e9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fe07006ac30 0x7fe07006d140 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 --2- 172.21.15.55:0/2637444383 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe078003a10 0x7fe0901e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 >> 172.21.15.55:0/2637444383 conn(0x7fe0900b5730 msgr2=0x7fe09006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 shutdown_connections 2024-05-10T03:36:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.661+0000 7fe096000640 1 -- 172.21.15.55:0/2637444383 wait complete. 2024-05-10T03:36:06.693 INFO:tasks.ceph:Scrubbing osd.4 2024-05-10T03:36:06.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:06.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.984+0000 7f72a1200640 1 Processor -- start 2024-05-10T03:36:06.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.985+0000 7f72a1200640 1 -- start start 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f72a1200640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f729c0cb900 0x7f729c0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f72a1200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c071b80 con 0x7f729c15f210 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f72a1200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0cc250 con 0x7f729c071e70 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f72a1200640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c072280 con 0x7f729c0cb900 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f7293e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f729c15f210 0x7f729c06b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42770/0 (socket says 172.21.15.55:42770) 2024-05-10T03:36:06.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.987+0000 7f7293e00640 1 -- 172.21.15.55:0/3586273924 learned_addr learned my addr 172.21.15.55:0/3586273924 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729be00640 1 --2- 172.21.15.55:0/3586273924 >> v2:172.21.15.162:3300/0 conn(0x7f729c0cb900 0x7f729c0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059154854 0 0) 0x7f729c071b80 con 0x7f729c15f210 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7280003520 con 0x7f729c15f210 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540628133 0 0) 0x7f7280003520 con 0x7f729c15f210 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 >> v2:172.21.15.162:3300/0 conn(0x7f729c0cb900 msgr2=0x7f729c0cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 --2- 172.21.15.55:0/3586273924 >> v2:172.21.15.162:3300/0 conn(0x7f729c0cb900 0x7f729c0cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.988+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 >> v1:172.21.15.162:6789/0 conn(0x7f729c071e70 legacy=0x7f729c15c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c15ca60 con 0x7f729c15f210 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7284001670 con 0x7f729c15f210 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 >> v1:172.21.15.55:6789/0 conn(0x7f729c15f210 legacy=0x7f729c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7280003970 0x7f7280003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72800047c0 0x7f7280004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f728000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728000c170 con 0x7f7280005210 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728000c2e0 con 0x7f7280003970 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728000c4a0 con 0x7f72800047c0 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729be00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f728000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.989+0000 7f729b400640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7280003970 0x7f7280003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f7293e00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72800047c0 0x7f7280004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f729be00640 1 -- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72800047c0 msgr2=0x7f7280004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f729be00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72800047c0 0x7f7280004bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f729be00640 1 -- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7280003970 msgr2=0x7f7280003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:06.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f729be00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7280003970 0x7f7280003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.990+0000 7f729be00640 1 -- 172.21.15.55:0/3586273924 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280003520 con 0x7f7280005210 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.991+0000 7f729be00640 1 -- 172.21.15.55:0/3586273924 reap_dead start 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.991+0000 7f729be00640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f728000b980 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7294009fc0 tx=0x7f72800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e1fd21f5dc3f10 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.991+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294003b90 con 0x7f7280005210 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.991+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400f040 con 0x7f7280005210 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.991+0000 7f729aa00640 1 -- 172.21.15.55:0/3586273924 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294004020 con 0x7f7280005210 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 -- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 msgr2=0x7f728000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f728000b980 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7294009fc0 tx=0x7f72800057f0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 -- 172.21.15.55:0/3586273924 shutdown_connections 2024-05-10T03:36:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 --2- 172.21.15.55:0/3586273924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f728000b980 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 -- 172.21.15.55:0/3586273924 >> 172.21.15.55:0/3586273924 conn(0x7f729c06c070 msgr2=0x7f729c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 -- 172.21.15.55:0/3586273924 shutdown_connections 2024-05-10T03:36:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.992+0000 7f72a1200640 1 -- 172.21.15.55:0/3586273924 wait complete. 2024-05-10T03:36:06.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.995+0000 7f72a1200640 1 Processor -- start 2024-05-10T03:36:06.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.995+0000 7f72a1200640 1 -- start start 2024-05-10T03:36:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f729c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f729c0c9120 0x7f729c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f729c15aa20 0x7f729c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c0b2060 con 0x7f7280005210 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1f38d0 con 0x7f729c15aa20 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f72a1200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c15b140 con 0x7f729c0c9120 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f729b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f729c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f729b400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f729c1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54788/0 (socket says 172.21.15.55:54788) 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.997+0000 7f729b400640 1 -- 172.21.15.55:0/670992999 learned_addr learned my addr 172.21.15.55:0/670992999 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f7293e00640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f729c0c9120 0x7f729c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 -- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f729c0c9120 msgr2=0x7f729c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f729c0c9120 0x7f729c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 -- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f729c15aa20 msgr2=0x7f729c0b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f729c15aa20 0x7f729c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c15b420 con 0x7f7280005210 2024-05-10T03:36:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f729b400640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f729c1542d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f728c00e480 tx=0x7f728c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0052e0 con 0x7f7280005210 2024-05-10T03:36:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c005440 con 0x7f7280005210 2024-05-10T03:36:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c003830 con 0x7f7280005210 2024-05-10T03:36:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.998+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c0b2270 con 0x7f7280005210 2024-05-10T03:36:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:06.999+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c15ae30 con 0x7f7280005210 2024-05-10T03:36:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.000+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f729c15fed0 con 0x7f7280005210 2024-05-10T03:36:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.001+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f728c003990 con 0x7f7280005210 2024-05-10T03:36:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.002+0000 7f7293400640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f727806ac30 0x7f727806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.002+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f728c089160 con 0x7f7280005210 2024-05-10T03:36:07.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.002+0000 7f7293400640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f7278071200 0x7f7278073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.002+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7278073d60 con 0x7f7278071200 2024-05-10T03:36:07.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.002+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f728c08b740 con 0x7f7280005210 2024-05-10T03:36:07.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.003+0000 7f7293e00640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f727806ac30 0x7f727806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.003+0000 7f729be00640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f7278071200 0x7f7278073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.004+0000 7f7293e00640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f727806ac30 0x7f727806d140 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7284001200 tx=0x7f7284009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.004+0000 7f729be00640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f7278071200 0x7f7278073690 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.007+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== osd.4 v2:172.21.15.162:6800/35542 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7278073d60 con 0x7f7278071200 2024-05-10T03:36:07.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.021+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 --> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f729c15fbf0 con 0x7f7278071200 2024-05-10T03:36:07.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.022+0000 7f7293400640 1 -- 172.21.15.55:0/670992999 <== osd.4 v2:172.21.15.162:6800/35542 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7294031470 con 0x7f7278071200 2024-05-10T03:36:07.026 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:07.026 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:07.026 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.022+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f7278071200 msgr2=0x7f7278073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.022+0000 7f72a1200640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6800/35542,v1:172.21.15.162:6801/35542] conn(0x7f7278071200 0x7f7278073690 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f727806ac30 msgr2=0x7f727806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f727806ac30 0x7f727806d140 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7284001200 tx=0x7f7284009040 comp rx=0 tx=0).stop 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 msgr2=0x7f729c1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 --2- 172.21.15.55:0/670992999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7280005210 0x7f729c1542d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f728c00e480 tx=0x7f728c00e940 comp rx=0 tx=0).stop 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f729be00640 1 -- 172.21.15.55:0/670992999 reap_dead start 2024-05-10T03:36:07.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 shutdown_connections 2024-05-10T03:36:07.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 >> 172.21.15.55:0/670992999 conn(0x7f729c06c070 msgr2=0x7f729c0715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:07.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 shutdown_connections 2024-05-10T03:36:07.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.023+0000 7f72a1200640 1 -- 172.21.15.55:0/670992999 wait complete. 2024-05-10T03:36:07.055 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-10T03:36:07.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.339+0000 7f4840400640 1 Processor -- start 2024-05-10T03:36:07.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.339+0000 7f4840400640 1 -- start start 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.341+0000 7f4840400640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f483815fe50 0x7f4838141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.341+0000 7f4840400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48380b2550 0x7f483806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.341+0000 7f4840400640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483806ad20 con 0x7f48380b2140 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.341+0000 7f4840400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4838141f50 con 0x7f48380b2550 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.341+0000 7f4840400640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483806a110 con 0x7f483815fe50 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48380b2550 0x7f483806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48380b2550 0x7f483806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54792/0 (socket says 172.21.15.55:54792) 2024-05-10T03:36:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 -- 172.21.15.55:0/4180060982 learned_addr learned my addr 172.21.15.55:0/4180060982 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:07.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483da00640 1 --2- 172.21.15.55:0/4180060982 >> v2:172.21.15.162:3300/0 conn(0x7f483815fe50 0x7f4838141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 -- 172.21.15.55:0/4180060982 >> v2:172.21.15.162:3300/0 conn(0x7f483815fe50 msgr2=0x7f4838141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:07.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 --2- 172.21.15.55:0/4180060982 >> v2:172.21.15.162:3300/0 conn(0x7f483815fe50 0x7f4838141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 -- 172.21.15.55:0/4180060982 >> v1:172.21.15.162:6789/0 conn(0x7f48380b2140 legacy=0x7f4838142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:07.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.342+0000 7f483e400640 1 -- 172.21.15.55:0/4180060982 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483806a3f0 con 0x7f48380b2550 2024-05-10T03:36:07.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.344+0000 7f483e400640 1 --2- 172.21.15.55:0/4180060982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48380b2550 0x7f483806b1b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f483806e520 tx=0x7f483002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d059b5d82218a03 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.344+0000 7f4837e00640 1 -- 172.21.15.55:0/4180060982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830031600 con 0x7f48380b2550 2024-05-10T03:36:07.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.344+0000 7f4837e00640 1 -- 172.21.15.55:0/4180060982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483002d050 con 0x7f48380b2550 2024-05-10T03:36:07.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.344+0000 7f4837e00640 1 -- 172.21.15.55:0/4180060982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830031980 con 0x7f48380b2550 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 -- 172.21.15.55:0/4180060982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48380b2550 msgr2=0x7f483806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 --2- 172.21.15.55:0/4180060982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48380b2550 0x7f483806b1b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f483806e520 tx=0x7f483002a780 comp rx=0 tx=0).stop 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 -- 172.21.15.55:0/4180060982 shutdown_connections 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 --2- 172.21.15.55:0/4180060982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48380b2550 0x7f483806b1b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 --2- 172.21.15.55:0/4180060982 >> v2:172.21.15.162:3300/0 conn(0x7f483815fe50 0x7f4838141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 -- 172.21.15.55:0/4180060982 >> 172.21.15.55:0/4180060982 conn(0x7f48380cbb90 msgr2=0x7f48380cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 -- 172.21.15.55:0/4180060982 shutdown_connections 2024-05-10T03:36:07.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.345+0000 7f4840400640 1 -- 172.21.15.55:0/4180060982 wait complete. 2024-05-10T03:36:07.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.347+0000 7f4840400640 1 Processor -- start 2024-05-10T03:36:07.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.347+0000 7f4840400640 1 -- start start 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48380b2140 0x7f48381565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48380b2550 0x7f483813a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838138df0 con 0x7f483815fe50 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483815a960 con 0x7f48380b2140 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f4840400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483815c270 con 0x7f48380b2550 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.349+0000 7f483e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54808/0 (socket says 172.21.15.55:54808) 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 -- 172.21.15.55:0/3442482108 learned_addr learned my addr 172.21.15.55:0/3442482108 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483da00640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48380b2140 0x7f48381565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 -- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48380b2550 msgr2=0x7f483813a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483d000640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48380b2550 0x7f483813a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48380b2550 0x7f483813a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 -- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48380b2140 msgr2=0x7f48381565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48380b2140 0x7f48381565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 -- 172.21.15.55:0/3442482108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009b30 con 0x7f483815fe50 2024-05-10T03:36:07.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.350+0000 7f483e400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f483002ad70 tx=0x7f4830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.351+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483003d860 con 0x7f483815fe50 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.351+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483003d9e0 con 0x7f483815fe50 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.351+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483003dd90 con 0x7f483815fe50 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.351+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483815c4c0 con 0x7f483815fe50 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.351+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483815ca80 con 0x7f483815fe50 2024-05-10T03:36:07.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.352+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838138f30 con 0x7f483815fe50 2024-05-10T03:36:07.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.353+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4830048460 con 0x7f483815fe50 2024-05-10T03:36:07.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.355+0000 7f4835600640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 0x7f481806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.355+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f48300aa4a0 con 0x7f483815fe50 2024-05-10T03:36:07.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.356+0000 7f483da00640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 0x7f481806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.357+0000 7f483da00640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 0x7f481806d110 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f482c006fd0 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.360+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4838138f30 con 0x7f483815fe50 2024-05-10T03:36:07.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.560+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f483806a9b0 con 0x7f481806ac00 2024-05-10T03:36:07.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.561+0000 7f4835600640 1 -- 172.21.15.55:0/3442482108 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f482c00c050 con 0x7f481806ac00 2024-05-10T03:36:07.565 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 4 to deep-scrub 2024-05-10T03:36:07.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 msgr2=0x7f481806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 0x7f481806d110 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f482c006fd0 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-05-10T03:36:07.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 msgr2=0x7f483813a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f483002ad70 tx=0x7f4830004a40 comp rx=0 tx=0).stop 2024-05-10T03:36:07.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 shutdown_connections 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f483815fe50 0x7f483813a950 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48380b2550 0x7f483813a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f481806ac00 0x7f481806d110 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.563+0000 7f4840400640 1 --2- 172.21.15.55:0/3442482108 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48380b2140 0x7f48381565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.564+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 >> 172.21.15.55:0/3442482108 conn(0x7f48380cbb90 msgr2=0x7f48380af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.564+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 shutdown_connections 2024-05-10T03:36:07.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.564+0000 7f4840400640 1 -- 172.21.15.55:0/3442482108 wait complete. 2024-05-10T03:36:07.594 INFO:tasks.ceph:Scrubbing osd.5 2024-05-10T03:36:07.594 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:07.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.877+0000 7f503ea00640 1 Processor -- start 2024-05-10T03:36:07.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.877+0000 7f503ea00640 1 -- start start 2024-05-10T03:36:07.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.880+0000 7f503ea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f50381407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.880+0000 7f503ea00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50380cea90 0x7f503815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.880+0000 7f503ea00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380b44a0 con 0x7f503815e0e0 2024-05-10T03:36:07.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.880+0000 7f503ea00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5038140cf0 con 0x7f50380cea90 2024-05-10T03:36:07.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.880+0000 7f503ea00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380b3a00 con 0x7f50380b35f0 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.881+0000 7f503e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50380cea90 0x7f503815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.881+0000 7f503d600640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f50381407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.881+0000 7f503e000640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50380cea90 0x7f503815e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.55:50942/0 (socket says 172.21.15.55:50942) 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.881+0000 7f503e000640 1 -- 172.21.15.55:0/791914350 learned_addr learned my addr 172.21.15.55:0/791914350 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f5037e00640 1 -- 172.21.15.55:0/791914350 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383770464 0 0) 0x7f50380b44a0 con 0x7f503815e0e0 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f5037e00640 1 -- 172.21.15.55:0/791914350 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5020005730 con 0x7f503815e0e0 2024-05-10T03:36:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f503e000640 1 -- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 msgr2=0x7f50381407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f503e000640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f50381407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f503e000640 1 -- 172.21.15.55:0/791914350 >> v1:172.21.15.162:6789/0 conn(0x7f503815e0e0 legacy=0x7f50380ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f503e000640 1 -- 172.21.15.55:0/791914350 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380b3ee0 con 0x7f50380cea90 2024-05-10T03:36:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.882+0000 7f503d600640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f50381407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.883+0000 7f503e000640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50380cea90 0x7f503815e4f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5030009e50 tx=0x7f503002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef076319dff38d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.884+0000 7f5037e00640 1 -- 172.21.15.55:0/791914350 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030031600 con 0x7f50380cea90 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.884+0000 7f5037e00640 1 -- 172.21.15.55:0/791914350 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503002d050 con 0x7f50380cea90 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.884+0000 7f5037e00640 1 -- 172.21.15.55:0/791914350 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030031960 con 0x7f50380cea90 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 -- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50380cea90 msgr2=0x7f503815e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50380cea90 0x7f503815e4f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5030009e50 tx=0x7f503002a6a0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 -- 172.21.15.55:0/791914350 shutdown_connections 2024-05-10T03:36:07.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50380cea90 0x7f503815e4f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 --2- 172.21.15.55:0/791914350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f50381407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 -- 172.21.15.55:0/791914350 >> 172.21.15.55:0/791914350 conn(0x7f503806c070 msgr2=0x7f503806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 -- 172.21.15.55:0/791914350 shutdown_connections 2024-05-10T03:36:07.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.885+0000 7f503ea00640 1 -- 172.21.15.55:0/791914350 wait complete. 2024-05-10T03:36:07.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.888+0000 7f503ea00640 1 Processor -- start 2024-05-10T03:36:07.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.888+0000 7f503ea00640 1 -- start start 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.890+0000 7f503ea00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f503815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503815e0e0 0x7f503813f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503ea00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 0x7f50381f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503ea00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503815a0e0 con 0x7f50381588f0 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503ea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50381f4fe0 con 0x7f503815e0e0 2024-05-10T03:36:07.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503ea00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038159210 con 0x7f50380b35f0 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 0x7f50381f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 0x7f50381f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54822/0 (socket says 172.21.15.55:54822) 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 -- 172.21.15.55:0/2156386688 learned_addr learned my addr 172.21.15.55:0/2156386688 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503d600640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f503815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503cc00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503815e0e0 0x7f503813f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 -- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 msgr2=0x7f503815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50380b35f0 0x7f503815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 -- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503815e0e0 msgr2=0x7f503813f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f503815e0e0 0x7f503813f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.891+0000 7f503e000640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c009400 con 0x7f50381588f0 2024-05-10T03:36:07.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f503e000640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 0x7f50381f41e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f503002ac90 tx=0x7f5030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503003b610 con 0x7f50381588f0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503003b770 con 0x7f50381588f0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503003bae0 con 0x7f50381588f0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5030009b30 con 0x7f50381588f0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.892+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5038159460 con 0x7f50381588f0 2024-05-10T03:36:07.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.893+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50380b1bb0 con 0x7f50381588f0 2024-05-10T03:36:07.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.894+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5030038030 con 0x7f50381588f0 2024-05-10T03:36:07.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.895+0000 7f5035600640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac30 0x7f501806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.895+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f50300a9940 con 0x7f50381588f0 2024-05-10T03:36:07.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.896+0000 7f5035600640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7f5018071200 0x7f5018073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:07.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.896+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5018073d60 con 0x7f5018071200 2024-05-10T03:36:07.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.896+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50300ae2a0 con 0x7f50381588f0 2024-05-10T03:36:07.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.896+0000 7f503d600640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac30 0x7f501806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.897+0000 7f503cc00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7f5018071200 0x7f5018073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:07.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.898+0000 7f503d600640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac30 0x7f501806d140 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f502c009ae0 tx=0x7f502c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.898+0000 7f503cc00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7f5018071200 0x7f5018073690 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:07.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.900+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== osd.5 v2:172.21.15.162:6808/35543 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5018073d60 con 0x7f5018071200 2024-05-10T03:36:07.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.914+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 --> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5038143f80 con 0x7f5018071200 2024-05-10T03:36:07.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.915+0000 7f5035600640 1 -- 172.21.15.55:0/2156386688 <== osd.5 v2:172.21.15.162:6808/35543 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5024001210 con 0x7f5018071200 2024-05-10T03:36:07.919 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:07.919 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:07.919 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:07.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7f5018071200 msgr2=0x7f5018073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6808/35543,v1:172.21.15.162:6809/35543] conn(0x7f5018071200 0x7f5018073690 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac30 msgr2=0x7f501806d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f501806ac30 0x7f501806d140 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f502c009ae0 tx=0x7f502c008040 comp rx=0 tx=0).stop 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 msgr2=0x7f50381f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.916+0000 7f503ea00640 1 --2- 172.21.15.55:0/2156386688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50381588f0 0x7f50381f41e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f503002ac90 tx=0x7f5030004a40 comp rx=0 tx=0).stop 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.917+0000 7f503e000640 1 -- 172.21.15.55:0/2156386688 reap_dead start 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.917+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 shutdown_connections 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.917+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 >> 172.21.15.55:0/2156386688 conn(0x7f503806c070 msgr2=0x7f503806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.917+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 shutdown_connections 2024-05-10T03:36:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:07.917+0000 7f503ea00640 1 -- 172.21.15.55:0/2156386688 wait complete. 2024-05-10T03:36:07.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-10T03:36:08.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.235+0000 7f9471e00640 1 Processor -- start 2024-05-10T03:36:08.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.235+0000 7f9471e00640 1 -- start start 2024-05-10T03:36:08.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f9471e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f946c0adbc0 0x7f946c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f9471e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c071f20 0x7f946c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f9471e00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c1623a0 con 0x7f946c0ad7b0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f9471e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c072330 con 0x7f946c0adbc0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f9471e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0cdd60 con 0x7f946c071f20 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f946aa00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f946c0ad7b0 0x7f946c161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42784/0 (socket says 172.21.15.55:42784) 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f946aa00640 1 -- 172.21.15.55:0/4275755918 learned_addr learned my addr 172.21.15.55:0/4275755918 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.238+0000 7f946a000640 1 --2- 172.21.15.55:0/4275755918 >> v2:172.21.15.55:3300/0 conn(0x7f946c0adbc0 0x7f946c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f946b400640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c071f20 0x7f946c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f9469600640 1 -- 172.21.15.55:0/4275755918 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438160532 0 0) 0x7f946c1623a0 con 0x7f946c0ad7b0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f9469600640 1 -- 172.21.15.55:0/4275755918 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9450003520 con 0x7f946c0ad7b0 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f946a000640 1 -- 172.21.15.55:0/4275755918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c071f20 msgr2=0x7f946c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f946a000640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c071f20 0x7f946c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f946a000640 1 -- 172.21.15.55:0/4275755918 >> v1:172.21.15.55:6789/0 conn(0x7f946c0ad7b0 legacy=0x7f946c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.239+0000 7f946a000640 1 -- 172.21.15.55:0/4275755918 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0ce040 con 0x7f946c0adbc0 2024-05-10T03:36:08.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.240+0000 7f946a000640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c0adbc0 0x7f946c0719e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f94580097d0 tx=0x7f945802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6179f2559518d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.240+0000 7f9469600640 1 -- 172.21.15.55:0/4275755918 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945802ae40 con 0x7f946c0adbc0 2024-05-10T03:36:08.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.240+0000 7f9469600640 1 -- 172.21.15.55:0/4275755918 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458032440 con 0x7f946c0adbc0 2024-05-10T03:36:08.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.240+0000 7f9469600640 1 -- 172.21.15.55:0/4275755918 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580327d0 con 0x7f946c0adbc0 2024-05-10T03:36:08.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 -- 172.21.15.55:0/4275755918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c0adbc0 msgr2=0x7f946c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c0adbc0 0x7f946c0719e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f94580097d0 tx=0x7f945802a130 comp rx=0 tx=0).stop 2024-05-10T03:36:08.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 -- 172.21.15.55:0/4275755918 shutdown_connections 2024-05-10T03:36:08.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c071f20 0x7f946c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 --2- 172.21.15.55:0/4275755918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c0adbc0 0x7f946c0719e0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 -- 172.21.15.55:0/4275755918 >> 172.21.15.55:0/4275755918 conn(0x7f946c0cbb90 msgr2=0x7f946c0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:08.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 -- 172.21.15.55:0/4275755918 shutdown_connections 2024-05-10T03:36:08.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.241+0000 7f9471e00640 1 -- 172.21.15.55:0/4275755918 wait complete. 2024-05-10T03:36:08.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.244+0000 7f9471e00640 1 Processor -- start 2024-05-10T03:36:08.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.244+0000 7f9471e00640 1 -- start start 2024-05-10T03:36:08.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.245+0000 7f9471e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f9471e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f946c0ad7b0 0x7f946c1f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f9471e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c0adbc0 0x7f946c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f9471e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c14b080 con 0x7f946c071f20 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f9471e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c14ab30 con 0x7f946c0ad7b0 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f9471e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c145020 con 0x7f946c0adbc0 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54840/0 (socket says 172.21.15.55:54840) 2024-05-10T03:36:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 -- 172.21.15.55:0/1532509281 learned_addr learned my addr 172.21.15.55:0/1532509281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946b400640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c0adbc0 0x7f946c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946a000640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f946c0ad7b0 0x7f946c1f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 -- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c0adbc0 msgr2=0x7f946c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c0adbc0 0x7f946c1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 -- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f946c0ad7b0 msgr2=0x7f946c1f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.246+0000 7f946aa00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f946c0ad7b0 0x7f946c1f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f946aa00640 1 -- 172.21.15.55:0/1532509281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458009400 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f946aa00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c430 tx=0x7f945c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c005ce0 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c005e40 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0072d0 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c15b190 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.247+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1f3cd0 con 0x7f946c071f20 2024-05-10T03:36:08.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.249+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c066f60 con 0x7f946c071f20 2024-05-10T03:36:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.250+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f945c007430 con 0x7f946c071f20 2024-05-10T03:36:08.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.251+0000 7f9463400640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 0x7f944c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.252+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f945c0887b0 con 0x7f946c071f20 2024-05-10T03:36:08.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.252+0000 7f946a000640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 0x7f944c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.253+0000 7f946a000640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 0x7f944c06d110 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9458034000 tx=0x7f945802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.256+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f945c053aa0 con 0x7f946c071f20 2024-05-10T03:36:08.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.453+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f946c161e40 con 0x7f944c06ac00 2024-05-10T03:36:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.455+0000 7f9463400640 1 -- 172.21.15.55:0/1532509281 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9458006e10 con 0x7f944c06ac00 2024-05-10T03:36:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 5 to deep-scrub 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 msgr2=0x7f944c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 0x7f944c06d110 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9458034000 tx=0x7f945802e000 comp rx=0 tx=0).stop 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 msgr2=0x7f946c1440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c430 tx=0x7f945c00c8f0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 shutdown_connections 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f946c0adbc0 0x7f946c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f944c06ac00 0x7f944c06d110 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f946c0ad7b0 0x7f946c1f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 --2- 172.21.15.55:0/1532509281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f946c071f20 0x7f946c1440e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 >> 172.21.15.55:0/1532509281 conn(0x7f946c0cbb90 msgr2=0x7f946c0cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:08.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 shutdown_connections 2024-05-10T03:36:08.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.457+0000 7f9471e00640 1 -- 172.21.15.55:0/1532509281 wait complete. 2024-05-10T03:36:08.490 INFO:tasks.ceph:Scrubbing osd.6 2024-05-10T03:36:08.490 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.779+0000 7f6dab000640 1 Processor -- start 2024-05-10T03:36:08.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.779+0000 7f6dab000640 1 -- start start 2024-05-10T03:36:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.781+0000 7f6dab000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 0x7f6da41407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6dab000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da415e0e0 0x7f6da415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6dab000640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da40b44a0 con 0x7f6da40ce380 2024-05-10T03:36:08.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6dab000640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da4140cf0 con 0x7f6da415e0e0 2024-05-10T03:36:08.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6dab000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40b3a00 con 0x7f6da40b35f0 2024-05-10T03:36:08.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6da9200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da415e0e0 0x7f6da415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6da9200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da415e0e0 0x7f6da415e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54842/0 (socket says 172.21.15.55:54842) 2024-05-10T03:36:08.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6da9200640 1 -- 172.21.15.55:0/4114384299 learned_addr learned my addr 172.21.15.55:0/4114384299 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.782+0000 7f6da9c00640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 0x7f6da41407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.783+0000 7f6da9200640 1 -- 172.21.15.55:0/4114384299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 msgr2=0x7f6da41407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.783+0000 7f6da9200640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 0x7f6da41407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.783+0000 7f6da9200640 1 -- 172.21.15.55:0/4114384299 >> v1:172.21.15.162:6789/0 conn(0x7f6da40ce380 legacy=0x7f6da40ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.783+0000 7f6da9200640 1 -- 172.21.15.55:0/4114384299 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da40b3ee0 con 0x7f6da415e0e0 2024-05-10T03:36:08.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.783+0000 7f6da9c00640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 0x7f6da41407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.784+0000 7f6da9200640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da415e0e0 0x7f6da415e4f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6da406bd60 tx=0x7f6d9002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4cb5874d123805 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.784+0000 7f6da3e00640 1 -- 172.21.15.55:0/4114384299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9002ae70 con 0x7f6da415e0e0 2024-05-10T03:36:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.784+0000 7f6da3e00640 1 -- 172.21.15.55:0/4114384299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90032440 con 0x7f6da415e0e0 2024-05-10T03:36:08.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.784+0000 7f6da3e00640 1 -- 172.21.15.55:0/4114384299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900327d0 con 0x7f6da415e0e0 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 -- 172.21.15.55:0/4114384299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da415e0e0 msgr2=0x7f6da415e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da415e0e0 0x7f6da415e4f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6da406bd60 tx=0x7f6d9002a160 comp rx=0 tx=0).stop 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 -- 172.21.15.55:0/4114384299 shutdown_connections 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da415e0e0 0x7f6da415e4f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 --2- 172.21.15.55:0/4114384299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da40b35f0 0x7f6da41407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 -- 172.21.15.55:0/4114384299 >> 172.21.15.55:0/4114384299 conn(0x7f6da406c070 msgr2=0x7f6da406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 -- 172.21.15.55:0/4114384299 shutdown_connections 2024-05-10T03:36:08.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.785+0000 7f6dab000640 1 -- 172.21.15.55:0/4114384299 wait complete. 2024-05-10T03:36:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.788+0000 7f6dab000640 1 Processor -- start 2024-05-10T03:36:08.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.788+0000 7f6dab000640 1 -- start start 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da40b35f0 0x7f6da413f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 0x7f6da4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da415e0e0 0x7f6da415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4157bb0 con 0x7f6da40ce380 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40aea10 con 0x7f6da40b35f0 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.789+0000 7f6dab000640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4156dd0 con 0x7f6da415e0e0 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 0x7f6da4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 0x7f6da4157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54844/0 (socket says 172.21.15.55:54844) 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 -- 172.21.15.55:0/1841810056 learned_addr learned my addr 172.21.15.55:0/1841810056 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:08.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9c00640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da40b35f0 0x7f6da413f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6daa600640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da415e0e0 0x7f6da415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 -- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da415e0e0 msgr2=0x7f6da415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6da415e0e0 0x7f6da415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 -- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da40b35f0 msgr2=0x7f6da413f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da40b35f0 0x7f6da413f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9200640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900094d0 con 0x7f6da40ce380 2024-05-10T03:36:08.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.790+0000 7f6da9c00640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6da40b35f0 0x7f6da413f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6da9200640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 0x7f6da4157220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d900090b0 tx=0x7f6d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90030b80 con 0x7f6da40ce380 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90030d00 con 0x7f6da40ce380 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9003d680 con 0x7f6da40ce380 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da40aca80 con 0x7f6da40ce380 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.791+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da40ad040 con 0x7f6da40ce380 2024-05-10T03:36:08.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.792+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6da40b09b0 con 0x7f6da40ce380 2024-05-10T03:36:08.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.794+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d90039040 con 0x7f6da40ce380 2024-05-10T03:36:08.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.794+0000 7f6da1600640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.795+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f6d900a9fd0 con 0x7f6da40ce380 2024-05-10T03:36:08.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.795+0000 7f6da1600640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f6d84071200 0x7f6d84073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:08.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.795+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d84073d60 con 0x7f6d84071200 2024-05-10T03:36:08.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.795+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d900ac630 con 0x7f6da40ce380 2024-05-10T03:36:08.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.795+0000 7f6da9c00640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.796+0000 7f6daa600640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f6d84071200 0x7f6d84073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:08.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.797+0000 7f6daa600640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f6d84071200 0x7f6d84073690 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.797+0000 7f6da9c00640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d98006fd0 tx=0x7f6d98008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.799+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== osd.6 v2:172.21.15.162:6816/35544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6d84073d60 con 0x7f6d84071200 2024-05-10T03:36:08.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.814+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 --> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6da4143f80 con 0x7f6d84071200 2024-05-10T03:36:08.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.815+0000 7f6da1600640 1 -- 172.21.15.55:0/1841810056 <== osd.6 v2:172.21.15.162:6816/35544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6d9c007ca0 con 0x7f6d84071200 2024-05-10T03:36:08.819 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:08.819 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:08.819 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.815+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f6d84071200 msgr2=0x7f6d84073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.815+0000 7f6dab000640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6816/35544,v1:172.21.15.162:6817/35544] conn(0x7f6d84071200 0x7f6d84073690 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 msgr2=0x7f6d8406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f6d8406ac30 0x7f6d8406d140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d98006fd0 tx=0x7f6d98008040 comp rx=0 tx=0).stop 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 msgr2=0x7f6da4157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 --2- 172.21.15.55:0/1841810056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da40ce380 0x7f6da4157220 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6d900090b0 tx=0x7f6d900046c0 comp rx=0 tx=0).stop 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6daa600640 1 -- 172.21.15.55:0/1841810056 reap_dead start 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 shutdown_connections 2024-05-10T03:36:08.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 >> 172.21.15.55:0/1841810056 conn(0x7f6da406c070 msgr2=0x7f6da40af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:08.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 shutdown_connections 2024-05-10T03:36:08.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:08.816+0000 7f6dab000640 1 -- 172.21.15.55:0/1841810056 wait complete. 2024-05-10T03:36:08.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-10T03:36:09.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.124+0000 7f3922a00640 1 Processor -- start 2024-05-10T03:36:09.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.124+0000 7f3922a00640 1 -- start start 2024-05-10T03:36:09.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 0x7f391c144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c06d7e0 0x7f391c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f391c0b2fc0 0x7f391c15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c06d350 con 0x7f391c0b2fc0 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c06c740 con 0x7f391c144710 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c06ca20 con 0x7f391c06d7e0 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f391c0b2fc0 0x7f391c15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f391c0b2fc0 0x7f391c15eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54858/0 (socket says 172.21.15.55:54858) 2024-05-10T03:36:09.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3922000640 1 -- 172.21.15.55:0/3680116354 learned_addr learned my addr 172.21.15.55:0/3680116354 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3920c00640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c06d7e0 0x7f391c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.127+0000 7f3921600640 1 --2- 172.21.15.55:0/3680116354 >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 0x7f391c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3922000640 1 -- 172.21.15.55:0/3680116354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c06d7e0 msgr2=0x7f391c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3922000640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c06d7e0 0x7f391c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3922000640 1 -- 172.21.15.55:0/3680116354 >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 msgr2=0x7f391c144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3922000640 1 --2- 172.21.15.55:0/3680116354 >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 0x7f391c144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3922000640 1 -- 172.21.15.55:0/3680116354 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c06cd00 con 0x7f391c0b2fc0 2024-05-10T03:36:09.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.128+0000 7f3921600640 1 --2- 172.21.15.55:0/3680116354 >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 0x7f391c144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:09.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.129+0000 7f3922000640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c0b2fc0 0x7f391c15eff0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f391c072240 tx=0x7f391402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880485dc21d09ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.129+0000 7f391be00640 1 -- 172.21.15.55:0/3680116354 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914031600 con 0x7f391c0b2fc0 2024-05-10T03:36:09.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.129+0000 7f391be00640 1 -- 172.21.15.55:0/3680116354 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391402d050 con 0x7f391c0b2fc0 2024-05-10T03:36:09.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.129+0000 7f391be00640 1 -- 172.21.15.55:0/3680116354 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914031980 con 0x7f391c0b2fc0 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 -- 172.21.15.55:0/3680116354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c0b2fc0 msgr2=0x7f391c15eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c0b2fc0 0x7f391c15eff0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f391c072240 tx=0x7f391402a610 comp rx=0 tx=0).stop 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 -- 172.21.15.55:0/3680116354 shutdown_connections 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c0b2fc0 0x7f391c15eff0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 --2- 172.21.15.55:0/3680116354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c06d7e0 0x7f391c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 --2- 172.21.15.55:0/3680116354 >> v2:172.21.15.162:3300/0 conn(0x7f391c144710 0x7f391c144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 -- 172.21.15.55:0/3680116354 >> 172.21.15.55:0/3680116354 conn(0x7f391c06a110 msgr2=0x7f391c06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:09.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.130+0000 7f3922a00640 1 -- 172.21.15.55:0/3680116354 shutdown_connections 2024-05-10T03:36:09.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.131+0000 7f3922a00640 1 -- 172.21.15.55:0/3680116354 wait complete. 2024-05-10T03:36:09.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.133+0000 7f3922a00640 1 Processor -- start 2024-05-10T03:36:09.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.133+0000 7f3922a00640 1 -- start start 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.134+0000 7f3922a00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.134+0000 7f3922a00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c0b2fc0 0x7f391c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3922a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f391c144710 0x7f391c142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3922a00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0ac3b0 con 0x7f391c06d7e0 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3922a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0aa650 con 0x7f391c144710 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3922a00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0aa8a0 con 0x7f391c0b2fc0 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54864/0 (socket says 172.21.15.55:54864) 2024-05-10T03:36:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 -- 172.21.15.55:0/610778767 learned_addr learned my addr 172.21.15.55:0/610778767 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3920c00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c0b2fc0 0x7f391c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3922000640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f391c144710 0x7f391c142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 -- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c0b2fc0 msgr2=0x7f391c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c0b2fc0 0x7f391c15ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 -- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f391c144710 msgr2=0x7f391c142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f391c144710 0x7f391c142b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.135+0000 7f3921600640 1 -- 172.21.15.55:0/610778767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914009b30 con 0x7f391c06d7e0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3921600640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f391000c7f0 tx=0x7f391000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910007820 con 0x7f391c06d7e0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910007980 con 0x7f391c06d7e0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c0aa490 con 0x7f391c06d7e0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910007d30 con 0x7f391c06d7e0 2024-05-10T03:36:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.136+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1410f0 con 0x7f391c06d7e0 2024-05-10T03:36:09.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.138+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c066ce0 con 0x7f391c06d7e0 2024-05-10T03:36:09.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.139+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f39100113f0 con 0x7f391c06d7e0 2024-05-10T03:36:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.139+0000 7f3919600640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 0x7f38fc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.140+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f3910089270 con 0x7f391c06d7e0 2024-05-10T03:36:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.140+0000 7f3920c00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 0x7f38fc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.142+0000 7f3920c00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 0x7f38fc06d110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3908006fd0 tx=0x7f3908008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.144+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3910054410 con 0x7f391c06d7e0 2024-05-10T03:36:09.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.344+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f391c0aa490 con 0x7f38fc06ac00 2024-05-10T03:36:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.345+0000 7f3919600640 1 -- 172.21.15.55:0/610778767 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f391c0aa490 con 0x7f38fc06ac00 2024-05-10T03:36:09.349 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 6 to deep-scrub 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 msgr2=0x7f38fc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 0x7f38fc06d110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3908006fd0 tx=0x7f3908008040 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 msgr2=0x7f391c0a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f391000c7f0 tx=0x7f391000ccb0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 shutdown_connections 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f38fc06ac00 0x7f38fc06d110 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f391c144710 0x7f391c142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f391c0b2fc0 0x7f391c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 --2- 172.21.15.55:0/610778767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391c06d7e0 0x7f391c0a9f50 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 >> 172.21.15.55:0/610778767 conn(0x7f391c06a110 msgr2=0x7f391c0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 shutdown_connections 2024-05-10T03:36:09.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.348+0000 7f3922a00640 1 -- 172.21.15.55:0/610778767 wait complete. 2024-05-10T03:36:09.382 INFO:tasks.ceph:Scrubbing osd.7 2024-05-10T03:36:09.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-10T03:36:09.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.667+0000 7f3dc9200640 1 Processor -- start 2024-05-10T03:36:09.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.667+0000 7f3dc9200640 1 -- start start 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.670+0000 7f3dc9200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3dc40b3ff0 0x7f3dc406a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.670+0000 7f3dc9200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc4072310 con 0x7f3dc415d710 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.670+0000 7f3dc9200640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc406b9b0 con 0x7f3dc4161150 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.670+0000 7f3dc9200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc406bc90 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.670+0000 7f3dc2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3dc40b3ff0 0x7f3dc406a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc2a00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3dc40b3ff0 0x7f3dc406a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54876/0 (socket says 172.21.15.55:54876) 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc2a00640 1 -- 172.21.15.55:0/3871130009 learned_addr learned my addr 172.21.15.55:0/3871130009 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:09.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc3e00640 1 --1- 172.21.15.55:0/3871130009 >> v1:172.21.15.55:6789/0 conn(0x7f3dc415d710 0x7f3dc415db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42794/0 (socket says 172.21.15.55:42794) 2024-05-10T03:36:09.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc2a00640 1 -- 172.21.15.55:0/3871130009 >> v1:172.21.15.162:6789/0 conn(0x7f3dc4161150 legacy=0x7f3dc406a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc2a00640 1 -- 172.21.15.55:0/3871130009 >> v1:172.21.15.55:6789/0 conn(0x7f3dc415d710 legacy=0x7f3dc415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.671+0000 7f3dc2a00640 1 -- 172.21.15.55:0/3871130009 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc406bf70 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.672+0000 7f3dc2a00640 1 --2- 172.21.15.55:0/3871130009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc406a820 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009870 tx=0x7f3dac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c98471b30690607 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.672+0000 7f3dc2000640 1 -- 172.21.15.55:0/3871130009 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac032440 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc2000640 1 -- 172.21.15.55:0/3871130009 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac0325a0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc2000640 1 -- 172.21.15.55:0/3871130009 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac032930 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 -- 172.21.15.55:0/3871130009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 msgr2=0x7f3dc406a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 --2- 172.21.15.55:0/3871130009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc406a820 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009870 tx=0x7f3dac02a270 comp rx=0 tx=0).stop 2024-05-10T03:36:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 -- 172.21.15.55:0/3871130009 shutdown_connections 2024-05-10T03:36:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 --2- 172.21.15.55:0/3871130009 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc406a820 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 -- 172.21.15.55:0/3871130009 >> 172.21.15.55:0/3871130009 conn(0x7f3dc40710f0 msgr2=0x7f3dc4071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.673+0000 7f3dc9200640 1 -- 172.21.15.55:0/3871130009 shutdown_connections 2024-05-10T03:36:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.674+0000 7f3dc9200640 1 -- 172.21.15.55:0/3871130009 wait complete. 2024-05-10T03:36:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.676+0000 7f3dc9200640 1 Processor -- start 2024-05-10T03:36:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.676+0000 7f3dc9200640 1 -- start start 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc415d710 0x7f3dc41551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3dc4161150 0x7f3dc41405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4156e60 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc41418a0 con 0x7f3dc415d710 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc9200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4155740 con 0x7f3dc4161150 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc4143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54890/0 (socket says 172.21.15.55:54890) 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 -- 172.21.15.55:0/1330627527 learned_addr learned my addr 172.21.15.55:0/1330627527 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc2a00640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc415d710 0x7f3dc41551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 -- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3dc4161150 msgr2=0x7f3dc41405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3dc4161150 0x7f3dc41405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 -- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc415d710 msgr2=0x7f3dc41551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3dc415d710 0x7f3dc41551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.678+0000 7f3dc3400640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac0094d0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dc3400640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc4143a80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c760 tx=0x7f3db400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4003b70 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db4003cd0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db400aa30 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc41579a0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.679+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc4157070 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.680+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3dc406c530 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.682+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3db400ab90 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dbb400640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3da006ac30 0x7f3da006d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f3db40890e0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dbb400640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f3da0071200 0x7f3da0073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3da0073d60 con 0x7f3da0071200 2024-05-10T03:36:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db408b7a0 con 0x7f3dc40b3ff0 2024-05-10T03:36:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.683+0000 7f3dc2a00640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3da006ac30 0x7f3da006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.684+0000 7f3dc3e00640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f3da0071200 0x7f3da0073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:09.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.685+0000 7f3dc2a00640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3da006ac30 0x7f3da006d140 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.685+0000 7f3dc3e00640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f3da0071200 0x7f3da0073690 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.688+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== osd.7 v2:172.21.15.162:6824/35545 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3da0073d60 con 0x7f3da0071200 2024-05-10T03:36:09.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.703+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 --> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3dc4067930 con 0x7f3da0071200 2024-05-10T03:36:09.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.704+0000 7f3dbb400640 1 -- 172.21.15.55:0/1330627527 <== osd.7 v2:172.21.15.162:6824/35545 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3dc4067930 con 0x7f3da0071200 2024-05-10T03:36:09.707 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-05-10T03:36:09.708 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-10T03:36:09.708 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f3da0071200 msgr2=0x7f3da0073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6824/35545,v1:172.21.15.162:6825/35545] conn(0x7f3da0071200 0x7f3da0073690 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3da006ac30 msgr2=0x7f3da006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f3da006ac30 0x7f3da006d140 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3dac034000 tx=0x7f3dac02e000 comp rx=0 tx=0).stop 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 msgr2=0x7f3dc4143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 --2- 172.21.15.55:0/1330627527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dc40b3ff0 0x7f3dc4143a80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c760 tx=0x7f3db400cc20 comp rx=0 tx=0).stop 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc3e00640 1 -- 172.21.15.55:0/1330627527 reap_dead start 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 shutdown_connections 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 >> 172.21.15.55:0/1330627527 conn(0x7f3dc40710f0 msgr2=0x7f3dc406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 shutdown_connections 2024-05-10T03:36:09.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:09.705+0000 7f3dc9200640 1 -- 172.21.15.55:0/1330627527 wait complete. 2024-05-10T03:36:09.734 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-10T03:36:10.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.010+0000 7f7b0ba00640 1 Processor -- start 2024-05-10T03:36:10.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.010+0000 7f7b0ba00640 1 -- start start 2024-05-10T03:36:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.012+0000 7f7b0ba00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b0415ab30 0x7f7b040715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.012+0000 7f7b0ba00640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040cd710 con 0x7f7b0415af40 2024-05-10T03:36:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.012+0000 7f7b0ba00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040cdfb0 con 0x7f7b040cdba0 2024-05-10T03:36:10.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.012+0000 7f7b0ba00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0406b4d0 con 0x7f7b0415ab30 2024-05-10T03:36:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.013+0000 7f7b09a00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7b0415af40 0x7f7b040b2fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42808/0 (socket says 172.21.15.55:42808) 2024-05-10T03:36:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.013+0000 7f7b09a00640 1 -- 172.21.15.55:0/2485594887 learned_addr learned my addr 172.21.15.55:0/2485594887 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.013+0000 7f7afbe00640 1 --2- 172.21.15.55:0/2485594887 >> v2:172.21.15.162:3300/0 conn(0x7f7b0415ab30 0x7f7b040715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.013+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760385307 0 0) 0x7f7b040cd710 con 0x7f7b0415af40 2024-05-10T03:36:10.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003440 con 0x7f7b0415af40 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1449216142 0 0) 0x7f7ae8003440 con 0x7f7b0415af40 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 >> v2:172.21.15.162:3300/0 conn(0x7f7b0415ab30 msgr2=0x7f7b040715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 --2- 172.21.15.55:0/2485594887 >> v2:172.21.15.162:3300/0 conn(0x7f7b0415ab30 0x7f7b040715e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 >> v1:172.21.15.162:6789/0 conn(0x7f7b040cdba0 legacy=0x7f7b0406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0406b1f0 con 0x7f7b0415af40 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3433871650 0 0) 0x7f7afc0019f0 con 0x7f7b0415af40 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 >> v1:172.21.15.55:6789/0 conn(0x7f7b0415af40 legacy=0x7f7b040b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 0x7f7ae8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae80088f0 0x7f7ae800add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae800b670 0x7f7ae800db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0406b4d0 con 0x7f7ae8003a60 2024-05-10T03:36:10.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800e460 con 0x7f7ae80088f0 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800e620 con 0x7f7ae800b670 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.014+0000 7f7b09000640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 0x7f7ae8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7afbe00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae80088f0 0x7f7ae800add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09a00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae800b670 0x7f7ae800db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09000640 1 -- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae800b670 msgr2=0x7f7ae800db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09000640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae800b670 0x7f7ae800db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09000640 1 -- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae80088f0 msgr2=0x7f7ae800add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09000640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae80088f0 0x7f7ae800add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09000640 1 -- 172.21.15.55:0/2485594887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8003440 con 0x7f7ae8003a60 2024-05-10T03:36:10.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.015+0000 7f7b09a00640 1 -- 172.21.15.55:0/2485594887 reap_dead start 2024-05-10T03:36:10.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.016+0000 7f7b09000640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 0x7f7ae8003ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7af4001720 tx=0x7f7ae80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15a811334395f50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.016+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40052d0 con 0x7f7ae8003a60 2024-05-10T03:36:10.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.016+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4005430 con 0x7f7ae8003a60 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b03e00640 1 -- 172.21.15.55:0/2485594887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402ab90 con 0x7f7ae8003a60 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b0ba00640 1 -- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 msgr2=0x7f7ae8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 0x7f7ae8003ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7af4001720 tx=0x7f7ae80036f0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b0ba00640 1 -- 172.21.15.55:0/2485594887 shutdown_connections 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/2485594887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ae8003a60 0x7f7ae8003ed0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.017+0000 7f7b0ba00640 1 -- 172.21.15.55:0/2485594887 >> 172.21.15.55:0/2485594887 conn(0x7f7b040cbb90 msgr2=0x7f7b040cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.018+0000 7f7b0ba00640 1 -- 172.21.15.55:0/2485594887 shutdown_connections 2024-05-10T03:36:10.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.018+0000 7f7b0ba00640 1 -- 172.21.15.55:0/2485594887 wait complete. 2024-05-10T03:36:10.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.020+0000 7f7b0ba00640 1 Processor -- start 2024-05-10T03:36:10.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.020+0000 7f7b0ba00640 1 -- start start 2024-05-10T03:36:10.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae8003a60 0x7f7b0415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b041f3dc0 0x7f7b04151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04151340 con 0x7f7b0415d1f0 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04152db0 con 0x7f7ae8003a60 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b0ba00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415c390 con 0x7f7b041f3dc0 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7afbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54898/0 (socket says 172.21.15.55:54898) 2024-05-10T03:36:10.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7afbe00640 1 -- 172.21.15.55:0/367533765 learned_addr learned my addr 172.21.15.55:0/367533765 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:10.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b09000640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae8003a60 0x7f7b0415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.022+0000 7f7b09a00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b041f3dc0 0x7f7b04151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 -- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b041f3dc0 msgr2=0x7f7b04151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b041f3dc0 0x7f7b04151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 -- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae8003a60 msgr2=0x7f7b0415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae8003a60 0x7f7b0415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 -- 172.21.15.55:0/367533765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0415c5e0 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7afbe00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0114e0 tx=0x7f7aec0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec015870 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec0159d0 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec015d40 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.023+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0415c8c0 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.024+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b04151550 con 0x7f7b0415d1f0 2024-05-10T03:36:10.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.025+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04151710 con 0x7f7b0415d1f0 2024-05-10T03:36:10.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.026+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7aec013400 con 0x7f7b0415d1f0 2024-05-10T03:36:10.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.027+0000 7f7b01600640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 0x7f7ae006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.027+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f7aec0897f0 con 0x7f7b0415d1f0 2024-05-10T03:36:10.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.028+0000 7f7b09000640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 0x7f7ae006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.029+0000 7f7b09000640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 0x7f7ae006d110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7af4032000 tx=0x7f7af402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.032+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7aec054900 con 0x7f7b0415d1f0 2024-05-10T03:36:10.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.232+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0415cde0 con 0x7f7ae006ac00 2024-05-10T03:36:10.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.234+0000 7f7b01600640 1 -- 172.21.15.55:0/367533765 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7b0415cde0 con 0x7f7ae006ac00 2024-05-10T03:36:10.237 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 7 to deep-scrub 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.236+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 msgr2=0x7f7ae006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.236+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 0x7f7ae006d110 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7af4032000 tx=0x7f7af402e000 comp rx=0 tx=0).stop 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.236+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 msgr2=0x7f7b041f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.236+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0114e0 tx=0x7f7aec0119a0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 shutdown_connections 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b041f3dc0 0x7f7b04151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7b0415d1f0 0x7f7b041f3880 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f7ae006ac00 0x7f7ae006d110 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 --2- 172.21.15.55:0/367533765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae8003a60 0x7f7b0415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 >> 172.21.15.55:0/367533765 conn(0x7f7b040cbb90 msgr2=0x7f7b040ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:10.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 shutdown_connections 2024-05-10T03:36:10.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.237+0000 7f7b0ba00640 1 -- 172.21.15.55:0/367533765 wait complete. 2024-05-10T03:36:10.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:36:10.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.549+0000 7f0ec3200640 1 Processor -- start 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.549+0000 7f0ec3200640 1 -- start start 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40b4170 0x7f0ec40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec3200640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ec40ccc80 0x7f0ec40cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec3200640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec40b4ac0 con 0x7f0ec415a5d0 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec3200640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec415ad30 con 0x7f0ec40ccc80 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec415b010 con 0x7f0ec40b4170 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec1e00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0ec415a5d0 0x7f0ec40cb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42810/0 (socket says 172.21.15.55:42810) 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec1e00640 1 -- 172.21.15.55:0/2163441327 learned_addr learned my addr 172.21.15.55:0/2163441327 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.551+0000 7f0ec2800640 1 --2- 172.21.15.55:0/2163441327 >> v2:172.21.15.55:3300/0 conn(0x7f0ec40ccc80 0x7f0ec40cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec1400640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40b4170 0x7f0ec40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec0a00640 1 -- 172.21.15.55:0/2163441327 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106404394 0 0) 0x7f0ec40b4ac0 con 0x7f0ec415a5d0 2024-05-10T03:36:10.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec0a00640 1 -- 172.21.15.55:0/2163441327 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea8003520 con 0x7f0ec415a5d0 2024-05-10T03:36:10.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec2800640 1 -- 172.21.15.55:0/2163441327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40b4170 msgr2=0x7f0ec40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:10.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec2800640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40b4170 0x7f0ec40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec2800640 1 -- 172.21.15.55:0/2163441327 >> v1:172.21.15.55:6789/0 conn(0x7f0ec415a5d0 legacy=0x7f0ec40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.552+0000 7f0ec2800640 1 -- 172.21.15.55:0/2163441327 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec415b2f0 con 0x7f0ec40ccc80 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.553+0000 7f0ec2800640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec40ccc80 0x7f0ec40cd090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009d90 tx=0x7f0ebc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c223b696e4c522a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec0a00640 1 -- 172.21.15.55:0/2163441327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc007740 con 0x7f0ec40ccc80 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec0a00640 1 -- 172.21.15.55:0/2163441327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc02d050 con 0x7f0ec40ccc80 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec0a00640 1 -- 172.21.15.55:0/2163441327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc007aa0 con 0x7f0ec40ccc80 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 -- 172.21.15.55:0/2163441327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec40ccc80 msgr2=0x7f0ec40cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec40ccc80 0x7f0ec40cd090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009d90 tx=0x7f0ebc02a660 comp rx=0 tx=0).stop 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 -- 172.21.15.55:0/2163441327 shutdown_connections 2024-05-10T03:36:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40b4170 0x7f0ec40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 --2- 172.21.15.55:0/2163441327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec40ccc80 0x7f0ec40cd090 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.554+0000 7f0ec3200640 1 -- 172.21.15.55:0/2163441327 >> 172.21.15.55:0/2163441327 conn(0x7f0ec406a110 msgr2=0x7f0ec406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.555+0000 7f0ec3200640 1 -- 172.21.15.55:0/2163441327 shutdown_connections 2024-05-10T03:36:10.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.555+0000 7f0ec3200640 1 -- 172.21.15.55:0/2163441327 wait complete. 2024-05-10T03:36:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.557+0000 7f0ec3200640 1 Processor -- start 2024-05-10T03:36:10.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.557+0000 7f0ec3200640 1 -- start start 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ec40b4170 0x7f0ec4159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40ccc80 0x7f0ec4151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4157c50 con 0x7f0ec415a5d0 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41598f0 con 0x7f0ec40b4170 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec3200640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41f4ba0 con 0x7f0ec40ccc80 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec2800640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54910/0 (socket says 172.21.15.55:54910) 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec2800640 1 -- 172.21.15.55:0/3444145883 learned_addr learned my addr 172.21.15.55:0/3444145883 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec1e00640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ec40b4170 0x7f0ec4159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.559+0000 7f0ec1400640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40ccc80 0x7f0ec4151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 -- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40ccc80 msgr2=0x7f0ec4151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40ccc80 0x7f0ec4151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 -- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ec40b4170 msgr2=0x7f0ec4159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ec40b4170 0x7f0ec4159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 -- 172.21.15.55:0/3444145883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009a70 con 0x7f0ec415a5d0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.560+0000 7f0ec2800640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009eb0 tx=0x7f0ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.561+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc03b820 con 0x7f0ec415a5d0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.561+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc03b9a0 con 0x7f0ec415a5d0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.561+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc03bd50 con 0x7f0ec415a5d0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.561+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec4159b40 con 0x7f0ec415a5d0 2024-05-10T03:36:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.561+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec41f4270 con 0x7f0ec415a5d0 2024-05-10T03:36:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.562+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec41f4d30 con 0x7f0ec415a5d0 2024-05-10T03:36:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.563+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ebc031400 con 0x7f0ec415a5d0 2024-05-10T03:36:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.564+0000 7f0eba000640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 0x7f0ea006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.564+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f0ebc0aa350 con 0x7f0ec415a5d0 2024-05-10T03:36:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.565+0000 7f0ec1e00640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 0x7f0ea006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.566+0000 7f0ec1e00640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 0x7f0ea006d110 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4005880 tx=0x7f0eb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.568+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ebc0754f0 con 0x7f0ec415a5d0 2024-05-10T03:36:10.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.763+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ec40b2110 con 0x7f0ea006ac00 2024-05-10T03:36:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.771+0000 7f0eba000640 1 -- 172.21.15.55:0/3444145883 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48533 (secure 0 0 0) 0x7f0ec40b2110 con 0x7f0ea006ac00 2024-05-10T03:36:10.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 msgr2=0x7f0ea006d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 0x7f0ea006d110 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4005880 tx=0x7f0eb4009040 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 msgr2=0x7f0ec40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009eb0 tx=0x7f0ebc004a40 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 shutdown_connections 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ec415a5d0 0x7f0ec40aebd0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ec40ccc80 0x7f0ec4151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0ea006ac00 0x7f0ea006d110 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 --2- 172.21.15.55:0/3444145883 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ec40b4170 0x7f0ec4159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.773+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 >> 172.21.15.55:0/3444145883 conn(0x7f0ec406a110 msgr2=0x7f0ec406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.774+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 shutdown_connections 2024-05-10T03:36:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:10.774+0000 7f0ec3200640 1 -- 172.21.15.55:0/3444145883 wait complete. 2024-05-10T03:36:10.808 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-05-10T03:36:09.102667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261128,"kb_used_data":12168,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713592,"statfs":{"total":773094113280,"available":772826718208,"internally_reserved":0,"allocated":12460032,"data_stored":6345904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003782"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":242,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-10T03:35:55.618556+0000","last_change":"2024-05-10T03:29:34.845830+0000","last_active":"2024-05-10T03:35:55.618556+0000","last_peered":"2024-05-10T03:35:55.618556+0000","last_clean":"2024-05-10T03:35:55.618556+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:35:55.618556+0000","last_undegraded":"2024-05-10T03:35:55.618556+0000","last_fullsized":"2024-05-10T03:35:55.618556+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T03:29:33.249200+0000","last_clean_scrub_stamp":"2024-05-10T03:29:33.249200+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:48:33.776782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.327,"15min":1.351},"min":{"1min":0.78100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8140000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.254}]},{"osd":1,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1459999999999999,"15min":1.143},"min":{"1min":0.78400000000000003,"5min":0.65400000000000003,"15min":0.59299999999999997},"max":{"1min":1.4850000000000001,"5min":1.853,"15min":1.853},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2649999999999999},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.679,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.012}]},{"osd":2,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2869999999999999,"15min":1.357},"min":{"1min":0.90000000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.9079999999999999,"5min":2.1349999999999998,"15min":3.0289999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.2929999999999999,"15min":1.3999999999999999},"min":{"1min":0.80800000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9930000000000001,"5min":2.4649999999999999,"15min":2.6000000000000001},"last":1.5620000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.222,"15min":2.222},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.355,"15min":1.335},"min":{"1min":0.94099999999999995,"5min":0.71899999999999997,"15min":0.70799999999999996},"max":{"1min":1.675,"5min":2.367,"15min":2.367},"last":1.4470000000000001}]},{"osd":4,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3740000000000001,"15min":1.4450000000000001},"min":{"1min":0.61099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.7110000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.49,"15min":1.4970000000000001},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.0680000000000001,"5min":2.323,"15min":2.323},"last":1.508}]},{"osd":5,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.5169999999999999},"min":{"1min":0.70199999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4660000000000002,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.6619999999999999,"15min":1.6870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.601,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5780000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.641,"15min":1.7010000000000001},"min":{"1min":0.73899999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.639,"15min":1.6359999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.2250000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.8120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.1839999999999999,"15min":1.1080000000000001},"min":{"1min":0.80200000000000005,"5min":0.57799999999999996,"15min":0.48699999999999999},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.2110000000000001,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.784}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.4399999999999999},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.60399999999999998},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":4.1429999999999998},"last":1.4330000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3919999999999999,"15min":1.2809999999999999},"min":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.53900000000000003},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.331},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3400000000000001,"15min":1.474},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.569},"last":1.6220000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4119999999999999,"15min":1.5680000000000001},"min":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.488,"5min":3.488,"15min":4.0869999999999997},"last":2.0459999999999998},{"interface":"front","average":{"1min":1.369,"5min":1.3759999999999999,"15min":1.327},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.371}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.468,"15min":1.4399999999999999},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5009999999999999,"15min":1.5640000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.966,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.9279999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.488,"15min":1.4199999999999999},"min":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.635,"15min":1.627},"min":{"1min":0.70499999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.516}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.655,"15min":1.6679999999999999},"min":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1200000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":2.1589999999999998},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.6659999999999999,"15min":1.6599999999999999},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.5469999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.64000000000000001},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.752},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1619999999999999,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7410000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.153,"15min":1.143},"min":{"1min":0.89400000000000002,"5min":0.66300000000000003,"15min":0.57699999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.206,"15min":1.22},"min":{"1min":0.86499999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.954},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.216,"15min":1.2170000000000001},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.7490000000000001,"15min":2.359},"last":1.365},{"interface":"front","average":{"1min":1.288,"5min":1.304,"15min":1.3300000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7909999999999999,"5min":1.913,"15min":1.913},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.347,"15min":1.3520000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4139999999999999,"15min":1.3240000000000001},"min":{"1min":1.1850000000000001,"5min":0.76800000000000002,"15min":0.68200000000000005},"max":{"1min":1.9390000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.272}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.91300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5429999999999999,"15min":1.587},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":3.7759999999999998},"last":1.55}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5329999999999999,"15min":1.518},"min":{"1min":0.875,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9670000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.556,"15min":1.554},"min":{"1min":0.86299999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.129}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.732,"15min":1.79},"min":{"1min":1.167,"5min":0.877,"15min":0.753},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":4.0899999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.653,"15min":1.6240000000000001},"min":{"1min":1.4299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.036,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.579}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.238,"5min":1.151,"15min":1.1439999999999999},"min":{"1min":0.72299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.681,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":1.048}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2130000000000001,"15min":1.1719999999999999},"min":{"1min":0.80600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.679,"5min":1.734,"15min":2.3839999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.2490000000000001,"15min":1.2729999999999999},"min":{"1min":0.997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1619999999999999},"last":1.5620000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.242,"15min":1.2709999999999999},"min":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.702,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.3740000000000001,"15min":1.399},"min":{"1min":0.94099999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.29},"last":1.667}]},{"osd":3,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.341,"15min":1.4370000000000001},"min":{"1min":0.753,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8859999999999999,"5min":2.379,"15min":2.6779999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5269999999999999,"15min":1.5680000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.7709999999999999},"last":1.4930000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4810000000000001,"15min":1.4990000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.512},"last":1.405},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4119999999999999,"15min":1.5209999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0579999999999998,"5min":2.5510000000000002,"15min":2.8109999999999999},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.5249999999999999,"15min":1.5720000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.141,"5min":2.4769999999999999,"15min":2.8530000000000002},"last":1.341},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.619,"15min":1.742},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.597,"5min":2.597,"15min":5.3529999999999998},"last":1.1819999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5880000000000001,"15min":1.6679999999999999},"min":{"1min":0.63600000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.177,"5min":2.6160000000000001,"15min":3.3479999999999999},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.694,"15min":1.7290000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.556,"5min":2.6349999999999998,"15min":2.8700000000000001},"last":1.6020000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.95999999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2130000000000001,"5min":1.6200000000000001,"15min":1.708},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.603},"last":1.137},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.625,"5min":0.377,"15min":0.377},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1559999999999999,"15min":1.177},"min":{"1min":0.63100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8360000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.2390000000000001,"15min":1.335},"min":{"1min":0.72299999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.762,"5min":1.919,"15min":2.2599999999999998},"last":1.085}]},{"osd":4,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3540000000000001,"15min":1.371},"min":{"1min":0.93799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.881,"5min":2.0910000000000002,"15min":2.3610000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4299999999999999,"15min":1.3919999999999999},"min":{"1min":0.95499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0790000000000002,"5min":2.1349999999999998,"15min":2.464},"last":1.5529999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4810000000000001,"15min":1.6299999999999999},"min":{"1min":0.97799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.839,"5min":2.4399999999999999,"15min":3.0470000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.446,"15min":1.478},"min":{"1min":0.998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9399999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.1000000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.544,"15min":1.569},"min":{"1min":0.88200000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.7569999999999999},"min":{"1min":0.92900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9319999999999999,"5min":2.4729999999999999,"15min":3.2280000000000002},"last":1.3440000000000001}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.51,"15min":1.5720000000000001},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.0670000000000002,"5min":2.2229999999999999,"15min":2.4399999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5780000000000001,"15min":1.6379999999999999},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.1429999999999998,"5min":2.29,"15min":2.6190000000000002},"last":1.4890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.88,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.998,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.434,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.643,"5min":1.746,"15min":1.746},"last":2.0529999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1699999999999999},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.1030000000000002},"last":1.048}]},{"osd":3,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.052,"15min":1.1060000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":2.169},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1930000000000001,"15min":1.244},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8380000000000001,"5min":1.9910000000000001,"15min":2.2229999999999999},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.506},"min":{"1min":0.71299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.8050000000000002},"last":1.306},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3580000000000001,"15min":1.504},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.6150000000000002},"last":1.589}]},{"osd":5,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9950000000000001,"5min":2.173,"15min":2.508},"last":1.792},{"interface":"front","average":{"1min":1.587,"5min":1.5589999999999999,"15min":1.607},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.379,"5min":2.379,"15min":2.6669999999999998},"last":1.7350000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.504,"15min":1.542},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.7280000000000002},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.4950000000000001,"15min":1.569},"min":{"1min":0.87,"5min":0.81899999999999995,"15min":0.78200000000000003},"max":{"1min":2.1720000000000002,"5min":2.3799999999999999,"15min":2.8860000000000001},"last":1.9339999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.579,"15min":1.639},"min":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2759999999999998,"5min":2.3149999999999999,"15min":2.8410000000000002},"last":1.994},{"interface":"front","average":{"1min":1.615,"5min":1.5700000000000001,"15min":1.649},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.58999999999999997},"max":{"1min":2.0870000000000002,"5min":2.468,"15min":2.7690000000000001},"last":2.1360000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.89600000000000002,"15min":0.91700000000000004},"min":{"1min":0.42699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.607,"15min":1.607},"last":1.157},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.45800000000000002},"max":{"1min":1.4770000000000001,"5min":1.996,"15min":2.2949999999999999},"last":1.026}]},{"osd":2,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96299999999999997,"15min":0.998},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":1.6910000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.171},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.627,"5min":2.0720000000000001,"15min":2.444},"last":1.3500000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2250000000000001,"15min":1.3029999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.4129999999999998},"last":1.49},{"interface":"front","average":{"1min":1.272,"5min":1.196,"15min":1.2749999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.7799999999999998},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.508},"min":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0230000000000001,"5min":2.3159999999999998,"15min":2.5299999999999998},"last":1.472},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4390000000000001,"15min":1.554},"min":{"1min":0.93899999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.9610000000000001,"5min":2.1139999999999999,"15min":4.2949999999999999},"last":1.4490000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.544},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9350000000000001,"5min":2.2450000000000001,"15min":3.8029999999999999},"last":1.294},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.603},"min":{"1min":0.90600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.1200000000000001,"5min":2.3290000000000002,"15min":2.6190000000000002},"last":1.5760000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.5469999999999999,"15min":1.6859999999999999},"min":{"1min":1.111,"5min":0.94999999999999996,"15min":0.91100000000000003},"max":{"1min":2.1549999999999998,"5min":2.4470000000000001,"15min":3.48},"last":1.407},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5329999999999999,"15min":1.6870000000000001},"min":{"1min":1.028,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0720000000000001,"5min":2.1230000000000002,"15min":3.6880000000000002},"last":1.524}]},{"osd":7,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6100000000000001,"15min":1.708},"min":{"1min":0.87,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.0409999999999999,"5min":2.2709999999999999,"15min":3.2869999999999999},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.619,"5min":1.5920000000000001,"15min":1.74},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.27,"5min":2.5449999999999999,"15min":4.0140000000000002},"last":1.2290000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.875,"15min":0.995},"min":{"1min":0.56899999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":2.1309999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.048,"5min":0.93899999999999995,"15min":1.137},"min":{"1min":0.77400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.532,"5min":1.6120000000000001,"15min":2.9390000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0609999999999999,"15min":1.1910000000000001},"min":{"1min":0.72799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.72,"5min":1.72,"15min":2.4390000000000001},"last":1.196},{"interface":"front","average":{"1min":1.258,"5min":1.0620000000000001,"15min":1.1879999999999999},"min":{"1min":0.755,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7190000000000001,"5min":1.8400000000000001,"15min":3.0009999999999999},"last":0.96199999999999997}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.014,"15min":1.089},"min":{"1min":0.66700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.9850000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2270000000000001,"15min":1.3069999999999999},"min":{"1min":0.68899999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.097,"15min":2.4750000000000001},"last":1.115}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.335,"15min":1.5649999999999999},"min":{"1min":1.071,"5min":0.745,"15min":0.745},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":3.1040000000000001},"last":1.419},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3460000000000001,"15min":1.4179999999999999},"min":{"1min":0.872,"5min":0.84099999999999997,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":3.0790000000000002},"last":1.153}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.3759999999999999,"15min":1.4570000000000001},"min":{"1min":0.95199999999999996,"5min":0.80700000000000005,"15min":0.57899999999999996},"max":{"1min":1.841,"5min":1.9710000000000001,"15min":2.4889999999999999},"last":1.22},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.444,"15min":1.5920000000000001},"min":{"1min":1.028,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0299999999999998,"5min":2.2120000000000002,"15min":2.9279999999999999},"last":1.401}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.552,"15min":1.6279999999999999},"min":{"1min":1.0600000000000001,"5min":0.98099999999999998,"15min":0.90900000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.7189999999999999},"last":1.323},{"interface":"front","average":{"1min":1.629,"5min":1.5029999999999999,"15min":1.6919999999999999},"min":{"1min":1.167,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":3.0539999999999998},"last":1.615}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5209999999999999,"15min":1.7370000000000001},"min":{"1min":1.3029999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.117,"5min":2.117,"15min":3.149},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5249999999999999,"15min":1.569},"min":{"1min":1.2709999999999999,"5min":0.86899999999999999,"15min":0.63700000000000001},"max":{"1min":2.0409999999999999,"5min":2.1509999999999998,"15min":2.3860000000000001},"last":1.2689999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:36:10.812 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-10T03:29:33.249200+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=10, tm_hour=3, tm_min=29, tm_sec=33, tm_wday=4, tm_yday=131, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=10, tm_hour=3, tm_min=36, tm_sec=2, tm_wday=4, tm_yday=131, tm_isdst=0) 2024-05-10T03:36:10.812 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-10T03:36:18.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:18.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:30.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T03:36:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.110+0000 7fb293e00640 1 Processor -- start 2024-05-10T03:36:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.110+0000 7fb293e00640 1 -- start start 2024-05-10T03:36:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.112+0000 7fb293e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29415a5d0 0x7fb2940cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.112+0000 7fb293e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2940cb730 0x7fb2940b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb293e00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 0x7fb29415df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb293e00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2940b4ac0 con 0x7fb2940cb730 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb293e00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29415ad30 con 0x7fb2940b46b0 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb293e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29415b010 con 0x7fb29415a5d0 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2940cb730 0x7fb2940b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2940cb730 0x7fb2940b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49880/0 (socket says 172.21.15.55:49880) 2024-05-10T03:36:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 -- 172.21.15.55:0/169372131 learned_addr learned my addr 172.21.15.55:0/169372131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292a00640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29415a5d0 0x7fb2940cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb293400640 1 --2- 172.21.15.55:0/169372131 >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 0x7fb29415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 -- 172.21.15.55:0/169372131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29415a5d0 msgr2=0x7fb2940cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29415a5d0 0x7fb2940cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 -- 172.21.15.55:0/169372131 >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 msgr2=0x7fb29415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 --2- 172.21.15.55:0/169372131 >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 0x7fb29415df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.113+0000 7fb292000640 1 -- 172.21.15.55:0/169372131 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29415b2f0 con 0x7fb2940cb730 2024-05-10T03:36:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.114+0000 7fb293400640 1 --2- 172.21.15.55:0/169372131 >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 0x7fb29415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.115+0000 7fb292000640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2940cb730 0x7fb2940b4170 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb29406fe50 tx=0x7fb28002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96272f1ff47364d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.115+0000 7fb291600640 1 -- 172.21.15.55:0/169372131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280032440 con 0x7fb2940cb730 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.115+0000 7fb291600640 1 -- 172.21.15.55:0/169372131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2800325a0 con 0x7fb2940cb730 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.115+0000 7fb291600640 1 -- 172.21.15.55:0/169372131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280032930 con 0x7fb2940cb730 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 -- 172.21.15.55:0/169372131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2940cb730 msgr2=0x7fb2940b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2940cb730 0x7fb2940b4170 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb29406fe50 tx=0x7fb28002a1e0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 -- 172.21.15.55:0/169372131 shutdown_connections 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 --2- 172.21.15.55:0/169372131 >> v2:172.21.15.162:3300/0 conn(0x7fb2940b46b0 0x7fb29415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2940cb730 0x7fb2940b4170 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 --2- 172.21.15.55:0/169372131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29415a5d0 0x7fb2940cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 -- 172.21.15.55:0/169372131 >> 172.21.15.55:0/169372131 conn(0x7fb29406a110 msgr2=0x7fb29406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:31.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 -- 172.21.15.55:0/169372131 shutdown_connections 2024-05-10T03:36:31.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.116+0000 7fb293e00640 1 -- 172.21.15.55:0/169372131 wait complete. 2024-05-10T03:36:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.119+0000 7fb293e00640 1 Processor -- start 2024-05-10T03:36:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.119+0000 7fb293e00640 1 -- start start 2024-05-10T03:36:31.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 0x7fb2941f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2940cb730 0x7fb294140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2941f8090 con 0x7fb29415a5d0 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29414ea70 con 0x7fb2940cb730 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.120+0000 7fb293e00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb294140a50 con 0x7fb2940b46b0 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49896/0 (socket says 172.21.15.55:49896) 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 -- 172.21.15.55:0/3474750892 learned_addr learned my addr 172.21.15.55:0/3474750892 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb292000640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2940cb730 0x7fb294140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb292a00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 0x7fb2941f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 -- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 msgr2=0x7fb2941f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 0x7fb2941f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 -- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2940cb730 msgr2=0x7fb294140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2940cb730 0x7fb294140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb293400640 1 -- 172.21.15.55:0/3474750892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800094d0 con 0x7fb29415a5d0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.121+0000 7fb292a00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 0x7fb2941f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb293400640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00ccf0 tx=0x7fb28c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00f680 con 0x7fb29415a5d0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c00f7e0 con 0x7fb29415a5d0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2941f7100 con 0x7fb29415a5d0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00fb90 con 0x7fb29415a5d0 2024-05-10T03:36:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.122+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294140f80 con 0x7fb29415a5d0 2024-05-10T03:36:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.123+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294066ce0 con 0x7fb29415a5d0 2024-05-10T03:36:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.124+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb28c00b040 con 0x7fb29415a5d0 2024-05-10T03:36:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.125+0000 7fb28aa00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 0x7fb27406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.125+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fb28c089cf0 con 0x7fb29415a5d0 2024-05-10T03:36:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.126+0000 7fb292a00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 0x7fb27406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.127+0000 7fb292a00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 0x7fb27406d140 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb294072370 tx=0x7fb284008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.130+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb28c054e90 con 0x7fb29415a5d0 2024-05-10T03:36:31.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.325+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 --> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb29415b8b0 con 0x7fb27406ac30 2024-05-10T03:36:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.331+0000 7fb28aa00640 1 -- 172.21.15.55:0/3474750892 <== mgr.4102 v2:172.21.15.162:6832/34944 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48406 (secure 0 0 0) 0x7fb28400c040 con 0x7fb27406ac30 2024-05-10T03:36:31.335 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:31.335 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-05-10T03:36:31.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 msgr2=0x7fb27406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 0x7fb27406d140 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb294072370 tx=0x7fb284008040 comp rx=0 tx=0).stop 2024-05-10T03:36:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 msgr2=0x7fb2940cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00ccf0 tx=0x7fb28c00a650 comp rx=0 tx=0).stop 2024-05-10T03:36:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 shutdown_connections 2024-05-10T03:36:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb29415a5d0 0x7fb2940cdd40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7fb27406ac30 0x7fb27406d140 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2940cb730 0x7fb294140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 --2- 172.21.15.55:0/3474750892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2940b46b0 0x7fb2941f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.334+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 >> 172.21.15.55:0/3474750892 conn(0x7fb29406a110 msgr2=0x7fb294070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.335+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 shutdown_connections 2024-05-10T03:36:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.335+0000 7fb293e00640 1 -- 172.21.15.55:0/3474750892 wait complete. 2024-05-10T03:36:31.364 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-05-10T03:36:31.109967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261112,"kb_used_data":12168,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754713608,"statfs":{"total":773094113280,"available":772826734592,"internally_reserved":0,"allocated":12460032,"data_stored":6345904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":254791224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003742"},"pg_stats":[{"pgid":"1.0","version":"113'119","reported_seq":256,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-10T03:36:12.496351+0000","last_change":"2024-05-10T03:36:12.496603+0000","last_active":"2024-05-10T03:36:12.496351+0000","last_peered":"2024-05-10T03:36:12.496351+0000","last_clean":"2024-05-10T03:36:12.496351+0000","last_became_active":"2024-05-10T03:29:34.843413+0000","last_became_peered":"2024-05-10T03:29:34.843413+0000","last_unstale":"2024-05-10T03:36:12.496351+0000","last_undegraded":"2024-05-10T03:36:12.496351+0000","last_fullsized":"2024-05-10T03:36:12.496351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'119","last_scrub_stamp":"2024-05-10T03:36:12.496287+0000","last_deep_scrub":"113'119","last_deep_scrub_stamp":"2024-05-10T03:36:12.496287+0000","last_clean_scrub_stamp":"2024-05-10T03:36:12.496287+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:55.993605+0000","scrub_duration":0.014062181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31848906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.389},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.327,"15min":1.351},"min":{"1min":0.78100000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.8140000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.425}]},{"osd":1,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1459999999999999,"15min":1.143},"min":{"1min":0.78400000000000003,"5min":0.65400000000000003,"15min":0.59299999999999997},"max":{"1min":1.4850000000000001,"5min":1.853,"15min":1.853},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.2390000000000001,"15min":1.2649999999999999},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.679,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":1.0649999999999999}]},{"osd":2,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2869999999999999,"15min":1.357},"min":{"1min":0.90000000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.9079999999999999,"5min":2.1349999999999998,"15min":3.0289999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2,"5min":1.2929999999999999,"15min":1.3999999999999999},"min":{"1min":0.80800000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9930000000000001,"5min":2.4649999999999999,"15min":2.6000000000000001},"last":1.804}]},{"osd":3,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.8720000000000001,"5min":2.222,"15min":2.222},"last":1.569},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.355,"15min":1.335},"min":{"1min":0.94099999999999995,"5min":0.71899999999999997,"15min":0.70799999999999996},"max":{"1min":1.675,"5min":2.367,"15min":2.367},"last":1.6499999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3740000000000001,"15min":1.4450000000000001},"min":{"1min":0.61099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.7110000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.302},{"interface":"front","average":{"1min":1.429,"5min":1.49,"15min":1.4970000000000001},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.0680000000000001,"5min":2.323,"15min":2.323},"last":1.7350000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.5169999999999999},"min":{"1min":0.70199999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4660000000000002,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.6619999999999999,"15min":1.6870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.601,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.6870000000000001}]},{"osd":6,"last update":"Fri May 10 03:35:45 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.641,"15min":1.7010000000000001},"min":{"1min":0.73899999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.639,"15min":1.6359999999999999},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.2250000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.8660000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.1839999999999999,"15min":1.1080000000000001},"min":{"1min":0.80200000000000005,"5min":0.57799999999999996,"15min":0.48699999999999999},"max":{"1min":3.1619999999999999,"5min":3.1619999999999999,"15min":3.1619999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2110000000000001,"15min":1.3120000000000001},"min":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":1.244}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.9460000000000002,"5min":3.9460000000000002,"15min":3.9460000000000002},"last":1.167},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.4399999999999999},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.60399999999999998},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":4.1429999999999998},"last":1.571}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3919999999999999,"15min":1.2809999999999999},"min":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.53900000000000003},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.3140000000000001},"last":1.323},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3400000000000001,"15min":1.474},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.5409999999999999,"5min":3.5409999999999999,"15min":3.569},"last":1.829}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4119999999999999,"15min":1.5680000000000001},"min":{"1min":0.82699999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.488,"5min":3.488,"15min":4.0869999999999997},"last":1.512},{"interface":"front","average":{"1min":1.369,"5min":1.3759999999999999,"15min":1.327},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":4.0490000000000004,"5min":4.0490000000000004,"15min":4.0490000000000004},"last":1.3839999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.468,"15min":1.4399999999999999},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.47},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5009999999999999,"15min":1.5640000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.966,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1.7250000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.488,"15min":1.4199999999999999},"min":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.635,"15min":1.627},"min":{"1min":0.70499999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.6779999999999999}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.655,"15min":1.6679999999999999},"min":{"1min":0.61199999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1200000000000001,"5min":2.8410000000000002,"15min":2.8410000000000002},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.6659999999999999,"15min":1.6599999999999999},"min":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":1.796}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.73199999999999998,"5min":0.69299999999999995,"15min":0.64000000000000001},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.88},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1619999999999999,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7410000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.2809999999999999}]},{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.153,"15min":1.143},"min":{"1min":0.89400000000000002,"5min":0.66300000000000003,"15min":0.57699999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.206,"15min":1.22},"min":{"1min":0.86499999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.954},"last":1.1140000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.216,"15min":1.2170000000000001},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.7490000000000001,"15min":2.359},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.304,"15min":1.3300000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7909999999999999,"5min":1.913,"15min":1.913},"last":1.2549999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.347,"15min":1.3520000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4139999999999999,"15min":1.3240000000000001},"min":{"1min":1.1850000000000001,"5min":0.76800000000000002,"15min":0.68200000000000005},"max":{"1min":1.9390000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7450000000000001}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.91300000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5429999999999999,"15min":1.587},"min":{"1min":0.73399999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":3.7759999999999998},"last":1.6559999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5329999999999999,"15min":1.518},"min":{"1min":0.875,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9670000000000001,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.508},{"interface":"front","average":{"1min":1.595,"5min":1.556,"15min":1.554},"min":{"1min":0.86299999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7160000000000002,"5min":2.7160000000000002,"15min":2.7160000000000002},"last":1.7010000000000001}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.732,"15min":1.79},"min":{"1min":1.167,"5min":0.877,"15min":0.753},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":4.0899999999999999},"last":1.78},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.653,"15min":1.6240000000000001},"min":{"1min":1.4299999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.036,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.4299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1299999999999999,"15min":1.163},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.238,"5min":1.151,"15min":1.1439999999999999},"min":{"1min":0.72299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.681,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2130000000000001,"15min":1.1719999999999999},"min":{"1min":0.80600000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.679,"5min":1.734,"15min":2.3839999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.2490000000000001,"15min":1.2729999999999999},"min":{"1min":0.997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1619999999999999},"last":1.4970000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.242,"15min":1.2709999999999999},"min":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.702,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.159},{"interface":"front","average":{"1min":1.472,"5min":1.3740000000000001,"15min":1.399},"min":{"1min":0.94099999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.29},"last":1.2969999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.341,"15min":1.4370000000000001},"min":{"1min":0.753,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8859999999999999,"5min":2.379,"15min":2.6779999999999999},"last":1.228},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5269999999999999,"15min":1.5680000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.25,"5min":2.25,"15min":2.7709999999999999},"last":1.629}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4810000000000001,"15min":1.4990000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.512},"last":1.208},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4119999999999999,"15min":1.5209999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0579999999999998,"5min":2.5510000000000002,"15min":2.8109999999999999},"last":1.331}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.5249999999999999,"15min":1.5720000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.141,"5min":2.4769999999999999,"15min":2.8530000000000002},"last":1.76},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.619,"15min":1.742},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.597,"5min":2.597,"15min":5.3529999999999998},"last":1.427}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5880000000000001,"15min":1.6679999999999999},"min":{"1min":0.63600000000000001,"5min":0.495,"15min":0.495},"max":{"1min":2.177,"5min":2.6160000000000001,"15min":3.3479999999999999},"last":1.726},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.694,"15min":1.7290000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.556,"5min":2.6349999999999998,"15min":2.8700000000000001},"last":1.911}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.93200000000000005,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.419},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.90800000000000003,"15min":0.95999999999999996},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2130000000000001,"5min":1.6200000000000001,"15min":1.708},"last":1.3620000000000001}]},{"osd":1,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.96399999999999997},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.603},"last":1.294},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1499999999999999,"15min":1.224},"min":{"1min":0.625,"5min":0.377,"15min":0.377},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1559999999999999,"15min":1.177},"min":{"1min":0.63100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8360000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.2390000000000001,"15min":1.335},"min":{"1min":0.72299999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.762,"5min":1.919,"15min":2.2599999999999998},"last":1.5289999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3540000000000001,"15min":1.371},"min":{"1min":0.93799999999999994,"5min":0.626,"15min":0.626},"max":{"1min":1.881,"5min":2.0910000000000002,"15min":2.3610000000000002},"last":1.5},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4299999999999999,"15min":1.3919999999999999},"min":{"1min":0.95499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0790000000000002,"5min":2.1349999999999998,"15min":2.464},"last":1.8460000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4810000000000001,"15min":1.6299999999999999},"min":{"1min":0.97799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.839,"5min":2.4399999999999999,"15min":3.0470000000000002},"last":1.71},{"interface":"front","average":{"1min":1.456,"5min":1.446,"15min":1.478},"min":{"1min":0.998,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.9399999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.667}]},{"osd":6,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.544,"15min":1.569},"min":{"1min":0.88200000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.633,"15min":1.7569999999999999},"min":{"1min":0.92900000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9319999999999999,"5min":2.4729999999999999,"15min":3.2280000000000002},"last":1.889}]},{"osd":7,"last update":"Fri May 10 03:35:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.51,"15min":1.5720000000000001},"min":{"1min":1.02,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.0670000000000002,"5min":2.2229999999999999,"15min":2.4399999999999999},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5780000000000001,"15min":1.6379999999999999},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.1429999999999998,"5min":2.29,"15min":2.6190000000000002},"last":1.74}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.88,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.998,"5min":0.92100000000000004,"15min":0.97299999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.84499999999999997}]},{"osd":1,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97899999999999998,"15min":0.97299999999999998},"min":{"1min":0.434,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.643,"5min":1.746,"15min":1.746},"last":1.52},{"interface":"front","average":{"1min":1.159,"5min":1.1180000000000001,"15min":1.1699999999999999},"min":{"1min":0.60599999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.1030000000000002},"last":1.2769999999999999}]},{"osd":3,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.052,"15min":1.1060000000000001},"min":{"1min":0.56499999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":2.169},"last":1.712},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1930000000000001,"15min":1.244},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8380000000000001,"5min":1.9910000000000001,"15min":2.2229999999999999},"last":1.1399999999999999}]},{"osd":4,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.401,"15min":1.506},"min":{"1min":0.71299999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.8050000000000002},"last":1.006},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3580000000000001,"15min":1.504},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.6150000000000002},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.9950000000000001,"5min":2.173,"15min":2.508},"last":1.9139999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.5589999999999999,"15min":1.607},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.379,"5min":2.379,"15min":2.6669999999999998},"last":1.4139999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.504,"15min":1.542},"min":{"1min":0.73799999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.7280000000000002},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.4950000000000001,"15min":1.569},"min":{"1min":0.87,"5min":0.81899999999999995,"15min":0.78200000000000003},"max":{"1min":2.1720000000000002,"5min":2.3799999999999999,"15min":2.8860000000000001},"last":1.827}]},{"osd":7,"last update":"Fri May 10 03:35:36 2024","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.579,"15min":1.639},"min":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.2759999999999998,"5min":2.3149999999999999,"15min":2.8410000000000002},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.5700000000000001,"15min":1.649},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.58999999999999997},"max":{"1min":2.0870000000000002,"5min":2.468,"15min":2.7690000000000001},"last":2.0230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32496,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1409024,"data_stored":645646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.89600000000000002,"15min":0.91700000000000004},"min":{"1min":0.42699999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3149999999999999,"5min":1.607,"15min":1.607},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.45800000000000002},"max":{"1min":1.4770000000000001,"5min":1.996,"15min":2.2949999999999999},"last":1.095}]},{"osd":2,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.96299999999999997,"15min":0.998},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":1.6910000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.159},{"interface":"front","average":{"1min":1.081,"5min":1.111,"15min":1.171},"min":{"1min":0.48899999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.627,"5min":2.0720000000000001,"15min":2.444},"last":1.0700000000000001}]},{"osd":3,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2250000000000001,"15min":1.3029999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.4129999999999998},"last":1.284},{"interface":"front","average":{"1min":1.272,"5min":1.196,"15min":1.2749999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.7799999999999998},"last":1.252}]},{"osd":4,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.508},"min":{"1min":0.84899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.0230000000000001,"5min":2.3159999999999998,"15min":2.5299999999999998},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4390000000000001,"15min":1.554},"min":{"1min":0.93899999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.9610000000000001,"5min":2.1139999999999999,"15min":4.2949999999999999},"last":1.4379999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.544},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.9350000000000001,"5min":2.2450000000000001,"15min":3.8029999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.603},"min":{"1min":0.90600000000000003,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.1200000000000001,"5min":2.3290000000000002,"15min":2.6190000000000002},"last":1.796}]},{"osd":6,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.6519999999999999,"5min":1.5469999999999999,"15min":1.6859999999999999},"min":{"1min":1.111,"5min":0.94999999999999996,"15min":0.91100000000000003},"max":{"1min":2.1549999999999998,"5min":2.4470000000000001,"15min":3.48},"last":1.756},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5329999999999999,"15min":1.6870000000000001},"min":{"1min":1.028,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":2.0720000000000001,"5min":2.1230000000000002,"15min":3.6880000000000002},"last":1.8500000000000001}]},{"osd":7,"last update":"Fri May 10 03:35:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.6100000000000001,"15min":1.708},"min":{"1min":0.87,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.0409999999999999,"5min":2.2709999999999999,"15min":3.2869999999999999},"last":1.9019999999999999},{"interface":"front","average":{"1min":1.619,"5min":1.5920000000000001,"15min":1.74},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.27,"5min":2.5449999999999999,"15min":4.0140000000000002},"last":1.5740000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":2002944,"data_stored":1236014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.875,"15min":0.995},"min":{"1min":0.56899999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":2.1309999999999998},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.048,"5min":0.93899999999999995,"15min":1.137},"min":{"1min":0.77400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.532,"5min":1.6120000000000001,"15min":2.9390000000000001},"last":1.4710000000000001}]},{"osd":2,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0609999999999999,"15min":1.1910000000000001},"min":{"1min":0.72799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.72,"5min":1.72,"15min":2.4390000000000001},"last":1.262},{"interface":"front","average":{"1min":1.258,"5min":1.0620000000000001,"15min":1.1879999999999999},"min":{"1min":0.755,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7190000000000001,"5min":1.8400000000000001,"15min":3.0009999999999999},"last":1.331}]},{"osd":3,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.014,"15min":1.089},"min":{"1min":0.66700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.9850000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2270000000000001,"15min":1.3069999999999999},"min":{"1min":0.68899999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.097,"15min":2.4750000000000001},"last":1.391}]},{"osd":4,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.335,"15min":1.5649999999999999},"min":{"1min":1.071,"5min":0.745,"15min":0.745},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":3.1040000000000001},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3460000000000001,"15min":1.4179999999999999},"min":{"1min":0.872,"5min":0.84099999999999997,"15min":0.67800000000000005},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":3.0790000000000002},"last":1.6419999999999999}]},{"osd":5,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.3759999999999999,"15min":1.4570000000000001},"min":{"1min":0.95199999999999996,"5min":0.80700000000000005,"15min":0.57899999999999996},"max":{"1min":1.841,"5min":1.9710000000000001,"15min":2.4889999999999999},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.444,"15min":1.5920000000000001},"min":{"1min":1.028,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0299999999999998,"5min":2.2120000000000002,"15min":2.9279999999999999},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.552,"15min":1.6279999999999999},"min":{"1min":1.0600000000000001,"5min":0.98099999999999998,"15min":0.90900000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.7189999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.5029999999999999,"15min":1.6919999999999999},"min":{"1min":1.167,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":3.0539999999999998},"last":1.5980000000000001}]},{"osd":7,"last update":"Fri May 10 03:35:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5209999999999999,"15min":1.7370000000000001},"min":{"1min":1.3029999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.117,"5min":2.117,"15min":3.149},"last":1.778},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5249999999999999,"15min":1.569},"min":{"1min":1.2709999999999999,"5min":0.86899999999999999,"15min":0.63700000000000001},"max":{"1min":2.0409999999999999,"5min":2.1509999999999998,"15min":2.3860000000000001},"last":1.724}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T03:36:31.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-10T03:36:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.636+0000 7f0ffbe00640 1 Processor -- start 2024-05-10T03:36:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.636+0000 7f0ffbe00640 1 -- start start 2024-05-10T03:36:31.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 0x7f0ffc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ffc06b120 0x7f0ffc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc0710f0 0x7f0ffc071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ffc15a900 con 0x7f0ffc06b120 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ffc0b2a30 con 0x7f0ffc15e920 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffbe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc15eec0 con 0x7f0ffc0710f0 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffa000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ffc06b120 0x7f0ffc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffa000640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ffc06b120 0x7f0ffc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49904/0 (socket says 172.21.15.55:49904) 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.639+0000 7f0ffa000640 1 -- 172.21.15.55:0/1516724594 learned_addr learned my addr 172.21.15.55:0/1516724594 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffaa00640 1 --2- 172.21.15.55:0/1516724594 >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 0x7f0ffc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffb400640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc0710f0 0x7f0ffc071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffa000640 1 -- 172.21.15.55:0/1516724594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc0710f0 msgr2=0x7f0ffc071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffa000640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc0710f0 0x7f0ffc071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffa000640 1 -- 172.21.15.55:0/1516724594 >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 msgr2=0x7f0ffc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffa000640 1 --2- 172.21.15.55:0/1516724594 >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 0x7f0ffc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffa000640 1 -- 172.21.15.55:0/1516724594 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc06ba70 con 0x7f0ffc06b120 2024-05-10T03:36:31.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.640+0000 7f0ffaa00640 1 --2- 172.21.15.55:0/1516724594 >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 0x7f0ffc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:31.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.641+0000 7f0ffa000640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc06b120 0x7f0ffc06b530 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc072190 tx=0x7f0fe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4e5f8f7f9078d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.641+0000 7f0ff9600640 1 -- 172.21.15.55:0/1516724594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8032440 con 0x7f0ffc06b120 2024-05-10T03:36:31.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.641+0000 7f0ff9600640 1 -- 172.21.15.55:0/1516724594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe80325a0 con 0x7f0ffc06b120 2024-05-10T03:36:31.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.641+0000 7f0ff9600640 1 -- 172.21.15.55:0/1516724594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8032930 con 0x7f0ffc06b120 2024-05-10T03:36:31.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 -- 172.21.15.55:0/1516724594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc06b120 msgr2=0x7f0ffc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc06b120 0x7f0ffc06b530 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc072190 tx=0x7f0fe802a1e0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 -- 172.21.15.55:0/1516724594 shutdown_connections 2024-05-10T03:36:31.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc0710f0 0x7f0ffc071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/1516724594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc06b120 0x7f0ffc06b530 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/1516724594 >> v2:172.21.15.162:3300/0 conn(0x7f0ffc15e920 0x7f0ffc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 -- 172.21.15.55:0/1516724594 >> 172.21.15.55:0/1516724594 conn(0x7f0ffc06c750 msgr2=0x7f0ffc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:31.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 -- 172.21.15.55:0/1516724594 shutdown_connections 2024-05-10T03:36:31.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.642+0000 7f0ffbe00640 1 -- 172.21.15.55:0/1516724594 wait complete. 2024-05-10T03:36:31.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.645+0000 7f0ffbe00640 1 Processor -- start 2024-05-10T03:36:31.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.646+0000 7f0ffbe00640 1 -- start start 2024-05-10T03:36:31.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc06b120 0x7f0ffc14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ffc0710f0 0x7f0ffc13a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc14df60 con 0x7f0ffc15e920 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc13bd40 con 0x7f0ffc0710f0 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffbe00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc13b130 con 0x7f0ffc06b120 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffb400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49914/0 (socket says 172.21.15.55:49914) 2024-05-10T03:36:31.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffb400640 1 -- 172.21.15.55:0/2188909300 learned_addr learned my addr 172.21.15.55:0/2188909300 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:31.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.648+0000 7f0ffaa00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc06b120 0x7f0ffc14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffa000640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ffc0710f0 0x7f0ffc13a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 -- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc06b120 msgr2=0x7f0ffc14f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc06b120 0x7f0ffc14f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 -- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ffc0710f0 msgr2=0x7f0ffc13a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ffc0710f0 0x7f0ffc13a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 -- 172.21.15.55:0/2188909300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80094d0 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ffb400640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400cc00 tx=0x7f0ff400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.649+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400ae80 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.650+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff400ce10 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.650+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4003b90 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.650+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc13b410 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.650+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc13b9d0 con 0x7f0ffc15e920 2024-05-10T03:36:31.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.651+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ffc04e290 con 0x7f0ffc15e920 2024-05-10T03:36:31.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.652+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0ff400f3f0 con 0x7f0ffc15e920 2024-05-10T03:36:31.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.653+0000 7f0ff2a00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 0x7f0fdc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:31.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.653+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f0ff40897f0 con 0x7f0ffc15e920 2024-05-10T03:36:31.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.654+0000 7f0ffaa00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 0x7f0fdc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:31.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.656+0000 7f0ffaa00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 0x7f0fdc06d110 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00a8d0 tx=0x7f0fec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:31.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.658+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0ff4054900 con 0x7f0ffc15e920 2024-05-10T03:36:31.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.859+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0ffc13be80 con 0x7f0ffc15e920 2024-05-10T03:36:31.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.867+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0ff40078a0 con 0x7f0ffc15e920 2024-05-10T03:36:31.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.867+0000 7f0ff2a00640 1 -- 172.21.15.55:0/2188909300 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0ff401d070 con 0x7f0ffc15e920 2024-05-10T03:36:31.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.870+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 msgr2=0x7f0fdc06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.870+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 0x7f0fdc06d110 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00a8d0 tx=0x7f0fec008040 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.870+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 msgr2=0x7f0ffc159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.870+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400cc00 tx=0x7f0ff400a5c0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 shutdown_connections 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc15e920 0x7f0ffc159370 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f0fdc06ac00 0x7f0fdc06d110 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ffc0710f0 0x7f0ffc13a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 --2- 172.21.15.55:0/2188909300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ffc06b120 0x7f0ffc14f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 >> 172.21.15.55:0/2188909300 conn(0x7f0ffc06c750 msgr2=0x7f0ffc13e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 shutdown_connections 2024-05-10T03:36:31.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:31.871+0000 7f0ffbe00640 1 -- 172.21.15.55:0/2188909300 wait complete. 2024-05-10T03:36:31.897 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T03:36:31.898 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T03:36:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.175+0000 7f846d400640 1 Processor -- start 2024-05-10T03:36:32.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.175+0000 7f846d400640 1 -- start start 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846d400640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f846815a5d0 0x7f84680cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846d400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84680cb730 0x7f84680b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846d400640 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84680b4ac0 con 0x7f84680b46b0 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846d400640 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680cdc90 con 0x7f846815a5d0 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846d400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846815ad30 con 0x7f84680cb730 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f8467e00640 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f846815a5d0 0x7f84680cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846ca00640 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f84680b46b0 0x7f846815df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57234/0 (socket says 172.21.15.55:57234) 2024-05-10T03:36:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.177+0000 7f846ca00640 1 -- 172.21.15.55:0/38367918 learned_addr learned my addr 172.21.15.55:0/38367918 (peer_addr_for_me v1:172.21.15.55:0/0) 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8467400640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84680cb730 0x7f84680b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8467e00640 1 -- 172.21.15.55:0/38367918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84680cb730 msgr2=0x7f84680b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8467e00640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84680cb730 0x7f84680b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8467e00640 1 -- 172.21.15.55:0/38367918 >> v1:172.21.15.55:6789/0 conn(0x7f84680b46b0 legacy=0x7f846815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8467e00640 1 -- 172.21.15.55:0/38367918 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846815b010 con 0x7f846815a5d0 2024-05-10T03:36:32.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.178+0000 7f8466a00640 1 -- 172.21.15.55:0/38367918 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075020608 0 0) 0x7f84680b4ac0 con 0x7f84680b46b0 2024-05-10T03:36:32.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.179+0000 7f8467e00640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846815a5d0 0x7f84680cb1f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f84500097a0 tx=0x7f845002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d1fbabcc7c7f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:32.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.179+0000 7f8466a00640 1 -- 172.21.15.55:0/38367918 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450032440 con 0x7f846815a5d0 2024-05-10T03:36:32.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f8466a00640 1 -- 172.21.15.55:0/38367918 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f84500325a0 con 0x7f846815a5d0 2024-05-10T03:36:32.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f8466a00640 1 -- 172.21.15.55:0/38367918 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450032950 con 0x7f846815a5d0 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 -- 172.21.15.55:0/38367918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846815a5d0 msgr2=0x7f84680cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846815a5d0 0x7f84680cb1f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f84500097a0 tx=0x7f845002a180 comp rx=0 tx=0).stop 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 -- 172.21.15.55:0/38367918 shutdown_connections 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84680cb730 0x7f84680b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 --2- 172.21.15.55:0/38367918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f846815a5d0 0x7f84680cb1f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.180+0000 7f846d400640 1 -- 172.21.15.55:0/38367918 >> 172.21.15.55:0/38367918 conn(0x7f846806a110 msgr2=0x7f846806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.181+0000 7f846d400640 1 -- 172.21.15.55:0/38367918 shutdown_connections 2024-05-10T03:36:32.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.181+0000 7f846d400640 1 -- 172.21.15.55:0/38367918 wait complete. 2024-05-10T03:36:32.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.183+0000 7f846d400640 1 Processor -- start 2024-05-10T03:36:32.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.183+0000 7f846d400640 1 -- start start 2024-05-10T03:36:32.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84680b46b0 0x7f84681592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 0x7f8468155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846814df20 con 0x7f84680cb730 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468157fe0 con 0x7f84680b46b0 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846d400640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8468158230 con 0x7f846815a5d0 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49928/0 (socket says 172.21.15.55:49928) 2024-05-10T03:36:32.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 -- 172.21.15.55:0/2932773036 learned_addr learned my addr 172.21.15.55:0/2932773036 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-05-10T03:36:32.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f846ca00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 0x7f8468155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 -- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 msgr2=0x7f8468155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467e00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84680b46b0 0x7f84681592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 0x7f8468155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 -- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84680b46b0 msgr2=0x7f84681592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.185+0000 7f8467400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84680b46b0 0x7f84681592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f8467400640 1 -- 172.21.15.55:0/2932773036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450009400 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f846ca00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 0x7f8468155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f8467400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f845800d5f0 tx=0x7f845800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580125f0 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8458012750 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458012b20 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8468158510 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.186+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8468158ad0 con 0x7f84680cb730 2024-05-10T03:36:32.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.188+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8468066ce0 con 0x7f84680cb730 2024-05-10T03:36:32.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.189+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8458012c80 con 0x7f84680cb730 2024-05-10T03:36:32.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.190+0000 7f845fe00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 0x7f844406d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T03:36:32.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.190+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f8458089a30 con 0x7f84680cb730 2024-05-10T03:36:32.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.190+0000 7f8467e00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 0x7f844406d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T03:36:32.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.192+0000 7f8467e00640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 0x7f844406d110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f845002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T03:36:32.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.195+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8458054ab0 con 0x7f84680cb730 2024-05-10T03:36:32.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.438+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f846804e290 con 0x7f84680cb730 2024-05-10T03:36:32.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.440+0000 7f845fe00640 1 -- 172.21.15.55:0/2932773036 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f84580569e0 con 0x7f84680cb730 2024-05-10T03:36:32.443 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-05-10T03:36:32.443 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2813909468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2813909468},{"type":"v1","addr":"172.21.15.162:6835","nonce":2813909468}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6307,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/920806179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":920806179},{"type":"v1","addr":"172.21.15.162:6837","nonce":920806179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/1414951448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1414951448},{"type":"v1","addr":"172.21.15.55:6833","nonce":1414951448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/364047900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":364047900},{"type":"v1","addr":"172.21.15.55:6835","nonce":364047900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2024-05-10T03:36:32.443 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 71 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 msgr2=0x7f844406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 0x7f844406d110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f845002e000 comp rx=0 tx=0).stop 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 msgr2=0x7f8468144f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f845800d5f0 tx=0x7f845800dab0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 shutdown_connections 2024-05-10T03:36:32.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f846815a5d0 0x7f8468155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84680cb730 0x7f8468144f60 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:6832/34944,v1:172.21.15.162:6833/34944] conn(0x7f844406ac00 0x7f844406d110 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 --2- 172.21.15.55:0/2932773036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84680b46b0 0x7f84681592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.442+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 >> 172.21.15.55:0/2932773036 conn(0x7f846806a110 msgr2=0x7f846814bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.443+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 shutdown_connections 2024-05-10T03:36:32.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-05-10T03:36:32.443+0000 7f846d400640 1 -- 172.21.15.55:0/2932773036 wait complete. 2024-05-10T03:36:32.472 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-10T03:36:32.472 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-10T03:36:32.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-10T03:36:32.472 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:32.507 INFO:tasks.ceph.mds.a:Stopped 2024-05-10T03:36:32.507 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-10T03:36:32.507 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:32.605 INFO:tasks.ceph.mds.c:Stopped 2024-05-10T03:36:32.605 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-10T03:36:32.605 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:32.658 INFO:tasks.ceph.mds.b:Stopped 2024-05-10T03:36:32.658 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-10T03:36:32.658 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:32.693 INFO:tasks.ceph.mds.d:Stopped 2024-05-10T03:36:32.693 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-10T03:36:32.693 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-10T03:36:32.693 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:34.313 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-10T03:36:38.795 INFO:tasks.ceph.osd.0:Stopped 2024-05-10T03:36:38.795 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-10T03:36:38.795 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:38.870 INFO:tasks.ceph.osd.1:Stopped 2024-05-10T03:36:38.870 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-10T03:36:38.871 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:38.948 INFO:tasks.ceph.osd.2:Stopped 2024-05-10T03:36:38.948 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-10T03:36:38.948 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:39.018 INFO:tasks.ceph.osd.3:Stopped 2024-05-10T03:36:39.018 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-10T03:36:39.018 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:39.115 INFO:tasks.ceph.osd.4:Stopped 2024-05-10T03:36:39.115 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-10T03:36:39.116 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:39.186 INFO:tasks.ceph.osd.5:Stopped 2024-05-10T03:36:39.186 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-10T03:36:39.186 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:39.279 INFO:tasks.ceph.osd.6:Stopped 2024-05-10T03:36:39.279 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-10T03:36:39.279 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:39.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-10T03:36:39.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-10T03:36:45.382 INFO:tasks.ceph.osd.7:Stopped 2024-05-10T03:36:45.382 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-10T03:36:45.382 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-10T03:36:45.382 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:45.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~6s 2024-05-10T03:36:48.604 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:48.612 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:50.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-10T03:36:50.796 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-10T03:36:51.484 INFO:tasks.ceph.mgr.y:Stopped 2024-05-10T03:36:51.485 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-10T03:36:51.485 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:51.576 INFO:tasks.ceph.mgr.x:Stopped 2024-05-10T03:36:51.577 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-10T03:36:51.577 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-10T03:36:51.577 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:51.637 INFO:tasks.ceph.mon.a:Stopped 2024-05-10T03:36:51.637 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-10T03:36:51.637 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:51.680 INFO:tasks.ceph.mon.b:Stopped 2024-05-10T03:36:51.681 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-10T03:36:51.681 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T03:36:51.728 INFO:tasks.ceph.mon.c:Stopped 2024-05-10T03:36:51.728 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-10T03:36:51.728 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'client session with non-allowable root' | head -n 1 2024-05-10T03:36:51.837 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:36:51.837 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-10T03:36:52.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:36:52.074 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-10T03:36:52.212 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:36:52.212 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-10T03:36:52.350 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:36:52.351 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-10T03:36:52.524 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:36:52.524 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-10T03:36:52.720 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:36:52.720 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-10T03:36:52.870 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:36:52.870 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-10T03:36:53.036 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:36:53.036 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-10T03:36:53.147 INFO:tasks.ceph:Archiving mon data... 2024-05-10T03:36:53.148 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/data/mon.a.tgz 2024-05-10T03:36:53.149 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-05-10T03:36:53.210 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.Rhf4JxX8tt 2024-05-10T03:36:53.210 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Rhf4JxX8tt 2024-05-10T03:36:53.433 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.Rhf4JxX8tt 2024-05-10T03:36:53.583 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.Rhf4JxX8tt is 405KB 2024-05-10T03:36:53.699 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.Rhf4JxX8tt 2024-05-10T03:36:53.758 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/data/mon.b.tgz 2024-05-10T03:36:53.758 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-05-10T03:36:53.817 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.3u2qcUCVAy 2024-05-10T03:36:53.817 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3u2qcUCVAy 2024-05-10T03:36:54.008 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.3u2qcUCVAy 2024-05-10T03:36:54.136 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.3u2qcUCVAy is 411KB 2024-05-10T03:36:54.244 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.3u2qcUCVAy 2024-05-10T03:36:54.301 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/data/mon.c.tgz 2024-05-10T03:36:54.301 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-05-10T03:36:54.352 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.B9E5Uk3xrL 2024-05-10T03:36:54.352 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.B9E5Uk3xrL 2024-05-10T03:36:54.528 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.B9E5Uk3xrL 2024-05-10T03:36:54.668 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.B9E5Uk3xrL is 411KB 2024-05-10T03:36:54.777 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.B9E5Uk3xrL 2024-05-10T03:36:54.834 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-10T03:36:54.834 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-10T03:36:54.842 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-10T03:36:54.927 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-10T03:36:54.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-10T03:36:54.933 DEBUG:teuthology.orchestra.run.smithi162:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-10T03:36:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-10T03:36:55.022 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-10T03:36:55.034 INFO:tasks.ceph:Archiving crash dumps... 2024-05-10T03:36:55.035 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi055/crash 2024-05-10T03:36:55.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-10T03:36:55.109 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi162/crash 2024-05-10T03:36:55.110 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-10T03:36:55.195 INFO:tasks.ceph:Shutting down logrotate 2024-05-10T03:36:55.195 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:55.203 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:55.324 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:55.416 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-10T03:36:55.498 INFO:tasks.ceph:Compressing logs... 2024-05-10T03:36:55.499 DEBUG:teuthology.orchestra.run.smithi055:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T03:36:55.506 DEBUG:teuthology.orchestra.run.smithi162:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T03:36:55.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-10T03:37:01.678 INFO:tasks.ceph:Archiving logs... 2024-05-10T03:37:01.678 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi055/log 2024-05-10T03:37:01.679 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T03:37:02.816 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi162/log 2024-05-10T03:37:02.817 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T03:37:03.574 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T03:37:03.584 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T03:37:03.585 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T03:37:03.593 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T03:37:03.727 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T03:37:03.737 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T03:37:03.737 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:37:03.745 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T03:37:03.796 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-05-10T03:37:03.801 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1692us[+1713us] +/- 40ms 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -2316us[-2295us] +/- 77ms 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -854us[ -833us] +/- 69ms 2024-05-10T03:37:03.807 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +1694us[+1690us] +/- 40ms 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -2301us[-2305us] +/- 77ms 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -839us[ -843us] +/- 69ms 2024-05-10T03:37:03.815 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T03:37:03.815 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T03:37:03.825 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T03:37:03.826 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T03:37:03.835 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T03:37:04.064 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T03:37:04.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:37:04.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:37:04.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T03:37:04.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:37:04.460 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T03:37:04.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T03:37:04.480 INFO:teuthology.task.internal:Duration was 1103.420747 seconds 2024-05-10T03:37:04.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T03:37:04.491 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T03:37:04.491 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T03:37:04.499 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T03:37:04.630 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:37:04.632 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T03:37:05.047 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T03:37:05.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2024-05-10T03:37:05.047 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T03:37:05.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-05-10T03:37:05.110 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T03:37:05.183 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T03:37:05.183 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T03:37:05.191 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T03:37:05.276 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T03:37:05.276 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T03:37:05.281 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T03:37:05.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T03:37:05.491 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T03:37:05.491 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T03:37:05.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T03:37:05.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T03:37:05.685 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T03:37:05.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T03:37:05.788 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2024-05-10T03:37:05.794 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-05-10T03:37:05.837 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T03:37:05.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:37:05.888 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T03:37:05.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T03:37:05.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T03:37:05.952 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T03:37:05.953 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi055 2024-05-10T03:37:05.953 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T03:37:06.058 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700476/remote/smithi162 2024-05-10T03:37:06.058 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T03:37:06.159 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T03:37:06.159 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T03:37:06.167 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T03:37:06.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T03:37:06.235 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T03:37:06.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T03:37:06.246 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T03:37:06.247 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T03:37:06.254 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T03:37:06.305 INFO:teuthology.orchestra.run.smithi055.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 03:37 /home/ubuntu/cephtest 2024-05-10T03:37:06.307 INFO:teuthology.orchestra.run.smithi162.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 03:37 /home/ubuntu/cephtest 2024-05-10T03:37:06.310 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T03:37:06.329 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T03:37:06.357 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e client.1-kernel-sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/sessionmap} duration: 1103.4207472801208 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-10T03:37:06.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T03:37:06.440 INFO:teuthology.run:pass